Promoting a deep approach to professional learning in the field
Development and evaluation
of electronic personalised learning
G Finnerty, M A Volante, L Rockingham,
J MacLaren and M O’Driscoll
Centre for Research in Nursing and Midwifery Education
Division of Health and Social Care
Faculty of Health and Medical Sciences
June 2008
Acknowledgements
We would like to thank:
All the students who participated in the project from Cohorts 1 and 2 from the Mentorship module.
The four students who gave of their time to speak to us about their mentor experience.
Colleagues in the Mentorship teaching team.
Colleagues from the University of Surrey e-learning units and in particular Dr Steve Rowett whose resourcefulness made the project possible.
Members of the Steering Group.
Dr Andrew Comrie and Professor Pam Smith for advice, guidance and support throughout the project.
This project was funded by the University of Surrey Fund for the Strategic Development of Learning and Teaching.
CONTENTS
1.0Summary
2.0Introduction
3.0Background
4.0Module design, development and delivery for personalised learning plans ……………………………………………………………………………………...…
5.0 Literature Review
6.0 Project approach and evaluation
7.0 Findings
8.0Discussion
9.0Conclusions and recommendations
APPENDIX ONE
Glossary of terms
E-tivities / Electronic activitiesICT / Technology
KSF / Knowledge and Skills Framework
PDA / Personal Digital Assistant
SPT / Specialist Practice Teacher
VLE / Virtual Learning Environment
1.0Summary
Aim of project
- To develop an e-portfolio to manage student progress and assist with authentic assessment on the Mentorship module(FHMS).
Objectives
- To evaluate student experience of peer-supported reflective learning using electronic portfolios
- To increase staff understanding and potential of e-portfolios as a way to engage students in meaningful learning.
Evaluation Methods
- Nominal group technique and e-learning questionnaire involving e-voting by students
- Semi-structured interviews with module tutors in FHMS and staff from e-learning unit
- Biographical narrative interviews with a sample of students following successful completion of the Mentorship Module.
Key Findings
- Mentor students managed their learning and progress through use of the Virtual Learning Environment (U-Learn). Support was presented through multiple strands, for example, handbooks, lectures, peer engagement, a help desk, IT services and the E-Learning Unit.
- Mentor students actively engaged in the reflection process to inform their learning and, importantly, their teaching and assessment.
- Tutors felt they had ownership of the module, which they described as organic and ‘living’.
- Learners described a deeper understanding of teaching styles, theories and new ways of performing in challenging mentoring roles. The blended learning approach was generally viewed as a positive innovation to inform mentoring as an art and a lifelong skill.
- Personalised learning within dynamic e-portfolio is a product of the learner’s own history, their future aspirations and how they engage with the design of learning activities, including the use of e-technologies.
- A route map which addresses learning strategies and content is part of the scaffolding necessary for learners to organise information into meaningful learning outcomes. This is demonstrated by the building of an e-portfolio of personalised learning products.
- Capturing student biographies is an effective approach to illuminate understanding the modification of personal learning schemata.
2.0Introduction
This report presents the outcomes of an education development and evaluation of technology supported personalised learning undertaken during 2005 – 2007. The work was funded by the University of Surrey Fund for the Strategic Development of Learning and Teaching. The aim of the project was to: develop an e-portfolio to manage student progress and assist with authentic assessment. The project used the continuing professional development ‘Mentorship module’ (30credits level 3) to pilot the development. The specific module context provided the project with a secondary aim of seeking to provide an effective and efficient learning experience that will enhance the supervision of the student experience in practice settings recognising the wider system in which mentoring and personalised learning plans are located.
Specifically the project had the following objectives
use e-portfolios to further develop the student experience of peer-supported reflective learning
support learner progress and assessment through the development of an e-portfolio as a tool for capturing evidence of learning outcomes and for ongoing professional development
enhance the supervision experience of mentor students and their learners on professional placements
increase staff understanding and capacity of e-portfolios as a way to engage students in a meaningful learning experience.
The aims and objectives were addressed using an action research design in which two members of the module teaching team worked with an education developer, and researcher to enquire into technology-supported personalised learning. The two teachers worked closely with an e-learning technologist and the other members of the module teaching team to design the planned e-tivities for students to build their e-portfolio. The action research design intended to facilitate staff engagement with the pedagogical practices necessary for the design and support of student centred e-learning. A programme of staff development for teacher moderated on-line discussion was put in place. A literature review on e-portfolios was undertaken and the e-tivities supporting personalised learning planswere evaluated using questionnaires and semi-structured interviews with staff. To understand the learner mentor experience within the wider context of the professional learning in the field, narrative interviews were undertaken after the completion of the module. This report summarises the outcomes of the development and evaluation as a basis for conversation and discussions with colleagues on e-developments for professional learning.
Section 3 provides contextual background to professional learning in the field.
Section 4 outlines the design, development and delivery of the e-learning education interventions to support personalised learning and the building of an ‘e-portfolio’.
Section 5 presents a review of selected relevant literature.
Section 6 descibes the project approach and methods for evaluation.
Section 7 presents findings using three perspectives. Firstly, the student perspective with findings from the standard module evaluation questionnaire, the e-learning evaluation tool and classroom field observations. Secondly, the teaching staff perspective with findings from the semi-structured interview and finally presenting the findings from learner mentor biographies using narrative interview.
Section 8 draws together the key issues emerging from the evaluation of electronic personalised learning plans.
Section 9 provides some conclusions and offers recommendations for action for consideration by relevant teaching and learning communities of the University.
3.0Background
The project used the mentor preparation module as a case for enquiry into the development of e-learning by means of an ‘electronic portfolio’ that supports learners to manage their progress and assist with meaningful assessment processes. A technological advantage of e-portfolios over traditional portfolios is that they are sharable, which facilitates interaction between student and student as well as teacher and student (Cotterill et al. 2004). The flexibility of the presentation methods and the embedding of interactive elements in the building of e-portfolios have been found to deepen learning for trainee teachers and to also reduce workload pressure in comparison with producing traditional portfolios (Woodward and Nanlohy 2002). Peer sharing and interaction was made possible through the discussion tool of ULearn.
Whilst it was intended to supply the students with their own on-line learning space to capture meaningful learning and build an e-portfolio, early in the project it became clear that this was not going to be feasible with the information technology support available. Instead, following discussion with the e-learning unit and Dr Rowett (in particular), the functions of the university‘s virtual learning environment (ULearn – Web CT Vista) were used to enable students to experience the use of technology to support personal learning intentions and to show the personal process of enquiry undertaken within the overall learning strategies of the module, for example, reflective essay incorporating feedback from e-mediated peer discussion. The decision was taken during the delivery of the module that the students would be asked to submit a conventional paper based portfolio. Whilst some of the students may have wished at the end of the module to have submitted their portfolio of learning electronically, consideration also needed to be given to where the teachers had reached in their understanding of e-learning as well as organisation processes for example, the examination office staff.
Undergraduate teaching and learning in the Division of Health and Social Care is delivered through programmes designed around classroom based learning in the university and the development of clinical competence through practice learning and reflection with mentors (workplace supervisors) in the local health economy. Methods of assessment are based on engaging students in meaningful learning activities, the products of which form a portfolio of evidence of professional knowledge, understanding including learning how to learn, and clinical competence. The portfolio is used to both track student progress and to assess student achievement of practice learning outcomes at the appropriate academic level. Mentors are a crucial resource in the assessment of practice learning outcomes and clinical competence and are required by the professional body of the Nursing and Midwifery Council (NMC), to undertake mentor preparation through University accredited and NMC approved courses (NMC 2006).
The Division of Health and Social Care is contracted for continuing professional development by the local Strategic Health Authority to provide mentor preparation (amongst other provision). The module is also used by other healthcare employers. Over one academic year the module is taken by approximately 400 learners. During the year 2006/07 the module ran 11 times. This project planned to use data from the first and second student cohorts of 2006/07 and the lessons learned through ongoing evaluation by the teaching team of the following cohorts has informed module design and delivery within the academic year.
4.0Module design, development and delivery for personalised learning
plans
In this section the e-learning educational intervention is presented and outlines the curriculum design and development of the module; the student constituency; the university teachers and staff development for e-learning, and the specific e-tivities supporting the building of a portfolio of meaningful learning experiences.
4.1Module development
•Blended campus and distance learning (15 credits at HE2/3)
•Assessed by profile of reflective writing and mentor assessed learning development plan (competency document)
Each cohort was run by a different pair from a group of 6 module tutors
4.2The student constituency
The majority of students who undertake the module are registered practitioners working in health services for example, nurses, midwives, operating department practitioners and ambulance service personnel. All are adults who are working and taking the module for professional development either in relation to their own personal aspirations or because of employer requirement. The range of ICT skills within health care is well known (Laurillard 2005) and a typical early posting on ULearn during the project reflects student difficulty:
“I am struggling with this way of learning, not sure it’s for me? It’s really quite stressful” .…
4.3The staff
The Mentorshipmodule is resourced by a core group of six teachers who have a particular interest in learning and teaching within practice settings and have been supporting the module over various lengths of time. The teachers had been involved in the working group undertaking the curriculum review and redesign of the module to meet the NMC standards for supporting learning and assessing in practice settings (NMC 2006).
The teachers were afforded a range of staff development in relation to ULearn and teacher supported peer mediated on-line group discussion. All university teachers are required to undertake the introduction to ULearn provided by the University e-learning unit in order to become a ULearn user. The project made funding specifically available for the six teachers to undertake the Oxford Brookes course peer learning on line. Four teachers took up the opportunity.Two members of the teaching team who supported cohorts 1 and 2 were key participants in the action research design for implementation of the e-portfolio development project. They were supported by an e-learning technologist from the university e-learning unit. The table overleaf summarises the structure and content of the e-tivities undertaken by the mentor students during Units 1 and 2 of the Mentorship Module
1
Table 1: Summary of e-tivities within overall module
Unit of learning / Coursework to be produced / E-tivities / Other Activities/ comments- Students to undertake pre-course checklist to ensure they have requisite knowledge and skills to be able to participate in the online learning environment
- ULearn log-in session
- Hopes, fears and expectations discussion thread
- Introduction to
- module café for social interaction and discussions
- sources of help including the ‘Help-at-hand’ area
- Discussion of library resources and online access of books, journals etc.
1 / Learning and Teaching in Practice Level 2 & 3
For this assignment you should develop a teaching plan that you will use with your learners. You should submit this plan with a 500 word reflective critique which explores the effectiveness of your plan in the light of relevant theoretical literature, and personal and professional experience. Within this unit you should include a discussion of how you used learning theories and domains of learning to support the development of your teaching plan. /
- Discussion thread: Priorities of workload and the learning environment (level specific - students access the discussion at the level they are studying)
- Uploading of draft assignments and receipt of feedback
1 and 2 / Collaborative online small group working to produce a seminar on their chosen topic. This combines enquiry based learning and group ULearn discussions for classroom presentation
- Group specific discussions with allocated online tutor
2 / Assessing and Evaluating Learning
Level 2
Prepare a 500 word reflective discussion which identifies two types of assessment used to assess learners in your clinical practice. You should compare and contrast these methods / strategies in relation to their strengths and weaknesses, paying particular attention to the concepts of reliability and validity.
Level 3
You should prepare a 500 word reflective critical analysis of the concepts of reliability and validity in assessment in relation to your assessment of a learner in your practice area. You should draw upon a range of resources, including personal and professional experience. /
- Level specific discussions based around individual learning journeys (My learning journey) in preparation for part 3 reflection ‘my learning journey’.
A written reflection: / Unit Three: My Learning Journey
Level 2 & 3
This final piece of work for your profile is a 1500 word reflection on your development as a mentor. The title of this is ‘Reflecting on my journey from the beginning of the course how do I now feel about the mentorship module and my practice as a mentor?’ You should draw upon your experiences of undertaking the module, in attending, studying and mentoring learners in your practice area. You should demonstrate evidence of appropriate reading around the subject of mentorship by referring to the relevant theories you have learnt about.
Your work should have a clear reflective structure, and clearly demonstrate description, analysis, synthesis, evaluation and self-awareness, and give an idea of how you feel you may further develop your mentorship whilst fulfilling all the learning outcomes. /
- My Learning journey and group discussions remain open
To be submitted (paper copy), 1 month following completion of the module with unit 1, 2 & 3 reflections
Learning and development plan / Competency assessment in practice Four proficiency statements must all be signed off and include comments on your practice at both the formative and summative stages.
1
5.0 Literature Review
5.1 Introduction
The purpose of this review of literature is to highlight the key elements which have informed the development of electronic resources within the Mentorship module in the Division of Health and Social Care. Evaluation of these new developments has entailed reviewing and analysing electronic portfolios used for medical training and teacher training. The literature review is organised by firstly stating the purpose of e-portfolios and defining terms. How e-portfolios are structured will be examined before discussing problems which have been reported through evaluations in Australia, the United States of America (US) and the United Kingdom (UK). This review then incorporates evaluations which include views of students, teachers and educators who have used these electronic records over the last decade. The intention is to complement other work in progress involving e-portfolios and electronic tools forrecording learning across the University.
5.2 Purpose of e-portfolios
The primary purpose of the e-portfolio is provision of a pedagogical tool for recording and managing learning. Garrett and Jackson (2006) suggest the electronic portfolio should be used as a tool to contextualise and embed knowledge in the workplace but Hennessy and Howes (2004) assert that, although useful as a collection of evidence of learning, the e-portfolio should not be merely a ‘transcription tool’ or ‘archive’ (Herbert, 1998). It would appear that e-portfolios are increasingly being used for job applications and promotions and also as a tool to show development of thinking (Butler, 2006). As Cambridge (2004) asserts, the electronic portfolio should represent what a person (or organisation) has learned over time through a collection of authentic and diverse evidence. Across the literature, it would appear that what has the most meaningful pedagogical influence is how learners are enabled to analyse their own evidence and take responsibility for their learning.