Transition
Evaluation Toolkit
Evaluation Tools Guide
Strategic Partnership
1. SP1: Partnership Meeting Observation Form Page (16)
2. SP2: Baseline Data Evaluation – Partners Annex (2)
3. SP3: Baseline Data Evaluation – Project Team, WLL Annex (3)
4. SP4: Health as a Cross Cutting Theme Questionnaire Available via WLL
5. SP5: Project Case Study Page (9)
Delivery Model – Trainer/IAG Adviser
1. SP2: Baseline Data Evaluation Annex (2)
2. DM1 Trainer/IAG Adviser Reflective Log Page (10)
3. DM2: Trainer/IAG Adviser Case Study Page (14)
4. DM3: Peer Observations of One to One meeting with Learner Page (17)
5. DM4: Observation of One to One meeting with Learner Page (18)
6. DM5: Quality Monitoring Visit Record (Partners) Page (19)
7. E1: Employer Survey
Participants
1. P1: Participant IAG & Session Evaluation Page (11)
2. P2: Participant Final Reflective Log Page (12)
3. P3: Participant Case Study Page (13)
Photographs
· PC1: Consent Form and Guidance Page (31)
Annex (1)
· Why evaluate the Transition project Annex (1)
Evaluation Tool / Person(s) Responsible / Frequency / Deadline for SubmissionSP1: Partnership Observation Form / Rinova, Julie Parish or representative / Every meeting / Within 3 days of every meeting
SP2: Baseline Data Form Evaluation: Partners / All partners / Once / By 18th July 2012
SP3: Baseline Data Evaluation: WLL / Wandsworth Lifelong Learning (WLL), project management team / Once / By 18th July 2012
SP4: Health and Wellbeing Questionnaire / Tim Brunt / Once / By August 2012
SP5: Project Case Study / Wandsworth Lifelong Learning / 3 over the lifetime of the project / 1. November, 2012
2. September, 2013
3. December, 2014
DM1: Trainer/IAG Adviser Reflective Log / All partners / Each partner to produce two, annually / By end of Nov 2012
By end of April 2013
By end of August 2014
DM2: Trainer/IAG Adviser Case Study / All partners / Each partner to produce one, annually / By end of April 2013
By end of August 2014
DM3: Peer Observation of One to One meeting with Learner / All partners / One to be produced for each partner, annually / By end of April 2013
By end of April 2014
DM4: Observation of One to One with Learner / WLL
All partners / One, for each partner, annually / By end of March 2013
By end of March 2014
DM5: Quality Monitoring Visit / WLL / Each partner to be visited at least once, annually / By March 2013
By March 2014
E1: Employer Survey
(Optional) / All partners, as appropriate / Once each year / By end of April 2013
By end of July 2014
P1: Participant IAG & Learning Evaluation / All partners / On completion of IAG support or learning intervention / Reported Quarterly, by given reporting deadline
P2: Participant Reflective Log / All partners / On exit from the project / Reported Quarterly, by given reporting deadline
P3: Client Case Study / All Partners / 1 produced Quarterly, min. two from each partner / Reported Quarterly, by given reporting deadline
How to use the Toolkit
The Toolkit has been designed for use by all of the Transition partners. It has been based on examples of surveys and questionnaires used from within the partnership, and from elsewhere.
We have adopted a colour guide to help you navigate the Toolkit and to select the right guidance and tool for the job.
Purple Guides: provides an overview of how to prepare for and write up a Case Study, and guides for the production of Reflective Logs.
Green Tools: are the templates that all project partners should use to capture feedback and effective practice.
Orange Observations: are the templates that will be used by peers and managers to capture effective practice.
Important Note: do customize these tools to fit with the project by using the Wandsworth, Skills Funding Agency and the ESF logos, and your own logo of course, if there is space.
You will also need to customize the surveys/questionnaires with the title of the service, activity or support that you have provided, but please do not change any of the questions. We will review questions used through our partnership meetings.
Thank-you!
Purple Guides
Using the Guides and Tools - creatively: please use the following guides to help you decide how you will use the Toolkit, for example, you might choose not to use for the forms with participants, but use the questions asked to capture their feedback on video or audio. You might want to ask a group to respond to the questions, rather than an individual, so you could create a ‘talking wall’ or a more creative level, a ‘mind map’ – to explore thoughts and opinions. Basically, you can be as creative as you like, you just need to use the questions provided.
You might choose to provide a Feedback Box – located in reception, for example, where the questionnaires are placed, or just simply provide envelopes to place them in – making the process of providing feedback confidential.
If you are feeling brave, you might want to introduce evaluation during inductions, exploring with tutors/IAG advisers and participants exactly what is going to be evaluated, why, and what else they would like to evaluate.
Working with the group, you might want to capture a group or individual ‘historic timeline’ to capture impact at key stages of the project.
Transition Project
Suggested Structure for Case Study Stories for Trainer/IAG Adviser
Essentially, for Trainer/IAG Adviser the case studies need to demonstrate their awareness of transferable skills, personal development and employability skills, how they acquired their own and how they weave such skills into their own teaching, and examples of successful outcomes with some of their participants. Successful outcomes for them might include:
· Planning transferable skills, personal development and employability skills into their teaching sessions
· Helping participants move on to more education/training
· Helping participants move into apprenticeships and/or sustainable employment
At least two of the delivery team should produce a Case Study Story for each delivery year of the TRANSITION project. The Case Study does not necessarily be written, it can be captured on video or audio, but these will need to be high quality, and should follow the proposed format.
Transition Project
Suggested Structure for Case Study Stories for Participants
For participants the case studies need to demonstrate their awareness of transferable skills, personal development and employability skills and why these are useful, which ones they have acquired and how, how they have been able to evidence them, and examples of successful outcomes: e.g. where they have been useful in helping them get work, or move on to more training and education, improving self confidence, and the like. Successful outcomes for them might include:
· Improvement in softer life skills
· Moving into more education/training
· Getting a job or an apprenticeship
· Understanding the value of various types of learning
At least two Case Study Stories should be produced for each delivery year of the TRANSITION project at key transition points within the delivery calendar.
Transition Project
Writing up the Case Study Story
Suggestion for pitch, tone and style
In writing up the case study we suggest that it should be:
· Written in simple, plain English
· Have a strong narrative thread to the story – particularly around transferable skills, personal development, and employability skills and successful examples thereof
· Written in a style familiar to the target audience
· Supported with a photograph, where possible
Structure
Try and give the case study a good headline.
If it helps to clarify the story, give subsequent paragraphs sub headlines.
Like many newspaper and magazine articles try and get the flavour of the key story element in the first paragraph and then expand in subsequent paragraphs.
We would suggest a word count of somewhere between 300 and 500 words. As with Trainer/IAG Case Studies, the Participant Case Studies can be captured on video or audio, and should be high quality, and follow the same format.
Transition Project
Reflective Practice
Trainer/Adviser Reflective Log for Delivery of Sessions with participants* and Final Reflective Log
Reflective practice in the Lifelong Learning sector is now widely recognized as an integral element of continuing professional development. It also provides essential information for evaluators – to capture and promote demonstrable evidence of change and adaptation, also improvements in practice that can be shared with others. In a professional setting, reflection is:
· Deliberate
· Purposeful
· Structured
· About liking theory and practice
· To do with learning
· About change and development
It is recommended that the delivery team complete at least one reflective log for each cohort of participants, or a minimum of twice each year, for the duration of the TRANSITION project.
Trainers/IAG Advisers are encouraged to use the Tools provided in this Toolkit. At least two reflective logs should be produced each year of TRANSITION project delivery, and each member of the delivery team will produce one final reflective log, each delivery year. Reflective logs can be produced in video or audio formats, be high quality, and follow the proposed format.
*Sessions can be either group or individual sessions e.g. one to one
Transition Project
Reflective and Inclusive Practice
Participant Final Reflective Workshop
Most of us find it quite a challenge to think about our own skills. It is hard enough to think about what areas we need to focus on developing, but it is even more difficult to acknowledge what we are really good at. The key features of effective reflection are identifying, analysing and exploring what we have done, understanding what we have done well and seeing how we can be more effective.
The Final Reflective Workshop has been designed to support the capture of wide-ranging aspects of the journey experienced by the young person participating in the Transition Project, along with Trainers and Advisers.
In an ideal world, each participant leaving the TRANSITION project should engage in a final reflective workshop, however, this is not always appropriate or possible. Therefore, we suggest that these workshops are embedded into delivery. It is advised that at least 30 per cent of those leaving TRANSITION will take part in a workshop. The Rinova team will facilitate at least 4 workshops during the lifetime of the project.
Green Tools
Transition Project
SP5 Project Case Study
Name of the project / Enter the name of the project and any “branding”Address of the project / Enter the address of the project, and list the partners and their address, if appropriate
Name of contact / Enter name(s) of people within your organisation who will be responsible for handling any external communications for the project
Contact details / Please provide telephone contact details, along with time(s) of availability, email address, and website address
Project aims and objectives / Use about 50 words to do this
Funding details / E.G. this project receives £xxx,xxx from the London Councils European Social Fund Co-financing programme. Funding is provided over 2 years and payments are made on the results achieved
Main beneficiaries / Describe, in brief, the main beneficiaries
What do you hope to achieve for the local community / Describe, in brief, what you hope to achieve for the community – is the project tackling a particular issue or addressing a problem, or does more positively, what solutions does it bring? Quotes help bring the narrative to life.
Outcomes / Please give a description of the specific outcomes for the project. What impact has been achieved, based on outcomes or experience from running similar projects? What are your future plans?
Please confirm who needs to approve the draft case study for your organisation:
Name:
Telephone:
Email address:
Signature:
Case study approval / confirmation for internal/external use:
Name:
Position:
Email:
Signature:
Transition Project
DM1 Trainer/IAG Adviser Reflective Log
Session and Venue / Enter Details hereBrief Description of what was delivered / Enter Details here
Number and type of Participant / If a 1-2-1 meeting, give example of the stage of support e.g. 2nd session out of 5, with a young person who has recently left school with no qualifications.
If a workshop or class based session, enter size of group, the stage of the programme they are at, if relevant, their general level, and a profile of the group.
Plans and Objectives / If a 1-2-1 give details of your plan(s) for the session and what the key objectives/goals of the meeting were.
If a workshop/session give outlines of the Lesson Plan and the Learning Objectives, noting if any of these were changed to meet with the needs of the individual or group.
Positive points of the session / Reflect on what went really well and why you thought it did, taking into consideration the group, etc.
Negative points of the session / Reflect on what did not go so well and why you thought it did not, taking into consideration the group, etc. Give a description of what happened. How can you modify your practice?
How did you include Transferable skills and was this successful? / Give examples and evaluate success of approach and outcome for participant
How did you include personal development/Soft Skills and was this successful? / Give examples and evaluate success of approach and outcome for participant
How did you include Employability Skills and was this successful? / Give examples and evaluate success of approach and outcome for participant
Outcomes Achieved / Were all learning objectives achieved and if not, what were the reasons? Were there additional objectives set, and were these achieved? If not, what were the reasons?
Conclusions: Key areas of development and/or improvement based on your own and participant evaluations / On reflection, what would you change based on your own analysis of the session and the feedback from participants?
Transition Project
P1 Participant IAG or Session Evaluation
Your Name:
Date:
Name of the College/Organisation:
Course/Project Title: IAG Adviser/Trainer:
Pease mark your answer with an XDid you find the Tutor/Trainer well prepared and easy to understand?
Do you think the Tutor/Adviser treated you/others fairly and with respect?
Has the session helped review and set your goals?
Has the session helped you set realistic plans for the future?
Has the session has helped improve your self-confidence?
Has the session helped you feel more positive about the opportunities that you have in life? / Yes
Yes
Yes
Yes
Yes
Yes / No
No
No
No
No
No
Would you recommend the session to someone you know? Yes No