Monitoring and Evaluation Plan

Monitoring and evaluation plan

Introduction

The project commits to 3 big targets relating to engaging people:

-500 volunteers

-10,000 active participants

-100,000 reached

These numbers are made up of people:

• producing new heritage or making existing heritage more accessible
• accessing the new/more accessible heritageproduced

All this should have:

-Outcomes for heritage – heritage is better managed, interpreted, explained, identified and recorded

-Outcomes for people – people develop skills, learn about heritage, enjoy themselves, volunteer time and change their attitudes or behaviour

-Outcomes for communities – more and a wider range of people engage with heritage, local communities in Wales are better places to live, work or visit and the WCIA will be more resilient.

How the plan is organised

This plan starts with the outcomes for people. It breaks down the numbers of volunteers, active participants and those reached to answer:

-What activities are all these people doing?

-What monitoring data will we collect to show they are doing these activities?

-What indicators will suggest these activities are leading to intended outcomes?

-What systems/paperwork need to be in place to ensure we capture this data?

-Who is broadly responsibly for capturing the data once systems are in place?

The plan then contains a simple checklist of the heritage to be produced and any additional data and systems required. This is simple because many of the outcomes relating to the heritage produced are inherent to the data and outcomes related to people. The outcomes related to communities are captured via the people and heritage. ‘WCIA more resilient’ is the additional outcome for community that the project may need to evidence separately / at the end.

The intention is that this data is sufficient to produce reports on a quarterly basis to feedback to partners and to HLF. It should also provide a broad and comprehensive starting point for annual evaluations and a larger external evaluation. The external evaluation which would include additional activities such as surveys and interviews with a selection of participants across the project from all engagement levels. Evaluations should seek to answer:

• Are we completing planned activities?
• Are the planned number and type of volunteer and active participant involved in the planned activities?
• Are the planned audiences being reached?
• Are the above activities with people leading to the intended outcomes?
• Are there any positive or negative externalities from the work?
• Might there be more efficient or effective ways of reaching the project outcomes?
• Are there any other lessons learned?
  

How to use this plan

You can use the plan quite flexibly, but essentially need to follow the monitoring and evaluation steps for each activity. To make things simpler, there is some standard monitoring to do for every volunteer, active participant and person reached which you can find at the top of each section. There may then be additional monitoring for certain activities which is detailed in the tables below. The following diagram contains step-by-step instructions for using the guide but it is understood that each person might use it slightly differently.

Partners and the WCIA

There are outcomes in the project bid pertaining to enabling the partners and the WCIA to become stronger and more sustainable as a direct result of the project. These changes are often qualitative and take place over periods of time and may include:

• Improved policies and procedures
• Better capacity to manage larger projects, teams or volunteer numbers
• Increase in audience engagement, for example, higher attendance at events and increased online participation
• Improved systems for managing data, financial management, monitoring and evaluation
• Skills development for the team and volunteers

To monitor this impact, the WCIA and partners will be asked to contribute to the HLF quarterly reports as much as possible. The WCIA as lead partner will contribute to each report and partners will be asked to contribute for periods in which they have been particularly active. Evaluations will include in depth interviews with project partners.

500 Volunteers

Across the 500 planned volunteers there are different levels of engagement that can be broadly categorised. Realistically, we can collect more data for those who contribute the most. Instead of listing the data and indicators for each volunteer activity, they are categorised by volunteer level. Only additional data required for specific activities is separately listed.

Standard systems should be used for all volunteers, only being amended where additional information is required for an activity. It is imperative that this is collected consistently and systematically across all partners – National Library, Urdd, People’s Collection Wales etc – as many volunteers contributing to WfP will not be under the WCIAs direct ‘line management’.

For clarity, our HLF reporting already recognises three ‘volunteering levels’ for match funding purposes (these calculate the ‘cash value’ of volunteering time, £50 / day for unskilled contributions; £150 / day for skilled contributions and £350 / day for professional contributions). Although these need to be captured, they are only relevant for the funding reports and all other volunteering M&E related to engagement and activities:

Our volunteer and contributor definitions are given levels 1-7 on the CRM to help us with reporting, communication, monitoring and evaluation. The levels do not indicate in any way the amount or value attributed to the contribution (although L1 / L7 in grey count towards HLF match funding but not our 500 volunteer target). The level of interaction with the WCIA team largely increases with the levels.

Level / Name / Description / Project example / Data collection / Indicators of success / Systems
1 / Beneficiary / Someone who gives their time for free to learn or benefit from a project. / Active participants / See active participant section / -90% Enjoyable experience
-Learned about heritage
-Geographical diversity achieved
-6% BME*
-2% NEETS*
-20% Welsh speakers
-Qualifications or skills gained or training completed as part of volunteering experience
-Qualifications, skills or training gainedmotivated by taking part as a volunteer
-Change in belief/view about peace/conflict
-Support into employment reported
-increase in interest levels in heritage
-repeat volunteering (with us or partners)
-Learning about partners
-Number and value of hours in line with HLF requirements
-reported benefits for community (greater understanding of shared heritage, intergenerational relationships, understanding of peace/conflict)
2 / Contributor / Someone who gives their time as an individual, but wouldn’t consider themselves a WCIA volunteer. / Speakers, guest writers for hidden histories / school resources, (unpaid) translators. / Hours and activities
Surveys / interviews of sample group / CRM volunteer records
Timesheets
3 / Community Volunteer / Someone who volunteers via another volunteer or partner organisation They may or may not consider themselves volunteers for the WCIA, but hopefully consider themselves volunteer for the specific project. / Local history groups, transcription volunteers, people who volunteer for community champions, school leads. / Hours and activities
Surveys / interviews with sample group
At discretion of partner/volunteer leader, other volunteer paperwork from levels 4-6
Evaluation of participation (impact on skills, impact on community, enjoyment)
Timesheets or sign in sheets as much as possible with tasks / CRM volunteer records
Timesheets
Volunteer paperwork as applicable
4 / Volunteer Leader / - they may be based in the community or in the WCIA offices and take responsibility for other volunteers or are in some kind of leadership role / Community champions, project leaders / Induction data on enquiry form
Support records
-hours completed
-personal development record
-achievements
-task log
Exit interview
-enjoyment
-skills learned
-into employment/training
-changes to attitudes/behaviour
-knowledge of/interest in heritage/heritage skills
-Community a ‘better place’ / impact on community
Follow up – same areas as exit interview + reflections on contribution to enjoyment/community/employability / CRM volunteer time and task records
Volunteer enquiry form
Personal development record
Timesheet
Exit interview
5 / Flexible Support Volunteer / Volunteer directly for the WCIA - may have ad hoc commitments,. Most office, events and communications volunteers fall into this category. / for example, helping out at events, or have a regular schedule with a variety of tasks / Induction data on enquiry form
Support records
-hours completed
-personal development record
-achievements
-task log
Exit interview
-enjoyment
-skills learned
-into employment/training
-changes to attitudes/behaviour
-knowledge of/interest in heritage/heritage skills
-Community a ‘better place’ / impact on community
Follow up – same areas as exit interview + reflections on contribution to enjoyment/community/employability / CRM volunteer time and task records
Volunteer enquiry form
Personal development record
Timesheet
Exit interview
6 / Interns / Project Assistants / Volunteer directly for the WCIA on a very specific task, project or set of outcomes. / Might apply to volunteers tasked with a specific piece of research or an Erasmus volunteer with clearly defined outcomes. / Induction data on enquiry form
Support records
-hours completed
-personal development record
-achievements
-task log
Exit interview
-enjoyment
-skills learned
-into employment/training
-changes to attitudes/behaviour
-knowledge of/interest in heritage/heritage skills
-Community a ‘better place’ / impact on community
Follow up – same areas as exit interview + reflections on contribution to enjoyment/community/employability / CRM volunteer time and task records
Volunteer enquiry form
Personal development record
Timesheet
Exit interview
7 / Partner / Someone who attends meetings & events, participates in decision making and governance (and is counted towards HLF’s Match Funding target). / Hours and qualitative feedback for quarterly reports / Sign in sheets

*not required (is for active participants) but desirable

Additional monitoring by volunteer activity

No. / Activity breakdown / Special characteristics / Data collection / Indicators of success / Systems / Responsibility / Vol Level
Champions and leaders – very high level of volunteering and ability to coordinate other volunteers
24 / Community Champions to establish peace heritage hubs in 12 areas and coordinate the uploading and tagging of local hidden histories trails or collection on PCW / From Peace Heritage Training Days / Location and details of hubs
Records of uploaded trails/collections
Training records/certificates / -12 hubs established
-24 of 100 Peace Heritage trainees become Community Champions
-Community Champions stay in roles for project duration
-24 Champions complete Agored accredited training
-Champions recruit of 51 local volunteers
-50 people interact with community output / Hub records
PCW monitoring system
Certificate collection process / Hanna
Fffion / 4
8 / Volunteer leaders in 2 pilot communities to lead local intergenerational work into hidden histories / 1 asylum seeker group
16-24 new to heritage / Training records/certificates
Details about pilot communities selected / 80% participants enjoy the experience
1 collection from each focus community uploaded onto PCW / Hanna
Fffion / 4
12 / Project leaders to lead intergenerational oral history projects (developing out of pilots above) / 1 asylum seeker group
16-24 new to heritage / Training records/certificates
Details about pilot communities selected / Total of 50 interviewees complete and 20 transcribed and deposited with National Screen and Sound Archive
80% interviewees enjoy experience / Hanna
Fffion / 4
2 / Community leaders to lead community-based research for DDMI journal / Community organisations who had done peace builder or oral history research / Copies of Davies papers / 12 hidden histories published in Davies papers / Ffion
Hanna / 4
Volunteers coordinated by leaders/community champions
51 / coordinated by champions to upload and tag Hidden History, trail or collection / 21 from Peace Heritage training days / Blog count and reach
Facebook post count and reach / 50 blog/facebook posts of hidden histories
300 items tagged on PCW
50 people interact with posts (commenting/sharing)
5000 view posts / Hanna
Fffion / 3
120 / involved in creating local content for Whole Story exhibition under the leadership of community champions / Exhibition panels
Training records and feedback forms / 90% training participants contribute ideas for community content
75% participants learn new interpretation and exhibition skills / Ffion
Hanna / 3
Office volunteering roles
2 / Office based volunteers supporting initial research into remembering for peace (previously soldiers’ stories) / Researched produced by volunteers / Office volunteer research examples used by and useful to other volunteer researchers / Ffion / 5
1 / Office volunteer managing survey process to debate questions raised during discussion events / Survey and interview questions and survey records / 200 respond to survey
20 respondents complete detailed interviews/blogs / Noam
Hanna / 5
1 / Office volunteer to research materials for inclusion in The Whole Story exhibition / Record of research completed / Ffion / 5
2 / writing information panels for exhibitions / Exhibition panel content written / Ffion / 5
Speakers/assemblies
12 / Speakers to give talks/lectures on Hidden Histories (10-20 min lectures in partnership with DDMI) / Lecture agendas and papers
Speaker soundbites and videos / 400 attend
200 watch online
75% say they learned something about peace heritage
75% say lectures were positive / Ffion / 2
12 / Speakers/lecturers in formal series / Lecture agendas and papers
Speaker soundbites and videos / 400 attend
200 watch online
75% say they learned something about peace heritage
75% say lectures were positive / Hanna / 2
12 / Speakers/workshop leaders at annual Remembering for Peace Conference / Agenda/session plans / 300 attendees / Jane / 2
30 / Speakers/workshop leaders in informal public events to draw attention to peace heritage and discuss peace and conflict / At least one with focus on refugees/refugee communities / Agendas/papers / 500 attendees
70% learned something
20% changed attitudes / Hanna / 2
5 / Deliver school assemblies / Topic summary
Organising teacher feedback / Jane / 2
University student volunteering roles
10 / Write essays/dissertations on peace heritage / Undergraduate students from Aberystwyth and Cardiff Unis / Essays topic, summary and marks / 1000 other project participants reached by this work / Ffion / 2
8 / Interns from Aber and Cardiff unis work with community organisations and/or schools / Undergraduate students from Aberystwyth and Cardiff Unis / Intern role descriptions / Internships all completed and each engage 10 participants
75% of participants say the project has helped them to meet learning outcomes OR develop employability skills / Ffion / 4
24 / Volunteer on any area of work (blogs, transcriptions, etc) / Undergraduate students from Aberystwyth and Cardiff Unis / Depends on role
Gather own evidence of engaged participants / Volunteers engage 200 participants
75% of volunteers have developed skills and feel they have added effectively to project’s capacity / Ffion
Hanna / 5
3 / Post graduate researchers in politics and gender, wales’ global role in the peace movement and memory and conflict / Post graduate students / Research titles / Successful applications submitted to AHRC
500 non academics reach this research / Ffion / 2
4 / Support young people to share hidden histories through digital storytelling / Undergraduate students from Aberystwyth and Cardiff Unis / You Tube channel hits
Support feedback / 24 KS4 and 5 pupils pilot resources (of 200 hidden history target)
75% enjoy creating digital stories of peace
At least 10 stories uploaded on You Tube
1000 online views / Hanna / 6
General volunteering roles – 6 of these should be office based
5 / Help to develop the transcription tool / Feedback on tool usability / Minimal technical complaints during transcription / Corrections to log in tool / Ffion / 2
95 / Transcribing and tagging the WW1 Book of Remembrance / Whether or not volunteers have done this kind of project before / 30% volunteers have not transcribed before
Complete, searchable record available online
75% volunteers rate experience positively / Transcriber surveys/polls / Ffion / 3
5 / Help to digitise100 items from ToP and DDMI onto PCW / Volunteers should be diverse / Feedback on digitisation system / 100 items digitised and available via PCW / Digitisation system with comments / Ffion
Hanna / 3
5 / Help to research the Urdd message history to get complete timeline of messages / Details of new artefacts
Project blog posts / 25 artefacts identified
Complete record of messages
10 articles on project blog / PCW record check system / Hanna / 3
3 / Produce 10 Hidden Histories / The hidden history details and which local archives/collections they link with / 10 produced and shared on PCW / Hanna
Ffion / 3
4 / Volunteer interviewers to interview peace activists commemorated in the ToP garden / 2 should be from volunteer youth group / Interview records
Training records / 80% of interviewees enjoy the experience / Noam
Ffion / Jane / 3
N/A / International volunteer / Should bring in active participants from local community / Sense of interaction with local community and shared learning – hours not recorded on CRM / Noam
Ffion / Jane
N/A / Local volunteer supporting international peace camp projects / Sense of interaction and learning from international volunteers / Noam
Ffion / Jane / 3
10 / Young people involved in planning/creating mosaic for peace garden / Mosaic pictures / Mosaic installed
75% young people enjoyed being involved / Noam
Jane / 3
40 / helping at exhibitions at 4 venues / 20 new
20 already involved elsewhere / Training records and feedback / Ffion
Hanna / 3
Teachers and young people volunteering roles
6 / Teachers on steering group to advise on development of education resources and activities / Teachers / Steering group agenda/minutes/actions / Action completion record / Action records / Jane / 2
14 / Peace school scheme leader (one per school) / Teachers / Data collection (baseline, monitoring and evaluation) developed with lead teacher based on school systems
School data (name, address, pupil numbers, free school meals) / 250 active learners in scheme
5000 teacher by scheme / Peace school scheme accreditation / Jane / 4
6 / lead teachers/volunteers to lead arts approach to hidden histories and what peace/conflict means / Teachers or volunteers working in non-formal education / List of schools/community groups along with group leader
Summary of approaches used
Assessment of enjoyment/engagement of young people in traditional teaching methods and arts based approach piloted
Details of roll-out schools/community groups / 150 young people in pilot
-80% of whom enjoy taking part
300 involved in their events/performances
Approach expanded to 6 more schools/community groups reaching 200 learners.
Scheme leaders report more enjoyment/engagement compared with traditional teaching approaches / Jane / 4
10 / Learners use Welsh Bac guidance for individual studies on hidden histories and share their studies on PCW / Individual study records and summaries / Jane / 3
6 / pupils on steering group to advise on development of education resources and activities – this 6 also plan events for young people (peace heroes etc) / Steering group minutes, agenda and action points / Action point completion / Jane
Noam / 4
24 / Young people to be ‘young ancestors’ and Tweet in what they would have Tweeted / Secondary school students / What they would have tweeted biographies
Participant list and activity records (Tweets)
Residential course records
In depth questionnaires/interviews with participants before, during and after
Retweets and other interaction with tweets / 18/24 young people enjoy the project and find it develops their skills and knowledge
1000 visit WTWHT mini site
100 download resources promoted by characters / Jane
Noam / 3
6 / Volunteers on a committee to develop workshops and activities for Urdd Eisteddfod around the Goodwill message / Welsh language communities / Committee notes/records
Urdd Eisteddfod activity plan
Visitor feedback / AT least 6 young people develop activities
Positive visitor feedback (anecdotal/sound bites) / Hanna
Noam / 4
6 / Young people volunteer at events/run events to celebrate achievements like Peace heroes / Steering group minutes, agenda and action points / Action point completion
50 people attend events / Noam
Hanna / 4
10 / Young people involved in planning/creating mosaic for peace garden / Mosaic pictures / Mosaic installed
75% young people enjoyed being involved / Noam
Jane / 3

10,000 Active Participants

The small table below includes the data to be gathered across the board. Where additional monitoring needs exist, they are detailed in the second table.