Evaluation of the Creative Employment Programme

June 11th 2013

Invitation to Tender

Evaluation of the Creative Employment Programme

Context

The Creative Employment Programme is an Arts Council England initiative which is being delivered by Creative & Cultural Skills. Arts Council England is funding this work in order to:

·  Create fairer entry routes to creative and cultural work

·  Generate new jobs

·  Support the next generation of arts entrepreneurs and freelancers

·  Drive growth by addressing skills shortages

As the organisation delivering this work, Creative & Cultural Skills wish to commission an independent evaluation to test whether the programme has successfully met the aims set out above. The investment, of up to £15m to support the creation of 6500 traineeships, formal apprenticeships and paid internship opportunities for young unemployed people aged 16-24 wishing to pursue a career in the arts and cultural sector[1], is a significant intervention.

For more details about the programme, FAQs and guidance supporting the application and selection process please visit: http://creative-employment.co.uk/the-creative-employment-programme.The programme begins in March 2013 and will be completed in 2015.

The Creative Employment Programme is a significant attempt to address some of the recurring themes and issues that historically have prevented real clarity and coherence around the available pathways into particular occupations. It seeks to achieve a step-change in employment practice in the creative and cultural sector by reducing an over-reliance on graduates. It also seeks to achieve this step-change by highlighting National Minimum Wage legislation in relation to the sector’s culture of unpaid internships, and to offer young people more choice and a greater sense of control over their own professional development and training.

Through the Creative Employment Programme, employers within the cultural sector will be able to access funding to contribute to the wage costs for new apprenticeship and internship roles. Employers are also invited to design work shadowing opportunities for 16-18 year olds through our Traineeship programme.

The Creative Employment Programme internships specifically target 18-24 year olds who are on Job Seekers Allowance, which is a deliberate attempt to decrease levels of youth unemployment and encourage employers to consider this pool of young people as entrants for the jobs they create.

Apprenticeships must be Government endorsed frameworks that are relevant and applicable to the sector, including creative and non-creative pathways. We want to raise the profile of on-the-job learning as a legitimate route to long-term success for young people and a channel through which employers can achieve an appropriately skilled and competitive workforce.

The Traineeship programme will be managed outside of the main grant programme and will target 16-18 year olds who are not in education, employment or training. The programme will offer time-limited work-focused training and work shadowing opportunities to help participants build the confidence and skills to progress into the workplace.

As part of the Creative Employment Programme a wider support academy will be available to young people and their employers[2]. This will provide employability training and development opportunities, access to an annual programme of inspirational speakers, peer network support, information on the hidden job market and development opportunities for employers who are new to employing young people.

Key evaluation questions

In this section we set out the focus for the evaluation, the range of impacts we wish see covered and the sorts of evidence and data we consider integral to the work.

The evaluation will comprise:

·  Process Evaluation – how effective and efficient is the system Creative & Cultural Skills has set up to administer this programme?

·  Impact evaluation – what range of impacts are manifest across a range of key stakeholders?

In combination these elements will provide a comprehensive analysis of the impacts and effects of the programme.

There should be a focus on the following areas:

Effectiveness of Creative Employment Programme delivery and administration

Did the processes of application, selection and programme management allow for the recruitment of engaged employers committed to the development of young people in their particular field, and lead to connections with unemployed people from a diverse range of backgrounds?

·  Did the programme reach a wide range of employers across the country?

·  Did the programme reach young people in geographical areas of highest youth unemployment?

·  Did assessor and panel roles support effective judgements around applicant selection?

·  Were appropriate quality criteria in place and were these applied consistently, in ways that underpinned the validity of the selection process?

Creative & Cultural Skills are developing a logic model which will visually represent key decision making moments and change mechanisms that allow for the programme’s design to influence participants. We would look to finalise this model with the evaluators once they are contracted.

Participant Impact Evaluation

a) Impacts on employers:

1)  Impact on business effectiveness and capacity

2)  Working practices and culture – what did apprentices/interns add to the work environment? Has it made them feel differently about the pools they recruit from?

3)  Longer term employer commitment to using apprenticeships and paid internships as methods of investing in growth, both aspirational and evidenced by changes to current practices where possible, as well as the extent to which they may access similar public funding streams.

b) Impacts on young people:

1)  Degree of connectedness to workplace and other workers, broader networking opportunities

2)  Content – what was learnt and relevance to work, the range of skills developed, and any further training or general career trajectory and aspirations

3)  Broader understanding of creative and cultural sector and future career trajectories with the option to assess employment prospects beyond the sector and for 12 months after the programme is complete.

c) Counterfactual

1)  Comparison to employers who were unsuccessful in participating in the Creative Employment Programme but who had shown an interest and made some plans – what did they go on to do outside the programme? Or employers who are recruiting beyond the programme. Did any recruit apprentices and interns in a different way?

2)  Comparison to the wider economy

3)  What other influencing factors can be accounted for in determining the Creative Employment Programme’s impact? Other programmes? Overall economic climate?

4)  Issues of deadweight and substitution/displacement – is there evidence to suggest some of the benefits of the Creative Employment Programme may have accrued even if the intervention had not taken place, or conversely that some of the benefits afforded to young people participating led to an opportunity cost for another young person, be they current or potential staff?

Social and Economic Impacts

An important aspect of this evaluation will involve a realistic assessment of the cost and benefits of the Creative Employment Programme as a potential approach to future training and recruitment, as well as a potential long term driver of growth in the creative and cultural sector.

Data will need to be gathered to compare spend per participant, job progression beyond the Creative Employment Programme where applicable, and other positive benefits directly attributable to the programme including:

·  further skills training,

·  work-related education,

·  significant shifts in attitude in young people towards work and employment

·  attitudes and preconceptions of employers in relation to unemployed young people

·  the balance between new job creation and filling pre-existing roles

·  an element of comparison to other work programmes

·  Progression routes onto education, further employment or other outcomes.

We are neither restricted by nor committed to only these elements and approaches but set them out as a means of further delineating the priority areas the evaluators will need to keep in focus. It is therefore important that evaluators specify how these will be addressed methodologically in their proposals.

Methodology

We anticipate a mixed method approach will be most effective in determining the impacts and effects of this programme. Given the ambitious aims we imagine some elements of the evaluation approach may include:

Desk research / use of existing statistics – we expect a thorough study of existing evidence relating to:

·  the impact of the Creative Employment Programme on Creative and Cultural labour markets locally, regionally, nationally and by sub-sector (in line with the Arts Council footprint)

·  an analysis of destinations for the Creative Employment Programme participants e.g. numbers returning to benefits, numbers moving into jobs, numbers moving onto further education or training.

·  other positive outcomes and points of learning, drawing on all relevant locally collated data available and national monitoring information

The relative complexity of this desk research and incorporation of existing statistical information will depend on accessing relevant data from a range of agencies (including Jobcentre Plus, the National Apprenticeships Service, FE Colleges) and localities. We are still in the process of exploring exactly what data can be collected from these partners and how robust this will be. Tenderers should demonstrate an awareness of this challenge and point to strategies they have adopted in prior work to manage this effectively. Responses to this tender should, therefore, present two possible scenarios: one where the data listed above is available and robust and another where some additional primary data collection is required.

Primary research – we expect primary research will be required, including the sampling of employers and young people for the purposes of survey, case study or interview. This may comprise combinations of face-to-face and telephone interviews with key persons as well as tracking and documenting the experiences of young people who might serve as case studies to add depth to more quantitative data. In combination we are particularly keen to see if this data demonstrates any shift in attitude, perception and intention that desk research and use of statistics may not wholly reveal. It should be noted that the contract for the evaluation extends to a full year after the funded life of the Creative Employment Programme itself. This will allow us to assess ongoing effects and impacts by monitoring young people’s destinations and progress for an additional 12 months. Methods and approaches for undertaking this element of the evaluation should be clearly indicated in tender submissions.

Project Outputs

The outputs for this evaluation project would be:

·  a final standalone report, the format of which will be agreed once the contract is awarded

·  A series of concise interim reports to inform the project steering group on progress and emerging findings

·  an annex of data tables from any surveys undertaken,

·  relevant statistical tests conducted on the data, as appropriate, to ensure the robustness of the work.

Intellectual Property

All Intellectual Property Rights in Pre-existing Works used by the researchers in delivering this project will remain with the researchers.

We would expect the researchers to grant Creative & Cultural Skills a perpetual, non-exclusive, royalty-free licence to use, reproduce, copy, modify, adapt and make available the Pre-existing Works in connection with the project.

Project Management

David Parker, Director of Research, Creative & Cultural Skills will Project Manage this evaluation on behalf of Arts Council England.

A Project Advisory Group will offer guidance throughout the project – and will be made up of a set of key stakeholders as well ACE as the overall funder. This group will steer the project, ensuring it meets to objectives set out in this invitation to tender and will receive draft materials and interim reports for comment.

Required skills and experience

Tenderers should be able to demonstrate skills and experience in:

·  quantitative and qualitative research

·  innovative approaches to addressing complex research questions

·  ability to present research findings comprehensibly to a range of audiences

·  ability to co-ordinate and negotiate with a range of stakeholders

·  understanding of the arts and cultural sector and the policy landscape around skills development and economic growth

·  ability and proven track record of undertaking Social Return on Investment and/or Cost Benefit Analysis

·  experience of gathering data from young people and of the methods best suited to engaging this group

·  experience of follow-up evaluation activity with key participants in programmes once the funded element is over

·  track record of undertaking research or evaluation that directly impacts on policy change and/or decision makers

Project Timetable

The proposed overall timetable is:

Brief to tender: 11 June 2013

Proposals received: 15 July 2013

Possible interviews: 22/23 July 2013

Agency appointed: 26 July 2013

Inception meeting: 29 July 2013

Interim meetings with steering group:

·  September 30 2013

·  January 24 2014

·  April 18 2014

·  August 29 2014

·  January 23, 2015

Draft report submitted to advisory group: 20 February 2015

Report at end of funded period of the Creative Employment Programme submitted to Creative & Cultural Skills: 27 March 2015

Ongoing tracking of sample of employers and young people for one year ‘post-programme’

Final report at end of 12-month follow on activity: March 2016

In addition, fortnightly telephone updates would be expected with the project manager, David Parker.

Tender Response Requirements

All tender proposals must clearly identify how you address all the requirements of this brief. Your proposal should include:

·  Understanding of the brief

·  Research methodology, including approach to sampling and recruitment and any particular frames, matrices or approaches used for analysis

·  Outputs

·  Timetable

·  Named personnel and responsibilities including detailed CVs of the proposed team and relevant organisational experience

·  Breakdown by task and staff member

Pricing/Budget and billing

The budget available for this project is up to £140,000 inclusive of VAT and all travel and other expenses. Value for money is a part of our evaluation criteria.

A contract will be awarded for the length of the project (from initiation to 31 March 2016). Payment for each element will be agreed in advance and linked to delivery of key milestones as per an agreed project plan.

Evaluation Criteria

Members of the steering group will assess and shortlist tenders. Overall assessment will be concerned with value for money, taking account of the quality and cost of tenderers' proposals. The tender will be assessed according to the following criteria and weighting: