Research Brief – Inspiring Change Manchester
LOT 1: Systems Change

Background to the research

Inspiring Change Manchester is a new Manchester partnership, led by the housing and homelessness charity Shelter, with an ambitious vision to help people improve their lives and sustain these improvements by transforming Manchester’s systems and services. Inspiring Change Manchester is one of 12 partnerships being funded by the National Lottery through Fulfilling Lives, the Big Lottery Fund’s £112m initiative to support people with multiple needs. Project delivery with beneficiaries will commence in July 2014.

The Big Lottery Fund has commissioned CFE Research to lead a national evaluation of the Fulfilling Lives programme. This will largely focus on outcomes for service users through the collection of a consistent set of data across all 12 local projects within the programme (data such as the Homelessness Outcomes Star and NDT assessment).

Inspiring Change Manchester will also undertake a local-level evaluation, dovetailing with the national evaluation to minimise duplication. The key focus for the local evaluation is to understand what is working, for whom and under what circumstances. This will help the project to be delivered more effectively over time in terms of its impact both on individual service users and on local systems change.

The overarching aim for the local evaluation of Inspiring Change Manchester is to assess the extent to which the project is achieving its four intended outcomes:

  • Manchester residents with three or more complex needs will have improved health, well-being, housing, employability, reduced re-offending, and these will be sustainable
  • Manchester residents with multiple and complex needs will have more opportunities for involvement in the services they need, and influence on decisions that affect them
  • Services will share more information and better co-ordinate interventions for people with multiple and complex needs
  • Commissioning of mainstream services will respond to project learning, funding cost-effective evidence-based interventions for people with multiple and complex needs.

In order to provide an independent assessment that will carry weight with commissioners and other organisations, the majority of local evaluation work will be commissioned from external organisations with relevant expertise.

The evaluation will comprise five key components over four Lots, as follows.

LOT 1: Systems change monitoring and analysis: to examine the extent to which the project successfully influences systems change in Manchester – i.e. how wider services are commissioned and delivered.

LOT 2a): Primary data collection with individual service users and volunteers to understand experiences and changes: building on CFE’s national set of outcomes data to understand more about client journeys and experiences and how these differ.

LOT 2b) Exploring the success and impact of unique project-specific elements: to understand more about local success and to inform national debates on service delivery. Elements to be researched may include the difference a Psychologically Informed Environment makes.

LOT 3: Value for money/cost effectiveness exercise:to influence senior commissioners by providing robust evidence, rooted in local data, showing the fiscal, economic and social benefits of the project’s approach.

LOT 4: Ad hoc research activities: to provide for one-off research needs.

This research brief refers only to LOT 1: Systems change monitoring and analysis. Appointed contractors will be required to work closely with organisations leading the evaluations of Lots 2a, 2b, 3 and 4. This is to reduce the burden placed upon service users and other stakeholders involved in the research and to remove any unnecessary duplication. The appointed contractor will also be required to attend data sharing and implementation meetings where relevant, which will be co-ordinated through the project’s “Innovation Lab”.

Inspiring change Manchester will plan to deliver its services and operate for 8 years. All local evaluation partners appointed through this tendering process will be contracted for the first 2 years after which time there will be a period of review. Subsequent evaluation activity may then be commissioned.

Content of the research – LOT 1, Systems Change Monitoring and Analysis

The Systems Change Monitoring and Analysis element of the local evaluation aims to examine the extent to which the Inspiring ChangeManchester project successfully influences and contributes to systems change in Manchester – i.e. how services are delivered and commissioned. Systems change monitoring will determine the achievement of the following project outcomes and indicators.

Outcome / Indicators
Services will share more information and better co-ordinate interventions for people with multiple and complex needs /
  • Services demonstrating improved accessibility and flexibility for people with complex needs, through changes to organisational culture and procedures
  • The number of organisations adopting common systems and actively sharing learning and information about their services

Commissioning of mainstream services will respond to project learning, funding cost-effective evidence-based interventions for people with multiple and complex needs /
  • Commissioning is improved, with services being commissioned in an integrated way, with increased service user involvement and supporting service flexibility which reduces exclusion

Systems change is a long-term process, and with this in mind, Inspiring Change Manchester developed a Theory of Change for local systems change (See appendix of this brief).It details the project’s overall aims forsystems change after eight years of delivery and highlights the steps that need to occur for these aims to be realised. In the short to medium term, this systems change component of the evaluation will monitor whether early steps in the Theory of Change are being met, thus ensuring the project is on track to deliver its longer-term aims. For example, in the short term the systems change evaluation will monitor whether representatives from the project’sStrategic Group sit on relevant boards and/or meaningfully feed in to them.

Budget

The table below shows the budget for systems change monitoring and analysis which is £25k over two years (excluding VAT). More resource will be available in Year 2 as more milestones in the systems change process will have been reached and much of the work in Year 1 will involve establishing a baseline for systems, i.e. mapping and understanding how current systems operate.

Component part of evaluation / Year 1 budget available / Year 2 Budget available
LOT 1: Systems change monitoring and analysis / 10k (excluding VAT) / 15k (excluding VAT)

Timelines for the research

The deadline for responding to this brief is Friday 27th June at 5pm. Received briefs will then be assessed and scored by the project team. Shelter may require potential contractors to attend an interview. A delivery partner for Lot 1 will be in place by July 2014. Local evaluation activity will also begin formally in July 2014, to coincide with the Inspiring Change Manchester project commencing delivery to beneficiaries. Evaluation contracts will end in June/ July 2016.

Role of the contractor

The contractor will initially attend an inception meeting at Shelter offices to finalise an agreed approach to take forward. The contractor will also be expected to attend at least one learning and data sharing workshop per year hosted by the project’s “innovation lab”. Finally, the contractor may be required to attend a planning session with organisations delivering the other evaluation lots. These requirements and additional project management should be factored in to the overall total maximum budget per year.

Systems change monitoring and analysis will likely involve two key elements as outlined below (i) systems mapping and (ii) stakeholder research. Note that the detail below is only a suggested methodology. Shelter will consider different and innovative methodological ideas from contractors which comprehensively meet the aims of this research LOT.

  1. Systems mapping: understanding and mapping the relevant local landscape of service delivery, partnerships, commissioning and data sharing agreements.

This task will likely be undertaken at project commencement and repeated at the 18-month mark to establish what has changed. As previously stated, systems change is a long-term process, so the 18-month point has been suggested to allow for a degree of systems change to have taken place but also to enable a period of analysis and review for service delivery beyond Year 2. The main output from the work is expected to be a matrix/ representative map of relevant local systems and service activity. This map may highlight linkages between organisations and the nature of those relationships.

The methodology will likely involve engaging/ consulting relevant stakeholders from local delivery organisations and commissioning bodies, and examining any documentation these organisations are able to share (in regards to delivery; partnerships; commissioning and data sharing agreements). The depth and breadth of any engagement/ consultation conducted could be agreed following a sourcing exercise (ie identifying organisations) which Shelter would support. The structure of the map/ matrix will also be supported by Shelter; their specific research aims and the Theory of Change.

The following list is indicative of the sorts of activities/ tasks that would take place under systems mapping. Shelter is however open to other suggested methodologies which meet our research aims.

  • Planning
  • Sourcing stakeholders (in conjunction with Shelter)
  • Stakeholder engagement/ consultation (we are open to suggestions for methodologies/ mechanisms for this)
  • Desk analysis (including documentation)
  • Producing matrix/ map representation
  • Recommendations for review of service delivery

Please outline in your proposal your suggested methodology for engaging/ consulting stakeholders and how many stakeholders you therefore envision you could engage based on the available budget, and the number of days you could allocate to any desk-based analysis.

  1. Stakeholderresearch: exploring the changing nature of service delivery and systems operation (including local relationships and partnerships) through research with key stakeholders from delivery organisations and commissioning bodies.

This research is expected to overlap with any conversations required as a part of the systems mapping element. Due to the aims of the research it is likely that stakeholder interviews will be involved but it may possible that certain elements of questioning could be conducted as a survey instead of interviews. This will be determined at the discretion of the appointed partner. You may wish to present different options in your proposal covering both interviews and a survey or other creative ideas.

The following list is indicative of the sorts of activities/ tasks that would take place under stakeholder research:

  • Planning
  • Topic guide development (with steer from Shelter)
  • Conducting interviews/ survey with local delivery organisations/ commissioners
  • Analysis
  • Case study development

Stakeholder research/ interviews will also likely take place at project commencement and around the 18-month mark. CFE Research (the national evaluators) are also seeking to speak to local stakeholders as a part of the national evaluation so this research will need to be mindful of minimising repetition in terms of both who is spoken to and the topics covered.

These interviews/ stakeholder research will gather evidence to address the questions below. A fuller topic guide will be designed in conjunction with the appointed evaluator. Some questions will only be relevant to Year 2.

  • What is local service awareness of the project’s protocols, and how often are they used?
  • Is there a change in the culture of service provision; the language services use; an increase in co-production AND does this actually improve impact?
  • How do voluntary agencies describe working with other agencies?
  • Has the sector begun to change – for example, is it more cohesive and integrated?
  • How well are partnerships working and what are stakeholder views of what is working and what is not?
  • Do representatives from the project’sStrategic Group sit on relevant boards and/or feed in to them?
  • Do relevant commissioners look at early project lessons and build this into their thinking, including Living Longer Living Better and CCGs’ provisioning side?

Alongside a succinct report, the main output from the interviews will be a range of case studies in order to illustrate change. These case studies will be supported and validated by the project but it is important in terms of their ability to influence that the case studies be developed primarily by the contractor.

The case studies will comprise:

  • two case studies of commissioners will be developed by the end of Year 2 to illustrate ways in which services are being commissioned in a more integrated way.
  • three case studies by the end of year 2 presenting an assessment of improved accessibility and flexibility in service delivery.
  • two case studies by the end of Year 2 to illustrate the greater use of service user involvement to deliver positive service or system change. (Note this will tie in heavily with LOT 2 of the local evaluation – ie primary data collection with individual service users and volunteers to understand experiences and changes)

Again, please indicate in your methodology the number of interviews/ survey respondents you believe to be feasible given the available budget.

Responding to this brief

Please outline as succinctly as possible:

  • Your understanding of the brief
  • Your proposed methodology for (i) systems mapping and (ii) stakeholder research over both Year 1 and Year 2. As stated, there is likely to be overlap between the two parts and the methodology suggested in this document is a guideline only. Shelter are open to other expert, creative ideas which meet our research aims.
  • Your relevant experience – including stakeholder engagement; desk-based analysis and case study development
  • Your knowledge of and experience researching service delivery and commissioning (local Manchester knowledge and experience an added strength)
  • Your capacity to deliver this brief over the two year period (including the start date of July 2014)
  • A breakdown of your costs
  • The project team that would undertake the research
  • Any perceived risks and how these would be mitigated

Please email your proposals to Sarah Colston , Senior Evaluation and Research Officer, by Friday 27th June at 5pm.

If you have any questions Sarah can be contacted on 0344 515 2129

1

Appendix: Theory of Change for Systems Change

1