INTEGRATED RESOURCE FRAMEWORK (IRF) EVALUATION – RESEARCH ADVISORY GROUP

MINUTE OF MEETING ON 26 MAY 2010 AT THE RAMADA JARVIS HOTEL, PRINCES STREET, EDINBURGH.

Attending:Kathleen Bessos, Scottish Government (chair)

Joanne Atkinson, NHS Ayrshire and Arran (representative for Ayrshire and Arran test site)

Alisdair McDonald, NHS Lothian (representative for Lothian test site)

Simon Steer, NHS Highland (via teleconference) (representative for Highland test site)

Roddy Ferguson, Fortuno Consulting (IRF evaluation project manager)

Marian Craig, Falcon Craig Consulting

Ron Culley, COSLA

Margaret MacLeod, NHS Information Services Division

Alison Taylor, Scottish Government

Christine Sheehy, Scottish Government Analytical Services Division

Iain Pearce, Scottish Government Analytical Services Division

Colin Blyth, Scottish Government (minute taker)

Apologies:Mike Brown, Vice-Convenor of ADSW Resources Standing Committee

Alison Wood, NHS Tayside (representative for Tayside test site)

Paul Leak, Scottish Government

Julie McKinney, Scottish Government

Nicola Fleming NHS Information Services Division

Lisa Reedie NHS Information Services Division

Gill McVicar, CHP Association representative

1. Introductions

1.1 Kathleen Bessos welcomed everyone to the first meeting of the Integrated Resource Framework Research Advisory Group.

2. Draft remit (Paper 3)

2.1 Kathleen Bessos spoke to paper 3 which outlined the key aims IRF evaluation, the key aims of the Research Advisory Group, and its proposed membership. There were no suggested amendments from the group and it was agreed this remit should be formally adopted.

3Ambitions for the research

3.1Alison Taylor provided some background to the IRF evaluation, which was commissioned by the Scottish Government’s Analytical Services Division on behalf of the IRF team in the Scottish Government. Invitations to bid for the evaluation work were sought through the Public Contracts Scotland website, and six tenders were received. Following assessment of the bids and subsequent interviews with two of the contractors, a consortium led by Fortuno Consulting Limited was awarded the contract in January 2010. The evaluation team were represented at the meeting by Roddy Ferguson and Marian Craig.

3.2 The evaluation brief specified clearly that a formative rather than a summative approach was required, particularly in light of the lack of existing evidence relating to the financial integration of health and social care. This also echoed early feedback from test site staff which suggested that a mechanism to let knowledge flow between the four test sites should be an integral part of the evaluation.

3.3 Another key consideration for the evaluation team will be to ensure that any learning is applicable to areas outside the current test site cohort. This is consistent with the need to maximise learning from the piloting of different approaches in each area, recognising that there is a growing appetite to adopt the IRF approach across Scotland as a whole.

4.Scope and mechanisms

4.1 Roddy Ferguson then updated the group on the scope of the research and mechanisms which will be used to monitor local progress. There are to be four monitoring frameworks developed – one for each test site –these will be based on a common national framework to monitor progress across the four tests sites as a whole. The current draft version of the Tayside evaluation framework was distributed to the group prior to the meeting and will be indicative of the approach taken in developing the frameworks in the other areas.

4.2 Capturing improved outcomes lies at the heart of the evaluation and with support from the Scottish Government these will now be incorporated as local measures for change. A key consideration in the development of local monitoring frameworks will be to demonstrate that any changes in outcomes are a direct result of changing patterns of resource use and staff behaviour in the test sites. The local monitoring frameworks will therefore include measures of change - agreed with the test sites – that only capture outcomes which can realistically be altered over the lifetime of the evaluation (18 months).

4.3It was suggested that using the Community Care Outcomes Framework – which incorporates measures covering emergency admission – to monitor outcomes could be a sensible approach, particularly given that all test site partnerships are members of the Community Care Outcomes Benchmarking Network. Roddy confirmed that this approach was considered, but had to be balanced alongside the need to support each area to pick their own set of appropriate output and process measures to reflect variations in approach and populations of interest.

4.4Roddy also raised the subject of attributing any changes in outcomes (or suitable proxies) to the implementation of revised IRF arrangements. The group agreed that it would not be feasible to establish a national control group, particularly in light of the fact that most partnerships are now mapping and could not be considered neutral.

5.Development of local monitoring frameworks

5.1Roddy Ferguson then outlined some key points in the development of local monitoring frameworks for each test site. Each framework will focus on capturing locally appropriate measures of change set against the context of local priorities and populations of interest. Marian Craig then spoke to the Tayside monitoring framework which was circulated prior to the meeting, with the following points emerging:

  • The level of engagement of key stakeholders has been a key consideration and this is being discussed at the Tayside IRF Project Team meetings. The level of engagement has been good so far.
  • A log of important issues is being kept and these will inform future refinements of the framework, and also influence the development of the other test site frameworks which will broadly adopt the same structure.

ACTION: Fortuno to share any big issues with the Scottish Government as they emerge.

5.2The group agreed that any emerging issues in test sites should be shared with the Scottish Government in order to facilitate a swift resolution to any local problems. The question of ‘action learning’ between test sites was then raised and Fortuno representatives then outlined their approach for sharing learning at every opportunity. This would take place at the following levels:

  • Within the test site – workshops are being arranged to feed back local issues. The exact format of these workshops has been left up to test sites to decide
  • Between test sites – through existing quarterly national IRF learning/sharing events for delivery staff.
  • Between test sites and non test sites – through a checklist of points to consider for those in the early stages of IRF development, to be published as part of the final evaluation which is due in November 2011.

5.3The group responded positively to this summary, noting that using existing opportunities for information sharing such as the quarterly learning/sharing events is a sensible approach. The development of a checklist for partnerships looking to develop was also welcomed by the group, particularly in light of the emphasis which has been placed on good local leadership as a key factor in embedding revised local financial arrangements.

6.Timescales

6.1 Roddy Ferguson then outlined key timescales for the development of local evaluation frameworks. These are broadly based upon the recent letter from Graeme Dickson to each test site Executive Lead which outlined key milestones for test sites between now and April 2011. By September 2010 a baseline position against measures in the framework will be established for each test site; an interim position will be established by February 2011; and the final position will be established by September 2011.

6.2In the subsequent discussions, the group agreed that these timescales struck a good balance between being both challenging and achievable. The following points also emerged:

  • There should be a degree of flexibility around the order in which test sites meet these key milestones, but equally progress should be monitored closely so that future meetings of the Research Advisory Group take place following key stages in the evaluation, and also fit in with meetings of the IRF Programme Board.
  • There was recognition that when the various local integrator structures are empowered to use revised financial mechanisms there will likely be a noticeable rise in issues emerging from the test sites.
  • Test site representatives asked for more clarity from the Scottish Government around how the Cost Book fits in with the parallel development of local tariffs and programme budgeting approaches.
  • Shifting the IRF from the status of a project to a change in culture will provide the biggest test of all, and learning from the evaluation will help to facilitate that change.

ACTION: Scottish Government to clarify the role of the Cost Book in relation to the development of local tariffs.

7.AOB

7.1There was no other business.

8.Next Meeting

8.1The date of the next meeting will shared with members of the group in due course. This will be chosen to coincide with the next meeting of the IRF Programme Board and the completion of the baseline measurement in each of the test sites.

1