DFID Somalia guidance to implementing partner on strengthening VfM in governance and conflict programmes
1. Background
1.1. Rationale for focusing on VfM
1. DFID is currently increasing its focus on proving that money is being used as well as possible – or value for money (VfM). VfM can be summed up as ‘delivering more with a given amount of money’, or ‘achieving certain objectives with less’. It is distinct but complementary to (i) reporting on results; and (ii) demonstrating good financial management. VfM combines results and money.
2. VfM indicators are ratios, often expressed in money terms, or outcomes/outputs. However, despite the desirability of demonstrating VfM through indicators such as unit costs, some sectors or programmes are less amenable to such quantification. This is especially the case at the outcome or impact level. Unit costs also need to be used carefully in different environments. For instance, there may be reasons for accepting that the cost of delivering an output is higher in one place than another.
3. DFID’s governance portfolio in Somalia consists of the most challenging interventions to demonstrate VfM: impacts are not readily monetised; the security environment is fragile and changeable; and socio-economic data is generally weak.
1.2. Basis for the VfM guidance
4. This guidance sets out DFID’s requirements of implementing partners for governance and conflict programmes in Somalia. It is based on analysis for DFID by ITAD[1] on how to assess VfM in governance and conflict programmes. The ITAD report needs to be understood in full to use this guidance.
5. ITAD provide a framework based on the UK Treasury’s ‘3 Es’ approach (economy, efficiency and effectiveness). The framework is made more suitable for interventions that are not easily monetised by allowing for normative judgements on programmes effectiveness. The framework also includes a method of scoring interventions according to the 3E categories and proposes aggregating these for a single summary score.
6. DFID Somalia has adapted this framework in the way presented in a workshop presented to agencies in January 2011. The main revision was to unpack from the proposed definition of VfM, elements that are standard management practice from what is VfM. This guidance also proposes a disaggregated overall assessment of VfM rather than the single index.
1.3. DFID Somalia’s requirements for demonstrating VfM
7. The approach required is to demonstrate how costs of inputs and activities are being managed down. The requirements include information in proposals (Business Cases) and reporting. DFID’s requirement for VfM in Business Cases is summarised in Section E of the internal How to note titled ‘Completing the Appraisal’.
2. VfM framework and putting it into practice
2.1. 3Es and ratings
8. The ITAD VfM report details the 3E methodology and proposes seven criteria against which to assess economy, efficiency and effectiveness. The following diagram is taken from the report and summarises how these VfM concepts map onto the levels of a logframe.
9. The seven criteria proposed (Section 4, pp 12 to 18) are:
· Effectiveness Theory of change
Leverage replication
Robustness of indicators
· Efficiency Productivity
Risk analysis and mitigation
· Economy Procurement
Unit costs
10. DFID Somalia is choosing to remove two of these from the summary VfM assessment – robustness of indicators and risk analysis and mitigation – as they do not relate uniquely to effectiveness and efficiency. These issues are, however, crucial to good project design and management and lay the basis for managing VfM. A third criterion, Theory of Change is also part of fundamental project design, although it is retained in the framework as it is central to demonstrating VfM effectiveness – that we are delivering the right outputs in order to achieve programme objectives.
11. The summary VfM assessment will include the criteria in bold in the list above. The 1-5 score will reflect a judgement reflecting the narrative in Figure 5 in the ITAD report. This in turn will be justified with a narrative describing progress against VfM indicators.
2.2. Selecting VfM indicators
12. Two to four VfM indicators will be selected for economy, efficiency and effectiveness. It is likely that there will be more indicators at economy and efficiency levels for governance and conflict programmes than at effectiveness level. While VfM indicators should map easily onto a logframe, we propose that they are listed in a separate table due to space constraints within a logframe.
13. ‘Good practice’ is well developed for choosing sound indicators (e.g. the SMART[2] test for indicators). For VfM, complementary checks include the following:
· Comparable – an indicator is useful if it can be compared with a standard or benchmark. Comparators would normally be from other geographic areas and other agencies. But if this is not practical, agencies may have to benchmark against their own programmes in other areas. A practical approach may also be to benchmark within the programme over time i.e. to show indicators improving over time. This is especially important where an agency effectively has a monopoly in operating in certain geographic areas.
· Selective and meaningful – indicators should be useful for programme management purposes, rather than for token reporting. Six to twelve indicators across the programme should be used.
· Measurable financially – indicators need to be monitorable. Often this will require generating financial reports linking resources with activities, outputs or outcomes. Proportionality will need to be exercised in deciding how much reconfiguring of financial reports and systems is justified to report on VfM indicators.
· Intuitive – finally, at least some of the VfM indicators should be usable in communication activities with external audiences, demonstrating how the agency is using resources well.
2.3. Integrating VfM within existing programme management (M&E, reporting)
14. VfM indicators will often combine progress on activities, outputs and outcomes with financial information (unit costs); or combine outcomes or outputs) in terms of activities (ratios). Monitoring VfM should be integrated within wider programme reporting. In practice, additional data may need to be gathered – or gathered in a different way i.e. with different periodicity or disaggregation. Financial information may need to be reported on differently to correspond with the unit that needs to be measured.
15. For programmes that have already started, M&E frameworks will likely need to be modified to report on VfM indicators. For new programmes, the M&E framework established as part of appraisal and start-up phases will need to incorporate VfM indicators into their design. Annex 1 below provides a template for a VfM section of a proposal.
16. Progress on VfM should be included in all programme reviews and reporting to donors. Adjustments to programmes should be made as they would be in response to lessons learned from other aspects of implementation performance. Substantive reviews such as mid term reviews will provide greater opportunity for making adjustments to monitoring frameworks and implementation.
17. Annex 2 below provides a basic template for reporting on VfM in reports. It is possible that material on effectiveness will repeat some analysis from elsewhere in the report.
18. VfM will also be an integral part of programme evaluations. The M&E framework needs to ensure that important evaluation questions around VfM can be evaluated at the end of programmes.
2.4. Multidonor and multiyear programmes
19. Funding proposals to DFID need to provide minimum information on how VfM will be monitored and managed. But most governance and conflict programmes involve more than donor. Most also last for longer than the period of DFID funding. This potentially reduces the incentives for agencies to adapt M&E frameworks to report on DFID’s VfM requirements. It also means that VfM reporting to DFID is potentially cosmetic in the sense that it is not informing programme management in a material way.
20. DFID hopes that all programmes that it funds will incorporate VfM reporting into the mainstream of its performance management at the earliest opportunity. This is likely to take place in stages, with some changes to VfM reporting possible immediately but some changes taking longer. Each review (e.g. six monthly reviews) provides an opportunity. Mid term reviews provide a greater opportunity for making substantive revisions. New programmes provide a full opportunity for incorporating VfM indicators.
21. DFID would like VfM reporting to become part of the reporting that agencies provide to all donors, rather than DFID reporting requirements being additional.
3. DFID VfM requirements from implementing partners
3.1. New funding arrangements
22. For all new proposals (see Annex 1)
· Develop VfM indicators of economy, efficiency and effectiveness, explain how they will be monitored and their value in improving programme performance and demonstrating VfM
· Include VfM in reviews and reporting
3.2. Existing DFID funding
23. Within 4 months (but as soon as practical):
· Develop VfM indicators for the programme.
· Start to monitor indicators and use in programme management
· Review VfM performance as part of programme reviews
· Include progress on monitoring and managing VfM in reporting to DFID from July 2011 (see Annex 2 below). First reports may only include progress on measuring VfM indicators, baselines and targets.
Annex 1 – Structure for VfM section in proposals (Business Cases)
1. Summary choice of VfM indicators
1.1. Selection of VfM indicators of economy, efficiency and effectiveness.
1.2. Explain links to logical framework and project design.
2. Economy
2.1. Indicators: Baselines, milestones and targets
2.2. Detailed description of what the indicators mean for performance
2.3. Measurement issues and fit with M&E framework and financial reporting
3. Efficiency
3.1. [As above]
4. Effectiveness
4.1. [As above]
4.2. Explanation of which aspects of Theory of Change will be reported as part of VfM analysis.
Annex 2 – Structure for VfM section in reports
1. Summary (1 page)
1.1. [Successes/challenges in monitoring VfM]
1.2. [Summary of areas with strong and weak VfM performance. Assessment of overall VfM picture across economy, efficiency and effectiveness]
1.3. [Summary of how this VfM analysis being used to improve VfM and programme performance]
2. Economy (1 page)
2.1. [VfM indicators: what they are, baselines and targets, measurement issues]
2.2. [Progress against targets: quantitative and qualitative with brief narrative]
2.3. [Implications for programme, measures being taken etc]
3. Efficiency (1 page)
3.1. [As above]
4. Effectiveness (1 page)
4.1. [As above]
[1] Measuring the impact and value for money of goverance and conflict programmes, Final report, Dec 2010, ITAD
[2] Specific, measurable, achievable, relevant and timebound