Value for Money

CARE UK DFID PPA Independent Progress Review

Background

CARE UK (CIUK) has secured £3.2 million per year through its 2011-2012 Program Partnership Arrangement(PPA) with DFID. These funds are focussed in the thematic areas of governance and social accountability, private sector engagement, climate change adaptation and peacebuilding across selected CARE Country Offices (COs), with a primary objective of supporting the testing, refinement and promotion of innovative approaches to development programming, which is expected to improve development practice across CARE and the development sector more broadly.

Under the terms of the PPA agreement, CIUK commissioned a team of three independent consultants[1]who conducted a mid-term evaluation (or “Independent Progress Review”) of the progress so far in the PPA. The Progress Review is also expected to assess the “value for money”(VfM) achieved through CIUK’s used of PPA funds as one of four primary objectives for the evaluation.

Defining “Value for Money”

Value for money is defined in various ways as demonstrated by the range of definitions presented here:

  1. “VfM is defined as the optimum combination of whole-of-life costs and quality (or fitness for purpose) of the good or service to meet the user’s requirement.” (HM Treasury[2], 2006)

In this definition, the VfMservices supported by the PPA would be assessed not solely on the basis of their cost, but also their “quality” relative to the needs of the “user”. Although the notion of quality is highly subjective, with this PPA it would involve the improvement of programming models that attracted wider use. Defining the users of PPA services is relative and requires clarification: CIUK’s PPA programme is not about direct implementation of activities but rather about promoting the innovation-testing-scaling process. Users of CIUK programme services are therefore defined as being primarily Country Offices and CARE International members, although the ultimate beneficiaries are obviously the impact groups which the development programming is expected to benefit and, as well, many of the tools and methodologies developed through the PPA would benefit the wider international development community. From the perspective of DFID’s theory of change, which sets the basic rationale for the PPA, the donor is also a user of CIUK services.

  1. “...effectiveness and value for money are inextricably linked... effectiveness involves achieving a sustained impact for intended beneficiaries; and value for money is the best use of resources to deliver the desired impact.” (Independent Commission for Aid Impact[3] ICAI, 2011)

CIUK’s PPA design points towards the “intended beneficiaries” being, again, Country Offices who undertake programming that ultimately benefits impact groups and CARE International members. Evidence of “sustained impact” of the PPA for these beneficiaries would be observed when the programming models initially supported by the grant were subsequently built upon, expanded or leveraged outside the context of the PPA. Here the idea is that the targeted PPA investment in CIUK technical support services and the associated financing of what amounts to action research results in products that are useful in other contexts. Examining what the “best use of resources” is to achieve this result is intrinsic to the ICAI’s notion of VfM although this would be very contextual, related to how far along the development of programming models in CIUK’s thematic areas has moved, as well as the complexity of the Country Offices’ portfolios.

  1. “A management approach to value for money: assessment of the extent to which key management processes and resource allocation decisions made at each stage of the implementation process results in the efficient delivery of higher value inputs, activities, outputs and ultimately outcomes and impacts.” (Coffey[4], 2012)

This management approach is non-committal about the actual component pieces that make up the PPA logical framework, and is focused instead on how well DFID and CIUK (and in effect CARE more widely) are steering the program process “efficiently”.Efficiency is defined as the relationship between outputs – in terms of goods, services or other results – and the resources used to produce them. Efficiency in this sense considers budget magnitudes in relation to results achieved. This management approach then examines the decision-making frameworks around which these budget envelopes are determined, and their results assessed.

These three different notions of the VfM concept remain very relevant to this IPM exercise. However balancing the three definitions, the core idea of VfM is that resources are used optimally to reach the desired outcomes[5]. An approach to assess VfM is defined in more detail below.

The IP Review’s Approach to Assessing Value for Money

The core idea of VfM is that resources are used optimally to reach the desired outcomes. According to its logical framework, the CIUK PPA has two desired outcomes:

  1. To enhance development impact in five sectors[6] through promoting good practice and wider take-up of CARE innovative programming models within CI and amongst the development community; and
  2. To enhance development impact by implementing innovative programming models in five specified sectors.

Innovation, promotion of good practice and wider take-up of successful programming models are intrinsic aspects of the CIUK PPA’s VfM proposition. Therefore it can be said that CIUK PPA funds expect to deliver value for money by:

  • Supporting processes that facilitate the packaging and promotion of innovative approaches/models for broader application and promotion. Without this additional investment, broader adoption of innovations is unlikely.
  • Reaching a much bigger audience than would otherwise be possible. This is achieved by investing relatively small amounts of resources to fine-tune innovative models. Successful models are then promoted for potential broader scale-up and adoption by CARE and other development actors, leveraging other funds and potentially multiplying the impact of DFID funds.
  • Improving the organisational effectiveness of CIUK, a key DFID partner, who administered total of £17.2m of other DFID funding in 2011-2012. Improvements in programming models supported by the PPA benefited this wider portfolio.
  • Adding learning to the development sector through “open source” materials created in collaboration with leading UK agencies.

PPA resources are actually spent by CIUK to achieve these outcomes in four broad categories of expenses:

  • Flexible funding provided by the PPA to directly support selected Country Offices in defined priorities related to the five sectors. These are the flexible annual grants.
  • Resources spent flexibly by the five thematic teams in initiatives that contribute to the development of programming models. These are the budgets managed by CIUK.
  • Compensating the time of CIUK technical specialists who support CO and UK-based activities.The costs of about twenty CIUK Advisors are substantially covered by the PPA.
  • Overall shared operational costs of CIUK, where these resources are managed. This is a fixed percentage shared by all grants covering overhead costs of CIUK’s operations.

The VfM equation therefore needs to assess the magnitude, efficiency and quality of these resources spent in relation to the achievement of the two desired outcomes. Therefore to test whether PPA funds are delivering value for money, CIUK needs to demonstrate[7]:

  1. That innovative models/approaches/concepts tested by CARE are adopted on a broader scale – whether internally or externally[8]
  2. That additional “flexible funding” is critical to this process.
  3. That apart from the relatively unrestricted PPA mechanism, there are no other ways of resourcing the development and packaging of innovative models for broader adoption[9]
  4. That the resource levels invested in model development processes are “reasonable”
  5. That there are management mechanisms in place to steer the appropriateness and/or need for the innovation investment[10]

The answers to these five questions are the core of the VfM assessment.

Methodology

In order to investigate VfM, the evaluators investigated CARE’s ability to innovate, pilot new models and take these to scale at the CO level. The team looked at three Country Offices cohorts differentiated on the basis of their access to flexible funding and technical support. In 2011 CARE International supported seventy-two COs in developing world supported by twelve national members, several of whom also manage operations in these COs[11]. The access COs have to flexible funding and technical assistance varies quite widely depending on several factors including which national member is providing the management services to the CO. The three CO cohorts or “pools” were:

CIUK focus Country Offices The review team visited a sample of four COs[12]and interviewed by telephone key informants from the other COs receiving support[13]. Based on the completeness of interviews and input received four CIUK focus countries were profiled: Peru, Bangladesh, Ethiopia and Sri Lanka. Each of these COs receives significant unrestricted resources from their management lead member. In addition, they receive flexible PPA funding and technical support through CIUK advisors.

COs which receive a significant annual allocation of flexible “unrestricted” funding from their lead member, although no PPA resources from CIUK The purpose of looking at this sample was to assess the particular value addition of CIUK involvement, and to assess the extent that innovation of new models and risk taking can occur where the pool of resources available to invest in this area is smaller or has different characteristics. Interviews took place with key CARE CO informants from Mozambique, Pakistan West Bank/Gazaand Vietnam[14]. These interviews focused particularly on the sources of technical assistance and knowledge management mechanisms in the five specific sectors (as these offices benefit from unrestricted resources, they have a different kind of access to the innovative models and lessons learned through the PPA’s innovation cycle).

COs that receive more limited “unrestricted” resourcesKey informants from Cuba, Zambia and Central Americawere interviewed.[15]Similar to the second cohort, the reviewers were trying to determine how innovation took place in these CO contexts, and how this was financed.

With the COs that do not receive CIUK funding, no data was collected having to do with the question of how “reasonable” the resource levels were in the development of new innovative models.The annex summarizes the findings of this inquiry.

Value for money assessment

This section answers the five CIUK questions or “tests” posed by CIUK to assess value for money, summarizing the interview findings with key informants. The section that follows considers the wider impact of PPA contributions to CARE.

1.Were innovative models/approaches/concepts tested by CARE adopted on a broader scale?[16]

The review has found numerous successful examples of the innovation-testing-scaling process taking place within CIUK partner COs which receive PPA 4 funding. However we note that other COs that do not receive PPA funding also consider innovation-testing-scaling as a modus operandi. Key informants in CARE Cuba and Vietnam specifically mentioned that sparking innovation and testing “new ideas” was now an expectation for INGOs by government counterparts. This process of scale-up is also an intrinsic aspect of the “programme approach” that CARE implements globally.[17] The real “prize”, of course, is not simply securing a follow-up grant to expand successful activities in a particular CO - and this is where CIUK has a particular role to play - but the encouragement of broader national, regional and global dissemination of these innovations. Models/approaches/concepts proven in one context need to be reworked for broader application. There is evidence that PPA-funded work has led to the development of consortium learning and capacity building programs (funded outside the PPA) such as the DFID conflict sensitivity consortium[18] and the European Community DME project[19], both of which have policy advocacy components which further extend their reach.

2.Is additional “flexible funding” critical to innovation-testing-scaling process?

Flexible funding plays a critical role in this process. By its nature the innovation process is very difficult to plan for and budget given its unpredictability. Key informants report that flexible funding is critical in several areas: retaining key staff and their knowledge when their project funding ends, building relationships with partners and convening coalitions, undertaking wider contextual or cross-cutting analysis and strategy development, establishing the evidence base (especially ex post) of the innovation’s impact, and effectively communicating these results in order to establish CARE’s credibility in the new area/field.

CIUK’s sustained funding effort through successful PPAs has also been critical to this innovation-to-scale process, where a significant return on the PPA investment has only been realised after several rounds of funding. In Peru a detailed analysis of investments in the area of health was undertaken[20] tracing the programme’s evolution from PPA 2 to date. Results and events that occurred along this timeline confirm the importance of using this semi-restricted resource in a strategic manner to build on the achievements of earlier investments, and, critically, to take advantage of opportunities that could not have been anticipated in the original design.

Focusing the PPA funding on particular sectors and themes also provides good value for money. While the funding is appreciated as being highly flexible by COs, CIUK has been able to effectively complement the CO field work with its own global technical contributions over time, which has further promoted the scaling-up process. Examples of this are found within the development of governance models, conflict sensitivity tools and approaches, and the on-going work in the area of financial inclusion.

3.Are there are no other ways of resourcing this process besides using the relatively flexible PPA 4 resources?

PPA flexible grants to COs are not the only means of resourcing the process. Several kinds of unrestricted and flexible funding resources are available to all CARE COs to varying extents. COs managed by CARE USA (including most of the PPA focus countries[21]) all receive an annual unrestricted grant of between $200,000-1.2 million per year[22]. In all cases, some portion of this funding is used for programme innovation. Some CIUK COs benefit from flexible resources provided by other CARE members: Denmark, Australia and Canada each have programme funding similar to a PPA that is made available to selected COs. Corporate and foundation resources are also frequently used to support innovation and knowledge sharing. Excellent examples of this are the Barclay’s “Banking on Change” programme, or various initiatives supported by the Bill and Melinda Gates Foundation.

Where particular and unique value addition occurs through the PPA is through the role of CIUK, and its provision of technical backstopping and support. The PPA is distinct from other flexible funding sources in that it supports clearly distinguishable outputs that contribute in a strategic way to wider programmatic objectives. Key informants in COs indicated that this contribution greatly accelerates the process of innovation and scaling-up. PPA is different from the unrestricted grants provided by CARE USA because they are tied to particular programme themes, and this provides structure and discipline in their use. PPA funds are very flexible (COs can propose a very creative range of activities for support) but they cannot normally be used to cover operational budget gaps, which distinguishes the PPA from other types of flexible funds. CIUK focus countries that are part of the PPA have the expectation that the funding will continue; this appears to influence their decision-making in terms of how to spend the flexible funding. The reviewers see evidence that PPA resources are invested in “foundational” initiatives, such as programme analysis, strategic planning and the development and promotion of business plans.

Even beyond the PPA focus countries, key informants indicated that CIUK technical support was invaluable to CARE. CO Representatives in Pakistan, West Bank/Gaza, Mozambique, Central America, Cuba, Zambia and Vietnam all specifically recognized the usefulness of the tools, methodologies and documentation made available by CIUK technical teams throughout the CI network.

4.Are the resource levels invested in the process “reasonable”?

The resource levels are reasonable. This review did not have the scope to investigate on a case-by-case basis, but the team did see sufficient evidence to assert that the PPA’s return on investment is very significant in terms of the additional resources it leverages. CIUK programme £3.2 million per year through its PPA. £1,880,000 was budgeted for activities during the first year which was focused on four thematic areas (with direct expenditures on activities worth £660,000) across fourteen COs (who budgeted £900,000 for activities). The balance £1,350,000 was used to support the CIUK programme implementation costs, including the PPA’s share of Advisory personnel costs.