Methodology
The overall aim of the 2015-Watch tool is to measure the contributions of donor countries and multilateral agencies to the achievement of the MDGs. The methodology analyses the extent to which donors’ development policies are oriented towards this internationally-agreed set of goals. It has been designed to allow comparative analysis over time and to monitor progress (or otherwise) in a donor’s contribution to the MDGs. The methodology also allows the performances of different donors to be compared.
1. Methodology for Chapter 1
The 2015-Watch tool is a diagnostic instrument. By using it to analyse donors’ development policies, it is possible to gauge the relative strengths and weaknesses of those policies in terms of their orientation towards the MDGs. It is also possible to pinpoint the precise stage(s) of the policy process which are insufficiently oriented towards the achievement of the goals.
The 2015-Watch methodology is based on the concept that the quality of donors’ development aid is directly related to the quality of their policy processes. The policy process is divided into four distinct phases: objective setting, allocation of resources, programming and implementation; and monitoring and evaluation (see diagram in Box 1 on p. 10 of 2015-Watch Report 5).
The scoring system accords an equal weight to all four policy phases. This reflects one of the fundamental concepts of the 2015-Watch methodology – that all four phases of the policy process are equally important in determining the overall quality of the donor’s development aid.
In 2008, the 2015-Watch methodology was slightly modified in order to give greater emphasis to MDG 1 (Eradicate extreme poverty and hunger). Whereas in previous years the methodology had focused on the inclusion of poverty eradication in donors’ development policies, it has now been expanded to analyse donors’ contributions to hunger eradication – the other part of MDG 1. The amendment to the 2015-Watch methodology is reflected in the inclusion of the new parameter ‘Eradication of poverty and hunger’ throughout the policy phases.
In addition to the increased focus on hunger, the 2015-Watch methodology has also been slightly amended to take into account the growing importance of budget support as a means of delivering development aid. This is reflected in the inclusion of the new parameter ‘MDG focus of budget support’ in the budget allocation phase (Phase II) of the policy process.
2. Selection of documents for Chapter 1
The documents analysed in policy phases I, III and IV are primarily official EU publications. Phase II involves analysis of aid statistics for which the main source of data is the OECD/DAC’s online Creditor Reporting System (CRS) database. The data for Phase IV are derived from independent evaluations of the European Commission’s country aid programmes.
The European Commission’s so-called “financing agreements” (budget support contracts) for nine countries were analysed for Phase II. The rationale behind their selection is explained in the methodology for Chapter 2.
All of the available country aid programmes for the period 2001/2 – 2006/7 were analysed for Phase III.
All of the 23 country evaluation reports which were published during the period January 2002 – May 2008 were analysed for Phase IV.
3. Methodology for Chapter 2
Chapter 2 is based on an analysis of a sample of the European Commission’s budget support programmes. The financing agreements for the following countries were analysed: Bolivia, Ghana, India, Malawi, Mozambique, Nicaragua, Tanzania, Uganda and Zambia.[i] These countries, which are all recipients of EU budget support, were selected primarily because they have all been the focus of Alliance2015 studies in recent years. Bolivia, Ghana and Nicaragua were all case countries in Alliance2015’s 2008 report Fostering Democratic Ownership – Towards Greater Impact on Poverty which examined the extent to which the Paris Declaration promotes ownership in specific country contexts. Malawi and Tanzania were both case countries in Alliance2015’s special report The European Commission’s contribution to HIV&AIDS in 2007. Zambia and Uganda were both case countries in Alliance2015’s other special report The European Commission’s commitment to education and the elimination of child labour. Thecontribution of the European Commission’s budget support to the achievement of universal primary education in Mozambique was the focus of a case study in 2015-Watch Report 3. The issue of child labour in India was also the subject of a case study in that report.
The analysis for Chapter 2 is based on an assessment of the objectives of the European Commission’s budget support programmes and the indicators selected to measure their achievement. Chapter 2 also includes an assessment of the level of democratic oversight of the European Commission’s aid programmes, including its budget support and the degree of transparency which exists on the EU side.
4. Methodology for Chapter 3
Chapter 3 is based on an analysis of the current generation of the European Commission’s aid programmes for the following countries: Bolivia, Ghana, India, Malawi, Madagascar, Mozambique, Nicaragua, Tanzania and Zambia.[ii]. In addition, Chapter 3 draws on the case studies in Alliance2015’s 2008 report Fostering Democratic Ownership – Towards Greater Impact on Poverty and the results of the European Commission’s own study into civil society consultation in African, Caribbean and Pacific countries.
Page | 1
[i]Madagascar was also a case country but no budget support financing agreements were available to researchers at the time of writing (May 2008).
[ii]Uganda was also a case country but the EU’s country programme for the tenth European Development Fund (2008 – 2013) was unavailable to researchers at the time of writing (May 2008).