WEST AFRICA AGRICULTURE PRODUCTIVITY PROGRAM
(WAAPP)
MONITORING & EVALUATION OPERATIONS MANUAL
VOLUME 1 – Main document
(Final Version)
January 2014
WAAPP M&E MANUAL
TABLE OF CONTENT
Page
1
WAAPP M&E MANUAL
1an overview of the WAAPP
1.1General context
1.2objective and components of the WAAPP
1.3Who are the beneficiaries and what are the target areas ?
1.4WAAPP Implementation arrangements
2introduction to some basic elements of m&e
2.1Purpose of M&E
2.2Monitoring versus Evaluation
2.3What is an Indicator ?
2.4The Results Measurement Framework (RMF)
3purpose of this M&E Manual
3.1Objectives of this manual
3.2Main contents and structure of the M&E Manual
4The M&E stakeholders roles and information needs
4.1Institutions involved in the implementation of WAAPP project
4.1.1At country level
4.1.2At Regional level
4.2The main actors of the WAAPP m&eis and their information needs
5THE m&e DATA: INDICATORS and the results measurement fRAMEWORK
5.1The WAAPP results measurement framework
5.2WAAPP results indicators
5.2.1Indicators baseline values: the initial values of indicators
5.2.2Most important steps for data collection
5.2.2.1Data Sources
5.2.2.2Method of Data Collection
5.2.2.3The Frequency of Data Collection
5.2.2.4Responsible Entities for Data Collection
6WAAPP M&E PROCEDURES
6.1GENERAL
6.2Operational planning
6.2.1Preparation of the Project Implementation Plan (PIP)
6.2.2Preparation of the Annual Work Plans and Budgets (AWPB)
6.2.2.1Content of the AWPB
6.2.2.2The AWPB preparation process
6.3Monitoring of the implementation of the AWPB (Process monitoring)
6.3.1Objective and scope
6.4Results Monitoring
6.4.1Organizational set up for data collection on results
6.4.2Source / Method of data collection
6.4.3Dashboard (tableau de bord) for results monitoring
6.5Reporting on the project progress towards results
6.5.1The Framework of reporting
6.5.2M&E data processing
6.6Evaluation of WAAPP project
6.6.1Overview
6.6.2Pre-Project or ex-ante evaluation
6.6.3Mid-term Evaluation
6.6.4Final Evaluations
7The tools of the m&e system
7.1Presentation of the tools
7.2The web based M&E software
7.2.1General characteristics
7.2.1.1System flexibility
7.2.1.2System scalability
7.2.1.3System security
7.2.1.4System simplicity
7.2.2The various modules to be provided
8Data quality assurance
1
WAAPP M&E MANUAL
LIST OF TABLES
Page
1
WAAPP M&E MANUAL
Table 1: Example of a Results Measurement Framework.
Table 2: Implementing entities for the WAAPP components and sub-components
Table 3: WAAPP stakeholders roles and their information needs
Table 4: Summary of M&E responsibilities
Table 5 : indicators data sources and frequency of data collection as indicated in the PAD
Tableau 6 – Template for AWPB Implementation Monitoring Form (IMF) for the Present Period (Quarter or Semester)
Table 7 – Template for accumulated values of AWPB Implementation Monitoring
Table 8: Template for Reporting & Data Quality Assessment - Regional M&E Unit level
1
WAAPP M&E MANUAL
LIST OF FIGURES
Page
1
WAAPP M&E MANUAL
Figure 1: Links between the logic of intervention and the chain of results.
Figure 2: M&E data sources
Figure 3: WAAPP Logic Model
Figure 4: The general architecture of the WAAPP M&E Information management System
1
WAAPP M&E MANUAL
LIST OF ANNEXES
(See M&E Operations Manual – Volume 2)
A.Annex 1: Results measurement frameworks for WAAPP
B.Annex 2 : Results Indicators Tracking Table
C.ANNEX3: Results indicators reference sheets
PDO indicators
Indicators for Component A
Indicators for Component B
Indicators for Component C
Indicators for Component D
d. ANNEX 4: Results monitoring forms (RM Forms)
FORMS FOR ODP INDICATORS
FORMS FOR COMPONENT A INDICATORS
FORMS FOR COMPONENT B INDICATORS
FORMS FOR COMPONENT C INDICATORS
FORMS FOR COMPONENT D INDICATORS
E. annex 5: Format report
F. annex 6: awpb format
E. annex 5: awpb monitoring
LIST OF ACRONYMS AND ABREVIATIONS
ARIs Agricultural Research Institutions
AWPB Annual Work Plan and Budget
CAADP Comprehensive Africa Agricultural Development Program
CARGS Competitive Agricultural Research Grant Schemes
CAS Country Assistance Strategies
CAVA Cassava-Adding Value for Africa
CERAAS Centre d’Etude Régional pour l’Amélioration de l’Adaptation à la Sécheresse / Regional Center for Drought Adaptation Studies
CGIAR Consultative Group on International Agricultural Research
CORAF/WECARDConseil Ouest et Centre Africain pour la Recherche et le Développement Agricoles/ West and Central African Council for Agricultural Research and Development
COSTAB Project’s Costs Table
CSIR Council for Scientific and Industrial Research in Ghana
CM&EOCountry M&E Officers
EAs Environmental Assessments
ECOWAP Economic Community of West African Agricultural Policy
ECOWAS Economic Community of West African States
FNRAAFonds National de Recherches Agricole et Agro-alimentaire in Senegal
FTMRFinancial and Technical Monitoring Report
IERInstitut d’Économie Rurale in Mali
ISRAInstitut Sénégalais de Recherche Agricole
ISTAInternational Seed Testing Association
IP National Implementing Partners
NCoSNational Centers of Specialization
NPCUNational Project Coordination Unit
NSCNational WAAPP Steering Committee
PIMProgram Implementation manual
PIPProgram Implementation Plan
RM&EORegional M&E Officers of WAAPP
RMFResult Measurement Framework
RITTResult Indicator Tracking Table
RPCUregional Program Coordination Unit
I. MAIN CONCEPTS UTILIZED[1]
Planninggreflects theprocess of identificationand implementationschedule fora series of actionsto be implementedtoachieve theobjectives.Operations may coverthe term of "components", "sub-components," "activities" or "sub-activities."
The project concept refers to a set of investment activities and operations planned in order to achieve an objective or set of objectives. This is a response mechanism chosen to achieve an objective or objectives identified during the planning process. The project includes a cycle that involves the identification, formulation, financing, implementation and monitoring and evaluation.
Monitoring and Evaluation Systemis defined asan integrated packagethat includessome aspects of theplanning process,ofsystematic data collection, analysis, exploitation, synthesis, anddissemination of information, and providesthemeans andskills toimplementthe systemto improvethe basis fordecision makingin themanagementandimplementation of a projectand thecapitalization ofexperiences and lessons learned.
Implementation Monitoring is a systematic and regular assessment of the resources used (input) based on forecasts on the one hand, the achievements (output) and the results of the project (results and intermediate results)in respect of targets and timing on the other. It is based on the development of Annual Work Plans and Budgets (AWPB). Its main deliverables are the technical reports, accounting and financial reporting, as well as activity progress reports.
Theimpact assessmentcomponentseeksto measure orunderstand the scopeof the project actionson the well-being of beneficiaries,and the major changesand outcomesinduced by theseactions.It includesthe establishment ofthe baseline, evaluations, surveys and periodic or specific indepth studies, as well as thefinal evaluations. The similarityof thepattern ofthe baseline surveyand the project completionsurvey is critical toenable a comparisonbetweenthe baselinedescribedat the beginning of the project andthe follow-up situationat the end ofit.
Results based Management (RGM):Results-based management (RBM) is a life-cycle approach to management that integrates strategy, people, resources, processes, and measurements to improve decision making, transparency, and accountability. The approach focuses on achieving outcomes, implementing performance measurement, learning, and adapting, as well as reporting performance.
Logic model (LM): Sometimes also called a "results chain," a LM is a description of the causal or logical relationships between inputs, activities, outputs, and the outcomes of a given policy, program or investment.
Result: A result is a describable or measurable change that is derived from a cause-and-effect relationship
Outcome: is a result statement that outlines what a policy, program, or investment is expected to achieve or contribute to.
Output: Products and services generated through inputs and activities
Beneficiary: The set of individuals and/or organizations that experience the change of state at the ultimate outcome level of a LM although they could also be targeted in the immediate and intermediate outcome levels.
Stakeholder: An individual, group, institution, or government with an interest or concern, economic, societal, or environmental inthe framework of the project.
Partner: The individuals and/or organizations that collaborate to achieve mutually agreed upon expected results.
1
WAAPP M&E MANUAL
1an overview of the WAAPP
1.1General context
Meeting the food needs of a rapidly growing population is one of the main challenges facing the countries of West Africa. The recent food, fuel and financial crises have demonstrated the need to focus food security in West Africa on the development of regional agricultural potential. This option would contribute both to economic growth and poverty reduction.
It is possible to achieve rapid growth in agriculture with emphasis on improving productivity. For example, the adoption of new varieties and improved farming practices would help increase millet yields by 30% (600 kg/ha to 780 kg/ha), the yield of irrigated rice by almost 50% (from 1.560 kg/ha to 1.950 kg/ha) and that of cassava yield of 40% (14 tons/ha to 20 tons/ha)[2].
The West Africa Agricultural Productivity Program (WAAPP) has been designed as a two-phase program with a total duration of 10 years, to support the implementation of CAADP’s agricultural research and development (R&D) pillar.
As a technical body of ECOWAS in the area of Agricultural research and Development, the Coordination the WAAPP was assigned to the West and Central African Council for Agricultural Research and Development (CORAF/WECARD).
Since 2007, the ECOWAS countries with the support of the World Bank and others donors ( ), funded three series of projects, including the WAAPP-1A (for Mali, Senegal, and Ghana) in 2007, the WAAPP-1B (for Burkina Faso, Ivory Coast and Nigeria) in 2010, and WAAPP-1C (for Benin, Guinea, Liberia, Niger, Sierra Leone, Gambia and Togo) in 2011. Based on the results obtained by the WAAPP-1A, the World Bank and countries have decided to support the extension of the program in its second phase for Senegal, Mali and Ghana(WAAPP-2A).
Thus, WAAPP-2A will continue to provide to the ECOWAS countries, a regional cooperation framework for the implementation of national and regional agricultural strategies for development, dissemination and use of technologies. These interventions are expected to improve agricultural productivity and economic growth while reducing poverty in recipient countries.
1.2objective and components of the WAAPP
As stated in the Project Appraisal Document of WAAPP 2A, the Development Objective (PDO) is “to scale-up the generation, dissemination and adoption of improved technologies in the participating countries’ priority agricultural commodity areas”. Reaching this objective implies:
(i)strengthening the regional cooperation in the technology generation and dissemination;
(ii)scaling-up dissemination and training programs focused on regional and national priority commodities;
(iii)developing collaborative mechanisms and synergies with the governments’ subsidy programs as well as with other projects and programs intervening along the targeted value chains.
As stated in the in the Results framework of the Project Appraisal Document, the achievement of the PDO can be verified by the following key outcome indicators:
- Direct project beneficiaries of which 40 % female disaggregated by country
- Beneficiaries who are using technologies generated by other countries - disaggregated by country
- Technologies generated by the Project with at least 15percent productivity increase over the control – disaggregated by country
- Producers with knowledge of technologies generated/released by the Project- disaggregated by country
- Area under new technologies disaggregated by country
- Processors/ producers who have adopted at least one new technology -disaggregated by country
The project activities are implemented through the following four components:
1. Component A:Enabling Conditions for Regional Cooperation in the Generation, Dissemination and Adoption of Agricultural Technologies.
This component finances support to strengthening the mechanisms and procedures for the exchange of technologies, including: (i) Implementation of ECOWAS common regulations on genetic materials, pesticides, fertilizers and veterinary products; and (ii) Implementation of regional WAAPP strategies, which consist of mainstreaming regional strategies into national action plans.
2. Component B:National Centers of Specialization
This component aims at strengthening regional cooperation in technology generation by aligning national priorities with the regional ones. It will support the upgrading of the National Centers of Specialization (NCoS) into Regional Centers of Excellence (RCoE). It will also strengthen relevant national research centers associated with NCoS in the development of the value chain and facilitate synergies with relevant Consultative Group on International Agricultural Research (CGIAR) and other international research centers. It will mainly finance construction and rehabilitation of core infrastructure, ISO certification of the NCoS, capacity building, grants to implement core research activities and capacity strengthening for CORAF/WECARD,
3. Component C: Support to Demand-driven Technology Generation, Dissemination and Adoption
This component finances the strengthening of priority-focused demand-driven agricultural R&D, scale-up technology dissemination and adoption within participating countries and facilitating access to improved genetic material. It includes three sub-components: (i) Demand-driven Technology Generation that will continue to strengthen the regional CARGS managed by CORAF/WECARD and the existing CARG scheme in each of the participating countries; (ii) Support to Accelerated Adoption of Released Technologies that will scale-up the adoption of released technologies to continue bridging the gap between farmers’ and researchers’ yields for selected priority commodities and (iii) facilitating Access to Improved Genetic Material that will increase the availability of and producers’ access to improved genetic materials (seed, planting materials, and animal breeds) for the targeted priority value chains.
4. Component DProject Coordination, Management and Monitoring and Evaluation.
The Project will continue to be coordinated: (i) at the national levels by existing national coordinating units; and (ii) at the regional level by CORAF on the basis of a well-defined mandate agreed by the Regional Steering Committee (RSC) under the Annual Work Plan and Budget.
1.3Who are the beneficiaries and what are the target areas?
The Project’s primary beneficiaries are the producers, processors and all other actors of the selected value chains. The beneficiaries include also the key participants in the generation and dissemination of technology, including researchers, extension staff, public and private extension services and advisory agencies, research institutions, universities, NGOs, and government agencies involved in value chains management.
1.4WAAPPImplementation arrangements
WAAPP is a regional programme. CORAF/WECARD has been mandated by ECOWAS and countries to coordinate the WAAPP at regional level. Each country has a national steering committee and a project coordination unit (PCU). A Regional Steering Committee, chaired by ECOWAS, is also established and meets once yearly to review the AWPB implementation progress of the current year, to approve the AWPB of the next year and to provide guidance for the project.
At national level, the institutional arrangements in each country are made for the Ministry in charge of Agriculture or Research to be responsible for the oversight of the Project; with the coordination of the Project being entrusted to a “Project Coordination Unit (PCU)”.The PCUs are responsible for the internal monitoring of Project indicators and outcomes as defined in the Results Framework. Each coordination unit (M&E officer)produces semi-annual progress reports along with notes synthesizing information on the project activities based (among others) on the reports produced by implementation agencies. The NCoS in participating countries will be responsible for measuring the impact of technologies released and transferred under Component 2.
At regional level, project coordination will continue to be entrusted to CORAF/WECARD which will have the responsibilities of monitoring program outcomes and impact at the regional level through:
(i)Carrying out update studies on the region’s agricultural productivity;
(ii)producing an annual consolidated report, based on the countries’ reports and their specific studies, to be shared with all ECOWAS countries and stakeholders;
(iii)informing participating countries on a yearly basis on implementation progress and the use of funds transferred from country proceeds,;
(iv)maintaining the web-based databases on grant administration and results while developing and maintaining an agricultural research resource database in terms of publications and research skills available in the region.
In order to speed up dissemination of project results and ensure achievement of the PDO, CORAF/WECARD will build further synergies with some regional organizationsnotably, the African Forum for Agricultural Advisory Services (AFAAS), the Network of Farmers' and Agricultural Producers' Organizations of West Africa (ROPPA), the West Africa Seed Alliance (WASA), the relevant CGIAR centers (IITA, ICRISAT, AfricaRice), the International seed testing association (ISTA), the African Seed Trader Association (AFSTA), the “Association des femmes de l’Afrique de l’Ouest” (AFAO), the Association pour la promotion de l’élevage au Sahel et en Savane (APESS), etc.
2introduction to some basic elementsof m&e
2.1Purpose of M&E
Monitoring and Evaluation (M&E) is essential for a results-based approach to program management. After being a key component of project design, M&E remains incorporated into all facets of the project cycle through its completion.The M&E system to be established will enable the WAAPPmanagement team to clearly demonstrate to the key stakeholders whether the project is achieving the stated goal, outcomes and outputs in accordance with the targeted timeframe.
Therefore, the M&E systems should provide the means to robustly analyse the relevant data and information in order to:
- monitor the project outcome indicators as reflected in the Results Measurement Framework;
- comprehensively track the implementation of the Annual Work Plan and Budget (the project inputs, activities and outputs), using mainly the set of indicators (or milestones) outlined under each component and sub-component at frequent intervals,
- Establish a process to alert implementers, stakeholders and WAAPP managers to any problems in project implementation and provide the basis for making the necessary adjustments.
- Outline the flow of data and information from the project sites through the various stakeholders, both for the use of the general public and to inform decision-making.
2.2Monitoring versus Evaluation
Even though monitoring and evaluationhave complementary roles, it is important todistinguish one fromthe other.
Monitoring isthe collection,analysis and systematicuse of information/data tofeed theproject management anddecision-makingin order to achieve theproject objectives.Throughmonitoring,we seek toestablish to what extentthe provision of inputs, work programs, andother necessary actionsand expected outcomesare achieved/executedso thatcorrective action canbe taken in timein case of'significant deviations from expectedquantitativeand temporal targets.
In general, through monitoring one candetermine whether the implementation of activitiesisgoing in the rightdirectionat the rightpace.It provideskey stakeholdersof a project withthe information needed to: