HEA Sahel Regional Project

HEA Sahel Regional Project

External Evaluation Terms of Reference

BACKGROUND

Most existing food and nutrition information systems across the Sahel fail to capture the changes in purchasing power (across seasons and over time) among poor households. Consequently, they fail to provide in-depth and reliable analysis of the nature and underlying causes of food insecurity and the determinants to malnutrition. Developing stronger information systems is a priority for the Sahel region because of the sustained high levels of acute and chronic child malnutrition and high food insecurity which are underpinned by chronic poverty and harsh climatic conditions. Therefore, there is a need to build and strengthen the analytical capacity of food and nutrition stakeholders with appropriate cost-effective and reliable methods, tools and measures. These will enable stakeholders to more accurately report the experiences of households faced with difficulties in accessing food, and design more effective and targeted interventions.

An important analytical framework which has contributed significantly to stronger information systems, and therefore more effective interventions, is the Household Economy Analysis (HEA). The HEA approach was developed by Save the Children (SCUK) and is a unique livelihoods-based framework that provides a clear and accurate representation of the inside workings of household economies at different levels of a wealth continuum, and in different parts of a country or region. This picture can be used for a wide range of purposes, including early warning, planning for and monitoring of poverty reduction strategies, as well as emergency response, early recovery, and food security policy formulation. HEA analysis is useful for both national food security information systems and for designing effective and appropriate interventions programs. There has been a growing interest in the Sahel sub-region in using the HEA to accurately assess food insecurity conditions. However, the expertise has been lacking at country level and where available, most of the experts are English-speaking and up-to-date studies carried out by SCUK were all led and conducted with the support of predominantly international English-speaking experts and not systematically translated to French.

Since 2010 Save the Children is leading the HEA Sahel regional project that is implemented in 7 countries of the Sahel region in partnership with the following partners:

Burkina Faso: Intermon Oxfam and the SAP (Systeme d’Alerte Precoce)
Chad: Intermon Oxfam and the SISAAP
Mali: Oxfam GB and the SAP
Mauritania: ACF and OSA (observatoire de la securite alimentaire)
Niger: Save the Children and the SAP
Nigeria: Save the Children
Senegal: WFP and CNSA (Conseil National de la Securite Alimentaire)

The project that has been funded originally funded by ECHO and since 2013 co funded by ECHO and OFDA has been implemented through different phases and the project experience its sixth phase. If the objectif of the project remains consistent over the time, the current phase funded by OFDA aims to improve the resilience of vulnerable population to shocks by strengthening Early Warning and Response systems in the Sahel region at national and regional level.

Phase I, which ended in September 2011, was characterized by regional level theoretical and practical trainings. Phase II was completed in June 2012 and focused on providing support to national Early Warning Systems (EWS) and humanitarian actors in the use of HEA analyses to inform food security and nutrition responses to the Sahel crisis. Phase III ended in June 2013 and focused on increasing capacity of key stakeholders to use HEA within early warning systems. The fourth and fifth phase seeked to further improve the institutionalize HEA within regional early warning systems through provision of more quality information and improve its use to inform national and regional response plans. The current sixth phase aims to continue the institutionalization process to start the exit strategy for this consortium.

Thus, in each of the country of implementation, food security staff members were trained in HEA baseline and Outcome Analysis and seasonal assessments of food access were carried out that timely inform the Cadre Harmonise process. The Food Economy Group (FEG) played an important role in the training component, as well as the review of baseline and OA results and reports. In order to improve project coordination at the regional level and facilitate the ownership of strategic issues with partners, a Technical Coordination Committee was established that includes Save the Children and the four implementing partners. The committee met once a year in Dakar

Moreover, since 2013 the finalized HEA baselines are compiled in an Sahel livelihoods Atlas that is developed with the support of FEG and that could be used as a source of basic for all stakeholders in the sub-region working on policy formulation, including the AGIR Sahel initiative, and in the development and humanitarian sectors.

On a medium / long term, the objective of this program is to get the HEA institutionalized to get HEA analysis conducted without support of the ngo partners of the current Sahel project.

PURPOSE OF THE EVALUATION

The objective of the current phase project funded by OFDA is to improve resilience of most vulnerable households to shocks in the Sahel by strengthening Regional Early Warning and Response Systems .

The purpose of this evaluation is to assess the project’s progress against the specific objective, and determine whether expected project results were achieved (refer to log frame for expected results and indicators). The evaluation will provide a summary of progress against expected results outlining: achievements and challenges faced.

The evaluation will assess the project activities implemented between October 2014 and end of 2017 in the 7 countries of the Sahel region (project funded by ECHO and OFDA), and examine whether recommendations from the external evaluation conducted in 2014, the internal evaluation of 2013 and the 2012 external evaluation were considered for this phase, as well as focus on lessons learned.

It will also consider these projects in the broader context of the HEA Sahel project that started in the region since 2010 providing short and medium term recommendations to achieve the ultimate goal of institutionalizing HEA in the Sahel region.

The findings of the evaluation could be used to:

improve quality of current programming
guide future programming initiatives, short and long-term
guide the strategic vision of the project in the long-term
to inform proposal development in related thematic areas such as resilience, livelihoods, social protection
lever additional resources from existing and new stakeholders to scale up and sustain the activities and benefits delivered by the project to date
inform the programmatic direction of HEA and EWER systems for donors, academic institutions, implementing partners, etc
demonstrate accountability for the funding received by ECHO and OFDA

Furthermore, this evaluation will assist Save the Children, partners and government to understand which activities require further development and which could be considered best practices and replicated or scaled for further impact.

OBJECTIVES AND KEY EVALUATION QUESTIONS

The objectives of the evaluation are to assess the relevance, effectiveness, efficiency, impact and sustainability of the project. The evaluator should consider the following topics when drafting the final evaluation questions: capacity-building; quality of HEA data and analysis; project management and delivery; and integration of HEA into EW and response systems; dissemination of information; cost effectiveness; institutionalization process.

The following are suggested lines of inquiry (a detailed list of evaluation questions will be prepared by the evaluator):

Relevance

Was the design of the project appropriate for its objectives and context? Did the HEA based analysis informed and improve EWS and response? Did the HEA based analysis used to better formulate long term programmes (including the AGIR / RP processes)?
What are the status and perception of the HEA within early warning systems and institutions in the region? What is the level of institutionalization of HEA in each country? How useful has HEA information been for supporting the design of humanitarian and development interventions?
Are key stakeholders using HEA analysis to inform broader frameworks of food and nutrition security and early warning systems (cadre harmonisé, country government response plans, SRPs etc.)?
If so, in what ways? If not, why not? What are the obstacles to integration?

Are the advocacy and communication tools relevant given our varied target audience?

Consider the Atlas, website, awareness sessions, etc

Effectiveness

Was the capacity building approach effective?

Were the right people targeted to be trained?
What is the retention knowledge of the people trained?
Did the trainees partake in class-room and field trainings?
Did decision makers increase understanding and awareness?
Is a critic mass of experts is achieved in each country?

Was baseline and seasonal information collected, communicated and used at an increased scale? (Compared to previous phases?)
How has quality evolved compared to previous phases (considering the involvement of FEG)?
Are there still obstacles to generating quality outputs (baselines and OA)? And if so, how can they be overcome?
How can the sharing of data and information be improved?

Was HEA information used to influence humanitarian and development plans and responses?
Did HEA information influence response plans? If so, how? If no, why not?
How can HEA information more effectively and efficiently contribute to response analysis and planning as well as to development plans?
Did HEA information influence development plans or programming? If so, how? If no, why not?

What type of advocacy work was developed and implemented (as part of the OFDA / ECHO HEA funded project or outside) and how was it contextualize by countries?

Are the HEA Working Groups effective at national level?

What type of activities do they implement on line with the ToRS?
What is the leadership of the WG in each country?

Was HEA information used to influence humanitarian and development plans and responses?

Efficiency

How efficiently was the project delivered/ implemented? What were some of the strengths and weaknesses?
Was the project implemented in a manner consistent with its design?
Was it timely? Were adjustments communicated appropriately and implemented efficiently?
Were the ambitions of the project in line with the capacity of the partners and the regional team (human, financial etc.) to implement the project? If not, how could this be mitigated?

How efficient was the technical support and management support from SC to its partners?

What was the process of technical verification and validation of results and was it efficient?
How could it be improved? How can recommendations for improvements of analysis be taken forward in the next phases?
How did the regional HEA team and the FEG contribute to the efficiency of the quality results?

Impact

Did we achieve the goal of better integrating HEA analysis into EWRS and pro-poor policy formulation in the Sahel?
If so, which elements should be scaled-up?
If not, how can we improve integration of HEA into EWR and policy formulation? (consider evaluation of HEA integration in the SAPs)
Which elements should be abandoned or redesigned?
Which elements require adjustment to achieve this goal?
How are the HEA focal points (NGO and SAP) involved in influencing policy and plans with HEA? How can this be improved?

What is the level of integration and interaction of the HEA Sahel project with others initiatives of the regional (ie. the Ecoagris project, Cadre Harmonize, resilience measurement lead by AGRHYMET, other)?

How was OA information included in 2016 and 2017 National Support Plans, UN SRP and NGO humanitarian and development programming (and strategies) and how this integration has evolved over time?
How can this key element be improved for the next phases of the project?

What is the Government’ uptake of the HEA and what are the costs dedicated to HEA in Ministries? What are the progresses made to secure longer term funding / get development donors on board/aware/interested?

Sustainability

What should be improved to ensure the sustainability of the approach (knowledge by trained HEA staff, staff retention within the partner organization, etc.)?

How can the relationship between SAP and NGOs focal points be improved to ensure sustainability?

How is the national and regional capacity to manage HEA framework (OA and baseline) without the external support? Are NGOs, the UN and government institutions capable of running HEA OA and using HEA information to inform early warning systems and response planning?

Are key stakeholders able to access HEA information?
If yes, how? If no, what barriers exist?

What is the value added of Save the Children?
What are the recommendations on the level of support still required from Save the Children, and how could this support be progressively phased out over the next phases of this approach to reach sustainability?

SCOPE OF THE EVALUATION

The evaluation will focus primarily on the experience of countries in building HEA capacity in the 7 countries of the project (Nigeria, Senegal, Niger, Chad, Burkina Faso, Mali and Mauritania) with a stronger focus on Niger, Nigeria, Chad and Burkina Faso

In order to triangulate data and answer specific evaluation questions, key informants in all project countries will be consulted via Skype or telephone.

The evaluation will remain at the institutional level and consider data collected from interviews with Save the Children, NGO partners, partner institutions and government. Community members will not be consulted for this evaluation.

EVALUATION DESIGN AND METHODOLOGY

This evaluation is a qualitative end-line inquiry that will use purposive sampling to select key informants in the designated countries. A more detailed evaluation plan that includes the finalized evaluation methodology and key guiding questions will be developed by the evaluator in consultation with the Program Manager, the HEA experts team, the Programme Coordinators, and the FSL Technical Advisor.

Qualitative research will be conducted through the following methods:

Review of project proposal; regional workshops reports; interim report; MoU with partners; budget and financial reports; internal and external evaluation report; HEA training materials and the dedicated website:
Semi-structured key informant interviews with key stakeholders;
Focus Group Discussions with key regional office staff in the West and Central Africa office;
Data will be analyzed, validated and shared through a final report and debriefing call with the Regional Office.

The OA results will also be assessed through the review of the CH reports and UN programmatic documentations (HNO, SRP).

The protocols used to conduct the face-to-face semi-structured interviews in Niger, Nigeria, Chad and Burkina Faso will also be employed for interviews with key informants in Senegal, Mali and Mauritania. Semi-structured interviews with country focal points in Senegal, Mali and Mauritania will be held, if time permits.

Indicative list of people to interview (non exhaustive):

- Regional FSL advisor of implementing partners (ACF, Oxfam GB, Oxfam Intermon, WFP)

- HEA focal points of implementing partners

- SAP HEA focal points

- Save the Children HEA Regional Team, FSL Advisor

- FAO

- FEWSNET

- CILSS

- ECHO

- OFDA

Indicative list of documents that could be reviewed:

- OA reports

- Baseline reports

- FEG documents on HEA

- FEG training reports

- CH national reports

- CH and PREGEC reports

- AGIR Regional Roadmap

- Government support plan

- SRP documents / UN CAP document

- WFP, Oxfam, ACF, SC capitalization documents HEA analysis related

- OCHA documents

- HEA regional workshop reports

- Donor reports (interim and final)

DELIVERABLES

The expected results of the consultancy are:

A final work plan with a list of stakeholders to be interviewed (to be shared prior to data collection)
Interview guides (to be shared prior to data collection)
A draft report, to be submitted for feedback and revised accordingly (to be submitted in English)
A final report in English which includes:
Executive summary
Introduction
Objectives of the evaluation and criteria used
Methodology and constraints
Results and analysis
Constraints and opportunities impacting the project
Conclusion and recommendations
Appendices (list of people interviewed and schedule, interview guides, list of documents reviewed etc)

QUALIFICATIONS AND BACKGROUND OF THE CONSULTANT

The qualification expected for this consultancy are the following ones:

Advance degree in one of the following areas: Agriculture, Rural development, project management, Food security, development economics, and social science.
Proven experience in evaluation
Knowledge in the Household Economy Analysis (HEA). A knowledge of the HEA Outcome Analysis process and challenges in the Sahel is highly desirable
Excellent written and oral communication skills
Good knowledge of Sahel regional and national early warning and response systems including the following initiatives and processes: “Cadre Harmonisé”, AGIR Sahel, Government and UN Response Plans
Fluent in French /or English with a good command of the second

BUDGET AND TIMEFRAME

The number of days expected for this consultancy stand at a max 49 days and will imply some travels in Niger, Nigeria, Chad and Burkina Faso. A travel might also be organized for briefing in Senegal with Save the Children Regional team.

The milestones and tentative dates of completion are the following: