Mid-Term-Evaluation on behalf of WHH Country Office Afghanistan
24.05.2017
1 Introduction
Country: / AfghanistanProject title: / Creation of Sustainable Livelihoods for Internally Displaced and Returning Afghan Families in Kabul Province.
Project No.: / AFG-1176 / 2015.3406.4
Project holder: / Welthungerhilfe
Partners: / ANAFAE, AECC
Approved budget: / 2.3 Mio EUR
Co-financer (line): / German Federal Ministry for Economic Cooperation and Development (BMZ Private Agencies)
Project period: / 15.07.2015-15.03.2019
Welthungerhilfe (WHH), formerly known as German Agro Action (GAA), is a German based, non-profit making, non-aligned international NGO that aspires to see a world without hunger and poverty. Welthungerhilfe's activities in Afghanistan began back in 1980 - immediately after the former Soviet Union's occupation - to provide emergency aid for refugees. In 1992, Welthungerhilfe further increased their emergency aid activities and services, as well as rehabilitation and reconstruction assistance.
Currently, WHH’s draws most of its funding from the German government as well as from donations of the German public with the aim to reduce the level of poverty while improving the livelihoods for the most vulnerable population.
Since July 2015, Central Regional Office of Welthungerhilfe Afghanistan in Kabul - in partnership with two local NGOs – has been implementing a project for Internally Displaced Persons (IDPs), Returnees and Host Community Residents in and around Kabul province. The project aims to empower marginalised and vulnerable IDPs, returnees and host community residents to become agents of their own change by giving them the skills to combat their marginalisation and providing them with an enabling environment for this endeavour.
Project Objective:
The project aims to empower marginalised and vulnerable IDPs, returnees and host community residents to become agents of their own change by giving them the skills to combat their marginalisation and providing them with an enabling environment for this endeavour.
Project Outcome:
23,485 IDPs, returnees and host community residents in Kabul Province are capacitated to improve their living conditions autonomously through (a) diversification of livelihoods, (b) integration of their children, (c) representation of their interests in political dialogue.
Outputs/expected results:
1-IDP, returnee and host community adults have improved their literacy skills up to a level equivalent to grade 3 in the Afghan school system.
Target Indicator:
Until the end of the project, 75% of the 1,000 enrolled adults have passed the final exam of literacy classes and have basic numeracy, reading and writing skills.
2-IDP, returnee and host community children have improved their communicative and expressive capacities, as well as their self-esteem.
Target Indicators:
a)Throughout the project, 9,000 children in 22 IDP, returnee and host community communities have regular access to child friendly spaces and social circus activities.
b)By the end of the project, 40 children and youths have been trained to mobilise their peers and conduct social circus activities in the camps.
3-IDP, returnee and host community women have profound knowledge in poultry keeping and fruit processing and are able to use their knowledge practically.
Target Indicator:
By the end of the project, 60% of the 810 trained IDP, returnee and host community women can contribute to household income.
4-IDP, returnee and host community farmers have improved access to irrigation water and enhanced their skills in agricultural production techniques.
Target Indicator:
a)Upon completion of construction works, the surface of land that can be reliably and constantly irrigated in the intervention area has increased by 30%.
b)By the end of the project, 75% of the 140 trained farmers apply productive agricultural techniques in the areas of vegetable, fruit and cereal production.
c)By the end of the project, 60 % of the 140 trained farmers have created new livelihoods through the selling of agricultural produce on local markets.
5-Government officials know the National IDP Strategy and show stronger engagement for and commitment to the problems of IDPs and their host communities.
Target Indicator:
a)By the end of the project, the MoRR and its DoRRs have started initiatives to implement the National IDP Policy; the initiatives take into account the results of the research project (study), the IDP Shuras and the 2 conferences organised as part of the project.
b)By the end of the project, a platform initiated by WHH for dialogue between IDPs and respective government representatives has been established.
The activities are being implemented in partnership with Afghan Educational Children's Circus (AECC) and Afghanistan National Association for Adult Education (ANAFAE). AECC is implementing activities related to Project Result 01 - Social Circus Activities (Psychosocial Support) for children in the KIS and outside of the KIS, while ANAFAE was implementing Project Result 02 which was running of Adult Literacy Classes in PD 13 of Kabul city and Paghman district of Kabul province. Welthungerhilfe is responsible for implementation of Result 03 – Poultry Keeping and Fruit Processing, Result 04 - Irrigation Water and Agricultural Techniques, and Result 05 – Internally Displaced Persons National Policy relevant activities.
The project holder would like to conduct a mid-term evaluation of the full project. The results will be used by the project holder to improve the work during the remaining half of the project. The evaluation results will be shared with the donor.
2 Task description
The main tasks for the evaluation team are drafted below. The suggested structure of the report is attached as an annex.
2.1 Questions of specific interest for Welthungerhilfe and its partners
The objective of this Mid-Term evaluation is to provide a comprehensive assessment of the project and an opportunity to critically assess administrative and technical strategies and issues. The evaluation gives recommendations to improve the potential of the Project to achieve expected outcomes and objectives within the Project timeframe.
The evaluation aims to determine:
- To analyse the design of the project; and evaluate the effectiveness and efficiency of the project implementation by the 3 partners within the agreed contractual conditions, time frame, work-plan, and budget.
- To analyse the level of coordination and cooperation between WHH, AECC and ANAFAE and find out the gaps, shortcoming, weaknesses and strength points.
Note to the evaluator: In the report, these specific questions might be integrated into the sections on the OECD/DAC criteria or/and the chapter conclusions.
2.2 Criteria for the assessment of the project
For assessment of the project, the evaluation team uses the OECD/DAC criteriaof relevance, effectiveness, efficiency, impact and sustainability (see annex 4).
In addition to the OECD/DAC criteria, the evaluator or team should analyse any other pertinent issues that need addressing or which may or should influence future project direction. Based on the observations and findings, the evaluator should come up with clear recommendations for the remaining time of the project.
Relevance: To what extent is the project suited to:
- The priorities and needs of the target groups.
- The donor and government priorities and policies relevant to IDPs and returnees in Afghanistan?
- Is the Project addressing the right target group in the targeted area?
- Are the Welthungerhilfe relevant standards, procedures and donor priorities considered?
- Are the activities and outputs of the projects consistent with the project primary goal and the attainment of the objectives?
Effectiveness: To what extent has the project achieved its stated outputs?
- IDP, returnee and host community adults have improved their literacy skills up to a level equivalent to grade 3 in the Afghan school system, and to what extend the intervention helped them on their daily life
- IDP, returnee and host community children have improved their communicative and expressive capacities, as well as they feel themselves with stable self-esteem?,
- IDP, returnee and host community women have profound knowledge in poultry keeping and fruit processing and can use their knowledge practically and to what extend they contributed/will contribute their household income?
- IDP, returnee and host community farmers have improved access to irrigation water, enhanced their skills in agricultural production techniques and to what extend the intervention improved/will be improved their livelihood.
- Government officials know the National IDP Strategy and show stronger engagement for and commitment to the problems of IDPs and their host communities, and to what extend they applied/will be applied IDP policy up to end of project
- To what extend the provincial IDP Shura activities were/are effective in term of dialog between government and IDPs, and to what extend the needs of IDPs and Returnees considered in MoRR/DoRR annual action plan.
- Efficiency:
- How cost efficient were the project activities?
- Were the project objectives detailed above achieved on time?
- Was the programme or project implemented in the most efficient way?
- Impact: What were the short term positive and negative changes produced by the project, directly or indirectly, intended or unintended?
- What outcome (positive, negative) can be observed and what impacts expected?
- What recommendations to achieve expected outcomes and objectives within the Project timeframe are made based on the “lessons learnt” and “good practices” are made?
- Sustainability: Assessment on whether the project will be environmentally as well as financially sustainable? Whether the benefits of the project activities are likely to continue after donor funding has been withdrawn.
- To what extent are the benefits of the project expected to continue once the donor funding has ceased?
- What are the major factors which have so far influenced the achievement or non-achievement of sustainability of the programme or project?
2.3 Description of context and project
With regard to questions and criteria the evaluation team describes the following: context, status quo of the relevant professional discourse, important actors, the project, experiences of similar projects onthe country or international level. This chapter provides information for the analysis in the chapter “relevance.”
2.4 Evaluation design and methodology
As part of the assignment, the successful evaluator will apply a mixture of different methodologies, including primary data collection and review of existing resources:
a)A desk review of all project-related documents, reports (narrative, financial and monitoring reports), review of proposal and budget and contract with donor and between WHH and its two implementing partners
b)Field data collection on a sample of outcomes (selection to be discussed by the evaluator with the project team) to verify data collected to date
c)Structured interview with direct & indirect beneficiaries with consideration of sufficient sample size. (Quantitative approach)
d)Focus group discussions with beneficiaries including girls, boys, men and women, parents, and teachers.
e) Key informants interview with community representative, relevant government officials and key staff. (Quantitative & Qualitative approach)
As part of the assignment, the evaluator will provide a detailed planning on proposed methodologies. WHH will review the planned methodologies proposed by the evaluator and provide feedback before the evaluation process begins. We would also like the evaluator to suggest any particularly innovative or interesting methodologies they may employ.
A critical assessment and discussion of the project’s strengths and challenges is welcome. Welthungerhilfe is ready to learn from mistakes. The study design should follow international standards concerning accuracy and reliability of data.
2.5 Reporting and documentation
a)Minutes of the kick-off meeting (a “lighter” form of an inception report), documenting decisions, specifications, agreed dates for field visits, reporting etc. emerging from the meeting.
b)Debriefing paper, at the end of the field mission that outlines the most important findings of the evaluation, 1-2 pages
c)Evaluation report, at the latest four weeks after the field mission, 30-45 pages maximum, plus annexes.
d)Draft management response; integration of the recommendations into Welthungerhilfe’s “management response” form,togetherwith the final version of the evaluation report.
Details of the reports and documentations are included as annex 1.
3 Timetable and phases of the evaluation
Evaluation to begin in the first half of July 2017 (time-frame to be discussed with evaluator), and work in the field to be undertaken around an approximately 20 days’ timeline with the final report to be submitted by August15, 2017 at the latest.
4 Profile of the evaluation team
The evaluator should meet the following criteria:
- An advanced degree or equivalent experience in economics, and/or relevant academic field.
- At least 4 years of experience working on and evaluating livelihood, social inclusion and education projects with expertise in livelihood and education, social sciences.
- Knowledge of the livelihood/education sector and challenges related to social inclusion, and education in Afghanistan.
- Knowledge of partnership models and assessing their effectiveness, especially in a North-South relationship
- Clear, effective writer in English/Dari
- Experience of working with participatory methodologies
5Quality Check of the evaluation report
Welthungerhilfe will check the quality of the evaluation report. The applied grid is documented as an annex.
6 Deadline and Place of Submission
Consulting firms or individual freelancing evaluator that fulfill the requirement shall submitas PDF:
a)Application letter, signed by lead applicant;
b)Financial and technical proposal with schedule;
c)Detailed CV of the consulting team;
d)References
Please note that applications in other format than PDF and applications with an application letter lacking a signature will not be considered.
These should be submitted to 5pm June6, 2017.
The technical proposal for the content of the evaluation must at least contain:
a)Methodology: detail of the following should be presented
-Type of study design
-Sampling technique
-Data collection technique
-Methods of data compilation, analyzing and interpreting
b)Understanding of the TOR
c)Organizational experience and capacity to undertake the assignment
d)The physical input (e.g. computer, printer, fax, telephone and internet access)which would be used by the team of consultant
e)Work schedule
f)Composition of the whole evaluation team including data collectors and supervisors
Financial part
The offer should containthe costs of personnel (lead and assistant consultant, enumerators, supervisors, etc) and inputs(stationary, photocopying, etc). Costs for unforeseen expenses should not be included in thecalculation.
When preparing budget, the following to be considered;
- Soft and hard copies of relevant documents will be provided by WHH
- Transportation will be provided by WHH
- WHH staff will facilitate community entry
- Laptop to be provided by the evaluator
Preferred budget structure:
Item / Total Cost (AFN / USD)Evaluator’s Fee: XX days @ XX AFN / USD Total = XXXX AFN
Other costs (please specify)
Total
7 Proposal evaluation
The evaluation will be done in two stages. During the first stage, the technical part of the proposal will be evaluated and rated. The technical part and the financial part will get 60% and 40% of the evaluation.
8 Termination
If the Client finds that the Consultant is not discharging its duties according to thisterms of reference or is violating Welthungerhilfe’s in-country security management; it may at any time unilaterally terminate the contract and holds theconsultants liable for all damages, financial and otherwise including advance payments.
9Reimbursement and support for proposal preparation:
The cost of preparation of the financial and technical proposals is neither reimbursable norcan be considered as a direct cost of the project.
Annexes
- Annex 1: Structure of reports and documents
- Annex 2: Management Response to Project Evaluation
- Annex 3: Quality check of the evaluation report (Welthungerhilfe)
- Annex 4: Information about DAC Criteria
Annex 1: Final Report Outline
Table of contents
List of abbreviations and acronyms
Lists of tables, illustrations, boxes, etc.
Acknowledgement and disclaimer
I. Summary
1Brief description of the project and framework conditions
2Relevance
3Effectiveness
4Efficiency
5Outcomes and impacts
6Sustainability
7Most important recommendations
8General conclusions and “lessons learnt”
II. Main text
0Prior remarks
0.1Reason and aim of the evaluation
0.2Evaluation process
0.3Evaluation methods
1Description of the project
1.1Brief project description
1.2Framework conditions
1.3Brief description of the target group
2Relevance
2.1Relevance to core problems of the target group
2.2Relevance to the objectives of Welthungerhilfe
2.3Relevance to the objectives of the partner organisation and the partner country
3Effectiveness
3.1Achievement of project purpose
3.2.Project preparation
3.2.1Quality of target group, actor and situational analyses
3.2.2Extent self-help approaches and structures are taken into account
3.3Project planning
3.3.1Quality of the project planning matrix
3.3.2Analysis of the results chain
3.3.3Realism of project purpose
3.3.4Appropriateness of staff, material and financial planning
3.4Quality of the analysis of project holder (partner(s) or external structure of the Welthungerhilfe)
3.4.1Organisational structure and focal points of work
3.4.2Staff, equipment and logistics
3.4.3Project steering and financial administration
3.5Steering by the head office, regional office and cooperation with the project holder
3.6Quality of the project execution
3.6.1Presentation and evaluation of the project outputs and activities
3.6.2Involvement of the target group and project cooperations
3.6.3Overall evaluation of the execution
3.6.4Occurrence of assumptions and risks
4Efficiency
4.1Cost/benefit ratio for the overall project
4.2Cost/performance ratio for the individual measures
4.3Quality of the internal project M&E system
4.4Other aspects
5.Direct and indirect, short, medium and long-term outcomes and impacts
5.1Use of Outputs / Outcomes