Term of reference for an evaluation of the food security and civic education projects in the ZOA Uganda programme in Amudat District.
Background
a.The context of Amudat
The population of Karamoja region are mostly semi-pastoralists. Pastoralism has always been the main backbone of the Pokot economy over the years. While pastoralism is still being considered as a most appropriate and sustainable livelihood system based on the ecological opportunities and limitations that exist in the area, increasingly people engage in crop production. This trend is also linked to a strategy of dealing with the loss of livestock during the conflicts. There is a significant land base available where livestock production can be combined with agriculture, especially in the wet and agro-pastoralist zones.
The Karamoja region has high poverty levels as studies suggest 80% of the population live below the poverty line (UNDP, Human Development Report Uganda, 2007). This poverty translates into high levels of food insecurity. Root causes of poverty are adverse weather conditions and years of conflict. Droughts in this region normally occur every three to five years. However, the trend has become more unpredictable due to climate change. The targeted Pokot communities in the Karamoja region have suffered for more than 4 decades from insecurity in the forms of cattle raids and the unlawful use of guns. Overcoming the socio-economic isolation, that results from lingering conflicts, is one of the region’s main challenges.
Amudat District became a separate District In December 2009. Not all key positions within the district have been filled and many departments are understaffed. Many of the positions at sub-county level are vacant. The traditional system of governance and management operates parallel to the administrative structure at community level. The traditional council of elders’ structure is generally more accessible and respected than the formal administrative structure (LC system).
In 2009 ZOA started itsprogramme in Amudat with a food security project. The programme has expanded ever since. The main objective of the ZOA programme in Amudat is to support the Pokot returnees and host communities in rebuilding their livelihoods. The main sectors ZOA is working in are: food security and civic education. The programme has been implemented in Amudat, Karita and Loroo sub-counties and Amudat town council in Amudat District[1], Karamoja region. ZOA has an office in Amudat town and ZOA has a small sub-office in Karita to accommodate the long distances in the Pokot area.
b.The food security sector in Amudat
The food security interventions aimto support returning Pokot and host population by rehabilitating their livelihoods through agricultural projects.The main agricultural/food security project “Food aid & Livelihoods Recovery in Amudat District “has been funded by CRWRC. The project runs from 1st of January 2011 to 31st of October 2012. It targets the Pokot returnees from Kenya together with their host communities of Pian, Pokot, Sabiny and Chekwi currently living in the sub counties of Amudat, Karita and Loroo in Amudat District.
The ultimate goal of this project is that communities in Amudat District are able to address their food and household needs in a semi pastoralist setting.It addresses both the immediate and intermediate food needs of approximately 13,500 people affected by drought and other adversities including challenges associated with returning and settling, such as lack of household assets. The project focuses on addressing the immediate food needs during the hunger gap and on enhancing food security through food-for-work and voucher-for-work activities. Through this approach, the community, with support from ZOA, is expected to be able to meet their current household food needs as well as acquiring seeds, skills and other inputs critical in enhancing agricultural production.
Specific objectives of the project are:
1. Participating households increase food consumption during the hunger gap;
2. Participating households increase crop production;
3. Participating households involved in agriculturally related IGAs increase income.
Atotal of 13,500 beneficiaries (2,250 households of 6 members each) were targeted by this project using the food for work (FFW) and voucher for work (VFW) approach. The table below provides an overview of the number of beneficiaries per sex and type of work. A household is defined as a wife with her children as there are normally more wives in one homestead as men are polygamous.
Number of beneficiaries for VFW and FFWin the 22 months of implementation
Type of beneficiary/work / Food-for-Work / Voucher-for- Work / Total number of unique supported beneficiariesBatch 1 (FFW and VFW) / 2,100 beneficiaries
- 1260 women
- 840 men / 2,100 beneficiaries
- 1260 women
- 840 men / 1,050
Batch 2
FFW / 150 vulnerable beneficiaries
not involved in VFW / 0 / 150
Total / 2,250 / 2,100 / 2,250
c.Civic education sector in Amudat
Under the civic education sector, ZOA implements two projects. One project focuses on functional adult literacy (FAL), strengthening advocacy for access services, and awareness raising about Female Genital Mutilation (FGM). This project isfunded by Diakonia. The civic educationsector also includes civil society strengthening, which is funded by the Dutch Ministry of Foreign Affairs (MFS-II).The civic education sector targets largely the same communities as the ones under the food security project.
FAL/FGM
The overall objectives of the part of the civic education project under Diakonia is to support Pokot returnees and host communities to enjoy their rights to access services and livelihoods and to be protected from violations.
the Immediate Objective(s) are:
- Pokot community members, including women and youth have the capacity to access livelihoods opportunities and to advocate for service delivery by being functional literate.
- Leaders and civic groups sensitize the Pokot communities on their rights and advocate for them
the planned outputs are the following
Planned Output 1:
FAL classes enhanced
Planned Output 2:
Capacity of local leaders and civic groups to promote rights strengthened
Target group/beneficiaries (gender specific):
This project will directly and indirectly target right holders: community members like women, men, elders and youth – especially women, girls and youth in the sub-counties of Amudat, Loroo and Karita. By targeting the different categories of primary beneficiaries, the project will be able to address any unique issues or deprivations including exclusions to ensure all community members are equally targeted. Aspects of the problem that targets a particular category will have to be responded to in an appropriate manner.
The project hopes to reach the following numbers of beneficiaries:
- The project targets to reach at least 30% of the total population with anti FGM and GBV messages
- A total of six (6) groups (women’s and youth groups) per year from the sub-counties of Karita, Amudat, Loroo and Amudat town council will be supported to carry out community campaigns against FGM and other negative cultural practices and also to run income generating activities (IGAs). In total 18 IGA groups will be supported.
- By the end of the project 24 FAL classes will have been supported). On average, each class will have 25 participants.
Civil society strengthening
The overall goal of MFSII is structural poverty alleviation through strengthening southern civil society organizations.
MSF-II is implemented in different areas where ZOA operates and targets to a large extent the same beneficiaries as the projects mentioned above. The results that the project in Amudat is contributing to are the following:
Result area 1: Greater participation in formal and non-formal (primary) education
Output 1.5:PTAs and/or SMCs have been prepared for getting part of sector policy dialogue
Result area 4: Agriculture has developed and food security has increased
Output 4.3: Farmers, IGA & VSL associations and/ or other civil society organisations (CSOs) have been prepared for getting part of sector policy dialogue
Result are 5: Local civil society organisations and alliances are organised in network/umbrella organisations and represent community based organisations (CBOs) and other actors
Output 5.1.: In targeted communities CBOs are equipped to become inclusive platforms for peace-building, networking and advocacy
Purpose of the assignment
The purpose of the assignment is toevaluate to what extent the project purpose and specific objectives are fulfilled
Specific objectives
- Objectively and representatively measure and calculate project indicators listed in this TOR by means of a survey of household members in a 10% random sample of the specific group of beneficiaries and of a control group of which the sample is at least half as large as the beneficiaries’ one.
- Use these indicators and other data to determine the extent to which the project purpose and specific objectives were realised. Support claimed effects by referring to the indicators set. Mention sources of verification used.
- Compare results of beneficiaries with project targets and with results in the control group
- Discuss possible reasons in case the project purpose was not (fully) realised. Where possible, differentiate between reasons at the level of (failed) project results and those related to the (limited) utilisation of these results by the target group(s).
- Discuss possible other effects (positive or negative)of the intervention.
- Evaluate the monitoring processes of the project and the sustainability of its interventions as described in this TOR
Indicators and aspects to be analysed and evaluated
To be able to measure the impact of the Amudat programme data from three different levels needs to be collected and analysed:
A)(Sample) beneficiaries
B)Community level
C)ZOA
The data needed is outlined below per level (ZOA may ask for additional data at the start of the evaluation).
a.Using data collected at the level of the beneficiaries
By means of a survey of household members in a 10% random sample of the specific group of beneficiaries of the food security sector and a control group of which the sample is at least half as large as the beneficiaries’ sample. These beneficiaries have also been targeted by the civic education sector to a great extent, so the same sample is used to measure results of both sectors. The following areas need to be analysed.
Food security sector:
- Food intake
Expected Results / Indicator / Targets / Data Source / Population covered
Participating households increase food consumption during the hunger gap / Number of meals consumed per day during the hunger gap / 2 / 24 hour recall food consumption survey. / 2,250 households in Amudat District
Data needed: food intake in the last 24 hours of each household member
Analysis: compare to preset caloric indicators (adult equivalent scales); compare to control group
- Perception of livelihoods situation by beneficiaries
Data needed: does the household think they are better off than 2 years ago (=pre-project)? Does the household think they have more cash income than 2 years ago (=pre-project)? Does the household think they are better off than others in the community? Does the household think they have never/rarely/sometimes/often/all the time difficulties providing basic needs for the household? (interview household head) (compare to 2 years ago for control group)
Analysis: compare to control group
- Access to water for production
Expected Results / Indicator / Targets / Data Source / Population covered
Beneficiaries have improved access to water for production / % of beneficiaries accessing water for production / 50% / FFW and VFW participants / 2,250 households in Amudat District
Data needed: improved access to water for agricultural and livestock production as compared to pre-project (compare to 2 years ago for control group)
Analysis: evaluate to what extent target of 50% of beneficiaries have improved access to water foragricultural and livestock production); compare to control group
- Access to improved roads
Expected Results / Indicator / Targets / Data Source / Population covered
Beneficiaries have improved road access to nearby markets / % of beneficiaries accessing improved roads / 50% / FFW and VFW participants / 2,250 households in Amudat District
Data needed: improved road access to nearby markets for agricultural and livestock products as compared to pre-project (compare to 2 years ago for control group)
Analysis: evaluate to what extent target of 50% of beneficiaries have improved road access to nearby markets for agricultural and livestock products; compare to control group
- Household income
Expected Results / Indicator / Targets / Data Source / Population covered
Participating households involved in agriculturally related IGAs increase income / Household income from the IGAs / 100,000UGX / Household IGA card / 2100 households in Amudat District
Data needed: list of income generating activities of all adult household members, specifically checking for beekeeping & honey production and income generated during the last year
Analysis: evaluate diversification of income portfolio and income generation while analyzing gender differences. Evaluate if target of 100000ugx per year generated by IGA is reached; evaluate to what extent target of 75% of beneficiaries apply new skills in IGAs (beekeeping & honey production; compare to control group.
- Crop production
Expected Results / Indicator / Targets / Data Source / Population covered
Participating households increase crop production / Kg cereals and kg beans grown per household / 30% increase / Household crop cards / 2100 households in Amudat District
Data needed: quantity of cereals and quantity of beans the household harvested in the last harvest (interview household head)
Seeds applied in Kg / Q (local measure: number of units) / Q (local measure: type of unit (eg bags, tins…) / Q (converted to kg) / Kgmaize
Sorghum
beans
Analysis: compare to baseline data and crop card data. Evaluate to what extent target of 30% increase has been reached; compare to control group
Civic education sector:
FAL/FGM
Under FAL/FGM the following indicators need to be measured among the sample beneficiaries:
Objectively verifiable indicators (OVI) / Data needed / AnalysisOverall Objectives
Support Pokot Returnees and host communities to enjoy their rights to access services and livelihoods and to be protected from violations /
- % of target group that has access to IGAs
- % of target group that has access to social services
- % reduction in cases of FGM among the Pokot people
- Participation in IGAs
- Access and use of social services (education, water, public info and formal health services)
- FGM head counts
Objective 1:
Pokot community members, including women and youth have the capacity to access livelihoods opportunities and to advocate for service delivery by being functional literate. /
- % of target group that can read and write
- Functional literacy and numeracy test results (test in sensible way)
Civil society strengthening
Under civil society strengthening the following indicator need to be measured at beneficiary level:
1.EDUCATION / Indicator / Data needed / AnalysisOutcome 1: Greater participation in formal and non-formal (primary) education / Enrolment rate of school-age children in targeted communities. / No of school going aged children in household of sample beneficiaries; school attendance of these children / Compare to baseline data; compare to control group
Additional data needed: (interview household head and female household members)
-Knowledge about rights and citizenship (test ability of interviewees to reproduce key information about legislation that is relevant for their specific lives and livelihoods and test ability to name institutions they can go to if they want to claim their rights and whether they have reported issues in the last year)
-Women participation in community meetings
-Views of interviewees on Female Genital Mutilation (FGM)and whether interviewees know girls/women who have undergone FGM in last year.
-Knowledge of other rites of passage for girls
Analysis: compare to baseline data; compare to control group
b.Using data collected at community level (community leaders; selected community members and CBOs)
Data to be collected at community level is outlined below for both sectors.
Food security sector:
-List and images of community assets that have been rehabilitated/constructed (subsurface dams, water ponds, roads...)
-Use and appreciation of community assets by community
Civic education sector:
FAL/FGM
The following indicators need to be measured at community/CBO/Civic group level:
OVI / Data needed / AnalysisObjective 2:
Leaders and civic groups sensitize the Pokot communities on their rights (and advocate for them) /
- # of sensitisation meetings/visits to communities by leaders
- # of sensitisation meetings by civic groups
- # of sensitisation meetings/visits to communities by leaders
- # of sensitisation meetings by civic groups
Civil society strengthening
Result area 5: Local civil society organisations and alliances are organised in network/ umbrella organisations and represent CBOs and other actors. / OVI / Data needed / AnalysisOutcome 5.1: Local civil society organisations and alliances are organised in network/ umbrella organisations and represent CBOs and other actors. / % of targeted CBOs that are actively involved in networking with other stakeholders and advocacy for improvement of service provision and livelihood conditions. / # of Number CBOs registered at sub-county/district level;
# of CBOs involved in networking and advocacy for improvement of service provision and livelihood conditions. / Compare to baseline data; compare to control group
% of targeted communities that have CBOs actively involved in conflict resolution and peace building. / # of target communities;
# of target communities with CBOs involved in conflict resolution and peace building / Compare to baseline data; compare to control group
Other data to be collected at community level:
-Type of civic groups and the level of participation in local governance. To what extent do these civic groups/CBOs have access to the local government? i.e.: how often do they meet the local government?