Contents
Executive Summary
List of Abbreviations
1.INTRODUCTION
1.1 Background to the Mid-Term Review and Evaluation
1.2The Mid-Term Review and Evaluation (MTRE)
2.0 Methodology
Table 1: Data Gathering Process
3. FINDINGS
3.1 Inception Activities
3.2 Needs Assessment/ Baseline Study
3.3 Developing Indicator Tools
3.4 Establishment and Support of Model Villages
3.5 Project Effectiveness
3.6 Efficiency
3.6 Impacts
4. General Conclusion
Executive Summary
This report summarizes the findings of a mid-term review of the Climate Based Change Adaptation. This 2-year, project funded by the EU, Nile Basin Discourse Forum Rwanda (NBDF Rwanda) is the lead contractor. Started in July 2013, the project completion date has been slated for 30th of June 2015. The project is a strategic Two (2) years pilot designed to bring about changes in empowering community groups and non-state actors to mitigate climate change risks through appropriate climate change adaptation activities for sustainable development. The core target groups are community groups in the four districts (Nyagatare ,Kirehe, Gatsibo and Bugesera districts in Eastern Province Rwanda), and non-state actors. The project intends to provide knowledge and skills for community members to be powerful advocates for change.
The mid-term review’s scope focused on evaluating progress to date, effectiveness and, to the extent possible, impact. Consultant assessed the extent to which the project is meeting its milestone targets and is on a trajectory that will lead to achieving its stated Objectives.
Key findings of the review include:
The project is within planned schedule activity schedule. The participation of diverse institutions working together and model village pilots adding value and will provide an important foundation in adaptation sustainability measures.
To date, the project’s activities are being performed in a vacuum with minimal knowledge sharing or dissemination, leading to a perception that CBCCA is not adding value. Outside direct beneficiaries, there is minimal understanding of what CBCCA is doing or how it is contributing to the government’s CCA policy agenda.
The evaluation team’s opinion is that the project has the potential to have a significant impact on target districts and national policy agenda, but that it is questionable whether it is on track to do so in 1st year yet key dissemination plansscheduled for year 2
Please refer to this report’s annexes for a list of meetings and documents reviewed, and MTR questionnaire.
List of Abbreviations
CBA / Community Based AdaptationCBCCA / Community Based Climate Change Adaptation
CBO / Community Based Organization
CCA / Climate Change Adaptation
EU / European Union
FGD / Focus Group Discussion
GoR / Government of Rwanda
KII / Key Informants’ Interviews
NBDF / Nile Basin Discourse Forum
NGO / Non-Government Organizations
PLA / Participatory Learning and Action
PRA / Participatory Rural Appraisal
RECOR / Rwanda Environment Conservation Organization
REMA / Rwanda Environment Management Authority,
SSF / Small Scale Farmers
1.INTRODUCTION
1.1 Background to the Mid-Term Review and Evaluation
Nile Basin Discourse Forum Rwanda (NBDF Rwanda)is the lead contractor ofthe European Union funded project ‘’Community Based Climate Change Adaptation (CBCCA’’. Started in July 2013, the project completiondate has been slated for 30th of June 2015. The project is a strategic Two (2) years pilot designed to bring about changes in empowering community groups and non-state actors to mitigate climate change risks through appropriate climate change adaptation activities for sustainable development. The core target groups are community groups in the four districts (Nyagatare ,Kirehe, Gatsibo and Bugesera districts in Eastern Province Rwanda), and non-state actors.
The project intends to provide knowledge and skills for community members to be powerful advocates for change. The project further gives emphasis to community groups and non-state actors’ adaptation needs in order to bring about positive benefits for the individual and the community. The CBCCA project addresses core needs of both the target groups and beneficiaries, as well as addressing key government policy set out in ‘Turning Vision 2020 in Reality’ (2007).
The Project is implemented in a decentralized manner with the members of Nile Basin Discourse Forum in Rwanda whose areas of action are within the four districts of the Eastern Province and together with RECOR (direct implementing partner of the project).
Objectives of the CBCCA Project
The overall objective of the project is to empower local communities and non-state actors to monitor the impact of poverty reduction and climate changes on environmental sustainability.
CBCCA project have four results:
- Community based organization leaders, and local authorities in the four districts will be able to monitor the impact of climate change in their villages and districts and adoptappropriate adaptation programs. Also, ownership and continuity of these programs will be strengthened.
- Community groups will have accurate information and links with policy makers in order to influence decisions to mitígate climate change risks, reduce poverty and meet their own needs sustainably.
- Non-state actors will have access to a range of indicator tools that support work towards a sustainable future.
- There will be increased participation, awareness and dialogue on climate change at decentralized level with policy makers, the donor community, international and local organizations.
1.2 The Mid-Term Review and Evaluation (MTRE)
The purpose of this evaluation is to assess the progress and achievements of the NBDF- EU project which is at a mid-point of implementation and to provide recommendations to guide the remaining implementation period. In accordance with the CBCCA project’s main document, the project issubject to a mid-term independent evaluation. The evaluation was undertaken in accordance with the Monitoring and Evaluation project element. This MTRE covers the first half of implementation however review highlights of entire project is carried out to strengthen the adaptive management and monitoring functions of the project and fast track acceleration of the implementation in 2nd half. Particular emphasis is put on the current project results and the possibility of achieving all the outcomes in the given timeframe, taking into consideration the performance and the speed, at which the project is proceeding.
Objectives of MTRE
The overall objectivesof MTRE is to assessthe mid-term level of performance achieved in the attainment of outcomes, identify shortcomings, to draw lessons, and to better redesign and implement targeted strategies for the rest of period.
As part of monitoring and evaluation framework, the mid-term evaluation is an independentreview to assess progress on outputs/outcomes and identify areas for strategic orprogrammatic adjustments, if necessary. Specifically, it is to:
- Assess the level of achievement of key indicators
- Identify the challenges of implementation and suggest solutions.
- Draw the lessons learnt during the implementation
- Propose the actionable recommendations for the rest of period of implementation
Limitations
This mid-term review relies on qualitative information generated through primary datagathering. Opportunities for triangulation were sought in order to facilitate validation of data through cross verification from two or more sources. However, not all data/informationcan be treated in this manner especially as certain key players have not responded to interview KII questionnaire 3despite having been contacted. The results from the Focus Group Discussion (FGD) and document review and observation of the activities on the ground should then be considered as key in these findings.
NB: This report does not attempt to cover areas which audit reports will have dealt with,especially concerning financial matters.
2.0 Methodology
The MTRE was conducted through a consultative process as well as an objective review and analysis of the project’s field visits, interim technical annual report, documents, NDBF/RECOR internal reports and summaries, project archives and relevant documentationwhich provided evidence to sufficiently describe the assessment parameters concerningthe project’s progress.
Apart from desk review, the methodology employed is essentially: (1) key informant interview (KII) using a questionnaire guide that focuseson particular agencies/institutions vis-a-vis outcome/output, (2) Focus Group Discussion(FGD) using a questionnaire guide, and (3) study or field visits where observation andinterview methods were utilized for more insights.
In some instances triangulation was sought throughassessment of the project implementation reports, FGDs and field visits to ascertain consistence with CBCCA project objectives and result indicator in project main document strategic interventions.
Table 1: Data Gathering Process
Methodology / Key Purpose / SourceDocument review / -Assessment of the project documents, baseline indicators surveys, field reports to ascertain consistence with the prevailing and forecasted indicators as stated in the project main documents/guidelines and strategic interventions. / - Project’s main document
-Field monitoring reports
-Needs assessment and baseline study
-Project’s stated means of verification
-semester based and annual report
- etc.
KII Questionnaire / To determine: (1) the effectiveness and efficiency of the implementation strategies or management systems; (2) what concerns need to be addressed into order to sustain the positive gains that the project has achieved; (3) how concerns can be addressed to achieve project outcome. / -Key Informants – steering committee members, District Authorities key staff, central government key staff, Project Technical Team
FGDs / -To find out the extent to which achievement/project impact is felt by target beneficiaries; / -Participants of field review meeting held in Gatsibo.
- beneficiaries testimonies
3. FINDINGS
Findings and Observations - Specific to Activity
3.1Inception Activities
The project was publically launched on 10th January 2014 in at Palast Rock Hotel in Nyamata sector in attendance of 34 participants representing multi-stakeholders; district authorities, partners, NGOs/CBOs/CSOs, FBOs, EU, and Academia. This was an important milestone as it increased publicity and dissemination of project details, learning from experience of other likeminded projects in the area and enhancement of collaboration and networking with other stakeholders in the project area. Introduction and orientation of project team to the stakeholder and partners in the project area as a foundation for collective advocacy and partnership through networking was also established.
Evaluation finds out that SC planning meetings were duly convened including discussions with partner organization. Purchases and Procurement of equipment was successfully executed and recruitment of district field officers and accountant were also done in effective manner.
Assessment of Activity year 1 delivery on Target indicators
ActivityInception Activities Planning, Purchases and Procurement, Recruitment of Staff, project introduction to partners and Districts, Signing MoU letters, Inception and planning. / Overall Target Indicator results / Status at MTRE / Unforeseen Results
-Planning meetings held with Steering Committee and Partner Organization,
-Purchases and Procurement of 4 laptop computers and printers, 4 motorcycles, furniture including office chairs and tables) were done,
-Recruitment of 4 district field officers and a Project accountant. / All listed milestones confirmed achieved. / -MOU signed enhanced collaboration, collective action planning and cooperation between NBDF/RACOR, FBOs/CBOs and government authorities in target districts.
MTRE Observation:
The steering committee is active in its oversight role. Since inception two steering Committee meetings have been held. The steering committee meetings were fruitful as they evaluated progress and gave recommendations on project implementation acceleration plans, sustainability and value addition. The composition of steering committee is also rich in diversity with management representatives from the key organizations involved in the project oversight and control, and represents key stakeholder groups that have special interest in the outcome of CBCCA project.
MTRE Delivery Rating: This activity was achieved 100%. It indicates that the project implementation planning was started up effectively judging from project output with risk factors in mind as well as implementation preparedness.
MTRE Recommendations:
Beyond holding project review meetings the steering committee needs to further act as an advocacy platform andpromote vertical and horizontal policy engagement with duty bearers. Steering committee needs to be vocal and visible enough in the project’s policy engagement activities. The role of the steering committee needs to be strengthened to generally help engage and influence policy engagement activities. In the next implementation phase (2nd year) the project could be re-strategized so that the steering committee can be involved in providing policy knowledge and partnership resources, assist in building sustainability plan such as securing funding and other roles as will be defined.
3.2 Needs Assessment/ Baseline Study
A collective effort by the consultant and NBDF Rwanda Staff resulted in carrying out a needs assessment and an indepth baseline indicator study. The baseline survey mapped basic indicators of welfare, information sources, livelihood/agriculture/natural resource management strategies, needs and uses of climate and agricultural-related information and current risk management in 4 project districts target. The study provides NDBF and RECOR –implementing partnerdirection on benchmark indicators to be monitoring during implementation. The study has further mapped information that is assisting in strengthening of the project’s monitoring and evaluation plan, development of ToTs training modules and capacity building interventions and information that is assisting in identifying relevant policies, guidelines and frameworks for supporting project implementation, advocacy and sustainability.
ActivityNeeds Analysis and Baseline Survey / Overall Target Indicator results / Status at MTRE / Unforeseen Results
-List of indicators on CBCCA
-No. & importance of specific issues tallied and addressed fromCommunity-Based needs - the poverty-environment linkages of governmental programs / 1 Baseline study report has been produced highlights specific issues tallied and addressed fromCommunity-Based needs - the poverty-environment linkages of governmental programs
1 List of indicators matrix produced / - identifies emerging knowledge on the exploration of social differentiation: CBCCA Knowledge Attitude Perceptions (KAPs) of the community members
- the indicator matrix produced is becoming benchmark for multi-stakeholder CBCCA project performance evaluation
Assessment of Activity year 1 delivery on Target indicators
MTREObservation:
Findings of the study are not widely shared with the community of climate change practice in Rwanda and elsewhere. Moreover, the findings would be helpful in evidence based policy engagement. Although NBDF expressed intentions to disseminate the findings to multi-stakeholders, no clear dissemination roadmap/plan was seen by MTRE. The MTRE also finds a gap in the project’s knowledge management sustainability area.
MTRE Delivery Rating:this activity is rated at Mid High 85% delivered15 remaining since generated information is not yet widely shared or used.
MTR Recommendation:
-There isurgent need to develop and implement communication strategy to sustain impact of such an important study. A dissemination plan is urgently needed to rollout sharing of project’s generated information to a wider coverage of multi-stakeholders.
-The indicator matrix produced out of the baseline study should be adopted in M&E plans as it is an evidence based guide to project performance monitoring and evaluation- it sets clear methodologies for tracking changes in project implementation.
3.3 Developing Indicator Tools
This activity is implemented in 5 phases. The key activities delivered so far include; Baseline study, an indicator matrix and ToTs indicator training modules toolkit for which qualify this activity to be at phase four 4. According to MTRE discussion with the NBDFproject coordinator, the remaining phase 5 is scheduled to be implemented in the 1st semester of 2nd year implementation before end of September 2014. The remaining activities include training of trainers on how to use indicator tools,publication of the indicator tools and dissemination.
ActivityDeveloping the indicator tools develop a series of indicator tools based on the needs analysis and partners experience. / Overall Target Indicator results / Status at MTRE / Unforeseen Results
-no. Of and quality of indicators
-no of publication shared
-% of community members using the indicator tool / 1 Baseline study report has been produced highlights specific issuesindicators
1 List of indicators matrix produced
1 indicator ToTs manual produced / None; for this activity unforeseen results may accrue from trainings and dissemination activities that will be undertaken in year 2.
Assessment of Activity year 1 delivery on Target Indicators
MTR Observation: The information generated towards developing indicator tool is yet to be utilized in 2nd year, yet it’s important that beneficiaries to get early capacity on how to interpret and utilize these indicators tools. ToTs need to be accelerated since some of the beneficiaries and project staff testified to MTRE that they lack some capacity and knowledge of climate change adaptation issues.
MTRE Delivery Rating:this activity is rated at High Rate100% delivery in accordance with year1 action plan.
MTR Recommendation: there is a need to put in place an implementation acceleration plan in the 2nd year of implementation especially focusing on capacity building of the key planned beneficiaries through urgent expedition of ToTs–during the MTRE there was expression of early capacity building demand driven by beneficiaries.
3.4 Establishment and Support of Model Villages
This activity is implemented in partnership with Rwanda Environment Conservation Organization (RECOR) through MoU established on 1st December 2013. The emphasis of this activity is on piloting best practices and models community based climate change adaptation actions on the ground. The model villages intend to serve as best practices in climate change adaptation and environmental protection towards poverty reduction. Activities are implemented in five (5) model villages selected in the four target districts.Activities are in support and linking other existing initiatives at district level supported by government programs like the rain water harvesting, agro forestry, post-harvest management systems, improved human resettlements, and environmental protection. land consolidation which is effective for land use and improved agro-productivity through use of organic manure, promote use of energy-saving stoves to reduce deforestation, and the promotion of the use of clean and renewable energy like solar.