Terms of Reference
Evaluation
Within and Without the State

Programme:

Within and Without the State (WWS) was established as a learning and innovation programme under Oxfam’s CHASE PPA, running from April 2011 – December 2016. WWS aims to improve the quality and effectiveness of civil society programming in fragile and conflict-affected contexts (F&CACs). Delivered in 5 countries (Afghanistan, DR Congo, Occupied Palestinian Territories, South Sudan and Yemen), Oxfam and partners have piloted a variety of approaches to working with civil society to promote more accountable governance and support peace building and state building in ways that are more likely to achieve beneficial results for all citizens. These initial pilots investigated the ways in which Oxfam can support the development of a healthy civil society in specific countries, complimented by a global learning and communications focus that aimed to identify and share good practice learned from WWS beyond the programme boundaries. For more information about the WWS programme, see the relevant pages on Oxfam’s Policy and Practice website.
Evaluation Purpose:
WWS is in its final year and Oxfam is keen to evaluate the initiative in order to determine what contribution, if any, the programme has made to strengthening civil society and promoting more accountable governance in the contexts in which it has been implemented, and to examine how this contribution was made.
Building on the findings of an interim evaluation in 2012, a gender review and annual progress reports to DfID, an evaluation is now being commission during the final year of the programme to document progress and learning from the programme and to inform future Oxfam’s governance programming in fragile and conflict affected contexts. The purpose of this evaluation is todocument the WWS programme’s design and delivery, to assess what has been achieved, and to identify what Oxfam can learn from its 5 years of implementation. With civil society space shrinking in many of the contexts it works, Oxfam is keen to learn more about what has worked well, what hasn’t and why in our efforts to strengthen governance and tackle the underlying causes of fragility. Specifically, Oxfam seeks to gain new insights on models of how WWS has enabled civil society development in specifics F&CACs, to shape future work in fragility and conflict and the operational challenges and opportunities that affect governance programming in these contexts.
There are a number of constraints to working in such contexts, they are complex and highly dynamic, access can be challenging, and we are often working to long term results horizons. It is expected that the evaluation will not only consider the effectiveness of the WWS programme, but importantly will seek to assess and document which strategies have proved effective and under what conditions in order to refine our understanding of what’s possible in these contexts.
While understanding the programme’s contributions to desired changes is an important foundation, the purpose of the evaluation is to critically reflect on our experience working with civil society actors and draw lessons to inform Oxfam’s future programming in fragile and conflict affected contexts – helping Oxfam to think about the value and effectiveness of different processes for engaging and working with civil society actors in these contexts, as well as about what kinds of outcomes we can and should be working towards in these challenging contexts. What are we learning about what ‘good’ looks like.
The objectives of this evaluation are:
  1. to better understand the contribution of WWS to tangible changes in people’s lives
  2. To capture learning about how to work effectively on governance issues in fragile and conflict affected contexts
  3. to take concrete lessons learned, both programmatically and operationally into the next phase of programme design for future conflict and fragility programming
  4. to test the extent to which WWS delivered its learning and innovation agenda
  5. to contribute to organizational learning on multi country programming models, and programming approaches in fragile and conflict affected contexts, including how programme management has affected outcomes at country and global levels
The evaluation will look across the WWS portfolio, but with a particular focus on work in OPTI, South Sudan and DRC.
Evaluation Questions:
  1. What kinds of outcomes were achieved by WWS in the different country contexts in which it operated?
  2. What were the contextual factors that appeared to affect the process and outcomes?How well have WWS projects adapted to the emergent and dynamic realities of the complex environments in which they’re working?
  3. What have been the main factors that have blocked or enabled WWS to achieve its objectives and scale up, related to
  4. Programme/project logic and theory of change
  5. Innovation in programme approaches (in particular in M&E, Action research, context and power analysis, and/ or partnerships)
  6. Institutional (Oxfam) constraints
  7. Individual country (social, political etc.) contexts
  8. What did Oxfam do, or should it have done to support country teams and partners to spot and capitalize on critical junctures in time during the programme?
  9. To what extent has WWS supported gender quality in the fragile and conflict affected contexts it operates in and what can we learn from what has worked and what hasn’t worked in how best to mainstream gender effectively in FCACs??
  10. To what extent has WWS successfully contributed to shared learning in Oxfam and beyond about how to programme effectively in fragile and conflict affected contexts?

Evaluation Design
The purpose of this evaluation is to encourage critical reflection within the programme, and promote learning for programme stakeholders, as well as Oxfam and other sector actors more broadly. Oxfam are therefore looking for an evaluation team able to work with them to design an innovative evaluation process thatpromotes reflective practice and engages stakeholders in a learning-focused enquiry as a team – through, for example, peer review processes, participatory field work, joint analysis etc.
The evaluation should be gender and conflict-sensitive, inclusive of all stakeholders, including people with a disability, culturally-sensitive and participatory, and adhere to principles of careful ethical practice, including informed consent, anonymity and confidentiality and non-harmfulness throughout data collection and storage. The evaluation design will be discussed and agreed in conversation with the successful evaluator(s), and will need to explicitly address these points.
In light of the challenging contexts in which WWS operates, proposed sub-contractors/ local evaluators will need to be approved by Oxfam.
Key Qualifications, Skills, and Capabilities
  1. In-depthunderstanding of governance and fragility programming, and the constraints and opportunities faced by the civil society sector in fragile and conflict affected contexts
  2. Practical experience in design and implementation of evaluations of governance, civil society capacity building and/or multi country programme evaluations, ideally in fragile and conflict affected contexts
  3. Demonstrable experience of producing high-quality, credible evaluations with track record undertaking evaluations in FCACs
  4. Excellent and proven quantitative and qualitative research skills, and familiarity with different methodologies for evaluation
  5. Excellent interviewing capacity and strong facilitation skills
  6. Excellent verbal/written communication skills and strong report writing skills
  7. Given the programme contexts, ability to work in English, Arabic, Lingala and French will beneeded. Oxfam can support with identification of suitable translators as required
  8. Familiarity or direct experience working in one of the WWS focal countries is preferred
  9. Capacity to organise all relevant logistics to effectively carry out the assignment.

Time Commitment
  • It is anticipated that the assignment will take approximately 40 working days to complete, though it is expected that this will be spread across approximately 3 months, as dictated by stakeholder availability
  • The evaluation will commence in June 2016
  • The final evaluation report should be completed byend of September 2016.

Deliverables
  • Inception report detailing evaluation design and field sites
  • Two draft reports for comment and feedback. To include country specific case studies
  • At least one face to face debrief on findings that are emerging with key stakeholders and a process to agree learning considerations and actions together
  • Final evaluation report of publishable quality that documents the research processes, details evidenced findings, including country specific case studies, and a summary of key lessons that can be used to inform the Oxfam’s work on fragility going forward. .

Indicative Budget
The indicative budget for this evaluation is approximately £20,000, as an agreed price for the totality of the work, with the evaluation team expected to manage the number of days spent on the evaluation andall expenses associated with the field trip to the region.
Submitting an Expression of Interest
Oxfam invites bids from individuals with the experience and skills described above.Tenders must include:
  1. A cover letter of no more than 4 pages introducing the evaluator and how the skills and competencies described above are met, with concrete examples as appropriate. Please also use this to indicate how the evaluator(s) would approach assessment against the key questions at country and meta (global) levels, illustrating the balance between rich learning and results at country level and the wider programmatic learning for Oxfam, from this being a multi country programme approach.
  2. A 1-page proposed budget covering all major anticipated costs (Oxfam prefers to pay an agreed price for the totality of the work including the field trips, and it is likely that this would be paid in phased instalments.)
  3. CVs for all evaluator(s)detailing relevant skills and experience of no more than 2 pages, including contactable referees
  4. One example of a relevant previous evaluation (one each for joint bids)
Tender should be sent to June 12, 2016.
Intellectual Property
As per clause 9 of Oxfam’s standard consultancy contract (see annex C) ownership and copyright of alldata, drafts and final products will be the sole and exclusive property of Oxfam Great Britain.