Testing the Theory of Change
Good programming principles state that the theory of change should be able to be tested so that we can plan for effective intervention strategies, and so that we clearly demonstrate the logical linkages between our contributions and the eventual changes in the lives of our chosen impact group.
This exercise is designed as a collective reflection process to examine how our specific theory of change can be tested, and thereby help in refining our system for measuring the impact of our programme.
Steps in the exercise
Step I – identify key components of the impact goal
Key question:
F Looking carefully at the impact goal, what are the key components of the overall impact that are envisioned within the goal?
Example:
CARE Burundi women’s empowerment program impact goal is: by 2025, poor and vulnerable women from rural areas will have regained their dignity and fully enjoyed their fundamental rights.
This goal envisions Aziza, a woman who:
§ has improved health, particularly sexual reproductive health;
and that she:
§ knows her rights and has the capacity to demand that they are upheld
§ has the confidence and capability to make decisions about her life, especially sexual relationships
§ is economically empowered to meet her basic needs and control productive assets
and lives in a context that has
§ laws that protect her rights and the laws are implemented
§ traditional laws and cultural norms are changed to remove the barriers to her empowerment
Step II – Explain causal relationship between each domain of change and the key components of the impact goal
Key questions:
F Look at each domain of change; how does each domain of change logically lead to one or more of the key components in the impact goal?
Example:
Step III - Examine the credibility of the identified causal relationships between domains and the components of the goal
Key questions:
F Which of these causal relationships have a well established body of evidence to support them (from long standing development practice, either CARE or others, that has been well researched and documented)
F Which of these causal relationships are less proven and need to be tested and documented?
Example:
The causal relationship between the availability of quality SRH services and measurable improvements in women health, especially sexual reproductive health, is well established and does not need to be tested.
The causal relationship which says that ‘when individual men are provided with information and dialogue around masculinity, this will lead to women’s empowerment’ is less well established and needs to be tested.
Step IV – identifying information needs to test the less proven causal relationships
Key questions:
Focus on the causal relationships that are less established:
F What key assumption(s) is the causal relationship based on?
F What are the essential questions that need to be asked in order to demonstrate the causal relationship?
F What are the sub-questions? [what specifically do we ‘need to know’]
Example:
Assumption to be challenged: ‘Engaging men by providing them information on SRH and changing their attitudes on masculinity will result in a change in their behavior and attitudes which enables the empowerment of women.’
Essential question: Does CARE-Burundi’s approach to men’s engagement result in women’s empowerment?
Sub-Questions:
· Does engaging men and changing the attitudes of individual men (rather than changes in broader social/cultural norms) actually empower women?
· Does providing men with information on specific topics (e.g., sexual reproductive health) lead to a change in attitude and behavior that empowers women?
· Does working with men on issues of masculinity result in a change in attitude and behavior that empowers women?
Step V – ensuring usefulness of the tested theory of change
Key questions:
F Who are the important stakeholders (audience) that the domainàimpact causal relationship needs to be demonstrated to, particularly outside of CARE?
F Keeping in mind the audience, what approaches/methods should be used to gather and analyze the information?
Example:
Audience / Objective / MethodologyCARE-Burundi
CARE International / Verifying effectiveness of the approach / Mix method of qualitative (KI, focus group, participatory evaluation) and quantitative (questionnaire and survey)
NORAD / Verifying effectiveness of the approach, fund raising / Mix method of qualitative (KI, focus group, participatory evaluation) and quantitative (questionnaire and survey)
EU, USAID, Gov Burundi / Evidence for advocacy and scale-up / Mix methods of qualitative (e.g., outcome mapping, interviews with boundary partners) and quantitative methods (e.g., survey of attitudes with a control group)
Step VI – looking for alternative explanations
Remember, identifying a causal relationship does not mean that it is automatically the exclusive or sole cause. After all the domains of change have been analyzed, it is also useful and important answer the following questions:
Key questions about alternatives:
F What might be an alternative explanation or causal relationship that could lead to the envisioned impact, but it is not presently in the proposed domains of change?
F Does the alternative explanation possibly have a stronger causal link then the one embedded in the domain?
F Why are we not using the alternative as part of our approach?
Example:
Step VII – Devising a feasible strategy to test the theory of change
This exercise can (and probably will) result in identifying a lot of unanswered questions that could be studied in detail. We must acknowledge, however, that addressing all of them immediately is usually not very feasible. Therefore, we have to establish a strategy that will help us focus on what “we need to know” versus using up our time and resources on issues that are only ‘nice to know’.
Key questions:
F Which questions could feasibly be addressed by CARE and/or its partners? At what scale and rigor?
F Which questions cannot be feasibly addressed by CARE or its partners and why? What strategy would be useful to counter this lack of information?
OMIT???
F How can we operationalize a learning strategy for the questions that CARE will address directly? When, how, resources?
1