CPWF Coordination and Change Projects Most Significant Change (MSC) storiesC&C stories- 2012
12 change stories from 5 (Andes, Limpopo, Mekong, Nile and Volta) CPWF basins
Selection process, results and feedback to C&C PLs
February- March 2012
Table of Contents
Introduction
The selection criteria
Individual Ranking
The most significant change stories selected
Individual Criteria and Ranking by Panel Members
Link to the stories
ANNEX 4- Feedback to the NILE C&C Project
Introduction
As a part of the CPWF Monitoring and Evaluation system, the Most Significant Change (MSC) approach is used to monitor and pick up on early progress, particularly initial and emerging outcomes in terms of changes in key stakeholders’ knowledge, attitude and practice (KAP) to which CPWF research has made a contribution.
The CPWF used MSC as part of six-monthly reporting in 2011. Basin Leaders were responsible for selecting the most significant stories from the technical projects, while CPWF Central was responsible for organizing a selection panel to choose the most significant change story from the C&C projects. For more detailed information, please seeuse of MSC in the CPWF.
After submissions on December 15, 2011, a program-level panel was formed by the Innovation and Impact Director to read, analyze, select and provide feedback to all BLs on the C&C stories. The panel met on 5th February, 2012. This document captures the output of that meeting, including:
- The four stories selected as most significant
- Selection criteria used
- Feedback on all the stories
Process followed
The members of the panel were asked to follow this process regarding the selection of significant change stories:
1. Everybody reads the stories (individually), and chooses the one that best answers the question: "From among all these significant changes, what do you think was the most significant change of all?"
2. Each panel member establishes their individual criteria used to choose a story
3. Each panel member grades all the stories (in this case, 12) against their individually established criteria
4. The group holds a conversation about the value of the reported changes, the stories themselves, and their individual choices
5. The group decides which stories are felt to be most significant (either by consensus or majority vote)
6. The reasons for the group’s choice(s) and the whole selection process are documented.
Panel members were also asked to keep in mind the following as they were reading and selecting the most significant stories of change:
- It is important to do this in an inductive way - that is tochoose the story first, then discuss the criteria. The process tends to produce richer discussion and reflection than if the criteria are set in advance.
- The selection criteria reflect what the panel value, which is important feedback for projects.
- Some stories will be better written and presented than others. It is also important that the selection panel evaluate the story on the story and not its presentation, communication value or just style.
On January 5, 2011, after reading the stories the panel met in Montpellier, France, to discuss the selections. Present in the meeting were: Sophie Alvarez (Facilitator), Boru Douthwaite, Tonya Schuetz, Amanda Harding, Michael Victor and Bing Bayot. Panel members Larry Harrington and Martin van Brakel provided their selections electronically. Panel members invited to select but not present or providing input due to timing conflicts were Alain Vidal and Simon Cook.
The selection criteria
In the meeting, each panel member presented their criteria for choosing stories. We then held a process of discussing these criteria, and agreed that our own values and considerations of “significance” as well as insights into project understanding of significance were present in all our criteria. Although each member of the panel analyzed the stories along their own individual criteria (see full list by panel member at end of document) and we did not agree on a single set of criteria, we identified some common criteria and common ideas about significance:
The change mentioned has substantial effect on external end users or stakeholders/ Is relevant for development (potential for impact– emerging, potential), beyond the scope of the BDC. This effect has good probability of repercussions/ growth/ expansion in the future.
Indicates that the C&C project is doing a good job in what should be its core internal business, for example cross-project coordination or communications/integration
Is clearly linked to basin/ CPWF Theory of Change.
Demonstrates quality of R4D approach, research
Implies a high level of collaboration with stakeholders/engagement with partners/ next users of research, presents a linking value (inter- project/ disciplines/ organizations)
Denotes unexpected results; surprises; unexpected success under adverse conditions- presents a high level of applied adaptive management/ innovation in responses
Is clearly reflected, has clear logic, responds to MSC questions, explains a clear process, highly attributable to C&C project
The discussion of significance generated a number of ideas:
A significant story is more than successfully addressing the basin development challenges; it is also being able to do innovation, catalyze change, and get more buy-in.
By significance we mean the scale, depth and sustainability of change along each of the criteria.
Many of the significant changes demonstrated:
- Good network management, knowing and talking to the right people
- Creating space/ infrastructure for dialogue, innovative approaches and processes
- Tapping into on-going or recent reforms
We selected the most significant stories by comparing our individual ratings. We selected four. However we found significant aspects in all stories and decided to give feedback on all stories rather than only the ‘most significant’, which had been our original intent.
Individual Ranking
Larry / Martin / Tonya / Boru / Sophie / Michael / AmandaMK5-3 / AN4-1 / MK5-1 / MK5-2 / MK5-1 / AN4-1 / N5-1
MK5-2 / L5 / MK5-2 / N5-1 / MK5-2 / MK5-3 / MK5-1
AN4-1 / MK5-2 / N5-2 / N5- 3 / N5-2 / N5-2
N5-2 / V5-1 / V5-2
The most significant change stories selected
The stories here represent the most frequently chosen by panelists along their individual criteria. The following are the four stories highlighted, in no particular order:
The story “National platform building on land, water and natural resource management” of the Nile C&C project was found to be a very good story with clear changes and implications for potential impact. The story explains the significance of the platform, well beyond the BDC. Although the change narrated is one that has been “long overdue” the C&C project was seen to take advantage of the accumulated events to create a platform with great potential, but whose true usefulness will only be seen in time. It was perceived to be very positive then that the story presents a well thought out follow up and monitoring plan of the results of the change.
The MSC story “From the Santa River Basin to the Cruz de Mayo community” presented by the Andes C&C shows the advantages of multi-disciplinary research, and the uses of social networks analysis. The links to the C&C project theory of change are strong, as well as the supporting documentation presented. The story also denotes a strong change in attitude within the BDC, an example of timely support to integration and a creative way to help projects improve their work in study sites.
“A vote in favor of process”, story presented by Mekong C&C project, presents a good example of the kind of linkages, opportunities and external impact that CPWF work in partnerships wishes to promote. The investment referred to in the story is significant, and although the relationship with the donor has been evolving over a long period of time, and is not attributable only to this project, the opportunity was well seized and expanded. The story shows a high potential of sustainability, through having a good partnership with common goals and understanding in place.
Finally, “Yea- sayers and nay-sayers” was found to relate well a story of how the opposition to work in hydropower in the Mekong was overcome through a big gamble on the part of the Mekong C&C and the CPWF. The story was found highly significant, eloquently written and with well – articulated learning. However, some panel members were confused by what the actual change was.
Individual Criteria and Ranking by Panel Members
LARRY
Unexpected results; surprises; unexpected success under adverse conditions
Substantial effect on external end users or stakeholders
Doing a good job in what should be the core internal business of the project, for example cross-project coordination or communications
BORU
Profundity of the change
Scope of the change
MICHAEL
Demonstrating coordination/integration
Demonstrating change linked basin OLM
Relevance for development (potential for impact)
Relevance for R4d approach
AMANDA
Significance of story for the BDC Team (internal – and related to quality of R4D and implicit outputs/outcomes)
Significance of story in terms of impact – emerging, potential (external to the BDC team but likely to be within the basin)
Significance of story beyond the basin and parameters set by the BDC
TONYA
Level of integration
Level of collaboration with stakeholders
Level of next users pro-activity triggered
Funding implications
Level of innovation
Level of applied adaptive management
MARTIN
Change in KAP (Knowledge, Attitudes and Practice) – is the change in KAP clearly reflected
Logic – is the logic behind the story clear; does it flow logically; clarity of responses to MSC questions
Actors – who are the actors changed; how relevant is this to the project / BDC
Process – is the change attributable to process within the project / BDC or is the process due to ‘external’ factors; is the process clear
SOPHIE
Main Criteria
Linking value (inter- project/ disciplines/ organizations)
Sustainability- probability of repercussions/ growth/ expansion of change in the future
Clear, strong ties to “greater” ToC - is it justified as part of CPWF business?
Bonus points for:
The follow- up monitoring of the change/ action plan is thought out and made explicit
Creative (if it goes beyond what could/ should anyway be expected of the C&C project)
Uses KM methods/ tools to develop/ share interventions/ change actions
Link to the stories
Please go to CPWF MSC C&C Stories 2011 to see the compilation of all the stories.
ANNEX 4- Feedback to the NILE C&C Project
N5 ChangeStoryN°1:Communicating Inside Out – The Nile Basin’s First Year
This story relates a clear change of practice across Nile BDC projects and beyond (CPWF as a whole; ILRI): putting in place internal communication mechanisms in support of building the NBDC as a coherent program,it is convincingand certainly highly relevant. It has a high significance in the internal adoption and outside recognition of this 'good practice'- it is also a change that is spreading and affecting many, and as such the story has much potential use.
As acknowledged by the author, it is still difficult to judge and assess how significant the changes are: some panelists thought that, although it is an important story, there isn’t a clear linkage to the changes we are seeking, and that the story would have benefited from better description of why/ how this contributes to overall BDC, and connection to research impact. The benefits of open sharing and good internal communication seem to be taken for granted, and are not explored explicitly.
N5 ChangeStoryN°2:National platform building on land, water and natural resource management: momentum for change
This was found to be a very good story with clear changes and implications for potential impact. The story explains the agreement by major development actors to integrate water into their agendas through a national level platform, and setting up the steering committee to run the platform. The significance of the platform is well described, and how it goes well beyond the BDC, and considers external context. The establishment of a stakeholder platform was timely but preceded by a longer and gradual change in attitude, particularly amongst policy-makers. Hence the story reports on a process which cumulated in an ‘event’: the actual establishment of the platform. The C&C project was seen to take advantage of the accumulated events to create a platform with great potential, but whose true usefulness will only be seen in time. Future changes to be catalyzed by this platform are inevitably ‘speculative’. It was perceived to be very positive then that the story presents a well thought out follow up and monitoring plan of the results of the change.
The story would benefit from a further analysis into the integration of a “new” platform into the existing interest and regional advances in the topic. What were the challenges and lessons learnt of doing this?
N5 ChangeStoryN°3:From Livestock-Water towards Consolidated RWM solutions in the Nile Basin
The change in this story - emergence of RWM framework based on years of water productivity research, especially on livestock water productivity- has its greater significance in its potential to become the framework to integrate and guide the work of the NBDC. Some behaviors have already changed - scientists adjusting methods, partners writing RWM proposals (although the example was about CG center (ILRI and IWMI joint appointments) scientists). The story shows awareness of issues beyond the BDC, into external influences, etc., and the implications on the design of the BDC (past). However, the story sets a context rather than relating significant change during project / BDC life. Also, the story overall seems to go way beyond the attribution of the BDC- going heavily into RWM concepts and previous work of other institutes, networks. The potential impact of the RWM framework remains a bit theoretical.