Webinar 3: Policy Forum- Questions asked:
Yeukai Mukorombindo: Semkae I am interested to know a little bit more about your journey towards selecting the appropriate learning tools as an organization and as a network. Any key tips for those currently trying to figure out the right learning tool/approach? For SAM practice
ANS: A MEL System must be fit for purpose – what change you are trying to achieve will determine the appropriate learning tools. We know now that when an organisation is told that a particular MEL framework and tools will work well with the kind of functions they execute, it more often than not fails to be useful. In the past we have used RBM, Outcome Mapping, fused the two, ToCs, etc but often in a straightjacket fashion which really did not speak to the specific change we wanted to see based on our realistic abilities, the nature of our work as a network and the Tanzanian contexts at both local and national levels. This applies for the SAM practice and more broadly the PRM system, we believe.
Yeukai Mukorombindo: For the benefit of everyone, can you give an example where PF applied small “p”(Politics) which yielded results?
ANS: We have had one recurring question from our colleagues in the PF network, International NGOs and other CSOs ask us how do we get audience with MPs so effectively and our meetings with them are well-attended even with some prominent parliamentarians? Sothey assume we pay them handsome Honoraria/sitting allowances to participate which is a big NO for us! The small “p” is to use the parliamentary committee clerks for the telephone calls and all other coordination actions required (you would have to cover for such costs, of course). MPs listen to the paid staff and there seems to be a strong trust for them. The official letter to the National Assembly Clerk will still have to be sent but you are likely to get a response well after you have met the MPs! This is a small “p” issue that we were glad to share with others and teaches us that the use of official channels of communication without a bit of appreciation and warmth towards mid-level officers in our context may get you wedged. Often these officials go through the trouble of describing to you the bottlenecks and where they lie and even help you how to put the right tone on your request letter.
Gertrude Mugizi: Do you therefore only use collaborative approaches now? Would you consider litigation to be confrontational. If so, have you had any success in using this to obtain movement on an advocacy goal and has this contributed to your conclusion that collaboration is better than confrontation?
ANS: We see advocacy as a continuum: on the extreme left is the hardcore activism and on the right engagement. It matters more how you navigate that continuum depending on the issue and context. Litigation would probably fall on the left side of the continuum and we would let those we are engaging with that it is an option we would not shy away from using if the space we currently enjoy was to vanish. The litigation action we have taken is currently stuck in the courts since we raised it 7 years ago (for further details on the seven civil organizations yesterday that filed a petition at the High Court seeking nullification of the newly enacted Constituency Development Catalyst Fund [CDCF] Act which among other reasons, compromises powers of the National Assembly to supervise the executive, see: ). We haven’t had any luck even discussing it with parliamentarians since then as we closed the doors after going to the courts and the executive has no interest either because in a way it was their “gift to MPs” to manage discretionary funds.
Esther Sharara: Semkae-from the perspective of funders, sometimes you get feedback like SAM brings very positive results at local level for very local level issues. But not convinced about its impact on national and other higher level issues. Could this also speak to issues of having a poor theory of change within the work/organisations? Because on the part of the implementers we are convinced that something is happening at the higher level (eg behavioural/attitudes of duty bearers, commitments to increase domestic resources for social for social sectors etc). Could our challenges be related to how we communicated our work and results? I just want to get your take on this issue communicate our work and results. I just want to get your take on this
ANS: A ToC that is not well thought out will lead to such feedback that we see only positive results at the local level but not the national. If you distinguish between the theory of change and theory of action, then the service delivery improvements at the local level will fall in the former. How such efforts at the local level are aggregated with others elsewhere will enable a mapping process that helps us see the big picture and eventually the national portrait which should include matters of say the attitudes of duty-bearers in commitments to social spending, for example. The problem is the lack of “collective narrative” which begins in the way our ToCs are framed. So I do agree how we communicate our work matters. I have been to CSO SDGs platform meeting where they issued a report on what NGOs in Tanzania have done on the SDGs and there was not a mention of Policy Forum whilst we had done numerous things in the area that the rest were not aware of and were so surprised we had done. We will find it harder to get the broader picture if we fail to find creative ways to communicate our work (beyond the odd posting on websites – useful as it may be, I still think we are speaking more to each other than others outside of our practices).
Gertrude Mugizi: Can you give us an example of a tactic that your PMEL will use to demonstrate value for money in a SAM intervention?I meant to assess value for money not 'demonstrate'.
ANS: We say if you would not pay your own money to attend a seminar/training workshop why should anyone else cover that cost for you? We have activities at the national level like the monthly breakfast debates we hold that are well-attended and a recent formal evaluation confirmed that the reasons for attending vary a lot: from keeping abreast of current policy issues for work to networking opportunities for new staffers employed at organisations. When also asked the PF member organisations whether we should shelve it we got a big NO as a response and the reasons were numerous and the knowledge of its value was more tacitly understood. The PMEL should be able to capture such tacit information in the future through the different MEL tools. At the local level this is more of a challenge as far as SAM is concerned but the PMEL tries to address this through use of the Zonal Reflection Meetings we will be holding in different parts of the country (planned – 2 a year).
Esther Sharara: Semkae-in your lessons you mention that service delivery results are there but not in a systemic and sustainable manner. Did I get you correct?. Could you please elaborate a bit more on this
ANS: We have seen numerous instances of improved service delivery in the time we were implementing the SAM partnerships with member organisations but the changes were not sustainable after the implementation ended due to the elapsing of the partnerships as agreed. In conjunction to this, there were some positive changes in attitudes and behaviour of local government officials in most of the districts where SAM had been implemented but not much evidence of enduring capacities hinting that the 2-year partnerships were perhaps too short.
Our understanding is that to see systemic change, you would need to see evidence at least of maintenance of the gains (not regression) and observe evidence of capacities beyond the few individual officers who interacted with PF and its partners. This is a major challenge for SAM as it was being implemented in a manner that made sustainability of the gains dependent on the persons holding those posts continuing with his/her role. What happens when they leave? SAM has to be systematised within government PRM processes.
Gertrude Mugizi: I really like your suggestions on what to do differently. I expect that this will be key area to explore in the regional SAM community.
ANS: Thank you. Let’s see how we can explore these further. I’m personally most keen on the storytelling.
Rachel Gondo: So if you have decided that as a network you will not focus on chnages to service delivery; where is your focus now? and how does this weave in to the collective narrative in the Tanzanian context?
ANS: Service – delivery is currently more of a long-term desired goal for us and not what we target directly through our interventions. I will explain what we mean:
After much soul-searching within the network, it bore down to the PF setup and niche as an organisation that convenes civil society groups. This meant resolving to revert the role of the secretariat to policy convener/facilitator as it was prior to the adoption of SAM in 2008 and organize away from its 2008 - 2016 role as SAM implementer albeit in partnership with member organisations. It has not meant dropping SAM but appreciating that to improve its impact and other social accountability approaches, we can take the lead in documenting and improving feedback loops in the network and learning about what contributes to positive changes, what does not and why. The evidence should then be shared widely to help deepen knowledge of sustainable impact.
Secondly, we have distinguished between our Theory of Change (ToC) that targets the long-term desired goal of service delivery and appreciates that many different actors and initiative contribute in some way to its achievement and our Theory of Action (ToA) where we put explicit our interventions and the relationships between different actions and intended results that will contribute to the ToC.
Yeukai Mukorombindo: Hi everyone i am having audio problems unfortunately sorry about that what i did want to say was i) thank you to PF and other organizations that participated openly about their MEL challenges. ithink part of the issue is that people are too scared or embarrassed to talk about MEL challenges but we can only learn by being open with our challenges and learning from those who have overcome them and i think PF are doing that so well. Then lastly expectations versus reality well put! MEL indicators for SAM shouldnt be linear ie do we have MEL indicators reflecting small p or relationships established for results How far has PF gone in developing these? Can you share an example?
ANS: In our experience, a lot of the small “p” issues are too numerous and ever-increasing, evolving and tacit that to create MEL indicators would be aiming at a moving target, tasking in terms of resources and not fit-for-purpose. But this does not mean we shouldn’t try. For instance the Breakfast Debate evaluation tool I mentioned above, although not designed to measure small “p” indicators, can pick the issues like “my new boss said that if I want to get to know the players in the policy field, the event is a must-attend”. I would not, however, design specific MEL small “p” indicators I would rather they emerge in the course of collection and analysing. I would be keen to hear from the community if they think otherwise.