Summary of the D-group discussion on performance monitoring
-September 2010
About two weeks ago we have concluded a three week email discussion around performance monitoring: monitoring progress towards ODF; how to define and measure toilet access, hygiene and use; and how to measure handwashing practices.
The results of the discussion have been used in the workshop on the same topics in Laos from 24 to 27 August 2010 and are also used to develop instruction sheets on performance monitoring and behavioural change communication.
For those who were not present at the workshop and also for those who would like to have the main elements of the discussion in one document, I have prepared a summary document with the introductions to the topic and the key issues discussed.
What did we discuss and who participated? 1
SUMMARY WEEK 1 (draft) 2
SUMMARY WEEK 2 (draft) 7
SUMMARY WEEK 3 (draft) 9
Additional info: handwashing tools and how to make soap 13
What did we discuss and who participated?
Through the D-group discussion we shared examples and developed ideas on what should be monitored for district sanitation planning (the key performance indicators), how (possible measuring methods and tools), who (possible departments, mass movements and/or, community organizations, and people that could be involved) and when (so as to meet both district and NGO programme requirements).
Participants of the D-group discussion were government partners (national, regional and district level), development partners (WaterAid, UNICEF, UNESCAP, GTZ, WSP?) and SNV advisors from Nepal, Bhutan, Laos, Cambodia and Vietnam, and some countries in Africa. The results of the discussion have been used in the workshop on the same topics in Laos from 24 to 27 August 2010 and are also used to develop instruction sheets on performance monitoring and behavioural change communication.
We have discussed 3 consecutive topics, which are part of most S&H programmes, including the programmes SNV is involved in:
1. Monitoring progress towards ODF: what is it, what should be monitored, who monitors, and how? – See summary week 1, p.1
2. How to define and measure toilet access, hygiene and use? – See summary week 2, p. 3
3. How to measure handwashing practices? – See summary week 3, p. 5
Below you will find the introduction and summary of key issues discussed for each topic.
SUMMARY WEEK 1 (draft)
Discussion on “Monitoring progress towards ODF, sustainability & technical quality of toilets in households”
There were 4 questions:
1. What are the principles for monitoring of access to sanitation and hygiene at district level?
2. What should be monitored regarding sustained ODF and technical quality of toilets?
3. Who could monitor ODF, types of toilets and technical standards, including the gender aspects?
4. How could these aspects be monitored, including involvement of women and men, and the poor?
Q1: What are the principles for monitoring of access to sanitation and hygiene at district level?
The following could be principles for monitoring of access to sanitation and hygiene at district level:
· Ownership of the monitoring programme by the local authorities;
· Low cost and low effort for all (including rural households, and schools);
· Monitoring should be as less “extractive” as possible for local people;
· Information on performance should be returned to communities;
· Monitoring and the resulting data and analysis should be gender and poverty specific;
· Monitoring should have low sensitivity to distortions due to power relations in communities.
Participants discussion
Regarding the first question on principles, Raj from Bhutan and Suchana from Nepal pointed to the importance of community ownership of monitoring activities. Also, they requested for early institutionalisation, and ownership of local governments. In addition to that Henk from Nepal pointed out that there are always different perspectives regarding what is to be measured and to be achieved. This calls for the involvement of more different people.
Quynh from Vietnam made a pledge to keep monitoring simple and user friendly, and limited to the minimum key indicators. He explains that is it often too ambitious.
In summary, these are the principles mentioned:
1. Local communities should own the monitoring (together with others) and it should not cause overburdening or tensions. Participatory methods are preferred.
2. Local governments should own the monitoring, and we should seek alignment with national criteria. However, where national standards are too high and do not allow for progress monitoring, “more steps on the ladder” should be included.
3. Frequency: may be initially high and reduce gradually to core indicators;
4. Gender- and poor-specific results and analysed data need to get the support from the highest government officials (see below)
Q2: What to monitor regarding sustained ODF and technical quality of toilets?
There are two questions underlying this question:
1. Should districts only monitor outcomes or also progress towards and sustaining of ODF?
2. Which toilets do we consider sanitary[1] and durable and how to count that, including ‘climbing the ladder’?
[1] For clarity’s sake, we have used the term sanitary toilet for toilets that meet minimal technical, environmental and health-related construction standards, and hygienic toilets for toilets that meet standards of hygienic operation and use. The first are discussed this week, the second in week 2.
Only monitor outcomes or also progress towards ODF?
In Kerala (India) and Indonesia, IRC supported programmes that monitor progress towards ODF. Promoters and/or village representatives keep lists of numbers of households (and out of them poor households), for the various stages of construction and ownership , for example:
Basic information / Total # and % of households in village without toilet or with insanitary toilet, incl. data for poor households;Demand raised / Total # and % of households without toilet and with insanitary toilet, , who have entered their names on toilet construction lists by (i) self or (ii) local mason, incl. for poor hhs;
Toilets constructed / Total # and % of households in village without toilet and with insanitary toilet, who have started to build their toilet, incl. data for poor households;
Toilets completed / Total # and % of households in village without toilet and with insanitary toilet, who have completed their toilet, incl. data for poor households;
Which toilets do we consider sanitary and durable and how to count that?
ODF status and/or toilet coverage obviously depends upon what you classify as a toilet. A starting point to define which types of toilets actually qualify as sanitary toilets is the definition below:
“hygienically separate human waste from human contact and the environment”.
Toilets that do not separate human waste from human contact or are open to water sources and/or flies would not be counted for coverage figures or ODF status under this definition.
There are two different ways to apply this in practice:
· Count only those toilets that comply with the official technology standards: Definitions of hygienic toilet technology standards in Nepal, Bhutan, Laos, Vietnam and Cambodia are still under development. It depends on the specific situation in the countries, and even within a country some technology types can be durable and environmentally adequate in one ecological zone and totally unmanageable in another.
· Count all toilets that comply with the agreed qualifying factors for a sanitary toilet: The SNV team in Bhutan has developed a toilet categorisation to address this issue. The reasoning is that technology may vary widely, as many toilets are built by households themselves.
In Bhutan simple pit latrines have long been considered acceptable standards, but this is now under revision. The Vietnamese government only considers double vault, pour flush and (semi-)septic tank toilets as hygienic. This is problematic as these technologies might be out of reach for most rural households, especially in more isolated areas. (The government is currently reviewing the standard). The Cambodian government considers five options: The Pour Flush Latrine (PFL), The Sealed Pit Latrine (SPL), The Ventilated Improve Pit Latrine (VIPL), The Dry Pit Latrine (DPL) and The Latrines for Disabled People (LDP). However, in practice preference is for PFL and the dry pit latrine has low acceptance with households due to its smell and the lack of ash. In the Indonesia programme, households sharing a sanitary toilet, e.g. 2 neighbouring and often related families which built and/or shared one toilet, became ‘co-owners’ and no longer counted as being without a sanitary toilet.Participants discussion
Most of the discussion concentrated on this question. The consensus is that progress towards ODF should also be measured, in particular if government standards are high. However, there are many different views of what should be included as progress:
· Bimal from Nepal mentioned 4 stages:
i. ODF1: no. of observed public places and trails with OD;
ii. ODF2: no. of households with unsanitary toilets, also called OD in fixed places;
iii. ODF3: over 90% HHs access and use ‘proper’ and hygienic toilets, others construct or share;
iv. ODF4: situation 3 is sustained over time AND other positive features (such as HWWS);
· Quynh from Vietnam suggests considering all toilets that include “collection”, “isolation from the environment” and “safe return to the environment of faeces”.
· Sanna-Leena from Nepal, RVWRMP project, suggests including the reasons why people build or not build toilets as a way to measure in progress.
· Christine gave an example of lists at village level where not only people who had built a toilet, but also those that committed to building one were included.
Then on what do we actually want to achieve or what do we actually consider ODF, there were people like Tiwari from Kenya and Vanny from Cambodia that suggested following the JMP criteria. Others think JMP is not specific enough about the sanitary and hygiene aspects of toilets, and suggest using a list of criteria. Different assessment lists were presented. First of all, to avoid the confusion due to different terminology, let’s agree on the following:
· Good toilets/ bad toilets is commonly used for construction quality (durability, strength) not for hygiene
· Sanitary/ unsanitary toilets refers to the toilet’s environmental quality, i.e. excreta contained away from animals (e.g. rodents, flies), humans and water sources;
· Hygienic/ unhygienic toilets would be sanitary toilets that are also well used and maintained.
John, Kencho and Raj from Bhutan shared a list of 8 factors of what they consider a sanitary and hygienically used toilet. In Cambodia, as shared by Dr. Chea Samnang, there is an assessment check list to declare ODF. In Nepal, as mentioned by Bimal, there is a list of 12 criteria as used in Kalikot district (?), and Christine shared a latrine self-score list used at village level in Indonesia.
Regarding what qualifies as a village and which area should be ODF to consider a community ODF, both John and Christine suggested to consider cat-san for fields, and focus on the inhabited areas. The definition of what are inhabited areas, if not defined by government, should be defined in consensus with the community Padam from Nepal pointed to the need to include institutions (schools!) and offices. A further suggestion was that when more households share a sanitary and hygienic toilet, either each household counts as served or the sharing households are a separate category;
A challenge is what to do about OD in very small and scattered hamlets and individual or a few scattered homesteads. Here risks of infection thru OD are lower, although flies may still transmit faecal germs to other homesteads. The consensus seemed to be that we monitor in what local government classifies as hamlet or village, or rely on either the formal or community-drawn boundaries, and disregard other very remote households in the ODF qualification (but not in the promotion activities of course).
Q3: Who could monitor ODF, types of toilets and technical standards, including the gender aspects?
Again there are two questions here:
1. Who should collect data?
2. Who should manage and analyse data?
One option for data collection is that specific village cadres, e.g. in Dept. of Health or Local Government, or leaders of local institutions such as women union, and headmasters, report the dates of agreed indicators to those responsible in their departments or union at district level. Reporting from village committees/schools to districts could even be by text message. Data management at district level would ideally be done by the district line agency or local authority. This agency would also share the data with other stakeholders (such as SNV) and feed back comparative summaries of the performance to the participating villages, so that they, too, can manage change at local level. The question here is what is sustainable in your country, considering constraints in time, resources and capacities that occur at district level?
Participants discussion
Who manages and uses the data. Most people who touched upon this point emphasised the importance that local community level organisations should be involved, as well as village level for example commune in Cambodia and VDC (village development committee) in Nepal. Sanna-Leena and Christine pointed out that it could be also a neighbourhood group (Ward), reporting to the VDC-wide institutions (such as VDC itself, or something like Health Post, main school, Water Resources Management Committee representing the whole VDC etc) In our working areas, there are many options, but no one-fit-for-all option.
At district level: ‘local government’ was mentioned by all. However we did not enter into the discussion whether this should be managed by line agencies (Health or Works or Rural Development) or by the local authorities, as this is very context specific. Raj emphasized that the discussion on institutional capacity and institutionalisation the data collection, management and analysis (the whole package) should start now. There are of course different monitoring efforts in the countries ongoing, so this is also context specific.
Q4: How could these aspects be monitored, including involvement of women and men, and the poor?
A complexity of social programmes, such as promoting and monitoring S&H is that the objectives include “community wide” achievement, and social inclusion. This implies that there is clarity about: