Monitoring Manual and Menu (MM&M) - the Manual

Monitoring Manual and Menu (MM&M) - The Manual

Monitoring Manual & Menu (MM&M)

forCBR and other community-based

disabilityinclusivedevelopment programs

Manual

Version 1.0

April 2014

Centre for Disability Research and Policy

University of Sydney

The Menu is available at

sydney.edu.au/health-sciences/cdrp/cbr-monitoring/

Cover Artwork:

The image is a collective work from artists supported by

Sunshine’s Community Access Program Art Studio

ACKNOWLEDGEMENTS

The University of Sydney team was: Ros Madden, Sue Lukersmith, Sally Hartley, Michael Millington, Charlotte Scarf, Nicola Fortune, Alexandra Gargett and Gwynnyth Llewellyn, of the Centre for Disability Research and Policy.

Key research partners and collaborators were:

SainimiliTawake / Secretariat of Pacific Community Regional Rights Resource Team (SPCRRRT) / Fiji
MariaiaMatakibau / Ministry of Health / Fiji
BounlanhPhayboun / COPE / LaoPDR
SengthongSoukhathammavong / COPE / LaoPDR
KhamkoChomlath / NRC/MoH / LaoPDR
BounphengPhetsouvanh / NRC / LaoPDR
Ben Theodore / Department for Community Development / PNG
Peter Sindu / Callan Services / PNG
Susan Goikavi / Callan Services / PNG
Goretti Pala / Solomon Islands College of Higher Education / Solomon Islands
Casper Fa'asala / Gender Alert and DPASI / Solomon Islands
Elsie NingaloTaloafiri / Ministry of Health and Medical Services / Solomon Islands
PenafranciaChing / College of Allied Medical Professionals (CAMP), University of the Philippines Manila / The Philippines
JoaquimSoares / KliburDomin Foundation / Timor Leste
Francisco de Deus Soares / ASSERT / Timor Leste
Phuong Anh Tran Thi / OGCDC / Vietnam
Phuong Tran Thi / OGCDC / Vietnam
Nhan Ha Chan / College of Medicine and Pharmacy / Vietnam
Cam Tu Huynh Thi / OGCDC / Vietnam

The MM&M was resourced principally by the University of Sydney.

The University of Sydney team working on the development of the International Classification of Health Interventions sponsored Nicola Fortune to work with the Sydney MM&M team during the closing stages of Menu refinement and prototype MM&M finalisation.

Key resources were provided by:

• Australian Aidwhose ALA Fellowship funding enabled CBR program stakeholders to inform and shape the design of the Menu as research partners
• CBM International which provided funding for work on the Manual from June to December 2013.

Thanks are also due to the Advisory Group (from June 2013): Sally Hartley, SainimiliTawake, JoaquimSoares, PenafranciaChing, Jörg Weber (CBM International), PimKuipers (Griffith University), Richard Madden, and Gwynnyth Llewellyn.

A number of reviewers also made valued contributions to the MM&M, including PriscilleGeiser (Handicap International), Theresa Lorenzo (University of Cape Town), Hasheem Manna (CBM-Nossal Institute) and Catherine Sykes (WCPT).

TABLE OF CONTENTS

INTRODUCTION

PRINCIPLES OF THE MM&M

DECIDING TO USE THE MM&M

Why is monitoring important?

Who can use the MM&M?

How can the MM&M help my program?

USING THE MM&M

Before you begin

Step 1: Prepare for collaboration

Step 2: Design information

Step 3: Develop the monitoring plan

Step 4: Undertake monitoring activities

Step 5: Analyse and use information

Step 6: Review monitoring activities & the monitoring plan

APPENDICES

Appendix 1: What is the difference between monitoring and evaluation?

Appendix 2: Outline of method for MM&M development

Appendix 3: Example of template for planning monitoring activities

Appendix 4: MM&M examples using the CBR Matrix

INTRODUCTION

The Monitoring Manual and Menu (MM&M) provides information on how to develop or improve monitoring, so that programs can collect the information they need. It has been designed for use by community based rehabilitation (CBR) programs and other community based disability inclusive programs. The development process involved collaboration with people involved with CBR, and research on CBR monitoring and evaluation (see Appendix 2). It is ready and freely available for testing or use, and can be downloaded from: sydney.edu.au/health-sciences/cdrp/cbr-monitoring/.

The MM&M is designed for use by program stakeholders
‘Stakeholders’ includes anyone interested in the success of the program. The most important stakeholders are the people with disabilities (participants) and their family – the people whom the program aims to serve. Other stakeholders are Disabled Persons’ Organisations (DPOs), other community groups, advocacy groups (e.g. those that support people with disabilities), community leaders, the organisation responsible for the program and its workforce (volunteers and staff), funders, and other local and regional organisations that work with, or could work with the program (for example women’s and youth groups).

The MM&M includes:

1. This Manual, which provides guidance on:
2. designing and planning monitoring activities
3. using the Menu
4. other resources and references
5. The Menu, which provides:
6. a list of information items
7. guidance on how information may be recorded.

The Manual and Menu go together. Read the Manual first and then use the Menu.

The Menu contains information items organised into four broad groups:

1.Person—personal profile and history, functioning and disability, environmental factors, and outcomes

2.Organisation—purpose, structure and strategy, resources, environment, and outcomes

3.Activities—what is done, and outcomes

4.Workforce—training undertaken, knowledge and skills, responsibilities and tasks, and quality of performance

No program is likely to collect information on all the items in the Menu. Users choose items that are relevant to their program goals and information needs.

The MM&M does not provide guidance on how to set up a CBR program or disability inclusive development program. You can find information on this through other resources such as the CBR Guidelines: 2010) and other disability inclusive guidelines and manuals (refer to resources referenceson p. 26).

PRINCIPLES OF THE MM&M

The following 7 principles are the basis of the MM&M.

Principle 1: Monitoring is locally owned and controlled

Community-based programs decide what information they will collect based on program needs and circumstances. The MM&M empowers people to undertake and manage monitoring activities. This means that program stakeholders need to work out their own monitoring plan, including:

• Why they need information;
• What information they need;
• How the information will be recorded;
• When and where the information will be collected, who will collect it, and how it will be stored;
• How the information will be analysed;
• Who will use the information and how.

The program stakeholders can be in charge of putting the monitoring activities into action and reviewing how well the monitoring activities are going.

Principle 2: Monitoring is participatory and inclusive

The MM&M provides a method for ensuring broad representation of stakeholders, including people with disability, in designing the monitoring plan, selecting information items, deciding how to record information, and implementing monitoring activities. Stakeholders can collect information, and use and report information. Their involvement helps to ensure that monitoring does what it is intended to do and provides the information stakeholders need. When stakeholders are involved, monitoring activities are more likely to be done well because everyone is interested in getting the information.

Include all stakeholders in monitoring in some way. Special attention should be given to the key stakeholders: people with disabilities and their families, and their representative organisations. These groups must have a strong voice in deciding what information is collected, how it will be collected, and for what purpose. Involving key stakeholders will strengthen connections in the broader community. Efforts should be made to include a range of potentially marginalised people to ensure diversity is recognised (e.g. people with intellectual or psychiatric disability, women, older people, youth representatives or people who belong to a cultural minority group); suitable environments must be created for people experiencing difficulties with communication, seeing, hearing, mobility, learning, playing or other activities.

Principle 3: Monitoring information is used for the benefit of stakeholders

It is important to collect only the information that will be used for the benefit of program stakeholders and the organisation. If no one is interested in a particular item of information, don’t collect it.

Information collected should be used regularly. It should be used only for the benefit of people with disabilities and key stakeholders. Monitoring produces information for making decisions and planning, understanding, measuring progress and outcomes, and ‘bigger picture’ analysis at local, national and international levels. When stakeholders can see how the information is being used, there is incentive for recording quality information.

All stakeholders should have access to monitoring information.

Principle 4: Information that does not change should be collected only once

Many programs already have information systems and processes in place. Monitoring should complement and build on these systems and processes, or replace existing systems, but not repeat or duplicate. Also, once information that does not change (e.g., date of birth) is collected, it should be recorded and stored so that a person is not asked the same question again later.

Principle 5: Monitoring is easy and part of everyday routine

Monitoring activities should not require lots of extra work or time. As much as possible, collecting and recording monitoring information should be a part of the usual routine of the people collecting it. Extra time might be needed sometimes, e.g., when all the items of information recorded are brought together for a review or a report, or when a special opportunity to advocate for change requires more information.

Collecting monitoring information should be simple and not hard to learn or sustain. It may be possible to involve other people in collecting the monitoring information, not just the program workforce (e.g., a teacher or parent of a child participant).

Making monitoring efficient
Ticking boxes and recording comments once a week is much easier to do and sustain over several months, than recording information which takes 15 minutes each day.

Principle 6: Use networks to sustain and build capacity for monitoring activities

Sustaining monitoring activities over time is important. One of the ways to sustain and build capacity for monitoring is to use networks, and strengthen and expand partnerships. Networks and partnerships can be in the local, national or international community. Aim for quality not quantity of networks.

The qualities that stakeholders should strive for in their engagement with networks and partners are described in the seven ‘C’s below (and illustrated in figure 1):

Coordination (bring together and managing between groups and activities)

Commitment (accept responsibility for action involving people with disabilities and monitoring)

Communication (share information and ideas, understanding each other, together use the information obtained from monitoring)

Creativity (use different ideas and existing resources to help to solve problems)

Collaboration (work with each other, build bridges between groups)

Cooperation (combine efforts for greater benefit)

Control (influence and understanding at the local level).

Figure 1: The Seven ‘C’s: Qualities of networks which produce good information

Funding organisations of community-based programs would also benefit from using networks. Such networks, adhering to these same seven ‘C’ qualities, can help to reduce the complexity of demands for information from community-based programs by streamlining information requirements to make them more consistent across funders.

Principle 7: Adopt ethical practices in conducting monitoring activities

Any program that collects and records information about people must do so respectfully and according to ethical principles.

The ethical principles of the MM&M are:

• Stakeholders, in particular people with disabilities and their families, are involved and have a strong voice in relation to the collection, use and dissemination of information.
• Only personal information that advances outcomes for people is collected.
• People providing personal information are told why it is needed. They should be given the choice to provide information or not. They have access to it, and can check and use the information. Supported decision making may be needed for some people, to help them decide if they want to participate in monitoring, or allow their data to be used.
• People providing personal information ownthat information. The information is only used with their permission, and they are informed about how it will be used. Personal information will be protected within the bounds of local law.
• In all collection processes, privacy is protected; where it is published or made public, information will be presented in a way that protects the anonymity of individuals.
• Data can be shared with other organisations only with the permission of the person. Sometimes this may seem to conflict with the principle of collecting information only once (Principle 4), but the decision is up to the person.
• Information should not be used in a way that discriminates against or unfairly disadvantages anybody.
• Think about how information is stored and who has access to it. Only those who need it and will be careful with it should have access.

Protecting the privacy of program participants
When talking with the program participant and their family to collect personal information, it is important for the person collecting the information to consider the location and who else is close by. If other people can see and hear the conversation and it is not private, then ask the participant and family whether they would like to move to a private place or would like you to ask others to move away so they cannot hear your conversation.

DECIDING TO USE THE MM&M

Why is monitoring important?

Monitoring supports the development and sustainability of community-based programs. Community-based programs need robust tools and resources like the MM&M to undertake and sustain monitoring activities (WHO et al 2010).

Monitoring provides information that can be used to:

reflect on how the program is going and make adjustments as needed

document and report on progress and outcomes for participants and the organisation

track service quality

learn from what has worked well and what has not, and identify aspects of the program that need to be modified or improved

monitor inclusion in mainstream services

help identify unmet need for participants, families and communities

support fundraising strategies

inform evaluation and monitoring of the CRPD

share learning on activities that have led to positive change

inform policy development at local, national, or even international levels

make comparisons across programs within a country, or between countries

identify and describe good practice.

Example: Using information for decision making (Solomon Islands)
The CBR principles of equality of opportunity, accessibility and participation and inclusion in society were influences on the Solomon Islands Department of Education which led them to adopt an inclusive education policy. The Department set up an inclusive education working committee. The national CBR program is involved. The CBR program provided information to the working committee on the number of children with disability in the villages where they had field workers. The government was given information on where the children lived, their age and disability and whether the child attended school. This information helped the Department of Education to justify setting up a National Learning Support Resource Centre. The Centre provides talking books, toys, equipment and other resources to support the teachers in mainstream schools to include children with disabilities. Information about the Centre and the resources was provided to teachers; schools where there were children that were not attending school were particularly targeted.

Who can use the MM&M?

A wide range of programs with diverse stakeholder interests, priorities, and information needs can use the MM&M. It is relevant to program managers, funders and service users, including individual participants, their families and communities, as well as government bodies and related services.

Example: CBR stakeholders (Timor Leste)
The core business of the CBR program in Timor-Leste, implemented by the NGO KliburDomin, is health, education, livelihood, social, and empowerment. The program supports people with disabilities, particularly children in their communities. A mobile rehabilitation team identifies children with disabilities in the villages. Children receive medical treatment, physiotherapy and occupational therapy from the mobile team whilst living at home, or stay at the St Damien’s respite Centre at KliburDomin for a few weeks for more intensive treatment. The CBR program also supports the children with wheelchairs, ramps and other equipment for home and so many now attend school for the first time.
The key stakeholders for CBR in Timor-Leste are:

• The DPO and their members and families
• Funders, including the Australian Government (DFAT formerly AusAID), Handicap International and Enablement from the Netherlands
• Community organisations and networks
• Other NGOs and CBR programs that also may provide special support for the children e.g.East Timor Blind Union (ETBU).
• The local school teachers and principals
• Government including the Ministry of Social Solidarity, Ministry of Education, Ministry of Health, Secretary of State for Training and Employment, National University of Timor-Leste
• Community volunteers
• Local leaders from national, district and village.

How can the MM&M help my program?

The MM&M is designed specifically to assist programs to plan, design and implement monitoring activities, with an emphasis on local control and stakeholder involvement. It is structured so that monitoring can start small and simply. The program can start with a few information items and then gradually build up over time and expand to include more information as capacity allows.