MONITORING TOOLS

FOR HUMANITARIAN ORGANISATIONS

2008

Graham White

Peter Wiles

The document has been financed by and produced at the request of the European Commission. The comments contained herein reflect the opinions of the consultants only.

1

DG ECHO - Monitoring Tools for NGOs

DISCLAIMER

The design of DG ECHO-funded interventions must be context-specific and any response should be informed by locally collected information on the needs, priorities and capacities of the affected population; wherever possible this information should be disaggregated by gender. DG ECHO is putting increased emphasis on a results based approach with measurable indicators. DG ECHO fully endorses the Sphere Standards in principal and considers them as a universally recognised set of benchmarks. Indicators should be based on Sphere standards, but adapted flexibly and must take full account of the local context, including national standards.

This document has been produced with the financial assistance of the European Commission’s Directorate-General for Humanitarian Aid – DG ECHO. The views expressed herein should not be taken, in any way, to reflect the official opinion of the European Commission.

DG ECHO, The Evaluation Partnership, TRANSTEC and Prolog Consult and the authors accept no liability whatsoever arising from the use of this document.

ACKNOWLEDGEMENTS

We gratefully acknowledge the help of staff and managers from DG ECHO, UN agencies and secretariat, the Red Cross/Crescent Movement and NGOs who were interviewed during the course of the preparation of this report. A full list of organisations consulted can be found in Annex 2.

We also thank those people who attended the workshops in Copenhagen and Nairobi and who shared invaluable comments on the draft documents. Finally our thanks go to António Cavaco, Director-General, at whose initiative this Study was undertaken, and Peter Cavendish, and Nicoletta Pergolizzi of DG ECHO, who gave invaluable support.

DG ECHO provided full funding for this report.

COPYRIGHT

Copyright for this document is held by the European Commission’s Directorate-General for Humanitarian Aid - ECHO. Copying of all or part of this document is permitted, subject to the disclaimer inside the front cover, provided that the source is acknowledged.

FURTHER COPIES

Further copies of this report may be requested by e-mail from or downloaded from ECHO’s website: see the ECHO home page at

NOTE

This is a Word version of the DG ECHO Monitoring Tools for NGOs, offered to users to allow easy adaptation. The primary version is in Adobe Acrobat (.pdf) format, available on CD-ROM and as a download from the internet.

TABLE OF CONTENTS

1.Start here

2.Information about these tools

COMMUNICATION TOOLS

3.How to introduce your agency

4.How to conduct an individual interview

5.How to conduct key informant interviews

6.How to conduct a focus group

7.How to hold a lessons-learned meeting

8.How to communicate with children

9.How to involve people throughout the project

ANALYSIS TOOLS

10.Gender analysis

11.Stakeholder analysis

12.Livelihood analysis

13.Risk analysis

14.Conflict analysis

OTHER TOOLS

15.Observation skills

16.Using indicators

17.Participatory methodologies

18.Cultural considerations in psycho-social interventions

19.Surveys and participatory assessment techniques

20.How to assess child-protection needs

21.How to carry out a field visit

22.How to set up a complaints and response mechanism

TABLE OF ABBREVIATIONS

ALNAP / Active Learning Network for Accountability and Performance in Humanitarian Action
ARC / Action for the Rights of Children
CRC / UN Convention on the Rights of the Child
CRC / The United Nations Convention on the Rights of the Child
DG ECHO / Directorate General for Humanitarian Aid
FAO / Food and Agriculture Organisation (UN)
GBV / Gender-based violence
HIV / Human Immunodeficiency Virus
IASC / Inter-Agency Standing Committee
ICRC / International Committee of the Red Cross
IDP / Internally displaced person
IHL / International humanitarian law
IHRL / International human rights law
INEE / Interagency Network for Education in Emergencies
NGO / Non-Governmental Organisation
NTFP / Non-TimberForest Product
OCHA / Office for the Coordination of Humanitarian Affairs (UN)
OHCHR / Office of the High Commissioner for Human Rights (UN)
PAT / Participatory Assessment Techniques
PLA / Participatory Learning and Action
PME / Participatory Monitoring and Evaluation
PRA / Participatory Rural Appraisal
TEP / The Evaluation Partnership
UN / United Nations
UNHCR / United Nations High Commissioner for Refugees (UN)
UNICEF / United Nations Children’s Fund (UN)
WFP / World Food Programme (UN)
WHO / World Health Organisation (UN)

The Evaluation Partnership 1

DG ECHO - Monitoring Tools for NGOs

1.Start here

1.1Background

This study on monitoring methodology for humanitarian aid has produced three documents: Monitoring Overview and Guidance, the Monitoring Templates (a compilation of monitoring standards and indicators), and this document – the Monitoring Tools.

1.2Aim of these tools

The aim of these tools is to help humanitarian organisations with monitoring the different aspects of their operations.

It does this by offering a compilation of tools that can be used in monitoring. Each toolprovides guidance on how to undertake a different aspect of monitoring work.There are a number of tools ranging from How to conduct an individual interview through to Risk Analysis.

Whilst this document covers an extensive range of tools, it does not cover every possible tool that might be required. Organisations are encouraged to identify other tools and develop them according to their needs.

It is not intended that these tools should represent a requirement for any organisation. Rather they are offered as suggestions which humanitarian organisations can adapt according to their own needs.

The coverage and presentation of the tools was decided after consultation with a wide range of humanitarian organisations.Accordingly, they aim tocombine the following qualities:

  • Based on existing good practice and recognised source documents
  • Simple, clear language
  • Accessible and easy to navigate, with detailed Table of Contents
  • Designed to be adapted by humanitarian organisationsto their particularneeds
  • User-friendly layout
  • Existing documents and resources are referred to where these may be of further help

While the primary intended audience is for those working in the field at the point of delivery, the tools may also be of use as a reference document to:

  • Field-based humanitarian managers
  • Managers at headquarters

1.3What thesetools cover

These tools cover a wide of subjects which can be classified into three broad groups: communication tools, analysis tools, and other tools. The division between the groups is not always clear-cut and some tools could appear in other groups.

Communication tools

  • How to introduce your agency
  • How to conduct an individual interview
  • How to conduct key informant interviews
  • How to conduct a focus group
  • How to hold a lessons-learned meeting
  • How to communicate with children
  • How to involve people throughout the project

Analysis tools

  • Gender analysis
  • Stakeholder analysis
  • Livelihood analysis
  • Risk analysis
  • Conflict analysis

Other tools

  • Observation skills
  • Using indicators
  • Participatory monitoring
  • Cultural considerations in psycho-social interventions
  • Surveys and participatory assessment techniques
  • How to assess child-protection needs
  • How to carry out a field visit
  • How to set up a complaints and response mechanism

There is an extensive body of documentation relating to the monitoring of humanitarian aid. The authors have sought to identify the most reliable sources. The toolsare based on existing good practice. They draw extensively from existing documents and full acknowledgements and references are given with each tool.

1.4How to use these tools

These tools provide practical guidance for undertaking a broad range of monitoring activities. It answers questions which might be asked such as:

  • What is a focus group and how does it operate?
  • How do I communicate with traumatised children?
  • What is stakeholder analysis and how does it operate?
  • What is risk analysis and how can it help our project?
  • What are indicators and how do I choose them?
  • How do you set up a complaints and response mechanism?

There are 20 tools in all, so this is just a sample of the kind of questions which this toolkit aims to answer.

DG ECHO's partners are free to use these tools and to adapt them to their own needs. The only condition is that they cannot be sold or distributed for profit.

2.Information about these tools

2.1Acknowledgements

We gratefully acknowledge the help of staff and managers from DG ECHO, UN agencies and secretariat, the Red Cross/Crescent Movement and NGOs who were interviewed during the course of the preparation of this study.A full list of organisations consulted appears in the Guide which accompanies these tools.

Many existing manuals and handbooks were helpful to the preparation of these tools.We acknowledge all source materials and specific references are provided with each tool.

We also thank those people who attended the workshops in Copenhagen and Nairobi and who shared invaluable comments on the draft tools. Finally our thanks go to António Cavaco, Director-General, at whose initiative this Study was undertaken, and Peter Cavendish, and Nicoletta Pergolizzi of DG ECHO, who gave invaluable support.

2.2Authors and date

The ECHO Monitoring Review, of which this is part, was undertaken by Graham White of The Evaluation Partnership and Peter Wiles of Transtec/Prolog. It was submitted to DG ECHO in June 2008. The compiler of these tools was Graham White. Websites for the companies that undertook the work are:

TEP

Transtec

PrologConsult

2.3Funding

DG ECHO provided full funding for this study.

2.4Copyright

Copyright for this document is held by the European Commission’s Directorate-General for Humanitarian Aid - ECHO.Copying of all or part of this document is permitted, subject to the disclaimer inside the front cover, provided that the source is acknowledged. Copying of specific tools is also subject to the copyright of the original source documents.

2.5Software and languages

These tools are available in English in pdf and Word formats.

2.6Further copies

Further copies of these tools may be requested by e-mail from or downloaded from ECHO’s website: see the ECHO home page at

2.7Sources

The following table provides a list of the tools together with references to the source documentation and the relevant websites.

COMMUNICATION TOOLS
Tool / How to introduce your agency
Agency / Emergency Capacity Building Project (
Document / Good Enough Guide Tool 1
Website /
Tool / How to conduct an individual interview
Agency / Emergency Capacity Building Project (
Document / Good Enough Guide Tool 5
Website /
Tool / How to conduct key informant interviews
Agency / USAID
Document / Conducting Key Informant Interviews
Website /
Tool / How to conduct a focus group
Agency / DFID
Document / Tools for Development (chapter 3.4)
Website /
Tool / How to hold a lessons-learned meeting
Agency / Emergency Capacity Building Project (
Document / Good Enough Guide Tool 11
Website /
Tool / How to communicate with children
Agency / Action for the Rights of Children
Document / Working with Children (Topics 1-3)
Website /
Tool / How to involve people throughout the project
Agency / Emergency Capacity Building Project (
Document / Good Enough Guide Tool 3
Website /
ANALYSIS TOOLS
Tool / Gender Analysis
Agency / Inter-Agency Standing Committee
Document / Women, Girls, Boys and Men: Different Needs – Equal Opportunities
Website /
Tool / Stakeholder Analysis
Agency / DFID
Document / Tools for Development (chapter 2)
Website /
Tool / Livelihood Analysis
Agency / The National Agriculture and Forestry Research Institute
Document / Livelihood Analysis: A Checklist
Website /
Tool / Risk Analysis
Agency / DFID
Document / Tools for Development (chapter 6)
Website /
Tool / Conflict Analysis
Agency / Swedish International Development Cooperation Agency
Document / Manual for Conflict Analysis
Website /
OTHER TOOLS
Tool / Observation skills
Agency / Action for the Rights of Children
Document / Situation Analysis (Topic 5)
Website /
Tool / Using indicators
Agency / DFID
Document / Tools for Development (chapter 5.6)
Website /
Tool / Participatory Monitoring
Agency / DFID
Document / Tools for Development (chapter 7)
Website /
Tool / Cultural considerations in psycho-social interventions
Agency / Action for the Rights of Children
Document / Working with Children (Topics 4-5)
Website /
Tool / Surveys and participatory assessment techniques
Agency / Action for the Rights of Children
Document / Situation Analysis (Topic 8)
Website /
Tool / How to assess child-protection needs
Agency / Emergency Capacity Building Project (
Document / Good Enough Guide Tool 8
Website /
Tool / How to carry out a field visit
Agency / Development Researchers’ Network
Document / The Monitoring and Evaluation Manual of the NGOs of the Forum Solint
Website /
Tool / How to set up a complaints and response mechanism
Agency / Emergency Capacity Building Project (
Document / Good Enough Guide Tool 12
Website /

The Evaluation Partnership 1

DG ECHO - Monitoring Tools for NGOs

COMMUNICATION TOOLS

The Evaluation Partnership1

DG ECHO - Monitoring Tools for NGOs

3.How to introduce your agency

This tool suggests good practice for introducing your agency to beneficiaries, government officials and others. It is based on Tool 1 in “Impact Measurement and Accountability in Emergencies: The Good Enough Guide”, developed by the EmergencyCapacityBuilding Project and available online at .

This checklist can be used to help make sure field staff know the answers to questions they are likely to be asked by beneficiaries, government officials, and others.

WHO ARE WE?

  1. What is an NGO?
  2. What is our mandate?
  3. Why is our agency here?
  4. Where do we get the money?

OUR AIM

  1. What can we do for people affected by the emergency in relation to:
  2. Water and sanitation
  3. Shelter
  4. Livelihoods
  5. Public health promotion
  6. Other kinds of project
  7. Why do we do this rather than other things?

THE PROJECT AND THE COMMUNITY

  1. What is our project area?
  2. Who decided?
  3. Who was involved in deciding project activities?
  4. What is the plan for the whole project?
  5. How long will it last?
  6. Who are the beneficiaries?
  7. Why were some people chosen and not others?
  8. Who was involved in deciding who the beneficiaries should be?
  9. How does the project work? How are beneficiaries involved?
  10. What will beneficiaries contribute?
  11. What will beneficiaries contribute?
  12. What do the materials cost us?
  13. What is the progress this month? What is the plan for next month?
  14. What are the main challenges for technical staff this month?
  15. What are technical staff doing to address these challenges?
  16. What exactly will beneficiaries receive?
  17. When will they receive it?

DEALING WITH PROBLEMS OR COMPLAINTS

  1. If something goes wrong with the project what can people do?
  2. If there is a problem with a community leader or community member working with beneficiaries, what can people do?
  3. If there is a problem with one of the NGO staff members (corruption, fraud, bad behaviour), what can beneficiaries do?

OTHER ORGANISATIONS AND THE GOVERNMENT

  1. Which other NGOs are working in the project location?
  2. What do they do?
  3. What government assistance is available? How do people access it?
  4. What other problems are people having? (For example, being displaced, no access to land, not being able to meet government officials to resolve problems.)

SOURCE

This tool is adapted by the publisher from Impact Measurement and Accountability in Emergencies: The Good Enough Guide, developed by the Emergency Capacity Building Project and published by Oxfam GB 2007 with the permission of Oxfam GB, Oxfam House, John Smith Drive, Cowley, Oxford OX4 2JY UK Oxfam GB does not necessarily endorse any text or activities that accompany the materials, nor has it approved the adapted text.

The Evaluation Partnership1How to introduce your agency

DG ECHO - Monitoring Tools for NGOs

4.How to conduct an individual interview

This tool suggests good practice for conducting interviews with individuals, whether an informal ten-minute conversation or a more structured formal meeting. It is based on Tool 5 in “Impact Measurement and Accountability in Emergencies: The Good Enough Guide”, developed by the EmergencyCapacityBuilding Project and available online at .

BACKGROUND

Individual interviews can be used during assessments orsurveys. An individual interview can mean a ten-minuteconversation during an informal visit or a longer and morestructured discussion, using a series of questions on aparticular topic. Whatever the case, focus on essentialinformation and build your interview around current concerns, for example, profiling and needs assessment,tracking changes, or seeking feedback.

  • Aim to interview people at times that are safe and convenient for both staff and interviewees. The time your interviewee has available should determine how long your interview lasts.
  • Make sure that people understand why you wish to talk to them and what you will do with the information they share.
  • Never use people’s names when using information without their express permission or that of their guardian.
  • Start with questions that are factual and relatively straightforward to answer. Move on to more sensitive issues, if necessary, only when the person you are interviewing is more at ease.
  • Make sure people know that you value their time and participation.
  • Don’t end the interview too abruptly.
  • Take responsibility for the effect on your interviewee if sensitive issues are discussed.
  • Record, store, and use information safely.

SOME ‘DO’S’ FOR INTERVIEWS

  • Do try to make sure you have a good translator.
  • Do locate elders/leaders first, explain who you are and what you are doing, and ask their permission to interview.
  • Do ask individuals’ permission to interview them; for example, ‘Is it OK if I ask you a few questions about the conditions here?’ Thank them afterwards.
  • Do try to prioritise discussions with women and children, and other people likely to be experiencing particular difficulty.
  • Do try and interview at least three families in each location in order to cross-check the information you are receiving.
  • Do make sure that you include people at the edge of a camp or site where you may find the poorest families living, quite literally, on the margins.
  • Do avoid large crowds following you around if possible, since this is likely to intimidate interviewees and interviewers.

SOURCE