Steps in Conducting Monitoring Activities
Introduction
In unit 2, you will recall that monitoring was defined as regular data collection, analysis and use of information to assist timely decision making, ensure accountability and provide the basis for evaluation. It was also mentioned that monitoring activities should commence before the start of programme implementation and should hence be incorporated into the project design stage. This unit will take you step by step through the process of conducting monitoring activities in a programme.
This is a very task oriented unit focusing on all the tasks involved in “doing monitoring”. It will be very intensive, but will reward you for practically engaging with it.
There are five Study Sessions in this unit:
Study Session 1: Preparations for monitoring activities
Study Session 2: Develop monitoring and evaluation objectives
Study Session 3: Develop / select indicators
Study Session 4: Methods of data collection
Study Session 5: Feedback strategies
Learning outcomes of Unit 4
By the end of this unit, you should be able to:· Outline and explain the steps involved in monitoring activities
· Develop monitoring and evaluation objectives
· Develop indicators for a Primary Health Care programme
· Select data collection methods
· Perform simple analysis and interpretation of monitoring data
· Outline feedback strategies for monitoring data
· Write a monitoring report
Here is an overview of the steps involved in monitoring:
STEP 1: Review existing information related to the project
STEP 2: Review (develop) Goals and SMART objectives of the programme
STEP 3: Determine activities needed to attain the objectives
STEP 4: Decide on the questions that need to be answered by the programme
STEP 5: Developing monitoring and evaluation objectives
STEP 6: Selecting indicators
STEP 7: Decide on the methods of data collection including data collection tools
STEP 8: Develop a Monitoring tool
STEP 9: Data collection
STEP 10: Analysis and interpretation of monitoring data
STEP 11: Decide on feedback strategies
STEP 12: Write a monitoring report
Unit 4 - Session 1
Preparations for conducting monitoring activities
Introduction
This session addresses the first steps in beginning monitoring. We will take you through the first steps outlined in the unit outline on the previous page.
The initial steps for monitoring including reviewing programme data including the problem that the programme was / or is to be developed to address. It also includes reviewing the programme goals and objectives ensuring that, should they already exist, they meet the criteria developed in Unit 3.
From goals and objectives flows information about the actual activities required to successfully implement the programme and meet the objectives, as well as questions that the programme must answer – these answers will be collected in the course of the monitoring. As a final activity of this session, we will look at these questions in preparation for the task of developing monitoring and evaluation objectives, which will be the subject of the next session.
Contents
1 Learning outcomes of this session
2 Readings
3 Overview of initial steps
4 Session summary
5 References
1 LEARNING OUTCOMES OF THIS SESSION
By the end of this session, you should be able to:· Review existing information about a programme to be monitored.
· Review the programme goals and SMART objectives.
· Make decisions on actions to take regarding the programme implementation
· Determine questions to be answered in the monitoring.
2 READINGS
There is one reading in this session.
Author/s / Publication detailsFeuerstein, M-T. / (1986). Ch 6 - Using Your Evaluation Results. In Partners in Evaluation: Evaluating Development and Community Programmes with Participants. London: Macmillan: 160-165.
3 OVERVIEW OF INITIAL STEPS
As we have mentioned, this unit takes you sequentially through the process of conducting monitoring activities; we will begin with a short section of the first steps.
The first step, “Reviewing existing information related to the project” allows you to get an overview of the project itself. Some of the information to be gained from this step may already exist, while some might not; in this initial step, you gain an overview of what information you have that will help you get to grips with the nature of the programme.
STEP 1: Review existing information related to the project
Reviewing the existing information about the programme will help you identify the goals and the objectives of the programme. You will also determine what the problem was that led to the introduction of the programme, and also whether the programme has been previously monitored or not.
FEEDBACK
Now refer back to unit 1: You will note that in that unit the problem that led to the development of the intervention was identified: improving the management of severe malnutrition. Further information (encapsulated in the first stage of the development of the conceptual framework – problem identification and cause analysis) includes the baseline data: including baseline case fatality rates of 50% and 28% in Mary Teresa and Sipetu hospitals respectively. Inadequate feeding, poor management of rehydration and infection, lack of resources, and a lack of knowledge and motivation among staff were identified as areas that needed attention.
In unit 3, we began developing the programme, including the goals and objectives of the programme, as well as the conceptual framework.
In summary, in your review of existing information, you will include (and note) – the problem, existing information about the problem, including an analysis of causes or contributing factors, baseline data to round up understanding of the problem.
Let us briefly look at what baseline data might consist of.
Baseline data is the information you have about the situation before you do anything. It is the information on which your problem analysis is based. It is very difficult to measure the impact of your initiative if you do not know what the situation was when you began it. (See also the toolkit on overview of planning, the section on doing the ground work.) You need baseline data that is relevant to the indicators you have decided will help you measure the impact of your work.
There are different levels of baseline data:
- General information about the situation, often available in official statistics e.g. infant mortality rates, school enrolment by gender, unemployment rates, literacy rates and so on. If you are working in a particular geographical area, then you need information for that area. If it is not available in official statistics, you may need to do some information gathering yourselves. This might involve house-to-house
surveying, either comprehensively or using sampling, or visiting schools, hospitals etc. Focus on your indicators of impact when you collect this information.
- If you have decided to measure impact through a sample of people or families with whom you are working, you will need specific information about those people or families. So, for example, for families (or business enterprises or schools or whatever units you are working with) you may want specific information about income, history, number of people employed, number of children per classroom and so on. You will probably get this information from a combination of interviewing and
filling in of basic questionnaires. Again, remember to focus on the indicators which you have decided are important for your work.
- If you are working with individuals, then you need “intake” information – documented information about their situation at the time you began working with them. For example, you might want to know, in addition to age, gender, name and so on, current income, employment status, current levels of education, amount of money spent on leisure activities, amount of time spent on leisure activities, ambitions and
so on, for each individual participant. Again, you will probably get the information from a combination of interviewing and filling in of basic questionnaires, and you should focus on the indicators which you think are important.
- Shapiro (2007)
Shapiro (2007) also suggests a way to deal with not having baseline data; she refers to this as “damage control”.
It is very difficult to go back and get this kind of baseline information after you have begun work and the situation has changed. But what if you didn’t collect this information at the beginning of the process? There are ways of doing damage control. You can get anecdotal information (see Glossary of Terms) from those who were involved at the beginning and you can ask participants if they remember what the situation was when the project began. You may not even have decided what your important indicators are when you began your work. You will have to work it out “backwards”, and then try to get information about the situation related to those indicators when you started out. You can speak to people, look at records and other written sources such as minutes, reports and so on.
One useful way of making meaningful comparisons where you do not have baseline information is through using control groups. Control groups are groups of people, businesses, families or whatever unit you are focusing on, that have not had input from your project or organisation but are, in most other ways, very similar to those you are working with.
For example: You have been working with groups of school children around the country in order to build their self-esteem and knowledge as a way of combating the spread of HIV/AIDS and preventing teenage pregnancies. After a few years, you want to measure what impact you have had on these children. You are going to run a series of focus groups with the children at the schools where you have worked. But you did not do any baseline study with them. How will you know what difference you have made?
You could set up a control groups at schools in the same areas, with the same kinds of profiles, where you have not worked. By asking both the children at those schools you have worked at, and the children at the schools where you have not worked, the same sorts of questions about self-esteem, sexual behaviour and so on, you should be able to tell whether or not your work has made any difference. When you set up control groups, you should try to ensure that:
- The profiles of the control groups are very similar to those of the groups you have worked with. For example, it might be schools that serve the same economic group, in the same geographical area, with the same gender ratio, age groups, ethnic or racial mix.
- There are no other very clear variables that could affect the findings or comparisons.
For example, if another project, doing similar work, has been involved with the school, this school would not be a good place to establish a control group. You want a situation as close to what the situation was with the beneficiaries of your project when you started
Clearly, this is all very difficult and less reliable; it is therefore strongly recommended that the baseline data collection be an integral part of the programme plan. Not only will it help in clearly defining the problem, but will also help in interpreting monitoring and evaluation data by providing a basis to make comparisons about the state of the problem from before the programme began to after the intervention was put into place.
Please read the following for more on strengthening monitoring by, among other things, ways to collect baseline data.
READING
Feuerstein, M-T. (1986). Ch 6 - Using Your Evaluation Results. In Partners in Evaluation: Evaluating Development and Community Programmes with Participants. London: Macmillan: 160-165.
If baseline data has been collected and interventions introduced but no monitoring activities have yet been carried out, you can proceed to the next step: Review the goals and objectives that the programme has been developed to meet and which, you will recall, emanate directly from the identified problem.
STEP 2: Review (develop) Goals and SMART objectives of the programme
We dealt extensively with this step in unit 3. In summary, you would need to review – or develop – the goals that the programme aims to achieve, and ensure that the objectives are clear and SMART. Review Unit 3 Session 1 again for a reminder of how to ensure clear goals and SMART objectives.
FEEDBACK
Once the goals and objectives – the end point you intend to reach – are clear, the next step is to derive programme activities. This is what the programme staff will actually do to ensure meeting the goals and captures day to day activities.
STEP 3: Determine the activities needed to attain the objectives
Once the programme objectives are clearly identified, the next step includes determining what activities are needed in order to achieve the programme objectives. In Unit 2 Session 1, we stressed that, “Monitoring is a process of continuous and periodic surveillance of the physical implementation of a programme”; therefore, the activities will determine what needs to be monitored. If we make the assumption – and we must make the assumption, if the programme has been logically developed (Unit 3, Session 2) – that the activities are what will cause the objectives to be met, then we need to keep track of all these activities, checking whether they are being carried out, since it logically follows that if they are carried out, then the objectives will be met, and the project be successful in its intended goals.