World Bank Impact Evaluation of Health Results-Based Financing (RBF) Programs

World Bank Impact Evaluation of Health Results-Based Financing (RBF) Programs

Field Manual

Guidelines forField Enumerators, Supervisors, Editorsand Data Entry Operators

Baseline Health Facility Survey[1]

Note to Principal Investigator:

• Customize instructions based on your questionnaire: modify instructions if needed, delete instructions on questions not used in your country, order the instructions in accordance with your questionnaire.
• Translate this document into the enumerators’ local language(s) if necessary.
• Use these instructions during the training of the enumerators and provide a copy to each enumerator that they can use in the field.

• Notes to principal investigators appear in blue boxes. These notes can be removed when the manual is adapted to the specific survey.
• Country-specific items that will clearly need adaptation, such as country name, appear highlighted in yellow.

Table of Contents

Introduction

1.Background on Results-Based Financing

2.Purpose of the study

3.Purpose and general structure of the survey

4.Purpose of this manual

5.Outline of manual contents

I.Team composition and team members role

1.Field supervisor’s role, overview

2.Field Editor’s role

3.Field enumerator’s role

4.Data Entry Operator’s role

II.General schedule

III.Materials required for fieldwork

IV.Quick Instrument review

V.General Instructions for Enumerators, Supervisors and Editors

1.How to administer surveys

a.Building rapport with the respondent

b.Tips for conducting the interview

c.Asking questions

d.Recording responses

e.Font Typestyle

f.Unique identification (ID) codes

g.Respondent versus subject

h.Reference period

i.Durations

j.Dates

k.The “other” category

l.Absence of information or response

m.Following skip patterns

n.Correcting mistakes

o.Watching for units

p.Non-Response

q.Grades and Levels of education

r.Recording Global Positioning System (GPS) coordinates

s.Checking completed survey forms

2.Responsibilities of the Field Supervisor

a.Preparation for the fieldwork

b.During the fieldwork

c.Maintaining motivation and morale

d.Monitoring enumerator performance

e.Overseeing the quality of the questionnaires

3.Responsibilities of the field editor

4.Field Data Entry Operator

VI.Health Facility Instruments

1.Preliminary instructions to Enumerators on Exit Interviews and Sampling

a.Eligibility

b.Sampling Patients for Exit Interviews

c.Introducing exit interviews to patients and obtaining consent

d.Allocating unique identification numbers (ID) to patients

e.Overview of exit interview questionnaires

2.Form F1: Health Facility Assessment

3.Form F2: Health Worker Interview

4.Form F3: Exit interview, Patient receiving Antenatal Care

5.Form F4: Exit Interview, Caregiver of Child under age 5

6.Form F5: Community Health Worker Interview

Introduction

1.Background on Results-Based Financing

Principal investigator: This section covers general RBF notions that can be adjusted to reflect the specific intervention, hypotheses and research objectives of your impact evaluation.

Results-Based Financing (RBF) for Health is a national-level tool for increasing the quantity and quality of health services used or provided based on cash or in-kind payments to providers, payers, and consumers after predetermined health results (outputs or outcomes) have been achieved. It combines the use of incentives for health-related behaviors with a strong focus on results. RBF is an umbrella term that includes output-based aid, performance-based financing, provider payment incentives, vouchers, contracting linked to particular targets, and conditional cash payments and transfers to households.

In some countries, RBF may take the form of paying a bonus to health facilities that meet certain quantity or quality targets, such as percent of women delivering their babies in a facility. Other countries are designing RBF mechanisms to provide incentives and support to the poor to overcome hidden barriers to using services, such as the cost of transport.

2.Purpose of the study

The objective of this study is to investigate how Results-based Financing (RBF) mechanisms may be used to cost-effectively maximize impacts on maternal and childhood health outcomes.

The main hypotheses of the study are that results-based financing (RBF):

• Increases the quantity of health services delivered: The interventions are designed to increase utilization of health services for maternal and childhood curative and preventative services by providing incentive payments to providers and/or users.
• Improves the quality of the services provided: There is a concern that incentive payments to increase quantity of services provided will lead to a decrease in the quality of services provided, particularly in rural areas with limited human and capital resources. For this reason, incentive payments must be tied not only to the quantity of services, but the quality of services as well.
• Improves the health status of the population: Through an increase in the use of preventive and curative health care, such as immunizations, pre-natal care, institutional delivery, or bed-net use campaigns, we anticipate a direct, positive impact on the child and maternal health status of treatment populations.
• Improves the motivation and behavior of the health providers: Supply-side RBF programs are designed to pay health care providers additional bonus payments based on their performance, which provides incentives to providers to improve their motivation in the workplace and quality of services delivered.
• Has no impact on the non-RBF services delivered: The study must also measure any externalities, positive or negative, associated with the RBF programs. There is concern that providers will shift their provision of care entirely to RBF services in order to increase the RBF bonus payments, at the expense of non-RBF services. For this reason, the study will also capture information on non-RBF services to identify if there is any shift in quality and quantity of non-RBF services as a result of the RBF program.

These hypotheses frame the content of the questionnaires that enumerators will administer.The results of the data analysis on each of these research questions will be used to inform maternal and child health policy recommendations. These analyses and policy recommendations can only be as good and accurate as the data they are based on.

3.Purpose and general structure of the survey

This health facility survey is designed to assess health system performance and health services utilization. Enumerators will collect information on health worker knowledge, training and practices; health worker satisfaction and motivation; patient satisfaction with, perceptions of and utilization of health services; patient financial status; and facility infrastructure, equipment, supplies and drug stocks. This information will be collected through interviews with health workers, patient exit interviews and an assessment of the heath facility. Enumerators will use five instruments to collect this information:

1. Health facility assessment (F1)
2. Health worker interviews (F2)
3. Exit interviews with patients receiving antenatal care (F3)
4. Exit interviews with caretakers of patients under 5 years old (F4)
5. Community health worker interviews (F5)

4.Purpose of this manual

The purpose of the field manual is to provide support to fieldenumerators. The field manual contains detailed information on procedures for carrying out fieldwork. All team members should prepare for fieldwork by familiarizing themselves with this field manual. Once in the field, the field manual serves as a detailed how-to-guide that describes:

• Procedures for sampling providers and patients at health facilities
• Conducting interviews with patients and providers
• Conducting facility assessments.

5.Outline of manual contents

The manual contains seven sections.

• Section Idescribesbackground information, the objectives of the study and survey, the purpose of the field manual, and outlines the manual contents.
• Section II describes the team composition and overall team members’ roles.
• Section III describes the general schedule for the survey.
• Section IV lists the materials required for fieldwork.
• Section V shortly reviews the instruments.
• Section VI provides useful general instructions to field enumerators, field supervisors and field editors, and describes the role of each of them in more detail than in section II. This section also includes information on exit interviews and sampling.
• Section VII goes through each form of the health facility instrument, describes the purpose of each instrument, how to administer it, to whom and in what time. It then goes through each form question by question and provides question-specific explanations of the terms and approach to be used.

I.Team composition and team members role

Principal investigator: You may need to adjust this section to reflect the organization of your survey field teams.

In the field, each team will have one field supervisor, three field enumerators and one field editor.

Team = 1 Field Supervisor + 1 Field Editor + 3 Field Enumerators + 1 Data Entry Operator

Field teams report to and are supervised by a field manager, who is not based in the field but manages and is responsible for several field teams.

Principal investigator: Please note the responsibilities of the field editor and field supervisor may be combined if the size of field teams has to be limited.
Principal investigator: If your survey team works with the Computer Assisted Field Entry (CAFE) method, the field editor will be responsible for data entry in the field. These responsibilities will have to be specified in the manual.

1.Field supervisor’s role, overview

Each field supervisor is the senior member of each field team. He/she is responsible for the well-being and safety of team members, as well as the completion of the assigned workload and the maintenance of data quality for that team. Each field supervisor receives his/her assignments from and reports to the field manager. The specific responsibilities of the field supervisor are to make the necessary preparations for fieldwork, to organize and direct the fieldwork, and to carry out data quality maintenance activities.

The field supervisor is also responsible for completing the Health Facility Assessment F1 form. Details of the responsibilities of the field supervisor are further described under sub-section2 of section VI.

The field supervisor reviews the forms deemed accurate (i.e. stamped and signed) by the field editor. The field supervisor will use red ball penfor making corrections in the forms s/he checks. If the field supervisor deems that a form is complete and accurate, s/he will stamp it with a red stamp and sign it. If the field supervisor does not find a form to be complete or accurate, the field supervisor will return it to the field editor for corrections. The field editor may then send the form back to the respective field enumerator to correct deficiencies with the respective patient or health worker. If it is not possible to correct a form’s deficiencies, the case will be discarded (if the deficiencies are extreme) and a new case will be sampled to serve as a replacement.

Note that in the case of the health facility assessment form F1, filled out by the field supervisor, the field editor still gets to review the form (see below for description of the field editor’s review process).

Satisfactory completion of work at all facilities in the geographic area (e.g. province, district) has to be certified by both the field supervisor and the field editor and submitted to the field manager for review and approval. Only when the field manager approves will the team members receive payment for their work. Without approval at these two levels, the team’s work will be deemed unsatisfactory and the team will not be paid.

2.Field Editor’s role

Each field editor’s primary responsibility is to ensure correct editing of all completed forms at a health facility. The field editor will also be responsible for completing at least one Health Worker interview.

As soon as a field enumerator, or field supervisor in the case of the health facility assessment form F1, has completed a form, the field editor will review it for accuracy and completeness. It is important to review forms as soon as possible, before patients and health workers leave the facility so that missing questions can be completed and questions that have been completed inaccurately can be redone in a correct manner. If the field editor deems that forms are not complete, he or she will send them back to the respective field enumerator to correct the deficiencies with the respective patient or health worker. If it is not possible to correct a form’s deficiencies, the case will be discarded (if the deficiencies are extreme) and a new case will be sampled to serve as a replacement. The field editor will use green ball pen for making corrections in the forms s/he checks. If the field editor deems that a form is complete and accurate, he or she will stamp it with a green stamp and sign it. The field editor will send all approved forms to the field supervisor for approval. Once all forms from a facility have been stamped and signed by the field editor and field supervisor, the field editor will certify the team’s work at the facility as being complete and satisfactory.

Note that in the case of a health worker interview filled out by the field editor, the field editor still conducts a quick review of his/her work, and then hands on his/her form to the field supervisor for approval. The rest of the review process is as described above.

3.Field enumerator’s role

Since they are the ones who collect information from respondents, enumerators occupy the central role in the survey process.

The responsibilities of the health facility enumerators include the following:

1. Completing all required interviews, observations and assessments, except for the health facility assessment form F1, which is completed by the field supervisor.
2. Checking completed survey forms to ensure that all questions were asked and all responses neatly and legibly recorded. This should be done at the end of each interview while the enumerators are still in the presence of the patient or health worker, enabling the enumerators to ask any questions that were inadvertently skipped or for which responses were recorded illegibly.

Field enumerators will fill out forms with blue ball pen.

4.Data Entry Operator’s role

The data entry operator is responsible for implementing Computer Assisted Field Entry (CAFE), and is therefore responsible for entering data from questionnaires as they come out of the field and for performing immediate consistency checks. The data entry operator is also responsible for the maintenance and technical operations of all hardware and software used to enter data.

II.General schedule

Each team will spend a minimum of one day in a facility before continuing survey work at the next site in the sample.

Principal Investigator: If your survey includes direct observation, adjust the instructions of the schedule to include direct observations. Direct observations should be conducted simultaneously with exit interviews (each patient being observed first, then interviewed at the end of the consultation).

• The enumerators will spend the first part of the visit conducting exit interviews; during the second part of the visit, they will interview the health facility’s health workers who come to the facility on the appointed day in order to be interviewed.
• The supervisor will spend the morning completing the facility assessment form F1. He/she will also assist the enumerators in conducting exit interviews.
• The enumerators will help with the facility assessment if they finish exit interviews before the field supervisorfinishes the facility assessment. [Following content specific to Community Health Worker surveys:] In some of the clinics one of the enumerators might have to travel to two selected health posts and interview two of the Community Health Workers.

It is imperative that the team arrive early at the facility—they should arrive well before the regular workday begins in the clinic, and should plan to spend a minimum of 1 day in each facility in order to complete all data collection. The whole team together, led by the field supervisor needs to greet the in-charge and other health workers upon arrival, give them the letter of introduction from theMinistry of Health, and explain to them the purpose of the survey and the steps involved in carrying it out. The team also needs to determine, through the help of facility staff, how many under-five patients and pregnant women are expected that day in order to figure out the appropriate sample scheme. This is explained in greater detail in sub-section1of section VII.

III.Materials required for fieldwork

Before leaving for the field, the field supervisor is responsible for collecting adequate supplies from the field manager of all materials the team will need in the field. These items are listed below:

1. Fieldwork documents
2. 5 copies of the Field Manual (one for the field supervisor, one for the field editor, three each for the enumerators)
3. Maps showing the location of all sampled facilities and all eligible communities within the catchment area
4. Letters of introduction to local authorities
5. Survey instruments in appropriate languages, including extra copies in case of wrong recording and including consent forms
6. Field Supervisor’s Checklist and Clinic Checklist (IE Toolkit)
7. Field supervisor’s Tracking Sheets (IE Toolkit)
8. One “Field supervisor’s Tracking Form - Log of Team Activity” per province
9. One “Log of Facility Assessment Work” per province
10. One “Facility Coversheet” for each facility being assessed
11. Patient Lists for Exit Interviews (IE Toolkit)
12. Cash management sheets (IE Toolkit)
13. Enumerator Evaluation Forms (IE Toolkit)
14. Laminated copies of the clinical case scenarios

Principal Investigator: The Results-Based Financing Impact Evaluation Toolkit provides several instruments that may be of use during field work.

1. Supplies
2. Blue pens for enumerators
3. Red pens for the field supervisor
4. Green pens for the field editor
5. Clipboards, notepads, blank paper
6. Staplers, stapler remover and staple pins
7. Plastic folders and rubber bands for completed questionnaires
8. Carrying bags
9. Backpacks
10. Flashlight and batteries
11. Cell-phone and top-up cards
12. Global Positioning System (GPS) device

If survey uses Computer-Assisted Field Entry (CAFE) method: