Mid-term Review
Window of Opportunity Program
Indonesia
Prepared for CARE USA Child Health and Nutrition Unit
Ruth Harvey, Consultant
May 14, 2011
List of Acronyms
CFA Community focused approaches
CHAN Child Health and Nutrition
DHO District Health Office
FGD Focus Group Discussion
IYCF Infant and young child feeding
IYCF-E Infant and young child feeding in emergencies
KMS Kartu Menuju Sehat, Child Health Card
MCH Maternal child health
M&E Monitoring and evaluation
MOH Ministry of Health
MTMSG Mother to mother support group
MTR Mid-term Review
NGO Non-government organization
TBA Traditional Birth Attendant
TTU Timor Tengah Utara
UNICEF United Nations Children’s Fund
WHO World Health Organization
Definition of Terms
Bidan: A midwife serving in a village who is employed by the government health system.
Kader: Community Health Volunteer recruited for conducting monthly growth monitoring sessions.
Puskesmas: Public Health Center that provides basic health facilities. One Puskesmas usually covers several villages within one district. Staff includes doctors, nurses, midwives, nutritionists and others.
Polindes: A village health center, run by the village midwife. This provides basic medical care, ANC, delivery services, post natal care and also conducts the monthly growth monitoring session.
Posyandu: Monthly growth monitoring sessions with other activities such as mass education, immunizations and other services with activities carried out by the kaders and midwife.
Bubur saring: A special porridge (bubur) prepared for young children made from ground rice or corn mixed cooked with water.
Contents
PROGRAM OVERVIEW 4
MID TERM REVIEW PROCESS 6
MTR Aims and Objectives 6
Methodology and Sampling 6
Training and Data Collection for the LQAS 7
FGD Data Collection and Analysis 7
RESULTS AND DISCUSSION 8
Quantitative Component – LQAS Survey 8
Changes in IFYC practices 8
Health Communication and Coverage of MTSG and Counseling Activities 10
Decision making for mothers 12
Qualitative Component 13
Focus Group Process 13
Findings from Focus Groups 14
Conclusions from the Focus Groups 22
Interviews with volunteers and health workers 24
Conclusions 27
Recommendations 29
Appendices 32
PROGRAM OVERVIEW
The goal of the Window of Opportunity program is to promote, protect, and support optimal infant and young child feeding (IYCF) and related maternal nutrition (rMN) practices. Specifically, the program’s interventions focus on improving the enabling environment, strengthening health systems to support optimal IYCF and rMN, and empowering communities and individuals to make optimal choices regarding breastfeeding, complementary feeding, and nutrition for women during pregnancy and lactation.
The Window of Opportunity (Window) program started in 2008 and is currently being implemented in five countries—Indonesia, Nicaragua, Sierra Leone, Bangladesh, and Peru.
Indonesia was one of the first countries under the Window’s program. From January 2007 to March 2008, CARE implemented the Infant and Young Child Feeding in Emergency (IYCF-E) project in two sub-districts of Timor Tengah Utara (TTU). The project’s aim was to prevent malnutrition in infants and young children under 24 months by increasing knowledge and skills of health workers and communities for optimal feeding practices. In 2008, additional funds were granted to expand the scope of the IYCF-E initiative into a more comprehensive program focused on development and emergency contexts, as well as maternal nutrition. Named Prima Bina, this IYCF project targets pregnant and lactating women in the same villages in TTU and an additional 8 villages in Belu district. The project now covers a total of 23 villages in these two districts of West Timor.
The goal of Prima Bina is to promote, protect and support infant and young child feeding and related maternal nutrition in the province of East Nusa Tenggara by 2010. Specific program objectives include building capacity of CARE staff, partners and volunteers; implementing a behavior change communication strategy; regular monitoring and evaluation of the program process and impact; and, institutionalization of best practices. Box 1 provides a brief timeline of key program activities to date. These include formative research, baseline assessments and planning during year one, and capacity building and training of local resources for group facilitators and counseling in year two.
Prima Bina uses two key strategies to provide support for mothers at the community level: Formation of Mother to Mother Support Groups (MTMSG), and IYCF Counseling – A community focused approach (CFA). By the time of the MTR, facilitators had been trained and activities started in 18 out of 23 villages. MTMSGs have been formed and operating in 14 villages (8 in TTU and 6 in Belu) and CFA counselors have been trained in four villages (in TTU).
Number of villages by Intervention TypeTTU / Belu / Total
MTMSG / 8 / 6 / 14
CFA / 4 / 4
None / 3 / 2 / 5
15 / 8 / 23
For the context of this evaluation, it is important to understand that training of local resources for support and counseling has been initiated but is not yet completed. Facilitators have been trained in 18 out of the 23 villages leaving five villages where interventions have yet to begin. Even in villages where facilitators have been trained, not all posyandu areas within the village are covered.
Timeline of Prima Bina Activities2008 / April - July / IYCF project for Emergency and Development, transition from IYCF-E to Prima Bina
July - September / Project staff recruited under IYCF-E Technical Specialist
June - August / Pilot of formative research
Project focus on learning from community
October / Project Manager hired
November / Formative research started, planning
Provincial workshop with DHO representatives and stakeholders from TTU and Betun
2009 / January / Consultant visit for formative research
February / Formative research, community and social mapping completed
March / 2-day workshop on Facilitation
March-May / Field staff "live and learn" in villages
April-May / Update Census of all children under 24 months
June to September / Prima Bina Baseline Survey
September - November / Community Facilitators do home visits to learn about IYCF
2010 / December - January / Project workshops - 1/2 day Orientation for stakeholders in all project villages
January / Facilitating MTMSGs - Training of Trainers for CFs and DHO partners
MTMSG training starts - first batch
March / Baseline survey report completed
IYCF for CFA- Training of Trainers, plus one village level training
April-July / MTMSG training continues - 5 batches
August-September / World Breastfeeding week activities for month
September / End of contracts, Replacement of 4 of 6 community Facilitators
October / Orientation and training of new project staff
Facilitating MTMSGs - Training of Trainers for new CFs
November / Update Census of all children under 24 months
November-December / Project Mid Term Review
MID TERM REVIEW PROCESS
MTR Aims and Objectives
The Window Program conducted a project Mid-term Reviews (MTR) in three countries which have completed baseline assessments and formative research and are in the process of rolling out IYCF and related maternal nutrition interventions. The aims of the review are to:
· Measure progress towards program IYCF objectives
· Look at coverage of key intervention activities - counseling, Mother to Mother Support Groups and group education,
· And also measure knowledge and awareness of interventions and attitudes towards Window programming.
Methodology and Sampling
The MTR consists of a quantitative LQAS survey and a qualitative component using FGDs in project communities. Draft and final versions of data collection instruments and proposed sampling methodologies were shared and revised in consultation with Window HQ staff. These are included in Annex A and B.
For the LQAS survey, a quantitative instrument was provided by Window’s headquarters staff and adapted based on experience in Nicaragua and consultations with the field team in-country. The English questionnaire was translated to Bahasa Indonesia with a few refinements -mainly clarifying wording for interviewers and adjusting the food list for common local foods. The final English version of the questionnaire used for data collection and the survey frequency results are attached as Annex A.
The LQAS Survey sample was drawn for six Supervision Areas (SA), three in TTU district and three in Belu. Prima Bina has six community facilitators (CFs) - field staff who are each responsible for a project area (SA). A sample of 19 locations was selected for each SA using population proportional sampling. The sampling framework is included in Annex C. At each location, two households were selected, one with a child under six months, and another with a child aged 6 to 11 months. The sample size is 6 x 19 = 114 locations x 2 children, for a total of 228 children under age one.
For the qualitative component, communities were selected using purposeful sampling to represent the different types of activities, the different types of environments and populations served. MTMSG locations were selected to include a strong area (TTU), and another area where groups are active but not the strongest (Belu), and the most active CFA area was chosen.
Village Location / District / Group Activity / Type of LocationOenenu Utara / TTU / MTMSG / Rural
Fatusene / TTU / CFA / Rural
Kabuna / Belu / MTMSG / Semi-urban/refugee
Training and Data Collection for the LQAS
Mid-term review preparation and data collection was a team effort of the Prima Bina Project team in West Timor working with an external consultant Ruth Harvey. The schedule for MTR activities and data collection is included as Annex D.
From November 12 to 17, Ruth provided training on the LQAS methodology and together with Project Manager Dr. Santi Wulandari and Assistant Project Manager Andarias Pagalla, trained the data collectors who were the CARE Community Facilitators. Survey training included a review of the concept of overage and key indicators, an overview of LQAS methodology, practice selecting households and respondents, interview techniques and an intensive review of the survey questionnaire. Training also included an afternoon practice session of interviewing respondents in a village setting.
Data collection was completed between November 18 and December 10th (November 18 to 24 and December 2 to 9 in TTU, and from December 2 to 9thin Belu.) The entire team travelled and worked together in each SA often splitting into two teams to cover two sample locations at once. The PM and APM worked as team leaders and field supervisors, and were responsible for selecting the starting households, supervising interviewers and checking questionnaires. Survey logistics were expertly handled by the PM and APM together with the project administrative and finance officers and provided good support for the survey activities. The final survey sample included all locations selected and two households per location. In one community, 20 instead of 19 locations were selected so the total number of respondents for the LQAS survey is 230.
Completed LQAS questionnaires were reviewed and checked in-country and then sent to CARE HQ for data entry, cleaning and initial analysis. Survey results in the form of an LQAS template spreadsheet and frequency distributions were sent to the consultant for program related analysis.
FGD Data Collection and Analysis
A smaller team consisting of the PM, consultant and two community facilitators with capacity in local language conducted the focus group discussions. FGD guides were based on the guides used in Nicaragua and some previously used by the consultant for similar work. For group facilitation, the initial plan was for the PM to lead the FGDs with the two community facilitators translating in local language. However, this arrangement did not to work well because a higher level of local language skills was required to fully appreciate the nuances of the discussion. The project’s Administrative Officer (HRAP), who was initially included as a translator for the consultant, was found to have the best local language skills, so it worked best for her to function as the FGD facilitator under the guidance of the PM and consultant, and for community facilitators to serve as note takers. The qualitative team met directly after each FGD, compiled notes, discussed findings and the completed a summary matrix.
When possible during survey visits to the community, the consultant with a translator conducted key informant interviews with trained village kaders or midwives. Staff and translator availability and scheduling were all challenges and only four interviews were completed.
Conducting both the LQAS survey and FGDs during the same period and involving some of the same staff in both activities posed a significant challenge. While this allowed the group to travel together, it added to logistical and supervisory burdens and limited the times available to schedule FGDs. Quality control of the LQAS survey did not suffer, however the quality of the FGD activities would have been better if scheduled for a different time. Language difficulties also hampered the ability to provide immediate input and guidance for FGDs which meant some opportunities to explore and probe further were lost.
RESULTS AND DISCUSSION
Quantitative Component – LQAS Survey
The purpose of the LQAS survey was to measure IYCF practice indicators to see how the project has progressed since baseline and to measure coverage and exposure to information, messages and support. The survey instrument was adapted from the MTR survey format developed at CARE Atlanta to measure infant feeding indicators consistent with WHO and KPC standards in a manner consistent with measurements of the project baseline survey.[1] Questions were added to measure coverage of program intervention activities and women’s decision making.
Changes in IFYC practices
While statistical comparison of the baseline and MTR surveys is not really appropriate[2], survey results do suggest a trend towards major improvements in key project indicators.
Proxy of Tendency for the Key IYCF Indicators (percentage)
Baseline[3] / Midterm Review / (95% CI)Timely Initiation of Breastfeeding / 55 / 72 / (66 – 78)
Exclusive Breastfeeding / 63 / 83 / (76 – 89)
Minimum Dietary Diversity / 11 / 22 / (14 – 29)
Minimum Meal Frequency / 70 / 87 / (81 – 93)
Minimum Adequate Diet / 7 / 20 / (13 – 27)
Early Breastfeeding – A large improvement is indicated for Timely Initiation of Breastfeeding. In addition, the large gap in coverage rates of the two districts found in the baseline is not seen in the MTR results. Timely initiation of breastfeeding in TTU is 74% and in Belu is 70% suggesting that improvements in Belu have contributed to these results.