1

In Cambodia, out-of-pocket health expenditures are disproportionately high compared to other similarly poor nations. Because there is no national social protection system and the private sector is poorly regulated, households risk impoverishment every time one of their members is sick. They may become indebted or sell land.

SKY micro-health insurance is one of several initiatives to address this issue, aiming to improve health and decrease the risk of poverty due to health shocks. After several years of operation, SKY is large enough for a rigorous evaluation. The impact evaluation will provide policy makers and researchers with some strong evidence about SKY’s effects on the rural poor.

  1. What is SKY?

SKY (an acronym for “Health for our Families”) is a micro health-insurance program run by the GRET (Groupe de Recherche et d’Echanges Technologiques, a French NGO). SKY offershouseholds a financial safety net against diseases and health shocks. Health services and treatmentsare free at public facilities in exchange for a monthly premium.

The program has been operating since 1998 and is still expanding. It currently (as of 2009) covers 40 000 persons inKandal, Takeo, Kampot and Kampong Thom provinces and in some parts of Phnom Penh. In these areas, it is typically the only insurance program offered to the households.

  1. The evaluation
  1. Main goals

AFD (the French Development Agency) initiated the SKY impact evaluation in 2007 and partnered with two research centers: Domrei, a Cambodian research institute specialized in health issues, and CEGA, the University of California,Berkeley’sCenter of Evaluation for Global Action.The evaluation aims to assess the efficiency of the program and raise stakeholders’ interest in the use of such an economic tool. It thus has some scientific and policy-oriented ambitions.

  1. What we want to measure

Measuring the impact of the program on household assets and living standards is the main concern: did SKY reach its goal in protecting families from impoverishment due to unexpected health expenses? Are households able to keep their land and savings?

TheImpact on health is also necessarily a core question as it is the primary goal of health insurance to improve health. Theevaluation should answer the following questions: Are SKY members in better health because they can now afford treatment every time they need it? Do they get better treatment? Do they recover faster?

The impact on public health facilitiesuse and quality is another concern. It is important to know ifSKY increases use of the public health system. This question is linked to the possible impact of the program on public sector quality. A higher utilization rate, coupled with capitation payments from SKY, may increase investments and quality.

From the investors and the policy-makers standpoint, the selection process issue iscrucial: being able to understand what the characteristics of a SKY member are and what is his risk behavior, are definitely some of the keys toward sustainability. The impact evaluation will provide valuable analysis about this issue also.

  1. The method

The evaluation examines SKY’s expansion in three areas in Kandal, Takeo and Kampot provinces. The basis of the study is a lottery for discount coupons of different values that takes place during the initial marketing meeting SKY organizes in each village. These coupons randomize the cost of the insurance. This randomization process permits a sample with a treatment group (high-coupon recipients, almost half of whom join SKY) and a very similar control group (low-coupon winners, fewer than 10 percent of whom join SKY). The impact of the program will be measuredthrough the estimation of the difference between the two groups’outcomes.

1

Several quantitative and qualitative tools will contribute to the evaluation.

- The main quantitative tool is the household (HH) survey. The baseline was conducted in August and December 2008. The first follow-up is being collected now (2009) andthe final round will be completed in 2010. This survey measuresassets and debt, self-perceived and objective health measures, health care utilization, health expenditures, risk behavior, trust in health institutions (staff, services and insurance), capability to pay for large health shocks and willingness to trade current for future income.

- The village chief surveysprovide information about health shocks and agricultural revenues at the village level.

- The village monographs are based on two waves of qualitative data collection in several villages with a long history of SKY insurance. Methods include interviews of members and SKY staff, maps of villages showing patterns of membership, and pictures. Interviews cover the perception of the program by SKY members, former members, those who never joined SKY, and those who work for SKY.

-The quality and utilization of the health centers will be assessed through two waves of clinic survey. This questionnaire measures the quantity and quality of the equipment (such as running water and sterilization tools) available in the facility, as well as the stock of medicine, the presence of staff, and the cleanliness of the facilities.

- At last, the logbooks are an innovative way to collect health data on a daily basis. They collect information on health, and health care, and costs (see Box 1).

  1. A deep sense of reality

The evaluation is deeply rooted in the Cambodian reality.

First, every questionnaire is pretested several times on small samples to improve clarity and relevance to the target audience before the general survey.

Second, the multiple sources of data make the collection very frequent: the database is updated on a regular basis, and evolves in line with the situation in the field. The longitudinal side of the study adds a valuable dimension to the understanding of the population.

Furthermore, the Khmer-speaking evaluation team also goes on field visits often, to carry out specific phases of the evaluation and to check out the situation in some villages.

last, the cooperation with the GRET and its SKY evaluation team is particularly close, which is another factor improving the usefulness and validity of the evaluation.

  1. Achievements

The first analyses and results are now available for some of these many data sources, with additional reports expected in 2009.[1]

  1. A first report on the clinic survey

The report on the Clinic Survey’s first round describes health center quality. Some aspects such as opening hours and drug stocks show some improvements relative to other reports, while hygiene and cleanliness show substantial room to improve. Complementary qualitative data from other sources and a second round of the clinic survey in 2010 will shed light on how SKY has influenced health centerquality.

  1. Quantitative data

More than 5 300 households have been interviewed during the first round of the household survey. The database has been cleaned and is now available for study.

More than 20 000 health events have been recorded in the logbooks. The data cleaning is in process. A research note on this instrument will be released in late2009.

  1. Qualitative data

Data collection for the baselineVillage Monograph occurred in January 2009. The interviews help us understand health care utilization, health seeking behavior and the motivations of insurance take-up/drop-out. A firstreport on Village Monographs will be available by the end of 2009.

  1. What’s next?

A first report on the baseline household survey will be released in early 2010. It will deliver the first measures of the impact of SKY on health, economic outcomes and health care facility utilization. An analysis of the selection process and the risk behavior of members will also be released in a few months.

1

Finally, the first half of the first follow-up of the household survey was completed in August 2009. The second half will be completed in December 2009, along with a second round of village chiefs’ interviews.

  1. Broader scope: lessons to be learned in a global health management framework

Health changes are generally difficult to capture and there are only a few studies on the impact of health-insurance programs. This evaluation thus aims at highlighting the potential of a scientific investigation and the interest it can have for policy-makers. Its goal is to produce rigorous results that clearly measure the causal relationship between health insurance and outcomes.

This relationship will constitute a very strong argument to advocate, according to the results, the launching of new phases in the insurance program. Stakeholders and policy-makers will be able to make choices, holding reliable scientific results in their hands. While no single study can provide unambiguous policy advice, the findings may be used as a tool giving the persons who need more knowledge about the capabilities of health insurance to help the rural poor.

It is crucial to have additional rigorous evaluations of both other health insurance plans and of related initiatives. For example, it is important to compare the effects of voluntary insurance such as SKY with the costs and benefits of Health Equity Funds(where donors purchase free insurance for the poor), If the different types of insurance and programs are evaluated rigorously, Cambodian policy-makers will have the information they need to create a coherent and efficient health care system.

1

1

[1] Evaluation reports available at: