Spanish Impact Evaluation Trust Fund (Sief)

7

THE WORLD BANK

SPANISH IMPACT EVALUATION TRUST FUND (SIEF)

IMPACT EVALUATION CONCEPT NOTE

PROVIDER PAYMENT REFORM
IN CHINA’S HEALTH CARE SYSTEM

December 16, 2008

Table of Contents

1. Summary 2

2. China’s provider payment reforms in context 3

2.1. International experience and research questions 3

2.2. Provider payment reforms in China 4

2.3. Provider payment reforms in the World Bank’s Health XI project 5

3. The SIEF-supported impact evaluation 7

3.1. The evaluation exercise as a constrained optimization problem 7

3.1.1. Objectives of the impact evaluation 7

3.1.1.1. Filling knowledge gaps for policymaking 8

3.1.1.2. Analytic rigor 8

3.1.2. Constraints facing the evaluation team 9

3.1.2.1. Nature, timing and location of project interventions 9

3.1.2.2. Likely interventions in non-project counties 10

3.1.2.3. Data availability 10

3.1.2.4. Scope for collecting new data 11

3.1.2.5. Budget 12

4. Options for the SIEF-sponsored evaluation 12

4.1.1. Sample size issues 12

4.1.2. Identification strategy options 13

4.1.2.1. Randomized phased rollout 13

4.1.2.2. Diffs-in-diffs, forming a control group from non-project facilities 14

5. Organization and timeline 16

5.1.1. Team composition 16

5.1.2. Settling on the evaluation design as Health XI moves ahead 17

5.1.3. Data—existing and new 18

5.1.4. Timeline 19

5.1.5. Budget 19

6. References 26

7

1. Summary

This concept note proposes an impact evaluation (IE) design for provider-payment innovations of the World Bank’s Health XI project. The Health XI project involves 8 provinces and 40 counties. It aims to help the Government of China (GOC) design and implement provider-payment and other health sector reforms in rural China, through piloting innovative reforms, evaluating them, and disseminating the lessons learnt. The goal of the SIEF-supported IE activities is to complement the project-supported evaluation activities. This concept note discusses the SIEF-supported evaluation objectives, study design, methodology, organization and timeline. It serves as a working document to guide the IE, with the final design to be determined after the Health XI project finalizes what provider payment innovations to pilot and evaluation studies to include.

To develop this concept note, we adopt a constrained optimization approach. The two broad objectives are to ensure: (a) the evaluation is as useful as possible to policymakers; and (b) it is as rigorous as possible. The constraints include: (a) the nature, timing and location of the interventions selected by the Health XI project; (b) the government’s plans vis-à-vis non-project counties; (c) the data already available and likely to be available in the future; (d) the scope for collecting new data; (e) and the SIEF budget.

The knowledge that this particular study seeks to generate concerns the impacts—on the quantity of health care, its quality, and the efficiency with which it is delivered—of the introduction of case-based payment in county hospitals and capitation-based payment for (THCs) and village clinics. A key secondary goal is to examine how impacts of basic provider payment reforms can be modified by (i) complementary reforms aimed at mitigating negative side-effects of the basic reforms and (ii) implementation of “enhancements” to the basic reforms, such as using weighted capitation formulas for primary care to reduce incentives for cream-skimming, and the introduction of explicit financial incentives for primary care facilities to provide preventive and public health services.

To identify the causal impact of the provider-payment innovations, we propose two identification strategies. The first and preferred method is a randomized phased rollout design, in which the interventions are randomly assigned to sub-sets of project counties through block randomizations and the interventions will be rolled out in two separate times. The second option is a difference-in-difference methodology with the counterfactuals constructed from facilities that have not been subject to the payment reforms and patients treated in (or households living in the catchment areas of) these facilities.

The data for IE will draw on existing facility based data, patient records and household surveys. Additional data collection will also be considered depending on access to existing data and their quality. Our initial power calculation shows that the minimum number of intervention THCs and county hospitals required to detect a 20% change in patient level measures (such as ALOS) with 80% power would be of the order of 30 and 10, respectively. Subject to budget constraints, we aim to expand the final sample size, especially through coordinating with the project-supported data collection activities.

The proposed SIEF team consists of experts with complementary roles. At the core is a Chinese team with major responsibilities in liaising with the Health XI project; monitoring and documenting the project interventions through appropriately designed questionnaires; working with the other PIs on designing appropriate survey instruments; fielding household and facility surveys, cleaning and analyzing the data; writing up the results; and disseminating the findings in China, including within the Health XI project.

2. China’s provider payment reforms in context

2.1.  International experience and research questions

Line-item budgets and fee-for-service (FFS) have long been the predominant methods for paying hospitals and primary health care providers in both developed and developing countries. Yet both methods are increasingly recognized as having important weaknesses. In an effort to enhance efficiency and cost-control, many countries have implemented or are considering new provider payment methods.[1] In the case of the hospital sector, the predominant approach has been to pay providers on the basis of the number of cases, with different reimbursement levels for different types of cases (e.g. the Diagnostic Related Groups (DRGs)). In the case of outpatient care, a commonly used alternative to FFS is to pay providers on the basis of the number of people that they serve (capitation). The attraction of both methods is that they transfer some of the financial risk to the provider, thus creating incentives for efficiency and cost-consciousness. But at the same time they create potentially adverse incentives—e.g. to increase volume, skimp on quality, refer patients to higher levels, and to “game” the payment system (e.g. through ‘upcoding’ cases in order to receive higher payments).

Despite extensive experimentation with provider payment reforms in both low- and middle-income countries, the evidence on their impact remains limited, most coming from the OECD countries. This evidence suggests that payment reforms can lead to improved efficiency and cost-control. But it also reveals the importance of measures to mitigate the adverse incentives introduced by reforms. Capitation payments, for example, are increasingly weighted to reflect at least the demographic characteristics of the enrolled population, if not other characteristics as well, such as prior health and socio-economic circumstances. Capitation is also increasingly supplemented by performance-related payments (P4P) that are sometimes linked to health status (e.g. blood pressure levels) but are more commonly linked simply to the delivery of specific preventive interventions (e.g. immunizations). DRGs can be accompanied by a FFS element: the DRG element would encourage cost-consciousness but the FFS element would discourage skimping and selection (the hospital can claim payment for extra services delivered to achieve higher quality or to compensate for a high-cost patient mix). Sometimes DRGs are also accompanied by a ceiling on DRG payments, or DRG rates are reduced once a target number of cases in the group have been reached.

In addition, the impact of provider payment reforms seems likely to depend on technical and management capacity in the health ministry and elsewhere in the health sector, the financing modalities, and regulations concerning provider autonomy. Even in the developed world, there are large variations across countries in the capacity to design and implement payment reforms, and there seems to be an emerging consensus that the theory of provider payment reform has gotten a little ahead of the practice. The identity of the payer and the accountability of the payer are also likely to be important factors. Finally, payment reform is unlikely to work if providers are not given the necessary autonomy to make the changes required to reduce their costs (e.g. if they lack the ability to hire and fire health workers).

2.2.  Provider payment reforms in China

Over the last few decades, China has seen a rapid escalation in health care cost, a gradual decline in health insurance coverage, and a rising share of health expenditure financed from out-of-pocket payments by patients over the last 20 years. Recently, the government has taken important steps towards addressing these issues, including the rolling out of a new health insurance scheme in rural areas. However, it is recognized that reforms on the demand-side are unlikely to succeed unless supply-side problems (e.g. inefficiency and provision of unnecessary care) are also addressed.[2]

A number of local governments have experimented with provider payment reforms, often combined with other measures to hold providers accountable for performance. In contrast to many other Communist countries, China shifted away from a budget-based payment model for hospitals many years ago, opting instead for a model where hospitals receive a basic budget paid from the government and are free to supplement this with fee income earned from charging patients, albeit using a fee schedule set by the government. Village doctors and township health centers (THCs) are also paid by FFS. The use of FFS has incentivized providers to deliver care, but the tilted fee schedule—which allows providers to make profits on drugs and high-tech care, but forces them to make losses on ‘basic’ care—has led to an expansion of profitable services at the expense of unprofitable ones. China’s drug spending—as a share of total spending—is one of the highest in the world, and high-tech equipment is widely diffused and overused. Reforms have been most common among urban hospitals, where—in the face of escalating costs—insurance schemes in some cities experimented with different forms of capped global budgets or payment per inpatient day with volume caps. Other cities have followed suit, some experimenting with somewhat different payment arrangements. By contrast, efforts to reform provider payments for the rural population—including the care they receive in urban hospitals—have been very few, with just a few examples of rural insurance schemes developing a case-based payment system for THCs and a salary-cum-performance-bonus scheme for village doctors.

These and related issues have been reviewed and discussed in a multi-year World Bank study of rural health reform that will be published in late 2008 or early 2009. The study inter alia assesses recent reforms (including the new rural health insurance scheme), and sets out some policy reform options for the future, with special reference to rural areas where reform has been slow to develop and where the bulk of the population still lives.

2.3.  Provider payment reforms in the World Bank’s Health XI project

The World Bank’s Health XI project, approved by the Bank’s Board of Directors in June 2008, aims to help the Government of China (GOC) design and implement provider-payment and other health sector reforms in rural China. The project, which is being financed almost equally by the Bank and Chinese Ministry of Finance (the UK’s department for International Development (DfID) is contributing 9% of the budget), is set to begin implementation at the end of October 2008 and to run until October 2013.

An explicit aim of the project is to pilot innovative reforms, evaluate them, and disseminate the lessons learnt. The project will focus on 40 rural counties in 8 of China’s poorer provinces (China has 22 provinces and just under 1500 rural counties[3]). Unusually, for a Bank project, the details of the reforms to be implemented are to be developed during the project rather than prior to negotiation. The local governments of the 40 project counties are to be given considerable latitude in deciding how they proceed. The choice is not unconstrained, however.

Counties must prepare proposals for reform pilots and have them approved in order to receive financial support from the project. Proposals are to be scrutinized by a panel of national and international Mandarin-speaking health reform experts. The process for scrutinizing and selecting proposals is to be determined by GOC in consultation with the Bank and DFID, but it will be GOC that will decide which proposals to proceed with, whether to request changes to the proposals, and how to allocate the budget across the counties. The project does, however, require that counties receive only half their approved budget in advance, with the rest being conditional on them hitting performance targets; there is also provision for the budgets of under-performing reformers to be reallocated in favor of over-performers.

Reform pilot proposals must include at the minimum provider payment reforms and certain other (sometimes related) reforms, namely: (a) reforms to enhance quality that link quality standards and procedures with financial and other incentives; (b) measures to improve equity of access to quality health services, including by gender; (c) measures to strengthen the design and management or the new rural health insurance program, and (d) institutional and financing reforms to improve public health functions and the delivery of public health services.

As far as the provider payment reforms are concerned, all 40 counties are expected to implement basic payment reforms covering financial interactions between the new rural health insurance scheme—the New Rural Cooperative Medical Scheme (NRCMS)—and general hospitals and lower-level facilities, namely THCs and village clinics. These basic reforms will involve NRCMS using case-based payment methods to pay general hospitals, and capitation-based payments to THCs and clinics. A rough estimate is that 182 general hospitals might fall within the counties covered by the project (an estimated 84 are government-run), 571 THCs, and nearly 3000 clinics (mostly privately run) (see Table 1). The NRCMS is likely to be willing to contract with non-government clinics (the vast majority of village clinics in China are now privately-operated) but whether it will contract with non-government general hospitals is not known.

In some counties, the basic payment reforms are expected to be complemented by other reforms or implemented in such a way as to mitigate the potentially negative consequences of these payment reforms. Complementary reforms are likely to include the introduction of clinical guidelines and enhanced quality monitoring. Implementation “enhancements” include the use of weighted capitation formulas for primary care to reduce incentives for cream-skimming, and the introduction of explicit financial incentives for primary care facilities to provide preventive and public health services.