A brief summary of the example case
The example case is drawn from a previously published study [1] that compared the properties of constant-sum paired comparison (CSPC) and discrete choice experiments (DCE) in eliciting societal preferences over the allocation of healthcare resources. The primary objective was to compare the response behaviours of the two methods in terms of completion rates, difficulty ratings, preference stability, and the incidence of dominant preferences, but marginal rates of substitution were estimated in order to establish the face validity of each method.
Attributes were identified though a process of empirical ethics: attributes had to have evidence of significant public support and had to be compatible with a theory of distributional justice, such as need, maximising, or egalitarian principles [2]. This ensured that the attributes were not only relevant, but also, in some sense, fair. This process identified five attributes: patient age, severity, final health state, duration of benefit, and distributional concerns [3]. To distinguish between severity as proximity to death and as a poor health state, this factor was decomposed into life expectancy without treatment and initial health state. Each attribute was assigned three levels, spaced as widely as possible across a plausible range. Levels of age tested preferences for young (age 10), middle age (40), and elderly (70) patients, while levels of initial and final health states tested preferences for very poor (utility 0.1), moderate (0.5) and very good (0.9) health. Levels of initial life expectancy (1 month, 5 years, or 10 years)and individual life year gains (1 year, 5 years and 10 years) tested preferences for prioritising patients facing imminent death while also being plausible in conjunction with the oldest age group. Distributional concerns were incorporated in terms of the number of patients treated: 500, 2500 or 5000.
A D-efficient optimal fractional factorial experimental design with 18 choice sets of 2 alternatives each was developed using SAS macros [4]. This design was then divided into two blocks of 9 choice sets each. As the primary objective of the study was to test response behaviours rather than estimate preferences, the design was simplified by using block 1 for the DCE questionnaire and block 2 for the CSPC questionnaire, rather than randomly assigning respondents to different blocks within each questionnaire. This simplified questionnaire administration, but violates the principles of optimal experimental design and limitsstatistical efficiency[5]. For this reason, estimates of marginal rates of substitution should be interpreted with caution.
Individuals were invited complete an online questionnaire via a mass email to students at The University of Sheffield, Sheffield, UK, and posters and electronic announcements to students, staff and faculty at Dalhousie University, Halifax, Canada, and physicians and staff atthe Capital District Health Authority, Halifax, Canada. Participantswere asked to imagine themselves as a societal decision maker responsible for allocating a fixed budget between two alternative healthcare programs. They were told that both programs had the same cost, and that the budget was large enough to fully fund one program or the other, but not both. By moving a slider, respondents could allocate percentage shares of the budget betweenprogram A or program B, including 100 percent to one program or the other, or an equal 50-50 split between both. One hundred and fifty participants submitted a completed CSPC questionnaire.
Coefficients from the tobit model of CSPC responses are shown in the main paper, and marginal rates of substitution (MRS) derived from these estimates, using individual life year gains as the numeraire, are shown in the table below (reproduced from the main paper).
Table 1: Marginal rates of substitution in terms of life year gains foregone, by attribute
Δx / MRS / SE / L95CI / U95CIΔ Age / 30.0 / -1.60 / 0.01 / -1.62 / -1.58
Δ Initial utility / 0.4 / -1.80 / 0.65 / -3.08 / -0.53
Δ Initial life expectancy / 5.0 / 0.47 / 0.05 / 0.38 / 0.56
Δ Final utility / 0.4 / 3.51 / 0.98 / 1.59 / 5.42
Δ Patients treated /1000 / 1.0 / 1.17 / 0.13 / 0.91 / 1.42
Δx=attribute change for which MRS was calculated; MRS: marginal rate of substitution; SE=standard error; L95CI/U95CI: lower/upper 95% confidence interval
The negative MRS for age and initial health state suggested that CSPC respondents preferred to prioritize younger patients and those in worse initial health states. Conversely, the positive MRS for final utility suggested respondents preferred to prioritize patients that could be returned to a better final health state. Likewise, the positive MRS for the number of patients treated suggested that respondents would prefer a smaller gain to more patients over a larger gain to fewer patients. These results are consistent with expectations from the literature [6–8]. Unexpectedly, though, respondents preferred to prioritize patients with greater life expectancies, suggesting an aversion to prioritizing those facing imminent death. This was contrary to expectations, although such a result has been observed elsewhere [9,10]. The overall consistency of these results with the literature supports the face validity of the CSPC method. It must be emphasized, though, that these preference results were a secondary objective of the study and were based on an experimental design with limited statistical efficiency in a relatively small sample. They are only reported here to illustrate the context of the case study.
References
1. Skedgel CD, Wailoo AJ, Akehurst RL. Choosing vs. allocating: discrete choice experiments and constant-sum paired comparisons for the elicitation of societal preferences. Health Expectations. 2013;n/a–n/a.
2. Richardson J, McKie J. Empiricism, ethics and orthodox economic theory: what is the appropriate basis for decision-making in the health sector? Social Science & Medicine. 2005;60:265–75.
3. Skedgel C. Societal Preferences in the Allocation of Healthcare Resources: An Empirical Ethics Approach. Medical Decision Making. 2011;31:E71.
4. Kuhfeld WF. Marketing Research Methods in SAS [Internet]. Marketing Research Methods in SAS. 2010 [cited 2010 Dec 1]. Available from:
5. Carlsson F, Martinsson P. Design techniques for stated preference methods in health economics. Health Econ. 2003;12:281–94.
6. Schwappach DL. Resource allocation, social values and the QALY: a review of the debate and empirical evidence. Health Expectations. 2002;5:210–22.
7. Dolan P, Shaw R, Tsuchiya A, Williams A. QALY maximisation and people’s preferences: a methodological review of the literature. Health Economics. 2005;14:197–208.
8. Stafinski T, Menon D, Marshall D, Caulfield T. Societal Values in the Allocation of Healthcare Resources: Is it All About the Health Gain? The Patient. 2011;4:207–25.
9. Shah KK. Severity of illness and priority setting in healthcare: A review of the literature. Health Policy. 2009;93:77–84.
10. Shah K, Tsuchiya A, Hole AR, Wailoo A. Valuing health at the end of life: a stated preference discrete choice experiment [Internet]. NICE Decision Support Unit; 2012 Dec. Available from: