Appendix

CEAC/EVPI analysis and statistical inferenceσσ

The CEAC can be constructed based on (re)sampling procedures, but also directly given a normally distributed net monetary benefit (NMB) estimator [39]:

CEAC(λ)= P{ > 0}(1)

Equation (1) thus yield the probability that the NMB estimator is positive. Based on the central limit theorem (CLT), the is normally distributed[48]. Based on the standard normal distribution with a Z score = ( – NMB(λ)) / σ ~ N(0,1) the CEAC (λ) is calculated as follows [48]:

CEAC (λ)= P{ > 0} = P{Z > - NMB(λ) / σ}(2)

CEAC (λ) = P{} = P{Z + / σ > 0}(3)

where Z = standard normally distributed random variable and σ = standard deviationof the NMB estimator.

A normal cumulative density function of the CEAC is then given by:

CEAC (λ) = Ф(/σ)(4)

where Ф (•) = standard normal cumulative density function.

An estimate of CEAC (λ) is then given by:

CEAC (λ) = Ф/σ)(5)

where and σ are sample estimates.

Because a parametric (1-α) one-sided lower-bound confidence interval (CI) has a lower bound – z1-ασ, the (1-α) CI where the lower bound of the interval is not above zero is given by the inequality nmb(λ) – z1-ασ ≤ 0. By determining the cumulative density functionof this term equation 6 follows:

Ф(/σ) ≤ 1- α(6)

From equation (5) and (6) it follows that the CEAC (λ) is less or equal to the chosen level of confidence. A one-sided lower bound confidence interval is then given by – z1-ασ and can be used to test the null hypothesis H0: ≤ 0 (i.e., control treatment should be continued) against the research hypothesis H1: > 0 (i.e., new treatment should be implemented). Equivalently, the p value of a test being below or above the pre-specified level of significance can be used. A frequentist interpretation of a CEAC is possible because the relationship between p value and CEAC can be expressed by p(NMB (λ)) = 1 – CEAC (λ). Graphically this is reflected by the fact that the CEAC is simply the mirror image of a p-value curve [39]. CEACs in the context of frequentist analyses can be regarded as a particular case of Bayesian inference with use of non-informative priors [48]. Hence, the prior probability of H0 (i.e., for a given willingness to pay a new intervention is not more cost-effective than its comparator) being false is assumed to be 0.5.

CEACs can also be applied using a non-parametric bootstrap approach [39]. The bootstrap makes no assumption about the distribution of data in the population. It works asymptotically, i.e., as the size of the original data sample increases, the bootstrap sampling distribution will tend to move towards the true sampling distribution. The bootstrap replicates of the NMB statistic nmb * b(λ), b = 1, ... , B, provide an estimate of the sampling distribution of the NMB estimator NMB (λ), defined as

F(a) = (

where F is a sampling distribution of the NMB estimator for any real value a and I denotes the standard indicator function. The bootstrap estimate of the CEAC can thus be calculated as:

CEAC (λ) = (8)

As shown for the parametric approach, there is a relationship between the CEAC and the bootstrap confidence intervals for calculating the NMB: a (1-α) one-sided lower-bound bootstrap confidence interval is given by α(λ), where α(λ) is the α-quantile of , i.e.,α(λ) = F-1(α). The quantiles of are given by the [(B+1)α] and the [(B+1)(1-α)] ordered values of the bootstrap estimates *b(λ), b = 1, . . . , B, respectively. The lower bound of a (1-α) confidence interval is not above zero if nmbCEAC(λ) = F-1(α) ≤ 0, or equivalently if α ≤ F(0) [39].

Because CEACs may mislead policy makers by providing insufficient information on the consequences of an incorrect decision [40,41] uncertainty is measured by the EVPI. Since perfect information would allow avoiding the chance of making a wrong decision when adopting a novel intervention, the EVPI can be defined as [8]:

EVPI = λ * σ * L(D0),(9)

where D0 = │NMB(λ)NT - I0│/ σ, λ = willingness to pay, σ = standard deviation of NMB, and L(D0) = unit normal loss integral for standardized distance D0, NMB(λ) NT = mean incremental net benefit of the new treatment, and I0 = point of indifference between new treatment and standard (I0 = 0).