Evidence-based Diagnosis:
A Workshop on Evaluating and Using Medical Tests
Small Group 4: Friday, June 10, 3:00 – 4:30
Problems with Answers
Objectives:A / Volunteer Bias
B / Lead Time Bias
C / Differing Natural History Bias
D / Stage Migration Bias
E / Pseudodisease
F / Total vs. Cause-Secific Mortality (Slippery Linkage and Sticky Diagnosis)
G / Number Needed to Screen
H / Number Needed to Screen × Screening Cost
Problem / Topic / Difficulty / A. Volunteer Bias / B. Lead Time Bias / C. Differing Natural History Bias / D. Stage Migration Bias / E. Pseudodisease / F. Total vs. Cause Specific Mortality / G. Number Needed to Screen / H. Number Needed to Screen× Screening Cost
1 / CT Screening for Lung Cancer / Intermediate / X / X / X / X / X / X / X
2 / Cervical Cancer Screening In India / Beginner / X
3 / Screening Mammography in Sweden / Beginner / X
4 / Prostate Cancer Screening / Beginner / X
5 / Colonscopy / TO BE WRITTEN
1. CT Screening for Lung Cancer (May replace with EBD Problem 6.2.)
The National Lung Screening Trial (NLST) randomized 53,464[T1] current and former heavy smokers (minimum 30 pack-years) aged 55 to 74 years to either helical CT scanning or chest-x-rays thrice yearly. Enrolment was in 2002-4 and the study was recently stopped for benefit by the Data Safety Monitoring Board. Although the study has not been published (as of 5/30/2011), press accounts indicate that after up to 5 years of follow-up, there were 354 lung cancer deaths in the CT screening group, compared with 442 in the comparison group, a 20.3% decrease. Press reports also indicate that about 25% of the deaths in the study were due to lung cancer, and that the CT screening group had 7% lower total mortality. Using these numbers and the "Goal Seek" tool in Excel, it is possible to estimate all of the group sizes and obtain P-values and confidence intervals for both lung cancer and total mortality, as shown below, where "Exposed" refers to the CT group:
Lung Cancer DeathYes / No / Total / Risk
CT / 354 / 26443 / 26797 / 1.321%
XR / 442 / 26225 / 26667 / 1.657%
ARR= / 0.336%
Death from Any Cause
Yes / No / Total / Risk
CT / 1538 / 25259 / 26797 / 5.739%
XR / 1646 / 25021 / 26667 / 6.172%
ARR= / 0.433%
- State whether each of the following statements is true or false; explain your answer
i.Because this is a randomized trial, the favorable effect of CT screening on lung cancer mortality can't be due to lead time bias, length bias, or volunteer bias.
True. Lead and length would only apply if you were limiting the comparison to those with lung cancer. This analysis includes everybody, both those with and those without lung cancer. In other words, it’s not survival time with cancer, it’s mortality, both cause-specific and total. Volunteer bias is not possible because it’s a randomized trial with a (presumed) intention-to-treat analysis.
The apparent benefit of CT for lung cancer mortality could be due to "Sticky Diagnosis Bias."
False. Sticky diagnosis bias is possible with cause-specific mortality, but it would bias the results against screening because those in the screened group would be more likely to have their deaths attributed to lung cancer.
ii."Slippery linkage bias" is unlikely to explain the apparent lung cancer mortality benefit.
True. Slippery linkage bias leads to underestimation of the harms of screening. In order for slippery linkage bias to explain the lung cancer mortality benefit, deaths due to lung cancer in the screened group would somehow need to have been attributed to other causes. If this had occurred, then the non-lung cancer death rate would be higher in the screened group, but it’s actually a little lower.
- The following is taken from the CBS News story about the study: (
" After 50 years of smoking, 67-year-old Steffani Torrighelli knew she was at high risk for lung cancer. Two years ago she enrolled in [the] study, and sure enough a CT scan picked up an early stage tumor before she had any symptoms… Since Torrighelli's lung surgery two years ago, she's cancer free and vigilant about screening."
Could Steffani's good outcome in this randomized trial be due to detection of pseudodisease? Explain.
Yes. There is no way to know if her early stage lung cancer would have caused her any problems. Although some lung cancer deaths appear to have been prevented, we don't know how many unnecessary operations may have occurred to achieve that benefit. The mortality benefit in this randomized trial can't be due to pseudodisease, but good outcomes in individual patients could be.
- It will be interesting, when the papers are finally published, to see how many scans, biopsies, lung resections, etc., were needed to obtain the observed mortality benefit, and what they cost. (I am hoping we will be able find this out, but the funding for the study did not cover any of these costs; participants with positive scans were told to follow-up with their usual providers and their own health insurance, if they had any.) We do know that about 25% of the participants had 1 or more positive scans that needed additional follow-up tests. Assume that the lung cancer mortality benefit resulted from an average of 4 years of CT scanning 3 times per year in the CT scan group.
- About how many screening CT scans were needed to defer one lung cancer death in the NLST?
ARR = 0.0036
NNT = 1/ARR= 1/0.0336= ~300
3 scans/year 4 years 300 = ~3600 (Exact answer is 3566.)
- Press reports say the scans cost about $300. What was the approximate cost of the screening CT scans per lung cancer death deferred?
$300 3600 = $1.08 million (Exact answer $1,069,887.) - No information from the study is available to estimate the additional cost of a positive scan, but a joint curbside consult from a faculty radiologist and chest physician suggests it might be $5000-$10,000. Using the low estimate of $5000, and assuming this cost applies to 25% of the CT scan group, what was the total approximate cost of further evaluating positive results in the NLST? (Assume, optimistically, that the 25% with a positive scan have only a single positive scan.)
25% ×26797 × $5000 = ~ $33.5 million (Exact answer $33,496,863.)
- Maintaining these same assumptions, what is the screening and evaluation cost per lung cancer death deferred?
The easiest way to do this is to compute the total cost per patient and multiply by the NNT. The cost per patient is 12 $300 for the screening scans, plus an average of 25% of $5000 = $3600 + $1250 = $4850. Multiply this by the NNT of 297 and the total screening and evaluation cost is about $1.44 millionper lung cancer death deferred.
- The Washington Post story ended with the following paragraph:
In addition to a reduction in lung cancer deaths among those scanned, the study found a 7 percent reduction in deaths from any cause. It remained unclear why that would be the case, but researchers speculated that it might be attributable to the scans detecting other cancers or illnesses, such as heart or lung disease.
What do you think of the speculation above about the basis for effect on total mortality? [2]
Since a significant chunk of the total mortality in these current or former heavy smokers is due to lung cancer, the first question to ask is whether there is any evidence of a benefit for non-lung cancer mortality, or whether the decrease in total mortality is easily explained by the decrease in lung cancer mortality. In fact, this is the case: there were 1184 vs 1204 non-lung cancer deaths. Excluding those who died of lung cancer, the data are shown below:
Yes / No / Total / Risk
CT / 1184 / 25613 / 26797 / 4.418%
XR / 1204 / 25463 / 26667 / 4.515%
RR / 0.979
ARR / 0.097%
So this RR of 0.975 provides minimal evidence for a benefit on non-lung cancer mortality, and there is no need to hypothesize benefits from early detection of other illnesses.
2. Cervical Cancer Screening In India
In the developing world, cervical cancer death rates have not decreased the way they have in the developed world. A cluster-randomized trial in India showed that a one-time HPV screening program reduced cervical cancer death rates compared with usual care.(1) Simplified results are shown in the following table:
Offered HPV Screening / ControlN / 34126 / 31488
Avg Follow-up / 7.9 / 7.9
Cervical Cancers Diagnosed / 127 / 118
Cervical Cancer Deaths / 34 / 64
Cervical Cancer Death Rate/10,000 / 10.0 / 20.3 / P = 0.0006
a) Number Needed to Screen: How many women need to be offeredone-time HPV screening to prevent one cervical cancer death at 7.9 years of follow-up. (For extra credit, compare this to your answer to (1)c(i) above.
Difference in death rate = (20.3 – 10.0)/10,000, so NNT = 10,000/10.3 = 965
Offer 965 Indian women a one-time HPV test to prevent a cervical cancer death.
Offer 300 American smokers (or ex-smokers) an average of 12 screening CT scans per patient to prevent one lung cancer death.
b) The above represents an “intention-to-treat” analysis. Roughly 80% of the women who were offered HPV screening received it and none of the control group received HPV screening. Would the true number needed to screen (vs. number needed to be offered screening) be higher or lower than the number calculated in part (a)?
Lower. Maybe even 0.8 × 965 = 772.
Roughly the same number of cervical cancers were diagnosed in the group offered HPV screening and the control group (127 vs. 118). The table below shows the distribution of cancer diagnoses by stage at diagnosis.
Stage At Diagnosis / HPV Screening or Control? / HPV Screening or Control?1A / 6% / 37%
1B / 22% / 26%
≥II / 69% / 31%
Unknown / 3% / 6%
c) Which column do you think represents which group? You have a 50/50 chance of labeling the columns correctly, but explain your choice.
The right-hand column is the HPV screening group because more cancers were diagnosed at earlier stages.
3. Screening Mammography
Based on the Swedish screening mammography studies(2), a woman has a 0.4% (4/1000) chance of dying of breast cancer between the ages of 55 and 64. Annual screening mammography appears to reduce this risk by 25%. How many women need to be screened annually between the ages of 55 and 64 to prevent one dying from breast cancer? (For extra credit, compare this to the 2 prior calculated numbers needed to screen.)
ARR = 0.4% × 0.25 = 0.1% NNT = 1/0.1% = 1000. (This is actually fairly accurate, even though it’s a round number.)
Offer 1000 55-year-old Swedish women annual mammography for 10 years to prevent one breast cancer death.
Offer 965 Indian women a one-time HPV test to prevent a cervical cancer death.
Offer 300 American smokers (or ex-smokers) an average of 12 screening CT scans per patient to prevent one lung cancer death.
4. Prostate Cancer Screening
Andriole et al (3) reported the results of a randomized trial of prostate cancer screening with combination of prostate-specific antigen (PSA) testing and digital rectal examinations compared with usual care (which was whatever the physician usually did). The subjects were men aged 55 – 74 years. After 7 years of follow-up the results of an "intention to treat" analysis" were as follows:
Diagnosis of Prostate Cancer / Death From Prostate Cancer / TotalRandomized To… / N / % / N / %
Annual Screening / 2820 / 7.35% / 50 / 0.130% / 38343
Usual Care / 2322 / 6.05% / 44 / 0.115% / 38350
There were significantly more patients diagnosed with prostate cancer in the group randomized to annual screening (116 vs. 95 per 10,000 person-years, risk ratio 1.21 95% CI 1.15 – 1.28). There were also more prostate cancer deaths in the group randomized to screening (2.0 vs. 1.7 per 10,000 person-years, risk ratio 1.14 95% CI 0.76 - 1.70).
a)If more prostate cancers were identified in the screening group, why didn’t fewer men die of prostate cancer?
The additional cancers identified were not cancers that kill patients, i.e. pseudodisease.
b)How would you advise a male patient aged 55 – 74 about whether to be screened for prostate cancer?
You may bring other evidence to bear, but based on this study, I would advise him not to be screened for prostate cancer. It increases his chances of getting biopsies and treatment without decreasing his risk of dying from prostate cancer.
c)Now read Box 12.1 on page 242 of EBD.
5. Screening Colonoscopy
References:
1.Sankaranarayanan R, Nene BM, Shastri SS, Jayant K, Muwonge R, Budukh AM, et al. HPV screening for cervical cancer in rural India. N Engl J Med. 2009 Apr 2;360(14):1385-94.
2.Nystrom L, Andersson I, Bjurstam N, Frisell J, Nordenskjold B, Rutqvist LE. Long-term effects of mammography screening: updated overview of the Swedish randomised trials. Lancet. 2002 Mar 16;359(9310):909-19.
3.Andriole GL, Grubb RL, 3rd, Buys SS, Chia D, Church TR, Fouad MN, et al. Mortality results from a randomized prostate-cancer screening trial. N Engl J Med. 2009 Mar 26;360(13):1310-9.
[T1]The only place I found something that looked like the actual (rather than rounded) N was here: