Chapter 5-16. Correlated Data: Analysis of Covariance (ANCOVA) Versus Change Scores
In this chapter we will compare the analysis of covariance (ANCOVA) approach to the change approach for analyzing repeated measures data. By ANCOVA, we are referring to specific analysis situation where where the baseline outcome variable is controlled for while testing for effects on one or more later outcome variable measurements.
Isoproterenol Dataset
We will again use the 11.2.Isoproterenol.dta dataset provided with the Dupont (2002, p.338) textbook, described as,
“Lang et al. (1995) studied the effect of isoproterenol, a β-adrenergic agonist, on forearm blood flow in a group of 22 normotensive men. Nine of the study subjects were black and 13 were white. Each subject’s blood flow was measured at baseline and then at escalating doses of isoproterenol.”
Reading the data in,
FileOpen
Find the directory where you copied the course CD
Change to the subdirectory datasets & do-files
Single click on 11.2.Isoproterenol.dta
Open
use "C:\Documents and Settings\u0032770.SRVR\Desktop\
Biostats & Epi With Stata\datasets & do-files\
11.2.Isoproterenol.dta", clear
* which must be all on one line, or use:
cd "C:\Documents and Settings\u0032770.SRVR\Desktop\"
cd "Biostats & Epi With Stata\datasets & do-files"
use 11.2.Isoproterenol.dta, clear
______
Source: Stoddard GJ. Biostatistics and Epidemiology Using Stata: A Course Manual [unpublished manuscript] University of Utah School of Medicine, 2010.
Listing it,
+------+
| id race fbf0 fbf10 fbf20 fbf60 fbf150 fbf300 fbf400 |
|------|
1. | 1 1 1 1.4 6.4 19.1 25 24.6 28 |
2. | 2 1 2.1 2.8 8.3 15.7 21.9 21.7 30.1 |
3. | 3 1 1.1 2.2 5.7 8.2 9.3 12.5 21.6 |
4. | 4 1 2.44 2.9 4.6 13.2 17.3 17.6 19.4 |
5. | 5 1 2.9 3.5 5.7 11.5 14.9 19.7 19.3 |
|------|
6. | 6 1 4.1 3.7 5.8 19.8 17.7 20.8 30.3 |
7. | 7 1 1.24 1.2 3.3 5.3 5.4 10.1 10.6 |
8. | 8 1 3.1 . . 15.45 . . 31.3 |
9. | 9 1 5.8 8.8 13.2 33.3 38.5 39.8 43.3 |
10. | 10 1 3.9 6.6 9.5 20.2 21.5 30.1 29.6 |
|------|
11. | 11 1 1.91 1.7 6.3 9.9 12.6 12.7 15.4 |
12. | 12 1 2 2.3 4 8.4 8.3 12.8 16.7 |
13. | 13 1 3.7 3.9 4.7 10.5 14.6 20 21.7 |
14. | 14 2 2.46 2.7 2.54 3.95 4.16 5.1 4.16 |
15. | 15 2 2 1.8 4.22 5.76 7.08 10.92 7.08 |
|------|
16. | 16 2 2.26 3 2.99 4.07 3.74 4.58 3.74 |
17. | 17 2 1.8 2.9 3.41 4.84 7.05 7.48 7.05 |
18. | 18 2 3.13 4 5.33 7.31 8.81 11.09 8.81 |
19. | 19 2 1.36 2.7 3.05 4 4.1 6.95 4.1 |
20. | 20 2 2.82 2.6 2.63 10.03 9.6 12.65 9.6 |
|------|
21. | 21 2 1.7 1.6 1.73 2.96 4.17 6.04 4.17 |
22. | 22 2 2.1 1.9 3 4.8 7.4 16.7 21.2 |
+------+
We see that the data are in wide format, with variables
id patient ID (1 to 22)
race race (1=white, 2=black)
fbf0 forearm blood flow (ml/min/dl) at ioproterenol dose 0 mg/min
fbf10 forearm blood flow (ml/min/dl) at ioproterenol dose 10 mg/min
…
fbf400 forearm blood flow (ml/min/dl) at ioproterenol dose 400 mg/min
In this dataset, each of the several occasions represents an increasing dose, so can be thought of as an effect across dose, rather than as an effect across time.
Paired Sample t Test (Change Analysis With Two Repeated Measurements)
The paired sample t test is a very popular approach for analyzing two correlated measurements. Let’s compare the no dose forearm blood flow, fbf0, to the initial dose (10 mg/min), ignoring race for now.
StatisticsSummaries, tables & tests
Classical tests of hypotheses
Mean comparison test, paired data
First variable: fbf10
Second variable: fbf0
OK
ttest fbf10 == fbf0
<or>
ttest fbf10 = fbf0
Paired t test
------
Variable | Obs Mean Std. Err. Std. Dev. [95% Conf. Interval]
------+------
fbf10 | 21 3.057143 .3864103 1.770755 2.251105 3.863181
fbf0 | 21 2.467619 .2532897 1.160719 1.939266 2.995972
------+------
diff | 21 .5895238 .1967903 .9018064 .1790265 1.000021
------
Ho: mean(fbf10 - fbf0) = mean(diff) = 0
Ha: mean(diff) < 0 Ha: mean(diff) != 0 Ha: mean(diff) > 0
t = 2.9957 t = 2.9957 t = 2.9957
P < t = 0.9964 P > |t| = 0.0071 P > t = 0.0036
The paired t test is identically the one sample t test on the absolute change scores (fbf10 – fbf0).
To verify this,
capture drop diff10gen diff10 = fbf10-fbf0
list fbf0 fbf10 diff10 in 1/5
+------+
| fbf0 fbf10 diff10 |
|------|
1. | 1 1.4 .4 |
2. | 2.1 2.8 .7 |
3. | 1.1 2.2 1.1 |
4. | 2.44 2.9 .46 |
5. | 2.9 3.5 .5999999 |
+------+
StatisticsSummaries, tables & tests
Classical tests of hypotheses
One sample mean comparison test
Variable name: diff10
Hypothesized mean: 0
OK
ttest diff10 = 0
One-sample t test
------
Variable | Obs Mean Std. Err. Std. Dev. [95% Conf. Interval]
------+------
diff10 | 21 .5895238 .1967903 .9018064 .1790265 1.000021
------
Degrees of freedom: 20
Ho: mean(diff10) = 0
Ha: mean < 0 Ha: mean != 0 Ha: mean > 0
t = 2.9957 t = 2.9957 t = 2.9957
P < t = 0.9964 P > |t| = 0.0071 P > t = 0.0036
Compared to paired t test from above,
Paired t test
------
Variable | Obs Mean Std. Err. Std. Dev. [95% Conf. Interval]
------+------
fbf10 | 21 3.057143 .3864103 1.770755 2.251105 3.863181
fbf0 | 21 2.467619 .2532897 1.160719 1.939266 2.995972
------+------
diff | 21 .5895238 .1967903 .9018064 .1790265 1.000021
------
Ho: mean(fbf10 - fbf0) = mean(diff) = 0
Ha: mean(diff) < 0 Ha: mean(diff) != 0 Ha: mean(diff) > 0
t = 2.9957 t = 2.9957 t = 2.9957
P < t = 0.9964 P > |t| = 0.0071 P > t = 0.0036
And we can easily see that the two forms of the t test are identically the same, which is a test that the mean difference, or change, is equal to 0.
The paired test takes the correlation structure of the data into account by being a test on difference scores. The standard error of the difference, by definition, includes the correlation coefficient of the two variables. The variance used in this formula is (van Belle, 2002, p.61):
where ρ is the correlation between the two variables, the sigmas represent
the standard deviations and the sigma-squared the variances.
Let’s verify this. First obtain the correlation coefficient between the baseline and time 10 measurements:
corr fbf0 fbf10(obs=21)
| fbf0 fbf10
------+------
fbf0 | 1.0000
fbf10 | 0.8927 1.0000
Using the standard deviations displayed in the paired t test output,
Paired t test
------
Variable | Obs Mean Std. Err. Std. Dev. [95% Conf. Interval]
------+------
fbf10 | 21 3.057143 .3864103 1.770755 2.251105 3.863181
fbf0 | 21 2.467619 .2532897 1.160719 1.939266 2.995972
------+------
diff | 21 .5895238 .1967903 .9018064 .1790265 1.000021
------
and converting them to variances for use in the formula for the change variance:
display 1.770755^2+1.160719^2-2*0.8927*1.770755*1.160719.81322181
Take the square root of the variance to get the standard deviation:
.90178812
which is identically the standard deviation of the difference shown in either t test output, accurate to 4 decimal places. (Accuracy was limited to 4 decimal places since that is all we used for the correlation coefficient in the calculation.)
Paired t test
------
Variable | Obs Mean Std. Err. Std. Dev. [95% Conf. Interval]
------+------
fbf10 | 21 3.057143 .3864103 1.770755 2.251105 3.863181
fbf0 | 21 2.467619 .2532897 1.160719 1.939266 2.995972
------+------
diff | 21 .5895238 .1967903 .9018064 .1790265 1.000021
------
Designing a Repeated Measures Experiment
There are two popularly used approaches to designing a pre-test vs post-test experiment.
Cross-Over Design
The first approach is the cross-over design, where the same subjects receive both treatments, one-half randomized to received treatment A first, and the other receives treatment B first, with a wash-out period in between.
treatment A (active) treatment B (placeblo)
baseline post-intervention wash-out baseline post-intervention
X0A X1A X0B X1B
A popular way to analyze these data are:
First, compute two change scores (differences):
diffA = X1A - X0A
diffB = X1B - X0B
Second, compare diffA to diffB using a paired t test.
Parallel Groups Design
The second approach is the parallel groups design, where different subjects are randomized to receive either treatment A or B.
pre (or baseline) intervention post-intervention
treatment A (active) X0A X1A
treatment B (placeblo) X0B X1B
A popular way to analyze these data are:
First, compute two change scores (differences):
diffA = X1A - X0A
diffB = X1B - X0B
Second, compare diffA to diffB using an independent groups t test.
Which study design leads to a more powerful statistical comparison? That depends on which version of the t test, paired or independent groups, is more powerful.
When Is the Paired t Test More Powerful than Independent Groups t Test?
The formula for the independent groups t test is (Rosner, 1995, p.259)
which when , reduces to
The formula for the paired t test is
where r is the Pearson correlation coefficient. The comparison involves three situations:
r = 0 : the two forms of the t test are identical
r > 0 : the paired t test is more powerful (larger value), since r will then shrink in the
denominator of the paired t test statistic [This is quantitative expression of the rule-of-
thumb that paired data increase precision.]
r < 0 : the independent groups t test is more powerful (larger value), since r will then inflate
in the denominator of the paired t test statistic
Note (Is it legitimate to switch to the independent groups t test when your repeated measures data has an r < 0?):
No. You are stuck analyzing your data with the paired t test, rather than an independent groups t test, since you have correlated data. That is, the independent groups t test assumes ρ = 0, and has no ability to account for any other correlation structure in the data.
Conclusion
Replacing with diffA and with diffB in the above discussion, and recognizing that r > 0 is the usual situation for a cross-over trial, the cross-over trial with its paired t test is the more powerful approach. [Still, cross-over designs are frequently criticized for other reasons. For example, one cannot be sure there is not a carry-over effect that remains even with a long wash-out period.]
Illustration
To illustrate the effect of the correlation structure on the paired and independent groups t tests, we will draw some random samples of size n=40 with one variable having mean 3 and SD 1.8, and a second variable having mean 2.5 and SD 1.2, similar to the estimates above. We will do this for various correlations between the two variables. Then, we will compare the two variables using both a paired and independent groups t test.
To draw random samples that are correlated, you pass the drawnorm command a vector of means, a vector of standard deviations, and a correlation matrix, which we do in the following Stata code.
capture drop log closelog using junk, replace
set seed 999
clear
forvalues r=-.75(.25).75 {
matrix m = (3 , 2.5) // mean1 = 3, mean2 = 2.5
matrix sd = (1.8 , 1.2) // sd1 = 1.8, sd2 = 1.2
matrix c = (1 , `r' \ `r' , 1) // correlation
drawnorm x1 x2 , n(40) means(m) sds(sd) corr(c)
sum
corr x1 x2 // Pearson correlation
ttest x1=x2 // paired t test
ttest x1=x2,unpaired // independent groups t test
drop x1 x2
}
log close
In this one iteration Monte Carlo simulation, the results are
correlation / means / paired t testp value / independent t test
p value
-0.78 / 3.36 vs 2.34 / 0.032 / 0.005
-0.28 / 2.75 vs 2.68 / 0.818 / 0.796
-0.22 / 2.87 vs 2.26 / 0.086 / 0.057
-0.18 / 3.15 vs 2.22 / 0.011 / 0.005
0.25 / 2.50 vs. 2.43 / 0.807 / 0.827
0.64 / 2.72 vs 2.43 / 0.313 / 0.510
0.77 / 3.01 vs 2.70 / 0.042 / 0.291
As expected, the independent groups t test consistently has a smaller p value (more powerful) when the data are negatively correlated and the paired t test consistently has a smaller p value (more powerful) when the data are positively correlated.
Analysis of a Parallel Groups Design Study
For the rest of the discussion, where we compare the ANCOVA approach to the change approach, we will consider only the parallel groups design (or an independent groups observational study).
pre (or baseline) intervention post-intervention
treatment A (active) X0A X1A
treatment B (placeblo) X0B X1B
Frison and Pocock (1992) discuss three methods for analyzing parallel groups design randomized controlled trials comparing two treatments with baseline and post-treatment measurements on the same subjects.
Post-treatment means (POST): use an independent groups t test to compare the two groups on
their post-treatment measurement (or mean summary measure of
post-treatment measurements)
i.e., compare X1A to X1B ,
and assume X0A = X0B as a result of randomization (and so
can be ignored)
Mean changes (CHANGE): use an independent groups t test to the compare the two groups on
their baseline to post treatment change (or difference between
mean of baseline measurements and mean of post-treatment
measurements)
i.e., first compute two change scores:
diffA = X1A - X0A
diffB = X1B - X0B
and then compare diffA to diffB
(has an intuitive appeal, since a paired t test uses change scores)
Analysis of Covariance (ANCOVA): compare the two groups using a regression model of the
post-treatment measurement (or mean summary measure of post-treatment measurements) using the baseline measurement (or mean summary measure of baseline measurements) as a covariate.