Homework 4 Notes

  1. Get a cross-tabulation of HI1*HI2. Calculate McNemar's test of symmetry for these related proportions. Use your user-defined formats for this table.

If you created temporary formats, you will need to recreate them for this exercise:

libname b510 v9 "e:\510\homework";

procformat;

value actfmt 1="1: <=1"

2="2: 2-4 hours"

3="3: >4 hours";

value hifmt 1="1:High"

2="2:Not High";

value rranfmt 1="1:Yes"

2="2:No";

value $genfmt "M" = "Male"

"F" = "Female";

run;

procfreq data=b510.combine2;

tables hi1*hi2 / agree;

format hi1 hi2 hifmt.;

run;

  1. What percent of students had high heartrate at time 1? What percent of students had high heartrate at time 2?

19 of the 65 students (or 20%) had high heart rate at time1.

31 of the 65 students (or 31%) had high heart rate at time 2.

  1. Was there a significant change in the percentage of students who had high heartrate at time 1 compared to time 2?Remember to include your test statistic, degrees of freedom and p-value.

Table of hi1 by hi2

hi1 hi2

Frequency ‚

Percent ‚

Row Pct ‚

Col Pct ‚1:High ‚Not High‚ Total

‚ ‚ ‚

ƒƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒˆ

1:High ‚ 11 ‚ 2 ‚ 13

‚ 16.92 ‚ 3.08 ‚ 20.00

‚ 84.62 ‚ 15.38 ‚

‚ 35.48 ‚ 5.88 ‚

ƒƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒˆ

2:Not High ‚ 20 ‚ 32 ‚ 52

‚ 30.77 ‚ 49.23 ‚ 80.00

‚ 38.46 ‚ 61.54 ‚

‚ 64.52 ‚ 94.12 ‚

ƒƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒˆ

Total 31 34 65

47.69 52.31 100.00

Frequency Missing = 14

Note: when we are looking at matched samples and comparing the proportion who had a particular outcome at time 1 vs at time 2, we are looking at the marginal proportions.

H0: proportion with high heart rate at time 1 = proportion who had high heart rate at time 2.

HA: proportion with high heart rate at time 1 ≠ proportion who had high heart rate at time 2.

The appropriate test for this problem is McNemar’s test of symmetry, since it is paired data (the same student before and after running). The result is highly significant McNemar’s test chi-square = 14.7273, with 1 df, p = 0.0001. The proportion who had high heart rate after running is significantly greater than the proportion who had high heart rate before running.

  1. Get a three-way cross-tabulation of RRAN*HI1*HI2. Use the user-defined formats for this problem. Calculate a McNemar's test of symmetry for these related proportions.

procfreq data=b510.combine2;

tables rran*hi1*hi2 / agree;

format hi1 hi2 hifmt.;

run;

  1. What is the stratifying variable in this problem?RRAN is the stratifying variable.
  2. What percent of students who did not run had high heartrate at time 1?

9of the 35(or 25.71%) students who did not run had high heart rate at time 1.

What percent of students who did not run had high heartrate at time 2?

8 of the 35 (or 22.86%) students who did not run had high heart rate at time 2.

Controlling for rran=2

hi1 hi2

Frequency ‚

Percent ‚

Row Pct ‚

Col Pct ‚1:High ‚2:Not Hi‚ Total

‚ ‚gh ‚

ƒƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒˆ

1:High ‚ 7 ‚ 2 ‚ 9

‚ 20.00 ‚ 5.71 ‚ 25.71

‚ 77.78 ‚ 22.22 ‚

‚ 87.50 ‚ 7.41 ‚

ƒƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒˆ

2:Not High ‚ 1 ‚ 25 ‚ 26

‚ 2.86 ‚ 71.43 ‚ 74.29

‚ 3.85 ‚ 96.15 ‚

‚ 12.50 ‚ 92.59 ‚

ƒƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒˆ

Total 8 27 35

22.86 77.14 100.00

Statistics for Table 2 of hi1 by hi2

Controlling for rran=2

McNemar's Test

ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ

Statistic (S) 0.3333

DF 1

Pr > S 0.5637

  1. Was there a significant change in the percentage of students who had high heartrate at time 1 compared to time 2 for those students who did not run?

No. The McNemar’s test was not at all significant (chi-square=0.333, df=1, p=0.5637). There was no evidence of a change in the proportion with high heart rate from time 1 to time 2 for those students who did not run.

  1. What percent of students who ran had high heartrate at time 1? What percent of students who ran had high heartrate at time 2?

4 of the 30 (or 13.33%) students who ran had high heart rate at time 1, while 23 of the 30 (or 76.67%) students who ran had a high heart rate at time 2.

Table 1 of hi1 by hi2

Controlling for rran=1

hi1 hi2

Frequency ‚

Percent ‚

Row Pct ‚

Col Pct ‚1:High ‚2:Not Hi‚ Total

‚ ‚gh ‚

ƒƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒˆ

1:High ‚ 4 ‚ 0 ‚ 4

‚ 13.33 ‚ 0.00 ‚ 13.33

‚ 100.00 ‚ 0.00 ‚

‚ 17.39 ‚ 0.00 ‚

ƒƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒˆ

2:Not High ‚ 19 ‚ 7 ‚ 26

‚ 63.33 ‚ 23.33 ‚ 86.67

‚ 73.08 ‚ 26.92 ‚

‚ 82.61 ‚ 100.00 ‚

ƒƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒˆ

Total 23 7 30

76.67 23.33 100.00

Statistics for Table 1 of hi1 by hi2

Controlling for rran=1

McNemar's Test

ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ

Statistic (S) 19.0000

DF 1

Pr > S <.0001

  1. Was there a significant change in the percentage of students who had high heartrate at time 1 compared to time 2 for those students who ran?

Yes, there was a highly significant change in the proportion who had high heart rate at time 1 compared to time 2 for those students who ran (McNemar’s chi-square = 19, df = 1, p <.0001). There was a highly significant increase in the proportion of students with high heart rate at time 2 vs time 1 among those who ran.

  1. Please explain your results for students who ran and did not run

Since those ran exercised between the two measurements, the proportion of students with elevated heart rate increased dramatically, from about 13% to almost 77%. For those students who did not run, there was very little change in the percent with elevated heart rate from time 1 to time 2, which is what we would expect, since they did not do any exercising between the two measurements.

.

  1. A (hypothetical) published report stated that 20% of graduate students at schools of Public Health had a resting heartrate greater than 85 beats per minute.
  2. Carry out a statistical test to determine if the proportion of students with a resting heartrate greater than 85 beats per minute in this class is consistent with this published report.
  3. What do you conclude?

This problem calls for a one-sample chi-square goodness of fit test. We want to compare the proportion of students with high heart rate to the hypothetical published value. The syntax is shown below:

procfreq data=b510.combine;

tables hi1 / chisq testp=(.20,.80);

run;

Note that the test proportions are based on the published report. We find that there is not a significant difference, based on the chi-square goodness of fit test (chi-square with 1 df = 0.0032, p=.9551). So, we have no evidence to think that the population from which this sample of students was drawn has a different proportion of students with elevated heart rate than was reported in this hypothetical study.

Test Cumulative Cumulative

hi1 Frequency Percent Percent Frequency Percent

ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ

1:High 16 20.25 20.00 16 20.25

2:Not High 63 79.75 80.00 79 100.00

Chi-Square Test

for Specified Proportions

ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ

Chi-Square 0.0032

DF 1

Pr > ChiSq 0.9551

Sample Size = 79

  1. The following (hypothetical) table was published in a hypothetical journal. It shows the relationship between eating a diet high in animal fat and risk of heart disease. The study was carried out among 50 males between the ages of 65 and 70.
  2. Carry out a chi-square test of significance for this table. What do you conclude about the relationship of a diet high in animal fat and heart disease?

Study 1: Relationship of High Fat Diet to Heart Disease
Diet / Heart Disease / Total
YES / NO
HIGHFAT / 15 / 15 / 30
LOWFAT / 6 / 14 / 20
Total / 21 / 29 / 50

The Pearson chi-square test for this study is 1.9704, with 1 degree of freedom, p = 0.1604. We don’t have sufficient evidence to reject the null hypothesis of no association between a diet high in fat and risk of heart disease from this study.

  1. Calculate the odds ratio and relative risk of a diet high in animal fat and risk of heart disease. Report the odds ratio and relative risk, along with their 95% confidence intervals. Please interpret each of these measures.

The odds ratio and relative risk and their 95% confidence intervals are shown below:

Case-Control (Odds Ratio) 2.3333 0.7067 7.7044

Cohort (Col1 Risk) 1.6667 0.7802 3.5606

Both of these measures are greater than one, meaning that the risk of having heart disease if you eat a high fat diet is higher than the risk of heart disease if you do not eat a high fat diet. However, both confidence intervals include 1, so we cannot reject the null hypothesis for this study.

  1. Another, larger hypothetical study was carried out by a group of researchers at another University, to study the same relationship. Their study examined this relationship among 500 males in the same age group.
  2. Calculate a chi-square test of significance for this table. What do you conclude based on this evidence?

The chi-square test for this study is highly significant (chi-square = 19.7044, with 1 df, p<.0001). We very strongly reject the null hypothesis, based on the results of this study, and conclude that those patients with a high fat diet are at greater risk of heart disease than those patients with a low fat diet.

  1. Calculate the odds ratio and relative risk for this table, and their 95% confidence intervals.

The odds ratio and relative risk for this study are the same as for the previous study, however, their 95% confidence intervals are much narrower, and do not include 1. This is in keeping with the highly significant results we saw for the chi-square test.

  1. Compare and contrast the results from question 4 above with the results from this question. Explain what is different about the results and why.

Even though the odds ratio and relative risk were identical for these two studies, the results from the first study were inconclusive about the relationship between diet and heart disease, whereas the results of the second study were highly significant. The larger sample size of the second study gave rise to more convincing evidence against the null hypothesis.

Study 2: Relationship of High Fat Diet to Heart Disease
Diet / Heart Disease / Total
YES / NO
HIGHFAT / 150 / 150 / 300
LOWFAT / 60 / 140 / 200
Total / 210 / 290 / 500
  1. Another hypothetical study compared the reliability of two raters in assessing tonsil size in a sample of 62 children at an emergency pediatric clinic.
  2. Assess the degree of agreement between these two raters in terms of their assessment of tonsil size. Is there significant agreement between the two raters? Report the results of both the asymptotic and exact tests.Report both kappa and weighted kappa.

Reliability of Assessment of Tonsil Size
Rater1 / Rater2 / Total
+ / ++ / +++
+ / 25 / 2 / 0 / 27
++ / 4 / 15 / 3 / 22
+++ / 1 / 2 / 10 / 13
Total / 30 / 19 / 13 / 62

Simple Kappa Coefficient

ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ

Kappa (K) 0.6960

ASE 0.0778

95% Lower Conf Limit 0.5436

95% Upper Conf Limit 0.8484

Test of H0: Kappa = 0

ASE under H0 0.0917

Z 7.5866

One-sided Pr > Z <.0001

Two-sided Pr > |Z| <.0001

Exact Test

One-sided Pr >= K 3.677E-13

Two-sided Pr >= |K| 3.677E-13

Weighted Kappa Coefficient

ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ

Weighted Kappa (K) 0.7472

ASE 0.0689

95% Lower Conf Limit 0.6122

95% Upper Conf Limit 0.8822

Test of H0: Weighted Kappa = 0

ASE under H0 0.1004

Z 7.4448

One-sided Pr > Z <.0001

Two-sided Pr > |Z| <.0001

Exact Test

One-sided Pr >= K 9.020E-14

Two-sided Pr >= |K| 9.020E-14

Sample Size = 62

For this study, we are estimating the reliability of the ratings of two different raters, for determining tonsil size. Both raters examine the same patients, so their ratings are matched. We want to estimate Cohen’s kappa for this problem. Higher values of Cohen’s kappa indicate higher agreement.

The value of Cohen’s kappa is 0.6960, with 95% Confidence interval of .5436 to .8484. The weighted kappa is somewhat higher, and is equal to .7472, with 95% confidence interval of .6122 to .8822.

In this problem, we are testing

H0: kappa=0

HA: kappa ≠ 0

So, we will use a two-sided p-value.

The result of the test for the unweighted kappa is highly significant (z-statistic = 7.5866, with two-sided p<.0001). The exact test is similarly highly significant (p< .0001). Therefore we strongly reject the null hypothesis and conclude that the kappa for these two raters is significantly greater than zero.

The results of the tests for the weight kappa are similar, with z=7.448, and p<.0001. Again, the exact test results are very similar, with p<.0001.

There appears to be good agreement between these two raters.

  1. In another hypothetical study, researchers were assessing the grip strength of elderly patients who had experienced a stroke, before and after grip strength training. The patients’ grip strength was rated initially in the morning, as their ability to squeeze together a 1 pound spring-loaded caliper. They were then trained in gripping techniques, and their grip strength was again tested after the training was completed later in the morning.
  2. What percent of patients could perform the task before training? What percent of patients could perform the task after training?
  3. Was there a significant change in the grip strength of the participants in the program after training?
  4. The researchers decided not to report these findings, and redesigned their study. Suggest a better way to design this study to assess the effects of grip strength training for these patients.

Grip Strength Before and After Training
Can Perform Task
Before
Training / Can Perform Task After Training / Total
YES / NO
YES / 17
/ 10
/ 27
NO / 3
/ 41
/ 44
Total / 20 / 51
/ 71

In this problem, we are again comparing the before proportion to the after proportion for the same patients, so we want to use McNemar’s test. The percent of patients who could perform the test prior to the training is 27/71 or 38.03%. The percent of patients who could perform the test after the training was 20/71 or 28%. This decline was not quite significant at the .05 level (McNemar’s chi-square = 3.7692, with 1 degree of freedom, p=.0522). We would not reject the null hypothesis in this case.

Since the percent of patients who could perform the test actually declined somewhat from the first measurement to the second one, does this mean that the grip strength training was not effective? Not necessarily. Since the patients were elderly and had suffered a stroke previously, their strength was not very great. The second grip strength was measured on the same day, after a training program. The patients may well have been fatigued after their training.

Another possible study design would be to randomly assign two groups of patients to either receive the strength training or not. The training could be carried out 3 times a week for 6 weeks, and then the proportion who could perform the test at the end vs. at the beginning of the study could be compared for the two study groups. This would allow the patients in the group who received the training to recover after their training and to get stronger. This would also avoid confounding the effects of fatigue and simple passage of time with the effects of the training.

1