Chapter 12: One-Way Analysis of Variance
- The General Logic of ANOVA (ANalysis Of VAriance)
Building on the previous chapters, here is how ANOVA can be conceptually placedwith respect to the parametric statistical tests we’ve already discussed.
DECISION TREE
How many groups are you comparing? / One Group(compared to a population) / Two Groups
(compared to each other) / Three or More Groups
(compared to each other)
z Test or tTest (depending on the sample size) / Independent Samples tTest
(two different groups) / Matched t Test
(same group measured twiceor matched groups) / ANOVA
(independent groups)
If you were interested in the difference in productivity of people who drink coffee versus those who do not, you could do a two-groupttest. But if instead you were interested in several different amounts of coffee and theirrelative effects on productivity, you would need to do an ANOVA. For example, if you thought there might be a threshold whereby too much coffee is no longer helpful, you could set up an experiment comparing the differences between those whodrank no coffee, had a tall coffee, had a grande coffee,orhad a venti coffee shortlybefore taking a pop quiz in a history class. You could compare all of the different amountswith one single action by calculating an ANOVA. Once you calculated it, you would know if there was at least one difference (if not more)among the groups. You wouldn’t know exactly which group(s) differ(s) significantly, but figuring that out will be covered in the next chapter.
If the ANOVA procedure didn’t exist, you would instead have to do ALL OF THESE t-tests:
No Coffee versus Tall Coffee
No Coffee versus. Grande Coffee
No Coffee versus Venti Coffee
Tall Coffee versus Grande Coffee
Tall Coffee versus Venti Coffee
Grande Coffee versus Venti Coffee
And although you are now a ttest guru, and you could actually do it—let’s be real, it would be really annoying! More important, it would lead to an inflated Type I error rate! And we don’t want that, do we? So we are lucky that Sir Ronald Fisher invented ANOVA.
To begin the ANOVA process, we need to divide the variance into two pieces: Between Groups and Within Groups. As the names suggest, the Between-Groups variance refers to the differences between the two or more groups that you have (in our example, that would be between the No Coffee, Tall, Grande and Venti Coffee groups). This is conceptually similar to the numerator component of the ttest, which was simply the difference between only two means. The Within-Groups variance, as the name implies, captures the variability within the groups—same as with the pooled variance in the denominator of the t test.
Once you have those values, you can plug them into the F ratio formula for ANOVA, which is supersimple:
If you ever forget which goes on top versus which goes on bottom, you can use the simple trick that they are listed alphabetically: B(between) before (over) W(within).
- Calculating the One-Way ANOVA for Equal and Unequal Sample Sizes
The ANOVA formula looks a bit different depending on whether you have equal sample sizes or different sample sizes.
If the groups are all exactly equal in size:
n = the number of people in each group
k = number of groups
= each group’s mean
(i.e.,the grand mean is a simple average of the group means)
(i.e., a simple average of the group variances)
s2 = each group’s variance
k = number of groups
Because there are several components required for the calculation of these next exercises, it can be quite helpful (and it is our recommendation) to put a key on the top of your page where you list all the components after calculating each one.
Here are the data for the coffee example so we can see this test in action:
Equal Sample Sizes:
A psych teacher asks his class of students to volunteer to participate in a study. He asks them to complete a quick survey right before the class is given a surprise quiz. The classroom sits right next to a Starbucks Coffee location. Students were not assigned to conditions, but instead they reported what size coffee they had that morning before the exam. Remarkably, of those who self-selected into the study, exactly fivestudents had no coffee, fivehad a tall coffee, fivehad a grande coffee and fivehad a venti coffee. Students’ scores on the exam were reported, on a scale of 1 to 10. The means and standard deviations of test scores are listed in the following table in groups by coffee cup size.
STEP 1: Get all your important info
/ s / n- No Coffee
- Tall Coffee
- Grande Coffee
- Venti Coffee
k = 4, n = 5, NT = 20
dfBet = k – 1 = 4 – 1 = 3
dfWithin = NT – k = 20 – 4 = 16
STEP 2: Calculate MSBet
STEP 3: Calculate MSW
STEP 4: Calculate F
When dealing with unequal sample sizes, a variation of the previous formulas must be used that gives more weight to larger groups:
where (i.e., a weighted average of the group means)
(i.e., a weighted average of the group variances)
Let’s say that another student walked into the class a little late who had a Venti coffee and did not do well at all on the exam (with a 4 out of 10).Of course,it is certainly possible that the poor exam grade received on this quizwas due to the student’s lateness as opposed to the Venti coffee, but for the sake of this example, we will allow this variation from ourexperimental procedure.
STEP 1: Get all your important info
/ s / n- No Coffee
- Tall Coffee
- Grande Coffee
- Venti Coffee
k = 4; NT = 21; dfBet = k– 1 = 4 – 1 = 3; dfWithin = NT – k = 21 – 4 = 17
STEP 2: Calculate MSBet
STEP 3: Calculate MSW
STEP 4: Calculate F
STEP 5: Check for Significance (see next section)
- The F Distribution and Testing an ANOVA for Significance
Just as we learned to check our ttests for significance with thetdistributions, the same can be done for our ANOVA with the alwaysusefulF distribution table. The F distribution works a bit differently, because you must look up TWO different degrees of freedom values. Also, the textbook provides a table for alpha equal to 0.05 and alpha = 0.01, so make sure you are checking the right one! And remember, the default is 0.05 unless you are told otherwise.
To determine your degrees of freedom, use the following DEGREES OF FREEDOM TREE:
dfTotalNT – 1
dfBet
k – 1 / dfWithin
NT – k
Example 1:
If you have four groups, each containingfive people (like our equal-n coffee experiment), you would have the following components:
k = 4n = 5NT = n * k = 5 * 4= 20
CALCULATE:
dfTotal = 20 – 1 = 19
dfBet = 4 – 1 = 3
dfWithin = 20 – 4 = 16
CHECK :
dfTotal = dfBet + dfWithin
19 = 3 + 16
GREAT! Those add up—we’re good to go.
Example 2:
Or if you had four groups, one with 10, another with 9, the third with 13, and the fourthwith 5:
k = 4 NT = n1 + n2 + n3 + n4 = 10 + 9 + 13 + 5 = 37
CALCULATE:
dfTotal = 37 – 1 = 36
dfBet = 4 – 1 = 3
dfWithin = 37– 4 = 33
CHECK :
dfTotal = dfBet + dfWithin
36 = 3 +33
YAY AGAIN! Those add up—we’re good to go!
So next, for each of the examples, we would want to look up these degrees of freedom values in the F table to get a critical F value to compare against the calculatedF value. In Example 1(the equal-n coffee study example), dfBet = 3 and dfWithin = 16, so our critical F would equal 3.24 at the 0.05 level, or 5.29 at the (alpha =) 0.01 level. Recall that the calculated F for that example was 19.87, which is larger than 5.29, so you can reject the null hypothesis (that the population means for the different amounts of coffee are all the same) even at the .01 level. This is shown in the first ANOVA summary table below.
Now, here are some exercises for you to do:
1.What would the critical F value be for Example 2, in whichdfBet = 3 and dfWithin = 33,if alpha = 0.05? ______If alpha = 0.01? ______
2.What would the critical F value be for a study with 60 people equally divided into 3 groups? dfBet = ___ and dfWithin = ___ and the critical F would therefore equal ______at the 0.05 level,and ______at the 0.01 level.
3.What would the critical F value be for a study with 96 people equally divided into 8 groups? dfBet = ___ and dfWithin = ___ and the critical F would therefore equal ______at the 0.05 level,and ______at the 0.01 level.
4. ANOVA Summary Tables
The results of a one-way ANOVA are typically presented in a summary table, which organizes all of the calculations and answers into one simple table.
Based on what we calculated for our coffee example with equal groups, this is what part of the summary table would look like:
Source / SS / df / MS / F / PBetween Groups / 3 / 14.40 / 19.86 / < .01
Within Groups / 16 / 0.725
Total / 19
NOTE: We know what you must be thinking:There are a lot of blank cells! We are not even halfway done! Well, no worries, some of those cells don’t need to get filled in. In fact, we’re going to X out the boxes that don’t need to be filled in, so you know how much more needs to be done.
Source / SS / df / MS / F / PBetween Groups / 3 / 14.40 / 19.86 / < .01
Within Groups / 16 / 0.725 / xxxxxxxxxxxxxx / xxxxxxxx
Total / 19 / xxxxxxxxx / xxxxxxxxxxxxxx / xxxxxxxx
Better? We thought so. So now you’ve only got to fill in the SS (aka Sum of Squares) information. However, because we calculated the MSs directly without calculating the SSs first, we don’t know what the SSs are. Fortunately, that part iseasy. It’s simply the MS in each row multiplied by the df for that row—and voilà, you have the SS. So, Step 1: MSBet * dfBet = SSBet; Step 2: MSW * dfW = SSW; andStep 3: SSBet + SSW = SSTotal.
Source / SS / df / MS / F / PBetween Groups / MSBet * dfBet = 43.20 / 3 / 14.40 / 19.86 / < .01
Within Groups / MSW * dfW = 11.60 / 16 / 0.725 / ------/ ------
Total / 43.20+11.60= 54.80 / 19 / ------/ ------/ ------
Practice your knowledge of summary tables by completing the following examples:
4.
Source / SS / df / MS / F / PBetween Groups / 1184.77 / 5
Within Groups / 65.27
Total / 2490.17 / 25
5.
Source / SS / df / MS / F / PBetween Groups / 80.00
Within Groups / 36 / 8.667
Total / 39
5. An Effect Size Measure for ANOVA
It’s great that the ANOVA for the coffee experiment is statistically significant (p < .01), but we are still not quite done yet. The large F ratio does not tell us automatically if our groups are really spread out quite a bit or if they overlap a lot. Even with a lot of overlap of scores between groups, which indicates a small effect size, it is possible to get a large F ratio by using large sample sizes. So it is very informative to combine your sample sizes with your F ratio to come up with a measure of the effect size in your data. The most popular measure of effect size is called eta squared (η2), and the formula for it is easy:
For the coffee example:
This is an extremely large effect size (the largest possible size is 1.0), because we had a very large F ratio with very small samples.
Okay, time to try an entire example yourself!
6.Three brands of macaroni and cheese are to be compared with respect to their taste quality. Six batches of each type will be cooked using each recipe. Given the following data, are these brands significantly different from one another?
/ SD / N- Kraft Mac & Cheese
- Annie’s Shells & Cheddar
- Easy Mac
STEP 1: Gather all your important info
k =
NT =
dfBet =
dfWithin =
STEP 2: Calculate MSBet
STEP 3: Calculate MSW
STEP 4: Calculate F
STEP 5: Check for Significance/Complete Summary Table
Source / SS / df / MS / F / PBetween Groups
Within Groups / ------/ ------
Total / ------/ ------/ ------
STEP 6: CalculateEta Squared
η2 =
6. The Kruskal-Wallis H Test
If you are worried that your data don’t fit the assumption of normal distributions in the population, and your samples are too small to rely on the Central Limit Theorem to save you, you know from Chapter 8 that youcan just rank your data, which takes care of any big gaps among your scores, or any extreme scores. After ranking, all of the data are nice and evenly spaced, but you can still see if some groups get consistently higher-ranked data than other groups. The trick is to rank all of the scores, from all of the groups, together, but to keep track of which score is from which group. Assuming you have studied the rank-sum or Mann-Whitney U test, you’ve already done this for two groups. Now, we’ll show you how to extend that procedure toa study that involves more than two groups. Let’s say you suspect that students in some majors watch way more movies than others (not including film majors, duh), but to demonstrate this you find out how many movies were watched in the last six months by 7 Chemistry majors, 6 English majors, and 5 Psychology majors. The data are in the next table.
Chemistry English Psychology
1 4 26
10 17 21
3 11 15
12 20 9
3 14 23
5 18
7
First, we will place all of the scores (# of movies) in order, but indicate which major each score is from with the first initial of that major underneath (C, E, or P). Under that, we will write the rank of the score.
1 3 3 4 5 7 9 10 11 12 14 15 17 18 20 21 23 26
C C C E C C P C E C E P E E E P P P
1 2.5 2.5 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Note how we dealt with the tie involving the two 3’s: First they were given ranks of 2 and 3 arbitrarily, and then we gave each score the average of 2 and 3. Next, we add the ranks separately for each major: TC = 35 (i.e., 1 + 2.5 + 2.5 + 5 + 6, etc.); TE = 66; TP = 70.
As a check, add these totals to make sure that together they equal the sum of all ranks from 1 to the total N (i.e., 18): Sum of all ranks = 35 + 66 + 70 = 171, which equals 18 (18+1)/2 = 171. Check! Now we are ready to calculate SSbet:
SSbet = 352 / 7 + 662 / 6 + 702 / 5 – [18 (192) / 4] = 175 + 726 + 980 – 1,624.5 = 1,881 – 1,624.5 = 256.5. Finally, we are ready for the last step – calculating the Kruskal-Wallis H statistic.
H = 12*SSbet / [NT (NT + 1)] = 12 (256.5) / (18 * 19) = 3,078 / 342 = 9.0
For the critical value we need to look in Table H of your text. (That’s not why it’s called Table H, but it sure is convenient, huh?) The df equals k – 1, where k is the number of groups, so we need to find the critical value of the chi-square statistic for 2 df, which is 5.99 for the .05 level, and 9.21 for the .01 level. Our value for H makes it at the .05 level (9 > 5.99), but just misses at the .01. By the way, we stole these data from exercise 3b in Chapter 12 of your text, so, as you can see from the answer key of your text, if you had performed a one-way ANOVA on these data instead, F = 8.77, p < .01. The F and H statistics are not usually that similar, but the p value for the K-W H test is usually not far behind the p value for the ANOVA.
The formula to find the eta squared that corresponds to your K-W H test is just:
η2 = (H – k + 1) / (NT – k)
For this example, η2 = (9 – 3 + 1) / (18 – 3) = 7 / 15 = .467, so nearly 47% of the variance in the movie variable is accounted for by college major.
Now try this example:
7. Suppose this time you asked 6 Psych majors, 4 English majors, and 4 Chemistry majors to rate their satisfaction with the social life on their campus on a scale from 1 to 10. Here are the data:
Psychology English Chemistry
6 8 7
10 10 5
7 7 9
8 8 4
8
9
a.Perform the Kruskal-Wallis H test on these data. Can you reject the null hypothesis with α = .05?
b. Calculate the appropriate measure of effect size, even if your results are not significant.
Answers to Exercises
1.Fcv(3,33) = 2.89, α=0.05;Fcv(3,33) = 4.44, α=0.01. [Note: cv stands for critical value.]
2.Fcv(2,57) = 3.16, α=0.05;Fcv(2,57) = 4.99, α=0.01.
3.Fcv(7,88) = 2.11, α=0.05;Fcv (7,88) = 2.85, α=0.01.
4.
Source / SS / df / MS / F / PBetween Groups / 1184.77 / 5 / 236.95 / 3.63 / < .05
Within Groups / 1305.40 / 20 / 65.27 / ------/ ------
Total / 2490.17 / 25 / ------/ ------/ ------
5.
Source / SS / df / MS / F / PBetween Groups / 80.00 / 3 / 240 / 27.69 / < .01
Within Groups / 312.01 / 36 / 8.667 / ------/ ------
Total / 392.01 / 39 / ------/ ------/ ------
6. k = 3; NT = 18; dfBet = 2; dfWithin = 15; 70.06
Source / SS / df / MS / F / PBetween Groups / 1072.12 / 2 / 536.06 / 5.56 / < .05
Within Groups / 1446.90 / 15 / 96.46 / ------/ ------
Total / 2519.02 / 17 / ------/ ------/ ------
η2 = (2*5.56)/[(2*5.56) + 15] = .426, or SSbet/SStot = 1,072.12/2519.02 = .426
7. SSbet = 502 / 6 + 35.52 / 4 + 19.52 / 4 – [14 (152) / 4] = 416.67 + 315.06 + 95.06 – 785.5 = 826.8 – 785.5 = 41.3; H = 12 (41.3) / (14 * 15) = 495 / 210 = 2.36 < 5.99, so the null hypothesis, that students in the three majors are equally satisfied with the campus social life, cannot be rejected. η2 = (2,36 – 3 + 1) / (14 – 3) = .36 / 11 = .033, which is a fairly small, but not very small effect size.