One-way ANOVAs Lab 51
One-Way ANOVA’s
We have already discussed the t-test. The t-test is used for comparing the means of two groups to determine if there is a statistically significant difference. The t-test allows us to determine the probability that we are making a Type I error (rejecting the null when it is true). If the probability is less than alpha (generally set at 0.05) then we reject the null hypothesis (conclude that there is a difference) with at least 95% confidence that the difference between the two conditions is not due simply to chance.
T-tests are appropriate for making comparisons when
- your independent variable (IV) is discrete and your dependant variable (DV) is continuous.
- there is only one IV in your study (it is not a factorial Design)
- there are only two levels of the IV
One-way ANOVAs are appropriate when
- your independent variable is discrete and your dependant variable is continuous.
- there is only one IV in your study (it is not a factorial design)
- there are more than two levels of the IV.
One-way ANOVAs allow us to determine if there is a difference AMONG three or more conditions.
The procedure is done in two parts.
1) Determine if there is a great enough difference AMONG the three or more conditions of your study.
When making comparisons between three or more conditions, we are really looking to see if there is a difference between at least 2 of the conditions. For Example, in a study conducted by a student in last years BR class three Background Sound(BS) conditions were used. These were White noise, Ambient Noise and Music. In each condition participantsare given spatial problems to complete. The dependant variable was the number of spatial problems they are able to solve correctly in the allotted time. We will refer to these as Ravens’ Matrix Scores (RMS)
The student wanted to compare scores from each of the conditions to each other.
1)White Noiseto Ambient noise.
2)White Noise to Music
3)Ambient noise to Music
He could do three t-tests to address each of these questions, however, for each t-test he conducts he is adding an additional 5% probability of making a Type I error. Over these three comparisons, he would have a .05 + .05 + .05 probability of making at least one Type I error. In other words, if he found significant difference between two conditions, he would really be only 85% certain that this is real difference and not simply due to chance. There alpha level for this set of comparisons is actually .15. This is much too high.
One way ANOVAs allow us to determine the likelihood of finding a difference between at least two of the conditions. The p value for a one-way ANOVA refers to the overall probability that at least one of the conditions will differ from at least one other condition. We want this probability .05 or less.
A one-way ANOVA is used to compare any number of group means and still maintain the probability of making a Type I error at 5%. An ANOVA is called an omnibus test becauseit looks at the amount the whole set of means differ from each other and determines if that pattern of differences is likely to have occurred less than 5 % of time by chance alone. If the entire set does not have a p value of greater than .05, than no one individual comparison can either.
Scientific Hypothesis - there are differences between at least 2 of the groups. (One of these things is not like at least one of the others).
Null Hypothesis – There are no differences among the groups.
The logic of this test is very simple (the mathematics is more complex – but the computer does it). Here is the basic idea.
Review: When we discussed Measures of Dispersionlast term, we talked about variance and standard deviations. Both of these are measures of how spread out the scores are in a distribution. Recall the variance is just the square of the standard deviation.
Sources of Variance
Each participant’sRMS can be seen as effected by two influences. First, all participants are individuals and come into the study with characteristics that are unique to them. For example, some might be in nasty moods while others may be poor test-takers. Variation due to differences between participant’s characteristics is called error variance. Error variance is the variation we would expect to find among any group of participants due simply to the fact that we have a group of people that differ from each other in innumerable ways. These differences effect their measurement on the DV but should effect the measurement of the DV about the same in each condition. The important thing to remember here is that error variance is variation that occurs among subjects that is not due to differences in the condition they are participating in.
The second possible source of variation among participants’ RMS is the influence of the different BSconditions. Thus, each participant’s RMS can be seen to be due to two separate influences. Error variance (characteristics of the individual) and variance due to the independent variable, BS. Assume that there is no effect of BS. We would still not expect to obtain exactly the same mean in each condition. There would be some variation between conditions just due to error variation. Since we are able to estimate Error variance we have an estimate of how much the means of the conditions are expected to vary from each other, due to chance alone.
When we did t-tests we were able to use the variance in each sample to estimate the variance we would expect to find between samples, if we repeatedly sampled from the same population. With t- tests that estimate was called the Standard Error of the Mean (). One way ANOVAs SPSS uses the variances from all the conditions to estimate of how much variation there is in the DV ratings that is due to factors other than the IV. In the Background Sound study there are three conditions, so SPSS finds the averaged variance within the three conditions which is called Within Group Variation. Within group variation, thus, can be seen as an estimate of how much variation we would expect to find between scores if the IV has no effect.
Next, we need to measure the amount of between groups variance. Conceptually, this is just the average amount the means of each condition vary from the mean of the entire group of scores. The way this is computed is pretty complex, but for this class we do not need to worry about it (SPSS will do it for us). What you need to know is the logic of what SPSS is doing. The means obtained from each condition differ from the Grand Mean due to both random variation and to variation due to the independent variable.
Between Group Variance = Error Variance + Variance Due to the Independent Variable
If there is no effect of the IV, then variance added due to the IV will be zero. The equation above will become
Between Group Variance = Error Variance + 0.
Or
Between Group variance = Error variance.
ANOVAs produce a statistic called an F ratio or F statistic. It is simply
If there is no effect of the IV, the numerator of this ratio should be very close to the denominator. Therefore, if there is no effect of the IV we expect the F ratio to equal 1 (any number divided by itself is 1). Both the numerator and the denominator of this ratio however are estimates, and we would expect there to be some difference between them simply due to chance. Remember, chance is predictable. If there is no effect due to the IV, then we will generally obtain an F ratio that is very close to 1. As the F ratio becomes larger, it is less likely that the difference was obtained solely due to chance. SPSS provides the probability of obtaining the F ratio you have, due to chance alone. This is the p value and it is provided in the SIG Column of the SPSS print out. If p<.05, then we can conclude that the variation among our groups is unlikely to be due to chance alone, and that the IV has an effect.
A significant one-way ANOVA (p < .05) tells us there is something other than chance influencing the differences among the scores we obtained. If the p value is not significant (p > .05) we must accept the null hypothesis, that there is no difference among our conditions. If we have controlled all influences other than error variance, that influence must be the IV. What a one-way ANOVA does not tell us, however, is which conditions are different from which others.
Step 2) The next step is to compare each condition to the other conditions individually. There are a large number of tests that do this. They are called Post hoc Multiple Comparisons and essentially they are all t-tests, with slight modifications. There are many reasons you might choose one of these test over the other (you will learn these reasons in GraduateSchool). Here we will simply use one of the most common tests for looking at differences between individual conditions.
For a within subject design the test is called a Least Squares Difference test, or an LSD. SPSS (if you direct it to) will provide an LSD test comparing each condition to each other condition. In the
Background Sound study there were three LSD’s conducted. EachLSD has a p value associated with it. You determine whether the difference between any two conditions is significant, by comparing the p value obtained to our alpha level (.05). For abetween subjects design you simply run follow up t-tests (just like in lab 5).
Q. Why do we need to run a one-way ANOVA first? The one-way ANOVA limits the chance of making a type one error to 5% for all the comparisons we are making. In the Background Sound Study the ANOVA controls for the fact that the researchers are making six comparisons. If the one-way ANOVA is significant, then we can be 95% certain that the difference AMONG the entire set of conditions is not simply due to chance. Therefore, we can also be 95% confident that the difference between any two of the means is not simply due to chance.
Q. What if my one-way ANOVA is not significant? Can I still do multiple comparisons? NO! If you do, you may well find significant differences, but they are not really significant. They are most likely due to the fact that you are allowing a 5% chance of making a Type I error with each comparison. Remember when comparing three conditions to each other, you really have a 15% chance you will find at least one significant difference among at least two of the condition means due solely to chance. If the one-way ANOVA is not significant, you do not interpret the multiple comparisons. You simply conclude that your IV did not have a significant effect on your DV -- that there was no significant difference found between your conditions.
Between Group and Within Groups Designs.
Just like with t-tests, the type of one-way ANOVA you run for a Within subjects design is different than for a Between Groups Design. When we run a Within Subjects design, the same subjects serve in all conditions of the study. Each group is equally affected by individual difference variables such as age, IQ, sex, income, family background, pervious experience etc. Any difference between the conditions can not be due individual differences (THEY ARE THE SAME PEOPLE IN EACH CONDITION). For a Between Subjects design you are comparing different groups of people to each other. Error variance for a Within Subject design therefore is much lower than for a Between Subjects design.
Therefore, the same difference between means might be significant if found using a within subjects design but not significant using a between subjects design. This makes the Within Subjects design much more powerful. (i.e., you have a much greater chance of finding a significant difference between conditions, when a real difference actually exists).
Lab 6
For this Lab you will find two sets of data. The first is called Lab6a and the second is called Lab6b. The data contained in LAB6a will be used to run a Between Subjects one-way ANOVA. The data in Lab6b will be used to run a Within Subjects ANOVA. SPSS refers to a within subjects ANOVA as a repeated measures design. This is because you are taking repeated measures from the same individual under different conditions.
Between Subjects One-way ANOVA.
The data for this analysis come from a study conducted by Corston and Colman (1996). Corston and Colman wanted look at the effects of audiences on females’ performance of a motor task. Since they were only interested in generalizing these results to women, only women were included as participants in this study. The motor task these women performed involved using a computer mouse to keep the cursor on a moving target presented on a computer screen. The women who participated in the study were randomly assigned to one of three conditions in which they performed the task either alone (with no audience), in the presence of a female audience, or in the presence of a male audience. The percentage of time spent on target in the three audience conditions were compared. Higher On-Target Percentages (OTPs) indicate better performance on the motor task and thus better motor skills.
The data entered into LAB6a contains the following variables. The first column contains the subject ID number. It is important when running a between subject’s design to be sure each row contains scores from different subjects. The second column indicates the audience condition the subject served in. SPSS refers to the IV as the Factor in the analysis. Remember, the word factor is just another name for IV. I have defined this variable for you. I named it AUDIENCE and then define the three levels (1 = alone; 2= female audience; 3 = male audience). The third column contains the percentage “on target scores “. Use the define variable menu to name this variable SCORES.
Procedure
1. From the menu at the top of the screen click on: Analyze, then click on Compare means then on One-way ANOVA.
2. Click on your dependant variable and move it into the box marked Dependant list by clicking on the arrow button.
3. Click on your independent variable and move it into the box labeled factor.
4. Click the Options button and click on Descriptives (this will instruct SPSS to provide descriptive statistics for each condition) and Homogeneity of Variance (this will instruct SPSS to include a Levene test of homogeneity of variance for your conditions). Then click on Continue.
5. Click on the button marked POST HOC. Click on LSD. This will instruct SPSS to conduct post hoc multiple comparisons for you using LSD tests. There are several multiple comparison tests available from SPSS. Be sure that you have selected only LSD or your output will provide a lot of stuff that you will not know what to do with.
6. Click onContinue and then on OK.
Output
Your output fie will contain four tables. The first provides the sample sizes (N), Means, Standard Deviations, standard error, Minimum and Maximums for each condition and for the entire data set (total). You should make sure that your minimum and maximum scores make sense. For this data set, the dependant variable is measured in percentages, therefore, the minimum can not be less than zero and the maximum can not be more than 100. If they are, this indicates that an error has been made in entering the data. You should also make sure that there are means only for the conditions you expect. If, in error. I had entered a 5 in the Audience column, SPSS would treat this as a separate condition, and it would severely mess up your results.
The second table provides the results of the Levene Test of homogeneity of variance. Like the t-test, in order for this test to be valid the variances between conditions should not statistically differ from each other. If there is a significant difference between the variances of the conditions the p value for this test will be less than .05. If you have equal numbers of subjects in each of your conditions, violation of the assumption of homogeneity of variance is not a problem and you can ignore it. In analyzing your study data, if the Levene test is significant and you do not have equal numbers of subjects in each condition, you should come see me and I will tell you how to deal with it. Don’t panic, it is an easy problem to deal with.
The third table provides the output for the ANOVA. The three columns that are important to you are the degrees of freedoms (df), the F statistic and the p value (SIG column). If the p value is less than .05 your One Way ANOVA is significant. This means that you can safely interpret the multiple comparisons provided in the next table. SPSS also provides Sum of Scores for Between Groups, Within Groups and Total. These Sums of Squares are used for determining the effect size for the One-way ANOVA.