Factorial MANOVA
MANOVA is used when you want to integrate multiple dependent variables into a factorial analysis. The procedure constructs a new “canonical variate” (a weighted linear combination of the DVs) for each effect that maximally discriminates among the conditions of that effect, then tests whether the discrimination (mean difference) is significant.
I like to start the analysis with separate factorial ANOVAs for each DV, to get an idea of the differential patterns of effects for the different DVs. Also, it is always good to know the bivariate effects, so you can determine whether or not the multivariate results have any “surprises.” Sometimes, if there are no interesting multivariate effects, it may be preferable to present the individual factorial ANOVAs, rather than the factorial MANOVA. Considerations such as your audience and the importance and representativeness of your individual DVs would all be part of this decision. SPSS/GLM gives these results in a very nice and easy-to-use form.
The next stage is to get the multivariate tests. SPSS/GLM also does this very nicely. However, when it comes to interpreting the multivariate effects and performing multivariate follow-ups, SPSS/GLM isn’t very helpful. The good news is that SPSS/MANOVA gives us some additional information, and we can write a bit of code to get a very useful approximation of the rest!
One challenge to these analyses is that there is a lot of output, and some of the tables are pretty large. So, this runs on a bit… Also, this example works with a very small sample; as with other multivariate analyses, larger sample sizes that give both statistical power and ample stability should be used whenever possible!
The Study:
The study sought to explore how different combinations of reinforcement (praising correct responses, criticizing incorrect responses, or no feedback) and task difficulty (easy or hard) on initial trials would influence three outcome variables, number of problems attempted, number of problems correct, and problem-completion speed on subsequent trials. Participants completed 20 “training trials” which were designed to produce a pattern of 10 correct and 10 incorrect answers. Whether the task was presented as “easy” or “hard” and the type of reinforcement given were randomly assigned. Participants were then given an opportunity to complete as may trials as they wanted, and were paid $5 for each trial on which they got the problem correct. Verbal feedback was not given on these trials.
Using GLM/multivariate to get univariate analyses and multivariate tests:
While it is possible to use GLM/univariate to get separate factorial ANOVAs for each DV, using GLM/multivariate organizes the output in a way that makes it easier to compare the effect patterns across DVS for each main effect and the interaction. I’m going to show you the output in a slightly different order than it comes out of SPSS. We’ll look at the univariate stats for each the DVS for each effect first and then look at the multivariate tests.
Univariate tests of individual DVs
SPSS code:
GLM numtry numcor speed BY task rein ç identifies DVs and IVs
/METHOD=SSTYPE(3)
/EMMEANS=TABLES(task) ç gets marginal means (k=2- follow-ups not necessary)
/EMMEANS=TABLES(rein) compare (rein) ç gets marginal means and follow-up analyses
/EMMEANS=TABLES(task*rein) compare (rein) ç gets SE of Rein for each Task difficulty (with follow-ups)
/EMMEANS=TABLES(task*rein) compare (task). ç gets SE of Task for each Rein type (with follow-ups)
Please note: SPSS includes confidence intervals that correspond to each of the means and means comparison analyses. In order to same some space, I’ve not included those in the output below.
Univariate significance tests:
Number of items attempted (numtry):· ME for Rein
· No ME for Task
· No Interaction
Number of items correct (numcor):
· ME for Rein
· ME for Task
· Interaction
Problem completion speed (speed):
· ME for Rein
· ME for Task
· Interaction
Looking at the Task main effect for each DV:
numtry marginal means look very similar.numcor & speed both have substantial mean differences, and both with simple > complex
So, thinking multivariately, we would anticipate that numcor and speed to be part of the task ME canonical variate, and participants in the simple condition to have higher scores on that variate.
Looking at the Rein main effect for each DV:
While there is a main effect for all 3 DVS, they do not all have the same pattern.numtry and speed both have the pattern è silence = praise > criticism
numcor has the pattern è praise = criticism > silence
So, thinking multivariately, we would anticipate a diffuse structure for this main effect, with two canonical variates each with different contributing variables and different marginal mean difference patterns.
Looking at the interaction for each DV:
I’ve decided to use the SE of task for each type reinforcement, but equivalent results are obtained if you use the SE of rein for each task. Ultimately you’ll need to look at both the check on the descriptiveness of each main effect, but to save a tree or two I’ll just present the one set of SEs. Similarly, I’ve left out the simple effect F-tests that SPSS produces for the interaction along with the pairwise comparisons.
There is no interaction for numtry. Numcor and speed have somewhat different simple effects patterns.For numcor è simple > complex for each type of reinforcement
For speed è simple > complex only for criticism
So, thinking multivariately, we would anticipate a diffuse structure for this interaction, with two canonical variates one involving numcor and the other involving speed. We expect that these variates will show different SE patterns from each other.
On to the multivariate analyses…
Multivariate Significance Tests
MANOVA assumes that the variance of each DV is the same across effect conditions and that the covariance among the DVs is the same across those conditions as well. Box’s test examines both of these assumptions. The desired outcome is to retain H0”:, support the homogeneity assumptions, as we found here. / While Wilk’s test is the most commonly used, Pillai’s is usually preferred for smaller.sample sizes, unequal sample sizes and if there are violations of the homogeneity assumption.Notice that while the Rein main effect and the interaction both could have diffuse structures, GLM only provides a single test of whether that effect is “significant”.
There is much we still don’t know from these GLM results…
· What is the composition of the canonical variates that are the basis of the multivariate tests just above? We can make some guess from the univariate results about the more and less important variables in each. But we also know better than to assume a direct relationship between bivariate and multivariate results – several kinds of collinearity effects could be operating. Said differently, we don’t have a description of these multivariate effects, we only know that they are “significant”.
· Both the Rein main effect and for the interaction involve multiple conditions, and, so, could have concentrated or diffuse structures. We don’t know whether one canonical variate is sufficient to maximally differentiate among the conditions, or if additional variates would contribute to our understanding of the pattern of condition differences for these effects?
· We know that the Rein main effect and the interaction are “significant.” But we don’t know the pattern of the mean differences of the canonical variate(s) for those effects. Said differently, we don’t have a description of these multivariate effects, we only know that they are “significant”.
Fortunately, there are ways to get more information out of SPSS…
Further Analyses of the Canonical Variates
Although not available via the SPSS GUI, the older MANOVA procedure can be accessed using syntax code to give use several additional pieces of information to help us further understand our results. Many textbooks and web sites suggest using discriminant function to obtain these same results. However, few statistical packages perform a factorial ldf.
SPSS Code:
manova numtry numcor speed BY task (1,2) rein (1,3) ç identifies the DVs & IVs
/ print = signif (multiv, eigen, dimenr) ç gets several multivariate tests & statistics
/ discrim stan cor. ç gets the weights to interpret and construct the canonical variates
To save a tree or two, I’ve left of the univariate tests that are part of this output. They are exactly the same as obtained from GLM. I’ve also left off the SE F-tests, because they are redundant with the pairwise follow-ups.
Examining the Interaction of Rein & Task:
EFFECT .. task BY rein
Multivariate Tests of Significance (S = 2, M = 0, N = 10 )
Test Name Value Approx. F Hypoth. DF Error DF Sig. of F
Pillais .79777 5.08745 6.00 46.00 .000
Hotellings 3.70942 12.98296 6.00 42.00 .000
Wilks .21057 8.64768 6.00 44.00 .000
Roys .78718
Note.. F statistic for WILKS' Lambda is exact.
Eigenvalues and Canonical Correlations
Root No. Eigenvalue Pct. Cum. Pct. Canon Cor.
1 3.69871 99.71123 99.71123 .88723
2 .01071 .28877 100.00000 .10295
Dimension Reduction Analysis
Roots Wilks L. F Hypoth. DF Error DF Sig. of F
1 TO 2 .21057 8.64768 6.00 44.00 .000
2 TO 2 .98940 .12319 2.00 23.00 .885
EFFECT .. task BY rein (Cont.)
Standardized discriminant function coefficients
Function No.
Variable 1
numtry -.15606
numcor .73197
speed .71367
Correlations between DEPENDENT and canonical variables
Canonical Variable
Variable 1
numtry .02545
numcor .65955
speed .73031
We know that the canonical variate that describes the Rein x Task interaction is made up of numcor and speed (weighted in the same direction). What we don’t know is the pattern of that interaction for this canonical variate. To get that we will need to construct the canonical variate, and then use it in a factorial ANOVA.
First we need to obtain z-score versions of each DV, then calculate a canonical variate score using the Beta weights shown above.
DESCRIPTIVES VARIABLES=numtry numcor speed ç defines the DVs
/SAVE. ç asks for the construction of Z-score versions of each
compute CV_int = (Znumtry * -.15606) + ( Znumcor * .73197) + (Zspeed * .71367).
GLM CV_int by rein task
/ EMMEANS = tables (rein*task) compare (rein). ç only want to examine the interaction effect
The interaction F-test will not match the F-values from the multivariate analyses, because they apply the df differently, the error term is calculated somewhat differently, and this model only includes the significant first canonical variate. Usually we will not report this interaction F-value The purpose of the analysis is to obtain the cell means for the canonical variate and the follow-ups, to allow us to describe the pattern of the simple effects making up this significant multivariate interaction.
Using this canonical variate, that is a combination of numcor and speed, we find no SE of Task for Praise, while for both Criticism and Silence, Easy tasks were performed significantly better than complex tasks. We would probably also note that the task effect is much larger for Criticism than for Silence (we could use mean difference or effect size differences to make this case).
Examining and “Checking” the Main Effect of Rein
As with the interaction, we know there is a significant multivariate main effect for Rein, but we do not know which DVs contribute to the canonical variate, nor do we knows the pattern of marginal mean differences for this effect. Also, because we have a significant interaction, we need to determine whether the multivariate main effect for Rein is descriptive for both Simple and Complex tasks.
EFFECT .. rein
Multivariate Tests of Significance (S = 2, M = 0, N = 10 )
Test Name Value Approx. F Hypoth. DF Error DF Sig. of F
Pillais 1.20210 11.55041 6.00 46.00 .000
Hotellings 9.25211 32.38238 6.00 42.00 .000
Wilks .07091 20.20539 6.00 44.00 .000
Roys .89812
Note.. F statistic for WILKS' Lambda is exact.
Eigenvalues and Canonical Correlations
Root No. Eigenvalue Pct. Cum. Pct. Canon Cor.
1 8.81537 95.27957 95.27957 .94769
2 .43674 4.72043 100.00000 .55134
Dimension Reduction Analysis
Roots Wilks L. F Hypoth. DF Error DF Sig. of F
1 TO 2 .07091 20.20539 6.00 44.00 .000
2 TO 2 .69602 5.02250 2.00 23.00 .015
Standardized discriminant function coefficients
Function No.
Variable 1 2
numtry .93466 -.09440
numcor -.57407 .91311
speed .65244 .47285
Correlations between DEPENDENT and canonical variables
Canonical Variable
Variable 1 2
numtry .59336 .20055
numcor -.20073 .86880
speed .50607 .47715
In order to explore the pattern and descriptiveness of this multivariate main effect, we have to compute the canonical variates, then employ them in ANOVAs with follow-ups.
compute CV_rein1 = (Znumtry * .93455) + (Znumcor * -.57407) + (Zspeed * .65244). ç compute 1st canonical variate
compute CV_rein2 = (Znumtry * -.09440) + (Znumcor * .91311) + (Zspeed * .47285). ç compute 2nd canonical variate
.
GLM CV_rein1 by rein task ç getting results for 1st canonical variate
/ EMMEANS = table (rein) compare (rein) ç follow-ups to describe pattern of Rein main effect
/ EMMEANS = tables (task*rein) compare (rein) ç follow-ups to check descriptiveness of Rein main effect
/ print descriptives.
GLM CV_rein2 by rein task ç getting results for 1st canonical variate
/ EMMEANS = table (rein) compare (rein) ç follow-ups to describe pattern of Rein main effect
/ EMMEANS = tables (task*rein) compare (rein) ç follow-ups to check descriptiveness of Rein main effect
/ print descriptives.
Describing the Main effect of Rein
Showing just the parts of the output that directly apply…
For the 1st canonical variate…For the 2nd canonical variate …
For the main effect of Rein, the first canonical variate is combination of numtry and speed, with Silence producing the best performance, Praise having intermediate performance and Criticism leading to the poorest performance.
The second canonical variate, which is a combination of numcor and speed, had the best performance from Praise, intermediate performance from Silence, and the poorest performance from Criticism.
There were two “surprises” in these effects – highlighting the value of multivariate analyses:
· Silence yielding the best performance on the first variate
· Speed being multi-vocal to both variates