SPSS ANOVA & MANOVA 1
SPSS ANOVA & MANOVA Assignment
Deborah Davis
Liberty University
Purpose
The purpose of this document is to provide a response to the fourth SPSS assignment that covers the use and understanding of the ANOVA and MANOVA test programs in statistics. The research scenario was given as a part of the assignment, and presumes leaders of colleges and universities are seeking ways to develop and maintain online and blended programs (Szapkiw, 2014, p. 2). This particular assignment required the use of SPSS to “choose the appropriate tests . . . run the tests and analyze the data” (Szapkiw, 2014, p. 2).
Structure
In that there are two research questions to this assignment and seven sections to each of them, and it is a small college paper, it will only use the first two levels of headings, and will be separated within each aspect.
Research Question 1
Is there a statistically significant difference between students’ total sense of community as measured by the Classroom Community Scale (CCS, Rovai, 2002) based on the type of three different mediums (video conferencing, chat systems, and Blackboard discussion forums)they used for class discussions in an online statistics course?
Null Hypothesis
There is no statistically significant difference between students’ total sense of community as measured by the Classroom Community Scale (CCS, Rovai, 2002) based on the type of three different mediums (video conferencing, chat systems, and Blackboard discussion forums) they used for class discussions in an online statistics course.
Alternative Hypothesis
There is astatistically significant difference between students’ total sense of community as measured by the Classroom Community Scale (CCS, Rovai, 2002) based on the use of video conferencing as opposed to chat systems or Blackboard discussion board forums when used for class discussions in an online statistics course.
Alternative Hypothesis
There is astatistically significant difference between students’ total sense of community as measured by the Classroom Community Scale (CCS, Rovai, 2002) based on the use of chat systems as opposed to video conferencing or Blackboard discussion board forums when used for class discussions in an online statistics course.
Alternative Hypothesis
There is a statistically significant difference between students’ total sense of community as measured by the Classroom Community Scale (CCS, Rovai, 2002) based on the use of Blackboard discussion board forums as opposed to chat systems or video conferencing when used for class discussions in an online statistics course.
Independent Variables – Communication Medium
Blackboard Discussion Board forums (1)
Chat systems (2)
Video conferencing (3)
All are nominally measured as there is no actual measure nor score nor rating in the three systems.
Dependent Variables – Community Connectedness Scale
Students’ total community score as measured by the Classroom Community Scale (Rovai, 2002): this is an interval/ratio measure as it contains a score which can be measured and can contain any numeral including zero.
Test to be Used
“A one-way between subjects (between-S) ANOVA is used in research situations where the researcher wants to compare means on a quantitative Y outcome variable across two or more groups” (Warner, 2008, p. 215). Since there is one dependent variable that is continuous and an independent variable with three independent levels, the one-way between groups ANOVA would be the appropriate statistical analysis. The term one-way indicates that there is only one independent variable while between groups indicates multiple groups within the dependent variable (Szapkiw, 2012, p. 26).
Assumptions
Warner (2008) explains the assumptions required as similar to those required for an independent samples t test. The quantitative dependent variable should be quantitative in nature, using an interval level of measurement. Scores should be distributed normally and observations should be independent. These elements are examined using three assumption tests that need to be completed: outliers, normality, and homogeneity. A box plot analysis showed no evidence of extreme outliers. The use of a histogram showed a regular bell-shaped curve which provides evidence of normality. The Levene’s test showed a p < .001 and “A significance level less than .05 means that variance cannot be assumed; that is, the assumption is not tenable” (Szapkiw, 2012, p. 26). This was the most appropriate analysis as it assists researchers in examining the strength and direction of the linear relationship between two variables, while controlling for another variable (Warner, 2008, p. 218).
See Figure 1.
Levene’s test of Homogeneity of VarianceLevene Statistic / df1 / df2 / Sig
14.54 / 2 / 37 / .00
Figure 1.
Descriptive Statistics
This assignment required elements to be presented from the data set provided. An SPSS analysis of the mean, standard deviation, and group number for community connectedness was performed against discussion board (M = 18.43, SD = 5.72, n = 14), chat systems (M = 25.42, SD = 8.67, n = 12) and video conferencing (M = 37.00, SD = 1.92, n = 14) and then against the whole population (M = 27.03, SD = 9.82, N=40). While not required, results are also provided in Table 1 for clarity.
Table 1.
Descriptive Statistics for Community Connectedness and Methods of Communication (N = 40)
Group / n / M / SDDiscussion Board / 1 / 14 / 18.43 / 5.72
Chat / 2 / 12 / 25.42 / 8.67
Video Conferencing / 3 / 14 / 37.00 / 1.92
Total / 40 / 27.03 / 9.82
Results
The tests reflect that it is appropriate to reject the null hypothesis: There is not a significant difference between students’ total sense of community as measured by the Classroom Community Scale (CCS, Rovai, 2002) based on the type of medium (video conferencing, chat systems, or Blackboard discussion boards) used for class discussions in the online Introduction to Statistics course. Tests also reflect that Alternative Hypothesis 1 is valid: There is a significant difference between students’ total sense of community as measured by the Classroom Community Scale (CCS, Rovai, 2002) based on the use of video conferencing as opposed to chat systems or Blackboard discussion boards when used for class discussions in the online Introduction to Statistics course.
Since the assumption of homogeneity of variances was violated, a more robust test of equality of means was used providing results of both Welch F(2, 27.97) = 70.90, p = 0.00, and Brown-Forsythe F(2, 20.28) = 32.23, p = 0.00. This also reflected a significance value of 0.00 but suggests the post hoc equal variances not assumed test (e. g. Games-Howell) would be more valid due to the small n’s and the inequality between them. See Table 2.
Table 2
Robust Tests of Equality of MeansCommunity_Connectedness_Scale
Statistica / df1 / df2 / Sig.
Welch / 70.900 / 2 / 17.968 / .000
Brown-Forsythe / 32.226 / 2 / 20.282 / .000
a. Asymptotically F distributed.
A review of the ANOVA F (2, 39) = 34.98, p 0.001 reflects a significance of less than 0.001. Since this is less than 0.05, there is a statistically significant result somewhere in the groups (Warner, 2008, p. 239).
Since the overall ANOVA was significant, a review the multiple comparisons table is required. This would not be needed were the overall ANOVA not significant. Tests revealed significant pairwise differences between the mean scores of community connectedness for those who participated via video conferencing and both students communicating via discussion forums and chat systems, p < .05. Students who communicated via discussion forums and chat systems reflected no significant difference in community connectedness. Post hoc comparisons to evaluate pairwise differences among group means were conducted. Significant pairwise differences between all groups were determined, but at different levels:between Blackboard Discussion Board Forum and Chat systems (p <.005); between Blackboard Discussion Board Forum and video conferencing (p <.001); and, between Chat systems and video conferencing (p <.001). Video conferencing students had significantly higher sense of community than those using discussion boards and chat. Chat students had significantly higher sense of community than discussion boardstudents.
Thus a comparison of the mean scores between the groups as reflected in the means plot appears dramatic, but a review of the means in the descriptive table reflects a less dramatic variance. See Figure 2.
Figure 2.
Effect Size
The effect size was computed using eta squared. The formula for eta squared is the sum of squares between groups divided by the total sum of squares. In this case, that would be 2456.63 divided by 3758.98 giving an eta squared value of 0.65 which is then evaluated based on Cohen (1988) as a standard. Effect size, based on Cohen (1988), was large, η2= .64. The strength of relationship between type of communication system and sense of community score was strong (power of 1.00 indicates 100% accuracy), accounting for 64 % of the variance of the dependent variable.
Post Hoc
Post hoc comparisons to evaluate pairwise difference among group means were conducted with the use of Tukey HSD test through equal variances were found untenable. Tests revealed significant pairwise differences between the mean scores of students who used video conferencing and students who used either discussion board or chat systems, p < .05. Students who used discussion board and chat systems did not significantly differ from each other.
Research Question 2
Is there a statistically significant difference in students’ total sense of community as measured by the Classroom Community Scale (CCS, Rovai, 2002) and perceived learning as measured by the Perceived CAP Learning Scale (Rovai et al. 2009) based on the type of medium (video conferencing, chat systems, or Blackboard discussion boards) used for class discussions in the online Introduction to Statistics course?
Null Hypotheses
There is not a significant difference between students’ total sense of community as measured by the Classroom Community Scale (CCS, Rovai, 2002) and perceived learning as measured by the Perceived CAP Learning Scale (Rovai et al. 2009) based on the type of medium (video conferencing, chat systems, or Blackboard discussion boards) used for class discussions in the online Introduction to Statistics course.
This hypothesis can be further broken into two sub-hypotheses based on the dependent variables. There is not a significant difference in students’ total sense of community as measured by the Classroom Community Scale (CCS, Rovai, 2002) based on the type of medium used(video conferencing, chat systems, or Blackboard discussion boards)for class discussions in the online Introduction to Statistics course. Also, there is no significant difference in students’ perceived learning as measured by the Perceived CAP Learning Scale (Rovai et al., 2009) based on the type of medium(video conferencing, chat systems, or Blackboard discussion boards)used for class discussions in the online Introduction to Statistics course.
Alternative Hypothesis
There is a significantdifference between students’ total sense of community as measured by the Classroom Community Scale (CCS, Rovai, 2002) and perceived learning as measured by the Perceived CAP Learning Scale (Rovai et al. 2009) based on the use of video conferencing as opposed to chat systems or Blackboard discussion boards when used for class discussions in the online Introduction to Statistics course.
Alternative Hypothesis
There is a significant difference between students’ total sense of community as measured by the Classroom Community Scale (CCS, Rovai, 2002) and perceived learning as measured by the Perceived CAP Learning Scale (Rovai et al. 2009) based on the use of chat systems as opposed to video conferencing or Blackboard discussion boards when used for class discussions in the online Introduction to Statistics course.
Alternative Hypothesis
There is a significant difference between students’ total sense of community as measured by the Classroom Community Scale (CCS, Rovai, 2002) and perceived learning as measured by the Perceived CAP Learning Scale (Rovai et al. 2009) based on the use of Blackboard discussion boards as opposed to chat systems or video conferencing when used for class discussions in the online Introduction to Statistics course.
Independent Variables – Communication Medium
Blackboard Discussion Board forums (1)
Chat systems (2)
Video conferencing (3)
All are nominally measured as there is no actual measure nor score nor rating in the three systems.
Dependent Variables
Sense of community as measured by the Classroom Community Scale (CCS, Rovai, 2002)
Perceived learning as measured by the Perceived CAP Learning Scale (Rovai et al. 2009)
Both are interval/ratio measures as both contain a score which can be measured and can contain any numeral including zero.
Test to be Used
“In a one-way MANOVA, mean scores on multiple quantitative outcome variables are compared for participants across 2 or more groups” Warner, 2008, p. 702). Since there aretworelated dependent variables that are continuous and an independent variable with three independent levels, the appropriate test for statistical significance is one-way MANOVA(Warner, 2008, p. 702).
Assumptions
Per Warner (2008), assumptions for the MANOVA include the following: “Observations on the Y outcome variables should be collected in such a ways that the scores . . . are independent;” “Each Y outcome should be quantitative and reasonably normally distributed;” “Associations between pairs of Y variables should be linear;” and “the variance/covariance matrices for the Y outcome variables should be homogeneous across the populations” (pp. 708-709).
The dependent variables(sense of community and perceived learning) aremeasured at the interval or ratio level. The independent variables (communications medium) consist of three categorical independent groups (discussion board, chat, and video). There is independence of observations as those in each of the discussion medium groups are in a singular group only. There should be no significant outliers, which the one-way MANOVA will detect.
A box plot was used to text for extreme outliers, and no concerns were observed. A check for normality is done usinghistograms. Though“MANOVA is robust enough to handle minor violations of normality” (Szapkiw, 2012, p. 31) when the sample size is at least 20 in each cell (TabacknickFidell, 2007, p.251), there is an exception when normality is affected by outliers. In this case, there are no significant outliers, thus there are no major concerns.
Correlation matrixes are used to check assumptions of multicolinearity.The correlation matrix reflect a significant association between the dependent variables, r = .96 ,p < .01. See Table 3.
Table 3.
Correlation MatrixPerceived CAP Learning Scale / Community Scale Total
Kendall’s tau b / Perceived CAP Learning Scale / Correlation Coefficient / 1.00 / .76**
Sig. (2-tailed) / - / .00
N / 40 / 40
Community Scale Total / Correlation Coefficient / .76** / 1.00
Sig. (2-tailed) / .00 / -
N / 40 / 40
**. Correlation is significant at the 0.01 level (2-tailed).
Scatterplots are used to test the assumption of linearity. While a curvilinear line would indicate the assumption is not tenable, in this case, there is a straight line that indicates linearity. See Figure 3.
Figure 3.
The assumption of equal variance is evaluated usingLevene’s test. The results of Levene’s test of equality of error provided evidence that the assumption of homogeneity of variance across groups was not tenable for perceived learning F (2,37) = 10.57, p < .001 or sense of community F (2,37) = = 14.54, p < .001. See Table 4.
Table 4.
Levene’s Test of Equality of Error VariancesaF / df1 / df2 / Sig.
Community Connectedness Scale / 14.537 / 2 / 37 / .00
Perceived CAP Learning Scale / 10.574 / 2 / 37 / .00
Tests the null hypothesis that the error variance of the dependent variable is equal across groupsa
- Design: Intercept + Communication Medium
Without homogeneity of variances, “the researcher may prefer to report Pillai’s trace instead of Wilks’s lambda as the overall test statistic; Pillai’s trace is more robust to violations of the homogeneity of variances and covariances” (Warner, 2008, p. 710). The Box’s M test is used to evaluate the assumption of homogeneity of variance-covariance (Warner, 2008, p, 668). The Box’s M test in this case shows this test is statistically significant p < .001), therefore the assumption of homogeneity of variance-covariance matrices is violated Box’s test M = 41.86, F (6, 29158.60) = 6.43, p < .001. See Table 5.
Table 5
Box’s Test of Equality of Covariance MatricesaBox’s M / 41.86
F / 6.43
df1 / 6
df2 / 29158.60
Sig. / .00
Tests the null hypothesis that the observed covariance matrices of the dependent variables are equal across groups.a
a. Design: Intercept + Communication Medium
With groups of similar sizes, analysis can proceeddespite a violation of the homogeneity of variances-covariance matrices (Warner, 2008, p. 710).
Descriptive Statistics
This assignment required elements to be presented from the data set provided. An SPSS analysis of the mean, standard deviation, for community connectedness(M = 27.03, SD = 9.82, N = 40) for discussion board (M = 18.43, SD = 5.72, n = 14), chat systems (M = 25.42, SD = 8.67, n = 12) and video conferencing (M = 37.00, SD = 1.92, n = 14). The same analysis was performed on perceived learning (M = 37.42, SD = 13.86, N = 40) for discussion board (M = 26.07, SD = 9.19, n = 14), chat systems (M = 35.25, SD = 13.41, n = 12) and video conferencing (M = 50.64, SD = 2.95, n = 14). See Table 6.
Table 6.
Dependent Variable Descriptive Statistics Disaggregated by the Independent Variable (N = 40)
Discussion Board( n = 14) / Chat
( n = 12) / Video Conferencing
( n = 14)
Variable / M / SD / M / SD / M / SD
Learning / 26.07 / 9.19 / 35.25 / 13.41 / 50.64 / 2.95
Community / 18.43 / 5.72 / 25.42 / 8.67 / 37.00 / 1.92
Results
The tests reflect that it is appropriate to reject the null hypothesis: If the null is rejected, there must be a significant difference between students’ total sense of community as measured by the Classroom Community Scale (CCS, Rovai, 2002) and perceived learning as measured by the Perceived CAP Learning Scale (Rovai et al. 2009) based on the type of medium (video conferencing, chat systems, or Blackboard discussion boards) used for class discussions in the online Introduction to Statistics course. Tests also reflect that Alternative Hypothesis 1 is valid: There is a significant difference between students’ total sense of community as measured by the Classroom Community Scale (CCS, Rovai, 2002) and perceived learning as measured by the Perceived CAP Learning Scale (Rovai et al. 2009) based on the use of video conferencing as opposed to chat systems or Blackboard discussion boards when used for class discussions in the online Introduction to Statistics course.
The Levene’s test shows an error variance of the dependent variable for community connectedness F(2, 37) = 0.00,p = 14.54 and for perceived learning F (2, 37) = 0.00, p = 10.57. This reflects that the assumption of homogeneity of variance (as p<.05) has been violated and a more robust measure is required. The results of the MANOVA indicated a significant affect among the groups for communication medium, Pillai’s Trace = .66, F = (4, 74) = 9.09, p < .001, ,partial 2 = .33. The observed power was moderate at .99. There was a 99% chance that the results could have come out significant. Based on these results, evidence wassufficient to reject the null hypothesis and conclude that students’ total sense of community as measured by the Classroom Community Scale (CCS, Rovai, 2002) and perceived learning as measured by the Perceived CAP Learning Scale (Rovai et al., 2009) significantly differed basedon the type of medium they used for class discussions See Table 7.
Table 7.
Multivariate TestsValue / F / Hypothesis df / Error df / Sig. / Partial Eta Squared / Noncent. Parameter / Observed Powerc
Pillai’s trace / .66 / 9.09 / 4.00 / 74.00 / .00 / .33 / 36.36 / .99
Wilks’ lambda / .35 / 12.52a / 4.00 / 72.00 / .00 / .41 / 50.08 / 1.00
Hotelling’s trace / 1.856 / 16.24 / 4.00 / 70.00 / .00 / .48 / 64.94 / 1.00
Roy’s largest root / 1.85 / 34.13b / 2.00 / 37.00 / .00 / .65 / 68.26 / 1.00
Tests the null hypothesis that the observed covariance matrices of the dependent variables are equal across groups.a
a. Design: Intercept + Communication Medium
Effect Size
The effect size was determined using the formula for Wilks’ Lambda for partial Eta SquaredF (4, 72) = 13.21, p <.01, ᾐ2 = .42. This, as reflected in Cohen (1988) would indicate a strong effect for this relationship.