Heck Ch 7 28
Multilevel analysis of Achievement or Personality Scale scores
Combining a number of observed variables, e.g., 10 answers on a test to measure knowledge in a particular area or 10 conscientiousness items to form a Conscientiousness scale score, is simply a linear combination of the items, analogous to, though not quite identical to, estimating the y-intercept and slope of a trajectory across the 10 items as if they were measures of some construct at 10 different time points.
In fact, the combination that we usually perform is the intercept of that trajectory. We don’t compute a slope because the item position is thought to be irrelevant and the estimated slope would change for each different order of items. But the intercept, if the slope is restricted to be 0, will not change. That intercept is the estimated “scale score” for an individual.
7 | X X
|
6 | X X X
|
5 | X X X X
| X
4 |
|
3 |
|
2 |
|
1 |
------
C1 C2 C3 C4 C5 C6 C7 C8 C9 C10
Time
So, the above person’s estimated level of C is 5.75, for example.
This chapter is about treating individual observations from a questionnaire, for example, as Level 1 observations in a fashion similar to treating observations at each time point as Level 1 observations in a longitudinal model.
The data file is Heck_Ch7_MLM1_pp. Below is an excerpt from it.
The construct is achievement. It is indicated by three measures described as reading, math, and language test scores in the text (p. 224).
So, person 1 scored 60.28 on reading, 42.48 on math, and 43.28 on language tests. (Strange combination of values. I would expect reading and language to be closer to each other than either is to math.)
The Level 1 model for a single construct
Yijk = pjk + eijk (Eq 7.4)
Yijk is the score on the ith indicator of the construct for person j in school k.
Where i = indicator score (e.g., reading score, math score, language test score)
pjk = overall value of construct for student j in school k.
j = student
k = school (This is a 3 level model)
Note that the Level 1 model has only one parameter – an intercept. It has no slope parameter.
The Level 2 model, for now
pjk = B0k + gjk Eq 7.5
B0k is the average across students at school k
gjk is random residual of the construct for person j within school k, it’s person-specific.
The Level 3 model, for now
B0k = g00 + u0k Note that u0k is a school-specific residual. Eq 7.6
The combined model
Yijk = g00 + u0k + gjk + eijk Eq 7.7
Note that there are three random effects – one for each level.
eijk is the random effect at Level 1
gjk is the random effect at Level 2
u0k is the random effect at Level 3.
The SPSS MIXED analysis
MIXED achieve
/CRITERIA=CIN(95) MXITER(100) MXSTEP(10) SCORING(1) SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE) PCONVERGE(0.000001, ABSOLUTE)
/FIXED=| SSTYPE(3)
/METHOD=REML
/PRINT=G SOLUTION TESTCOV
/RANDOM=INTERCEPT | SUBJECT(schcode) COVTYPE(ID)
/RANDOM=INTERCEPT | SUBJECT(schcode*Rid) COVTYPE(ID).
The output
MIXED achieve
/CRITERIA=CIN(95) MXITER(100) MXSTEP(10) SCORING(1) SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE) PCONVERGE(0.000001, ABSOLUTE)
/FIXED=| SSTYPE(3)
/METHOD=REML
/PRINT=G SOLUTION TESTCOV
/RANDOM=INTERCEPT | SUBJECT(schcode) COVTYPE(ID)
/RANDOM=INTERCEPT | SUBJECT(schcode*Rid) COVTYPE(ID).
Mixed Model Analysis
[DataSet1] G:\MdbO\html\myweb\PSY5950C\Heck_Ch7_MLM1_pp.sav
Model Dimensiona // Number of Levels / Covariance Structure / Number of Parameters / Subject Variables /
Fixed Effects / Intercept / 1 / 1 /
Random Effects / Intercept / 1 / Identity / 1 / schcode /
Intercept / 1 / Identity / 1 / schcode * Rid /
Residual / 1 /
Total / 3 / 4 /
a. Dependent Variable: achieve.
Information Criteriaa
-2 Restricted Log Likelihood / 56813.385
Akaike's Information Criterion (AIC) / 56819.385
Hurvich and Tsai's Criterion (AICC) / 56819.387
Bozdogan's Criterion (CAIC) / 56843.402
Schwarz's Bayesian Criterion (BIC) / 56840.402
The information criteria are displayed in smaller-is-better forms.
a. Dependent Variable: achieve.
Model 0: Yijk = g00 + u0k + gjk + eijk Eq 7.7
Fixed Effects
Estimates of Fixed Effectsa /Parameter / Estimate / Std. Error / df / t / Sig. / 95% Confidence Interval /
Lower Bound / Upper Bound /
Intercept / 58.180714 / .251485 / 305.101 / 231.348 / .000 / 57.685849 / 58.675580 /
a. Dependent Variable: achieve.
Covariance Parameters
Estimates of Covariance Parametersa /Parameter / Estimate / Std. Error / Wald Z / Sig. / 95% Confidence Interval /
Lower Bound / Upper Bound /
Residual / 33.001058 / .633165 / 52.121 / .000 / 31.783121 / 34.265667 /
Intercept [subject = schcode] / Variance / 11.520495 / 1.664309 / 6.922 / .000 / 8.679651 / 15.291146 /
Intercept [subject = schcode * Rid] / Variance / 55.869381 / 1.948537 / 28.672 / .000 / 52.177925 / 59.821998 /
a. Dependent Variable: achieve.
Random Effects Covariance Structures (G)
Intercept [subject = schcode]a // Intercept | schcode /
Intercept | schcode / 11.520495 /
Identity /
a. Dependent Variable: achieve.
Intercept [subject = schcode * Rid]a
Intercept | schcode*Rid
Intercept | schcode*Rid / 55.869381
Identity
a. Dependent Variable: achieve.
Model 1. Effects of Person and School characteristics on ACHIEVE p. 234.
The Level 1 model, same as above
Yijk = pjk + eijk Yijk is the score on the ith indicator of the construct for person j in school k.
The Level 2 model – relationship of overall level of ACHIEVe to characteristics of the students.
pjk = B0k + B1k*gmsesjk + B2k*gmacademicjk + gjk (Eq 7.8, p. 234)
gmsesjk is the grand mean centered SES of student j within school k
gmacademicjk is the grand mean centered level of academic course work of student j within school k.
gjk is random residual of the construct for student j within school k.
Both the intercept and the slopes may be school specific.
The Level 3 model, for now – characteristics of the school
B0k = g00 + g01*gmSES_meank + u0k (Eq 7.9, p. 235)
Only the intercept is affected by gmSES_mean. gmSES_meank = overall SES level of the school.
B1k = g10 Slope of relationship of average achievement to gmses is constant across schools.
(Eq 7.10, p. 235)
B2k = g20 Slope of relationship of average achievement to gmacademicjk is constant across schools.
The intercept depends of the relationship of average achievement to gmses and gmacademic depends on the gmSES_mean of a specific school, but the Level 2 slopes are constants across schools.
The combined model
Yijk = g00 + g01*gmSES_meank + u0k + g10*gmsesjk + g20*gmacademicjk + gjk + eijk
Individual average achievement depends on the mean SES of the school and mean SES of the person and mean acadmic level of the person.
The Syntax
MIXED achieve with gmSES_mean gmses gmacademic
/print=solution testcov
/fixed = intercept gmSES_mean gmses gmacademic
/random intercept |subject(schcode) covtype(ID)
/random intercept |subject(schcode*rid) covtype(ID).
Mixed Model Analysis
[DataSet1] G:\MdbO\html\myweb\PSY5950C\Heck_Ch7_MLM1_pp.sav
Model Dimensiona // Number of Levels / Covariance Structure / Number of Parameters / Subject Variables /
Fixed Effects / Intercept / 1 / 1 /
gmses_mean / 1 / 1 /
gmses / 1 / 1 /
gmacademic / 1 / 1 /
Random Effects / Intercept / 1 / Identity / 1 / schcode /
Intercept / 1 / Identity / 1 / schcode * Rid /
Residual / 1 /
Total / 6 / 7 /
a. Dependent Variable: achieve.
Information Criteriaa /
-2 Restricted Log Likelihood / 56146.528 /
Akaike's Information Criterion (AIC) / 56152.528 /
Hurvich and Tsai's Criterion (AICC) / 56152.531 /
Bozdogan's Criterion (CAIC) / 56176.544 /
Schwarz's Bayesian Criterion (BIC) / 56173.544 /
The information criteria are displayed in smaller-is-better forms. /
a. Dependent Variable: achieve.
Fixed Effects
Estimates of Fixed Effectsa /
Parameter / Estimate / Std. Error / df / t / Sig. / 95% Confidence Interval /
Lower Bound / Upper Bound /
Intercept / 57.970429 / .197919 / 291.705 / 292.900 / .000 / 57.580900 / 58.359959 /
gmses_mean / 2.775910 / .583165 / 563.511 / 4.760 / .000 / 1.630468 / 3.921352 /
gmses / 3.314677 / .266898 / 2480.822 / 12.419 / .000 / 2.791312 / 3.838043 /
gmacademic / 2.339894 / .134532 / 2658.163 / 17.393 / .000 / 2.076096 / 2.603693 /
a. Dependent Variable: achieve.
So achievement is positively related to school-level SES, to student SES, and to the academic orientation of the student’s schedule.
Covariance Parameters
Estimates of Covariance Parametersa /Parameter / Estimate / Std. Error / Wald Z / Sig. / 95% Confidence Interval /
Lower Bound / Upper Bound /
Residual / 33.000597 / .633149 / 52.121 / .000 / 31.782690 / 34.265175 /
Intercept [subject = schcode] / Variance / 5.499493 / 1.009772 / 5.446 / .000 / 3.837355 / 7.881580 /
Intercept [subject = schcode * Rid] / Variance / 42.983447 / 1.573672 / 27.314 / .000 / 40.007167 / 46.181144 /
a. Dependent Variable: achieve.
There is significant remaining variability in individual achievement scores around each student’s overall score and significant variability in students’ overall scores around predictions based on the person and school-level variables.
Model 2. A random school-level slope. P. 238
This model investigates whether the relationship of average achievement to students’ academic orientation (gmacademic) varies randomly across schools.
The Level 1 model (as above, in Model 1)
Yijk = pjk + eijk Yijk is the score on the ith indicator of the construct p for person j in school k.
The Level 2 model – characteristics of the students (as above, in Model 1)
pjk = B0k + B1k*gmsesjk + B2k*gmacademicjk + gjk (Eq 7.8)
gjk is random variation of the construct p of person j within school k, person-specific..
This is the same model as previously.
The Level 3 model, for now – characteristics of the school (again, as above, in Model 1)
B0k = g00 + g01*gmSES_meank + u0k (Eq 7.9, p. 235)
Only the intercept is affected by school level gmSES_mean as before.
B1k = g10 (Again, same as above in Model 1)
B2k = g20 + u2k (OK, this is different) (Eq 7.11, p. 238)
The red term is the new term here. The previous model assumed that the relationship of average achievement to gmacademic was the same for all students. This model allows it to vary randomly from school to school.
The combined model
Yijk = g00 + g01(gmSES_mean)k + u0k + g10(gmses)jk + (g20 + u2k) (gmacademic)jk + gjk + eijk
Yijk = g00 + g01*gmSES_meank + u0k + g10*gmsesjk + g20*gmacademicjk + u2k *gmacademicjk + gjk + eijk
The key dialog boxes
MIXED achieve WITH gmses_mean gmses gmacademic
/CRITERIA=CIN(95) MXITER(100) MXSTEP(10) SCORING(1) SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE) PCONVERGE(0.000001, ABSOLUTE)
/FIXED=gmses_mean gmses gmacademic | SSTYPE(3)
/METHOD=REML
/PRINT=G SOLUTION TESTCOV
/RANDOM=INTERCEPT gmacademic | SUBJECT(schcode) COVTYPE(UN)
/RANDOM=INTERCEPT | SUBJECT(schcode*Rid) COVTYPE(VC).
Model 2, p. 238
Yijk = g00 + g01*gmSES_meank + u0k + g10*gmsesjk + g20*gmacademicjk + u2k *gmacademicjk + gjk + eijk
Mixed Model Analysis
[DataSet2] G:\MdbO\html\myweb\PSY5950C\Heck_Ch7_MLM1_pp.sav
Fixed Effects
Estimates of Fixed Effectsa /Parameter / Estimate / Std. Error / df / t / Sig. / 95% Confidence Interval /
Lower Bound / Upper Bound /
Intercept / 58.051692 / .197206 / 288.010 / 294.370 / .000 / 57.663544 / 58.439841 /
gmses_mean / 2.955507 / .587556 / 562.187 / 5.030 / .000 / 1.801434 / 4.109580 /
gmses / 3.262756 / .266675 / 2478.883 / 12.235 / .000 / 2.739828 / 3.785684 /
gmacademic / 2.503287 / .152149 / 158.339 / 16.453 / .000 / 2.202784 / 2.803791 /
a. Dependent Variable: achieve.
Covariance Parameters
Estimates of Covariance Parametersa /Parameter / Estimate / Std. Error / Wald Z / Sig. / 95% Confidence Interval /
Lower Bound / Upper Bound /
Residual / 33.000587 / .633149 / 52.121 / .000 / 31.782681 / 34.265164 /
Intercept + gmacademic [subject = schcode] / UN (1,1) / 5.298109 / 1.011484 / 5.238 / .000 / 3.644303 / 7.702421 /
UN (2,1) / .893084 / .478497 / 1.866 / .062 / -.044752 / 1.830920 /
UN (2,2) / .733293 / .440893 / 1.663 / .096 / .225680 / 2.382665 /
Intercept [subject = schcode * Rid] / Variance / 42.223666 / 1.597367 / 26.433 / .000 / 39.206139 / 45.473440 /
a. Dependent Variable: achieve.
There is not officially(two-tailed significance level) random variation in the slope of the relationship of achievement to gmacademic from school to school (p=.096), although the text on p. 241 says that there is “. . . one-tailed p=.048).” Random variation in the slopes, if it were significant, would mean that for some schools there were steep slopes relating ACHIVEMENT to gmacadamic and for other schools the slopes were shallower. These differences are as yet unexplained.
This one-tailed significance led the authors to explore the reason for the differences in slope.
Model 3. Explaining Variation in slopes of the Level 2 gmses relationship. P. 241
The Level 1 model. As before, the Level 1 model is simply an average of test scores.
Yijk = ppjk + eijk Yijk is the score on the ith indicator of the construct p for person j in school k.
The Level 2 model – relating average achievement (pjk) to characteristics of the students
pjk = B0k + B1k*gmsesjk + B2k*gmacademicjk + gjk (Same as Model 2) (Eq 7.8, p. 234)
gjk is random variation of the construct of person j within school k, person-specific..
The Level 3 model – characteristics of the school affect the parameters of the Level 2 (person) model.