1

Supplementary Syntax for Estimating Intraclass Correlation Coefficients and Level Specific Fit

“Evaluating Factor Structures of Measures in Group Research: Looking Between and Within”

This online appendix includes SPSS syntax for estimating the ANOVA necessary to calculate the intraclass correlation coefficient (ICC) and an example ANOVA output table resulting from that analysis. The supplementary Excel workbook also includes a formula to calculate the ICC from the resulting ANOVA table.

This online appendix also provides computer code for estimating multilevel confirmatory factor analyses in Mplus, including partially saturated models for testing level specific fit in multilevel confirmatory factor analyses. Included separately is an Excel workbook for calculating level specific fit indices from the output of these analyses.

Included also in the supplementary materials is a sample of the data used for these analyses to illustrate the data structure.

Overview of syntax included:

Intraclass correlation coefficient: One-way ANOVA syntax

Model 1: Fully hypothesized model, with hypothesized factor structure specified both between and within groups

Model 2: Hypothesized factor structure between; saturated model within

Model 3: Independent model between; saturated model within

Model 4: Saturated model between; hypothesized factor structure within

Model 5: Saturated model between; independent model within

Overview of variables in models:

ID: ID unique to each client in the data set

GroupID: ID unique to each group in the data set

gq1-gg30: individual item scores from the Group Questionnaire

PosBond_L: member-leader positive bonding subscale

PosBond_M: member-member positive bonding subscale

PosBond_G: member-group positive bonding subscale

PosBond: positive bonding subscale

NegRel_L: member-leader negative relationship subscale

NegRel_M: member-member negative relationship subscale

NegRel_G: member-group negative relationship subscale

NegRel: negative relationship subscale

PosWork_L: member-leader positive working subscale

PosWork_M: member-member positive working subscale

PosWork: positive working subscale

Intraclass Correlation Coefficient

ONEWAY PosWork BY GroupID

/MISSING ANALYSIS.

ANOVA
PosWork
Sum of Squares / df / Mean Square / F / Sig.
Between Groups / 47588.203 / 194 / 245.300 / 2.367 / .000
Within Groups / 80944.182 / 781 / 103.642
Total / 128532.384 / 975

Model 1:

TITLE:

Model 1: Hypothesized between, hypothesized within

DATA:

File is GQwithGroups.dat;

VARIABLE:

Names are

ID GroupID gq1 gq2 gq3 gq4 gq5 gq6 gq7 gq8 gq9 gq10

gq11 gq12 gq13 gq14 gq15 gq16 gq17 gq18 gq19 gq20 gq21 gq22 gq23 gq24

gq25 gq26 gq27 gq28 gq29gq30PosBond_LPosBond_MPosBond_GPosBond

NegRel_LNegRel_MNegRel_GNegRelPosWork_LPosWork_MPosWork;

Usevar are

gq1 gq2 gq3 gq4 gq5 gq6 gq7 gq8 gq9 gq10 gq11 gq12

gq13 gq14 gq15 gq16 gq17 gq18 gq19 gq20 gq21 gq22 gq23 gq24 gq25 gq26

gq27 gq28 gq29 gq30;

Missing are all (-1, -9999) ;

Cluster = GroupID;

ANALYSIS:

Estimator is ml;

type = twolevel;

H1iterations=10000;

MODEL:

%WITHIN%

LdrBndW BY gq1@1

gq3 (1)

gq5 (2)

gq7 (3);

MemBndW BY gq2@1

gq4 (4)

gq6 (5)

gq8 (6);

GrpBndW BY gq26@1

gq27 (7)

gq28 (8)

gq29 (9)

gq30 (10);

LdrWrkW BY gq9@1

gq11 (11)

gq13 (12)

gq15 (13);

MemWrkW BY gq10@1

gq12 (14)

gq14 (15)

gq16 (16);

LdrNegW BY gq17@1

gq19 (17)

gq21 (18);

MemNegW BY gq18@1

gq20 (19)

gq22 (20);

GrpNegW BY gq23@1

gq24 (21)

gq25 (22);

PosBndW BY LdrBndW@1

MemBndW (23)

GrpBndW (24);

PosWrkW BY LdrWrkW@1

MemWrkW (25);

NegRelW BY LdrNegW@1

MemNegW (26)

GrpNegW (27);

LdrBndW with LdrWrkW;

LdrBndW with LdrNegW;

LdrNegW with LdrWrkW;

PosBndW with PosWrkW;

NegRelW with PosWrkW;

PosBndW with NegRelW;

gq1 gq3 gq5 gq7 gq9 gq11 gq13 pwith gq2 gq4 gq6 gq8 gq10 gq12 gq14;

gq15 gq17 gq19 gq21 pwith gq16 gq18 gq20 gq22;

%BETWEEN%

LdrBndB BY gq1@1

gq3 (1)

gq5 (2)

gq7 (3);

MemBndB BY gq2@1

gq4 (4)

gq6 (5)

gq8 (6);

GrpBndB BY gq26@1

gq27 (7)

gq28 (8)

gq29 (9)

gq30 (10);

LdrWrkB BY gq9@1

gq11 (11)

gq13 (12)

gq15 (13);

MemWrkB BY gq10@1

gq12 (14)

gq14 (15)

gq16 (16);

LdrNegB BY gq17@1

gq19 (17)

gq21 (18);

MemNegB BY gq18@1

gq20 (19)

gq22 (20);

GrpNegB BY gq23@1

gq24 (21)

gq25 (22);

PosBndB BY LdrBndB@1

MemBndB (23)

GrpBndB (24);

PosWrkB BY LdrWrkB@1

MemWrkB (25);

NegRelB BY LdrNegB@1

MemNegB (26)

GrpNegB (27);

LdrBndB with LdrWrkB;

LdrBndB with LdrNegB;

LdrNegB with LdrWrkB;

PosBndB with PosWrkB;

NegRelB with PosWrkB;

PosBndB with NegRelB;

!gq29(p1);

!MemBndB (p2);

!MemWrkB (p3);

!gq22 (p4);

!MODEL CONSTRAINT:

!p1>0;

!p2>0;

!p3>0;

!p4>0;

!Commented out syntax adds model constraints for Heywood cases;

OUTPUT:

stdyx;

Model 2:

TITLE:

Model 2- Hypothesized between, saturated within

DATA:

File is GQwithGroups.dat ;

VARIABLE:

Names are

ID GroupID gq1 gq2 gq3 gq4 gq5 gq6 gq7 gq8 gq9 gq10

gq11 gq12 gq13 gq14 gq15 gq16 gq17 gq18 gq19 gq20 gq21 gq22 gq23 gq24

gq25 gq26 gq27 gq28 gq29 gq30PosBond_LPosBond_MPosBond_GPosBond

NegRel_LNegRel_MNegRel_GNegRelPosWork_LPosWork_MPosWork;

Usevar are

gq1 gq2 gq3 gq4 gq5 gq6 gq7 gq8 gq9 gq10 gq11 gq12

gq13 gq14 gq15 gq16 gq17 gq18 gq19 gq20 gq21 gq22 gq23 gq24 gq25 gq26

gq27 gq28 gq29 gq30;

Missing are all (-1, -9999) ;

Cluster = GroupID;

ANALYSIS:

estimator is ml;

type = twolevel;

H1iterations=10000;

MODEL:

%WITHIN%

gq1-gq30 WITH gq1-gq30;

%BETWEEN%

LdrBndB BY gq1@1

gq3 (1)

gq5 (2)

gq7 (3);

MemBndB BY gq2@1

gq4 (4)

gq6 (5)

gq8 (6);

GrpBndB BY gq26@1

gq27 (7)

gq28 (8)

gq29 (9)

gq30 (10);

LdrWrkB BY gq9@1

gq11 (11)

gq13 (12)

gq15 (13);

MemWrkB BY gq10@1

gq12 (14)

gq14 (15)

gq16 (16);

LdrNegB BY gq17@1

gq19 (17)

gq21 (18);

MemNegB BY gq18@1

gq20 (19)

gq22 (20);

GrpNegB BY gq23@1

gq24 (21)

gq25 (22);

PosBndB BY LdrBndB@1

MemBndB (23)

GrpBndB (24);

PosWrkB BY LdrWrkB@1

MemWrkB (25);

NegRelB BY LdrNegB@1

MemNegB (26)

GrpNegB (27);

LdrBndB with LdrWrkB;

LdrBndB with LdrNegB;

LdrNegB with LdrWrkB;

PosBndB with PosWrkB;

NegRelB with PosWrkB;

PosBndB with NegRelB;

OUTPUT:

stdyx;

Model 3:

TITLE:

Model 3- Independent between, saturated within

DATA:

File is GQwithGroups.dat ;

VARIABLE:

Names are

ID GroupID gq1 gq2 gq3 gq4 gq5 gq6 gq7 gq8 gq9 gq10

gq11 gq12 gq13 gq14 gq15 gq16 gq17 gq18 gq19 gq20 gq21 gq22 gq23 gq24

gq25 gq26 gq27 gq28 gq29 gq30PosBond_LPosBond_MPosBond_GPosBond

NegRel_LNegRel_MNegRel_GNegRelPosWork_LPosWork_MPosWork;

Usevar are

gq1 gq2 gq3 gq4 gq5 gq6 gq7 gq8 gq9 gq10 gq11 gq12

gq13 gq14 gq15 gq16 gq17 gq18 gq19 gq20 gq21 gq22 gq23 gq24 gq25 gq26

gq27 gq28 gq29 gq30;

Missing are all (-1, -9999) ;

Cluster = GroupID;

ANALYSIS:

estimator is ml;

type = twolevel;

H1iterations=10000;

MODEL:

%WITHIN%

gq1-gq30 WITH gq1-gq30;

%BETWEEN%

gq1-gq30;

OUTPUT:

stdyx;

Model 4:

TITLE:

Model 4: Saturated between, hypothesized within

DATA:

File is GQwithGroups.dat;

VARIABLE:

Names are

ID GroupID gq1 gq2 gq3 gq4 gq5 gq6 gq7 gq8 gq9 gq10

gq11 gq12 gq13 gq14 gq15 gq16 gq17 gq18 gq19 gq20 gq21 gq22 gq23 gq24

gq25 gq26 gq27 gq28 gq29 gq30PosBond_LPosBond_MPosBond_GPosBond

NegRel_LNegRel_MNegRel_GNegRelPosWork_LPosWork_MPosWork;

Usevar are

gq1 gq2 gq3 gq4 gq5 gq6 gq7 gq8 gq9 gq10 gq11 gq12

gq13 gq14 gq15 gq16 gq17 gq18 gq19 gq20 gq21 gq22 gq23 gq24 gq25 gq26

gq27 gq28 gq29 gq30;

Missing are all (-1, -9999);

Cluster = GroupID;

ANALYSIS:

estimator is ml;

type = twolevel;

H1iterations=10000;

MODEL:

%WITHIN%

LdrBndW BY gq1@1

gq3 (1)

gq5 (2)

gq7 (3);

MemBndW BY gq2@1

gq4 (4)

gq6 (5)

gq8 (6);

GrpBndW BY gq26@1

gq27 (7)

gq28 (8)

gq29 (9)

gq30 (10);

LdrWrkW BY gq9@1

gq11 (11)

gq13 (12)

gq15 (13);

MemWrkW BY gq10@1

gq12 (14)

gq14 (15)

gq16 (16);

LdrNegW BY gq17@1

gq19 (17)

gq21 (18);

MemNegW BY gq18@1

gq20 (19)

gq22 (20);

GrpNegW BY gq23@1

gq24 (21)

gq25 (22);

PosBndW BY LdrBndW@1

MemBndW (23)

GrpBndW (24);

PosWrkW BY LdrWrkW@1

MemWrkW (25);

NegRelW BY LdrNegW@1

MemNegW (26)

GrpNegW (27);

LdrBndW with LdrWrkW;

LdrBndW with LdrNegW;

LdrNegW with LdrWrkW;

PosBndW with PosWrkW;

NegRelW with PosWrkW;

PosBndW with NegRelW;

gq1 gq3 gq5 gq7 gq9 gq11 gq13 PWITH gq2 gq4 gq6 gq8 gq10 gq12 gq14;

gq15 gq17 gq19 gq21 PWITH gq16 gq18 gq20 gq22;

%BETWEEN%

gq1-gq30 WITH gq1-gq30;

OUTPUT:

stdyx;

Model 5:

TITLE:

Model 5- Saturated between, independent within

DATA:

File is GQwithGroups.dat ;

VARIABLE:

Names are

ID GroupID gq1 gq2 gq3 gq4 gq5 gq6 gq7 gq8 gq9 gq10

gq11 gq12 gq13 gq14 gq15 gq16 gq17 gq18 gq19 gq20 gq21 gq22 gq23 gq24

gq25 gq26 gq27 gq28 gq29 gq30PosBond_LPosBond_MPosBond_GPosBond

NegRel_LNegRel_MNegRel_GNegRelPosWork_LPosWork_MPosWork;

Usevar are

gq1 gq2 gq3 gq4 gq5 gq6 gq7 gq8 gq9 gq10 gq11 gq12

gq13 gq14 gq15 gq16 gq17 gq18 gq19 gq20 gq21 gq22 gq23 gq24 gq25 gq26

gq27 gq28 gq29 gq30;

Missing are all (-1, -9999) ;

Cluster = GroupID;

ANALYSIS:

estimator is ml;

type = twolevel;

H1iterations=10000;

MODEL:

%WITHIN%

gq1-gq30;

%BETWEEN%

gq1-gq30 WITH gq1-gq30;

OUTPUT:

stdyx;