Lecture13FactorialDesigns
Topic: Theanalysisandinterpretationofdesignsemployingtwofactors.
Backingup...
Indesignswithonlyonefactor...
Otherfactorsare1)heldconstant,2)randomized,3)matched,etc.4)Ignored 5)Includedinresearch
Butwhatifyouwanttoincludeasecondfactorintheresearch.
QuestionHowshouldwecombinethelevelsofthesecondfactorwiththoseofthefirst?
Example
SupposetwoTypesofTrainingarebeingcompared–Lecturevs.CAIarebeingcompared.
Thesituationinvolvesteachingnewemployeesthebasicfactstheyneedtoknowworkinginanorganization. Thetrainingperiodlastsforoneweek.
OurinterestisinTypeofTraining,butitmightbethatthespecificjobinwhichanemployeewouldaffecthowmuchtheylearned. So,Jobisanextraneousvariable.
SowedecidedtoincludeJobintheresearch. Jobhasfourlevels–Clerical,Receptionist,Maintenance,andManagerial.
Sothisresearchinvolvetwofactors:
Factor1: TypeofTrainingwithtwolevels–LectureandCAI.
Factor2: Jobwithfourlevels–Clerical,Receptionist,Maintenance,andManagerial
Supposethedependentvariableisascoreonatestofamountlearnedduringatrainingsession.
ThemostefficientwaytoconductresearchinvolvingtwodifferentfactorsisadesigncalledaFactorialDesign. It’salsocalledacompletelycrosseddesign.
Inafactorialdesign,dataaregatheredatallcombinationsof levels ofbothfactors.
Thisdesignisbestconceptualizedusingatwowaytable,witheachdimensionofthetablerepresentingoneofthefactors...
Clerical / Receptionist / Maintenance / ManagerialLecture / Data / Data / Data / Data / Lecture mean
CAI / Data / Data / Data / Data / CAI
mean
Cler mean / Rec mean / Maint mean / Manag mean
Anyresearcherwouldcertainlyhavetwocommonquestionsconcerningtheresearch:
1. IsthereanyoveralldifferenceinperformanceofthosetaughtusingLecturevs.performanceofthosetaughtusingCAI?
Thisquestioncomparesperformanceofparticipantsinthefirstrowofthetwowaytablewiththatofparticipantsinthesecondrowofthetable. ThedifferencebetweenrowmeansiscalledtheMainEffectoftherowfactorintheabovedesign.
Clerical / Receptionist / Maintenance / ManagerialLecture / Data / Data / Data / Data / Lecture mean
CAI / Data / Data / Data / Data / CAI
mean
Cler mean / Rec mean / Maint mean / Manag mean
2. ArethereanyoveralldifferencesinperformanceofClericalworkers,Receptionists,Maintenanceworkers,andManagers?
Thisquestioncomparesperformanceinofparticipantsinthecolumnsofthetable. ThedifferencebetweencolumnmeansiscalledMainEffectofthecolumnfactorintheabovedesign.
Clerical / Receptionist / Maintenance / ManagerialLecture / Data / Data / Data / Data / Lecture mean
CAI / Data / Data / Data / Data / CAI
mean
Cler mean / Rec mean / Maint mean / Manag mean
AThirdQuestion
Thereisa3rdquestion,calledtheinteractionquestion,onethatisalittlelessobvious,butimportantnonetheless.
This question is emergent – it exists only because we’ve included two factors in our research in a factorial arrangement.
Itcanbeaskedintwoways:
3) Version1: DoesthedifferencebetweenLectureandCAIchangeasweconsideremployeeswhoareClerical,Receptionist,Maintenance,andManager?
3) Version2: DothedifferencesbetweenClericals,Receptionists,MaintenanceandManagerschangebetweenLecturepresentationandCAIpresentation?
ThisisaquestionaboutwhatiscalledtheInteractionoftheRowandColumnfactors.
Aninteractionexistswhentherowdifferenceschangefromonecolumntoanotherorequivalently,whenthecolumndifferenceschangefromonerowtothenext.
Clerical / Receptionist / Maintenance / ManagerialLecture / L-C Mean / L-R Mean / L-M Mean / L-B Mean / Lecture mean
CAI / C-C Mean / C-R Mean / C-M Mean / C-B Mean / CAI
mean
Cler mean / Rec mean / Maint mean / Manag mean
Or
Clerical / Receptionist / Maintenance / ManagerialLecture / L-C Mean / L-R Mean / L-M Mean / L-B Mean / Lecture mean
CAI / C-C Mean / C-R Mean / C-M Mean / C-B Mean / CAI
mean
Cler mean / Rec mean / Maint mean / Manag mean
Formal statistical tests oftheeffects
1) TestofRowMainEffect – the Row Main Effect is the difference between overall performance in Row 1 vs. overall performance in Row 2.
Each row is viewed as a group.
Themeanofallscoresineachrowiscomputed.
TheRowmaineffectistestedbyassessingthesignificanceofdifferencesbetweenthe marginal means of each row.
2) TestofColumnMainEffect – the Column Main effect is the difference between overall performance in Col 1 vs Col 2 vs. Col 3 vs. Col 4.
Each column is viewed as a group.
Themeanofallscoresineachcolumniscomputed.
TheColumnMainEffectistestedbyassessingthesignificanceofdifferencesbetween the marginal means of each column.
3) Testofinteractioneffect.
The differences between means within each column are compared with differences between mean within every other column.
Or
Thedifferencesbetweenmeanswithineachrowarecomparedwithdifferencesbetweenmeanswithineveryotherrow.
TheInteractionEffectistestedbyassessingthesignificanceofdifferencesofdifference.
Iftherowdifferenceschangefromonecolumntothenext,theInteractionissignificant.
Equivalently,ifthecolumndifferencechangefromonerowtothenext,theInteractionissignificant.
Usinggraphstovisualizemaineffectsandinteractions.
PlotCellmeansvs.levelsoftheColumnfactor
Connectmeansofcellswithinthesamerowwithaline.
Artificialdatawithnointeraction
Clerical / Receptionist / Maintenance / ManagerialMarginal
Lecture / 40,50
M=45 / 50,60
M=55 / 60,70
M=65 / 70,80
M=75 / 60
CAI / 30,40
M=35 / 40,50
M=45 / 50,60
M=55 / 60,70
M=65 / 50
Marginal / 40 / 50 / 60 / 70 / 55
TheplotofCellMeans
GraphicalRepresentationofRowMainEffect: Theaveragedifferenceinheightofthetwolines.
Notethatintheexample,thecontinuouslineisabovethedashedline,sothereis(ifsignificant)aRowMainEffect.
GraphicalRepresentationof ColumnMainEffect: Theaveragedifferenceinheightsofpointsateachcolumnlevel.
Thecolumnmaineffectisassessedbycomparingtheheightsofthefilledellipsesaddedtothefigureabove. Therearecleardifferencesintheheightsoftheellipses,suggesting(ifsignificant)thatthereisaColumnMaineffect.
GraphicalRepresentationofTheInteractionEffect
TheInteractionEffectistestedbycomparingthedifferencesbetweenrows–representedbythelengthsofthearrowsabove–ateachcolumn.
Thearrowsalllooklikethey’reaboutthesamelength,suggestingthattherowdifferencesarethesamefromcolumntocolumn. Thismeansthatthereisnointeraction. Notethatthelackofaninteractionmeansthatthelinesforthedifferentrowswillbeparallel.
Artificialdatawithaninteraction
Clerical / Receptionist / Maintenance / ManagerialMarginal
Lecture / 40,50
M=45 / 50,60
M=55 / 50,60
M=55 / 40,50
M=45 / 50
CAI / 30,40
M=35 / 40,50
M=45 / 50,60
M=55 / 60,70
M=65 / 50
Marginal / 40 / 50 / 55 / 55 / 50
Graphillustratinginteractionexample
GraphicalRepresentationofRowmaineffect: Compare“average”heightsoflines.
Wecanplainlyseethatthecontinuouslineisabovethedashedlinefor3columnsbutbelowthedashedlineforthe4thcolumn(Managers). ThismeansthatitmaynotmakesensetospeakofaRowMainEffect.
GraphicalRepresentationofColumnmaineffect: Compare“average”heightsofpointsateachcolumn
Wecanseethatthedifferencesbetweenthecolumnsarenotthesameforthedashedlineastheyareforthecontinuousline. AswasthecasefortheRowMainEffect,itmaynotmakesensetospeakofaColumnMainEffect.
GraphicalRepresentationofInteractioneffect: Comparedifferencesbetweenheightsofthelineateachcolumn.
Thedifferencesbetweenheightsofthelinesarenotthesamefromcolumntocolumn. Soifconfirmedbytheappropriatestatisticaltest,aninteractionmaybepresent.
GraphsofTypesofOutcomesofFactorialDesigns
BasedonAronAron,p.374,Table13-7.
1.
/ C1 / C2 / C3 / MarginalMeans
R1 / 10 / 10 / 10 / 10
R2 / 20 / 20 / 20 / 20
Marginal
Means / 15 / 15 / 15 / 15
Row
MainEffect / Column
MainEffect / Interaction
Yes / No / No
------
2.
C1 / C2 / C3 / MarginalMeans
R1 / 10 / 20 / 30 / 20
R2 / 10 / 20 / 30 / 20
Marginal
Means / 10 / 20 / 30 / 20
Row
MainEffect / Column
MainEffect / Interaction
No / Yes / No
------
3.
C1 / C2 / C3 / MarginalMeans
R1 / 10 / 20 / 30 / 20
R2 / 20 / 30 / 40 / 30
Marginal
Means / 15 / 25 / 35 / 25
Row
MainEffect / Column
MainEffect / Interaction
Yes / Yes / No
4.
C1 / C2 / C3 / MarginalMeans
R1 / 10 / 20 / 30 / 20
R2 / 10 / 20 / 60 / 30
Marginal
Means / 10 / 20 / 45 / 25
Row
MainEffect / Column
MainEffect / Interaction
Yes?? / Yes / Yes
TheperformanceinR2increasesmorefromC1toC2thandoesperformanceinR1.
ThedifferencebetweenR1andR2ischangesaswegofromC1toC3.
------
5.
/ C1 / C2 / C3 / MarginalMeans
R1 / 10 / 20 / 30 / 20
R2 / 30 / 20 / 10 / 20
Marginal
Means / 20 / 20 / 20 / 20
Row
MainEffect / Column
MainEffect / Interaction
No / No / Yes
Thisisaclassiccrossedinteraction. NeithertheRownortheColumnMainEffectisimportanthere. Theinteractionisthekeyfeature.
------
6.
/ C1 / C2 / C3 / MarginalMeans
R1 / 10 / 20 / 30 / 20
R2 / 20 / 40 / 60 / 40
Marginal
Means / 15 / 30 / 45 / 30
Row
MainEffect / Column
MainEffect / Interaction
Yes / Yes / Yes
ThisisasituationthatIwouldinterpretasrepresentingbothMainEffectsandaninteraction.
------
TwoWayFactorialANOVA
WorkedOutExampleBasedonMiniump.359
Thedata
ThedataarefromahypotheticalVerbalLearningExperimentinwhichparticipantswithLowAnxietylevelsandHighAnxietylevelsaregivenaverballearningtask. Somearegiveninstructionstoinducelittleifanypressure. Somearegiveninstructionstoinducemoderationpressuretoperformwell. Othersaregiveninstructionstoinducestrongpressuretoperformwell.
ThedatapresumablyillustratetheclassicinvertedUrelationshipoflearningtodrive/anxiety/motivation.
FactorialDesigns-18/17/3
id verblearn anxietypressure
1 40 1 1
2 64 1 1
3 46 1 1
4 56 1 1
5 46 1 1
6 46 1 1
7 39 1 1
8 38 1 1
9 44 1 1
10 69 1 1
11 61 1 2
12 54 1 2
13 55 1 2
14 40 1 2
15 43 1 2
16 47 1 2
17 57 1 2
18 51 1 2
19 40 1 2
20 55 1 2
21 50 1 3
22 48 1 3
23 60 1 3
24 63 1 3
25 83 1 3
26 63 1 3
27 53 1 3
28 60 1 3
29 73 1 3
30 69 1 3
31 41 2 1
32 34 2 1
33 37 2 1
34 48 2 1
35 57 2 1
36 47 2 1
37 55 2 1
38 33 2 1
39 42 2 1
40 38 2 1
41 48 2 2
42 58 2 2
43 42 2 2
44 40 2 2
45 49 2 2
46 49 2 2
47 56 2 2
48 41 2 2
49 35 2 2
50 57 2 2
51 56 2 3
52 35 2 3
53 43 2 3
54 39 2 3
55 29 2 3
56 32 2 3
57 54 2 3
58 43 2 3
59 49 2 3
60 49 2 3
FactorialDesigns-18/17/3
Conceptualization: Asa2(Anxiety)x3(Pressure)Factorial
Lowpressure: 1 / ModeratePressure: 2 / HighPressure: 3LowAnxiety: 1 / X / X / X
HighAnxiety: 2 / X / X / X
Theinterestsare:
1. IsthereaMainEffectofAnxiety. Dohighanxiouspersonsperformbetterorworsethanlowanxious?
2. IsthereaMainEffectofPressure. Overall,dopersonsunderdifferentamountsofpressureperformthistaskdifferently?
3. IsthereanInteractionofAnxietyandPressure: Doperformance differencesbetweenanxietylevelschangeatdifferentlevelsofpressure? Ordotheeffectsofdifferentlevelsofpressuredifferforpeoplewithhighanxietyvs.lowanxiety?
Theanalysis
Analyze->GeneralLinearModel->Univariate
SpecifyingPlots
SpecifyingPostHocs for any main effect with more than 2 levels –the column main effectin this example.
TheOutput
UNIANOVA
verblearn BYanxietypressure
/METHOD=SSTYPE(3)
/INTERCEPT=INCLUDE
/POSTHOC=pressure(BTUKEY)
/PLOT=PROFILE(pressure*anxiety)
/PRINT=DESCRIPTIVEETASQOPOWERHOMOGENEITY
/CRITERIA=ALPHA(.05)
/DESIGN=anxietypressureanxiety*pressure.
UnivariateAnalysisofVariance
G:\MdbT\P510511\P511L13-Factorial\FactorEGBasedOnMinP359.sav
TheMaineffectofAnxietywassignificant,witheta-squaredequalto.221.
TheMaineffectofPressurewasnotsignificant, although eta-squared equals .085.
Due to small sample size, test was not powerful enough to detect the fairly large difference.
TheInteractionwassignificant,witheta-squaredequalto.147.
Wheneveryouhaveasignificantinteraction,youshouldbeverycautiousininterpretingandreportingmaineffects. Thesignificantinteractionmayindicatethatthereisaneffectofavariable,butthatitisnotaMAINeffect.
PostHocTests
pressure
HomogeneousSubsets
ProfilePlots
ThereISaneffectofpressureinthesedata,butitisnotaMAINeffect. Instead,itwouldbebestcharacterizedasan“anxietyspecific”effect. Forlowanxietyparticipants,increasingpressureleadtoincreasingperformance.
Butforhighanxietyparticipants,increasingpressureleadtoincreasingperformanceonlyuptoapoint. Afterthat,furtherincreasesleadtoadecreaseinperformance.
Analysis of Change Between Pre- and Post-test Performance
Start here on 11/27/12
A combination between-group and within-groups design
Twobuildingsofanorganizationweregivenapretestmeasuringproductivity. ThenemployeesinBuildingAwereassignedtoworkinteamswhilethoseinBuildingBperformedessentiallythesameworkasbefore. After6months,posttestsofproductivitywereobtainedforeach. Thus,eachpersonwasmeasuredtwiceusingthesametest. TheinterestwasindeterminingwhetherpersonsinBuildingAincreasedproductivitymorethanthoseinBuildingB. This is a Pretest-posttest with nonequivalent groups design – Lecture 9, p. 12.)
TheDataEditor...
FactorialDesigns-18/17/3
idbldg prepost
1 1 52 48
2 1 50 49
3 1 62 54
4 1 63 56
5 1 35 26
6 1 82 72
7 1 39 46
8 1 39 38
9 1 60 64
10 1 59 53
11 1 66 67
12 1 35 26
13 1 64 56
14 1 37 37
15 1 59 49
16 1 49 48
17 1 39 34
18 1 44 35
19 1 53 50
20 1 46 35
21 1 18 21
22 1 75 74
23 1 64 54
24 1 58 64
25 1 29 29
26 1 56 46
27 1 67 71
28 1 60 50
29 1 42 41
30 1 24 21
31 1 61 69
32 1 17 8
33 1 34 35
34 1 58 54
35 1 69 68
36 1 62 69
37 1 46 37
38 1 45 45
39 1 56 57
40 1 36 43
41 1 46 43
42 1 61 60
43 1 45 35
44 1 73 77
45 1 62 61
46 1 54 51
47 1 35 35
48 1 46 41
49 1 51 58
50 1 50 54
51 1 37 26
52 1 50 54
53 1 32 27
54 1 79 73
55 1 49 46
56 1 48 42
57 1 36 38
58 1 34 42
59 1 30 23
60 1 64 62
61 1 54 50
62 1 52 56
63 1 58 50
64 1 23 29
65 1 56 53
66 1 43 45
67 1 51 53
68 1 35 37
69 1 46 39
70 1 43 44
71 1 29 33
72 1 63 63
73 1 45 41
74 1 49 38
75 1 62 50
76 1 49 53
77 1 51 51
78 1 31 20
79 1 71 76
80 1 72 78
81 1 50 53
82 1 31 21
83 1 19 25
84 1 66 71
85 1 42 42
86 1 65 56
87 1 47 39
88 1 35 29
89 1 40 30
90 1 36 39
91 1 56 45
92 1 48 42
93 1 47 47
94 1 69 58
95 1 44 35
96 1 58 59
97 1 20 20
98 1 35 25
99 1 35 31
100 1 57 61
101 0 42 44
102 0 35 39
103 0 42 62
104 0 61 75
105 0 49 51
106 0 55 67
107 0 54 72
108 0 44 51
109 0 39 59
110 0 66 70
111 0 43 63
112 0 48 53
113 0 48 54
114 0 44 51
115 0 43 56
116 0 62 77
117 0 36 51
118 0 44 47
119 0 53 65
120 0 47 56
121 0 59 76
122 0 52 71
123 0 50 59
124 0 55 70
125 0 39 56
126 0 39 48
127 0 56 65
128 0 50 68
129 0 74 94
130 0 48 67
131 0 58 73
132 0 53 69
133 0 45 58
134 0 55 72
135 0 58 63
136 0 50 61
137 0 54 62
138 0 49 65
139 0 57 67
140 0 47 64
141 0 30 36
142 0 41 46
143 0 58 63
144 0 51 70
145 0 54 69
146 0 50 65
147 0 64 79
148 0 66 77
149 0 33 35
150 0 69 72
151 0 45 53
152 0 40 57
153 0 54 66
154 0 55 65
155 0 45 67
156 0 50 70
157 0 41 56
158 0 42 62
159 0 59 72
160 0 56 64
161 0 63 71
162 0 49 57
163 0 41 56
164 0 51 71
165 0 62 65
166 0 56 78
167 0 85 90
168 0 54 61
169 0 44 55
170 0 51 63
171 0 39 57
172 0 31 37
173 0 57 71
174 0 45 56
175 0 40 43
176 0 34 48
177 0 45 66
178 0 34 42
179 0 52 61
180 0 33 50
181 0 46 63
182 0 46 51
183 0 40 49
184 0 47 66
185 0 67 76
186 0 60 77
187 0 44 54
188 0 59 71
189 0 61 81
190 0 62 78
191 0 47 56
192 0 57 60
193 0 56 77
194 0 57 77
195 0 54 70
196 0 58 62
197 0 48 53
198 0 52 71
199 0 51 56
200 0 48 52
FactorialDesigns-18/17/3
FactorialDesigns-18/17/3
The expected result – from Lecture 9 . . .
6. A salvage design: The Pretest-Posttest with Nonequivalent Groups Design
It is important to note that the pretest and posttest are the same instrument.
One of the most frequently employed designs in the social sciences.
Some outcomes lead to defensible arguments for treatment differences.
Others do not.
The ideal outcome:
If this pattern of results occurs – no difference on the pretest, difference favoring the treatment group on the posttest, most researchers would argue that it is evidence for the existence of a treatment effect.
Using our new knowledge of factorial designs, we can recognize this as one
Treatment Condition – T vs. C – is the Row factor.
Time – Pre vs. Post – is the Column factor.
In most applications of this design, the hoped-for result is a significant interaction with small, even nonsignificant row and column main effects.
Analyze -> General Linear Model -> Repeated Measures
Themaindialogbox
Specifying Plots
Put the Time factor (prepost in this case) on the horizontal axis.
Specify the Between-subjects factor (bldg) to be represented by separate lines.
The multivariate tests require the fewest restrictive assumptions. In this case, all tests lead to the same conclusions.
Results . . .
1. There is an overall difference between Pre and Post-test means.
But that may just be due to the huge increase in Building A.
2. There is an overall difference between Building A and Building B means.
But that may be an artifact of the huge increase in Building A.
3. There is an interaction of Building and Prepost. Thus, the difference between Pre- and Post-test means depends on which building is considered.
This is the key finding here – Building A performance increased, Building B’s did not.
Use the plots to interpret the interaction in greater detail.
This shows, confirmed by the significant interaction shown on the previous page that Performance in Building A increased from Pre to Post while performance in Build B decreased.
The conclusion is that assigning people to work in teams may have lead to increases in individual performance.
FactorialDesigns-18/17/3