Supplementary Information

Cognitive Priming and Cognitive Training:

Immediate and Far Transfer to Academic Skills inChildren

Bruce E. Wexler1, Markus Iseli2, Seth Leon2, William Zaggle, Cynthia Rush3, Annette Goodman, A., Esat Imal1, Emily Bo2

1Department of Psychiatry Yale University School of Medicine, 2National Center for Research on Evaluation, Standards, and Student Testing, CRESST / UCLA,3Department of Statistics, Yale University

SREP-16-11950-T

1SupplementaryInformation

2

3SupplementaryData1

Pearson Math Assessment First grade,95% Free Lunch

120

100

80

60

40

20

0

Pro@icientAtRiskBelowPro@icient

4

5

120

Pearson Reading Assessment 3rd grade

100

80

60

40

20

0

Pro@icientAtRiskBelowPro@icient

6

7

120

Pearson Math Achievement 2nd grade

ClassRecievedSpecialReadingProgramClassRecievedActivate

100

80

60

40

20

0

FallWinterSpringFallWinterSpring

Pro@icientAtRiskBelowPro@icient

8

9Inthisschooloneclasswasgivenaspecialreadingenhancementprogramandthe

10otherwasgivenourbrain-­‐trainingprogram.Theclassthatdidthebraintraining

11hadnon-­‐significantlygreatergainsinreadingthandidtheclassthatgotthespecial

12readingprogram.Shownhere,theclassthatgotourbraintrainingshowedmuch

13greatergainsinmath.N.B.,thisiscomparisontoanactivecontrol. 14

15

16

17

18

19

20

21

22

23

24

25

26

27

28SupplementaryMethods1

29

30CCReadingGameResponseOutcomes

31WeuseSamejima’sgradedresponsemodel1.InIRTgradedresponsemodelsthe

32responsevariablesmustbecodedinsuchawaythateachitemhasaresponsethatfalls

33alonganorderedscalewitheachvalueonthescalebeingpresentforeachitem.Inthis

34studywetreatthelevelsofCCgame-playasiftheywereitemsonanassessment.For

35easeofinterpretationwekeepthenumberoforderedresponsecategorieswithineachof

36thetwoCCgamesconsistentforalllevels/items.Thedegreetowhichthevaluesofthe

37performanceconstructsvaryacrosslevelsisdifferentintheCCmathgameascompared

38totheCCreadinggame.Asaconsequence,therecodingofthecontinuousperformance

39constructsrequiresdifferingapproachesthataredescribedforeachgameinthe

40supplementalmaterials.

41

42IntheCCreadinggametherewere31levelsthroughwhichthestudentsplayed.Students

43receivedacontinuousscoreoneachofthethreeperformanceconstructs(speed,accuracy,

44andcombinedspeedandaccuracy).Thecontinuousconstructswerethenbinnedinto

45quintilesbasedonperformanceacrossallthelevels.Thiscategorizationresultedinscore

46coverageforall5categoriesineachofthe31gamelevels(seeTable1).Settingthesame

47cutpointsforall31itemshasthebenefitofeasingthecomparativeinterpretationofthe

48IRTlevelparameterswithrespecttoleveldifficultysincethesamescoringrubricapplies

49ateachlevel.

50

51

52Table1.Cut-PointsforCCReadingGameResponseVariables

Speed-CorrectMovesPerMinute / Accuracy-% CorrectMoves / StandardizedSpeed andAccuracy
AllLevels
Cutpoint1 / <4.34 / <42% / <-0.69
Cutpoint2 / 4.34to 6.69 / 42%to 54% / -0.69to -0.22
Cutpoint3 / 6.70to 8.45 / 55%to 67% / -0.22to 0.19
Cutpoint4 / 8.45to 10.43 / 67%to 79% / 0.19to 0.66
Cutpoint5 / >10.43 / >79% / >0.66

53CCMathGameResponseOutcomes

54IntheCCmathgametherewere121levels.Themaximumnumberoflevelsengagedby

55anystudentwas115levels.InordertohaveadequatesamplesizecoverageforIRT

56analysisitwasnecessarytosubstantiallyreducethenumberoflevelsbycollapsinglevels

57withsimilargame-playcharacteristics.Forexampletheoriginalgamelevelsone,three,

58fourandfiveallsharedthesamegame-playconfigurationandproblemtype(addition

59problemwithasumlowerthanfive).Thesefouroriginallevelswererecodedintoa

60singlecollapsedlevel.Followingthisprocesswewereabletoreducetheoriginallevels

61toaworkablenumberoftwenty-fourcollapsedlevels.

62

63Intheprocessofattemptingtocreatebinnedquintilesfromthetwenty-fourcollapsed

64levelstherestillremainedunfilledcells.Thislackofadequatesamplecoveragesuggested

65thatthegame-playperformanceintheCCmathgamedecreasedmoresteeplyacross

66gamelevelsascomparedtotheCCreadinggame.Duetothisperformancedecreaseand

67inordertoensureadequatesamplecoverageacrossalltwenty-fourcollapsedlevelsitwas

68necessarytoemploythreeorderedresponsecategoriesintheCCmathgamein

69comparisontothefivecategoriesthatwereusedfortheCCreadinggame.

70

71Therewasalsoasharpdecreaseinperformancebeginningwithcollapsedlevelnumber

72elevenwhichhadthegame-playcharacteristicofaddingtentomultiplesoftenthrough

73ninety.Asaresultitwasnecessarytosetseparatecutpointsforcollapsedlevelsone

74through10andeleventhroughtwentyfour.Theresultingcutpointsforthethree

75responseconstructsareshowninTable2.InterpretationoftheIRTleveldifficulty

76parametersshouldtakethesecutpointsintoaccount.

77

78Table2.Cut-PointsforCCMathGameResponseVariables

Speed-CorrectMovesPerMinute / Accuracy-% CorrectMoves / StandardizedSpeed andAccuracy
Levels1-10
Cutpoint1 / <3.44 / <31% / <0.15
Cutpoint2 / 3.44to 5.51 / 31%to 48% / 0.15to 1.07
Cutpoint3 / >5.52 / >48% / >1.07
Levels11-24
Cutpoint1 / <1.92 / <10% / <-0.75
Cutpoint2 / 1.92to 2.99 / 10%to 16% / -0.75to -0.36
Cutpoint3 / >2.99 / >16% / >-0.36

79

80Thestructureofourdatasetisonethatiscommonlyseeninmulti-levelgrowthmodels.

81Eachstudentparticipatedinmultiplegame-playsessionsresultinginbasiclevels(or

82dimensions)inthedataset.ModellevelIrepresentstherepeatedsessionsthatarenested

83withinstudentsthuscapturingthewithin-studentvariation,andmodellevelIIcaptures

84thebetween-studentvariation.WeusetheflexMIRTsoftware2,whichimplementsthe

85Metropolis-HastingsRobbins-Monro(MH-RM)algorithm3-5whichallowsforthe

86estimationofhigherdimensionalmodelsandwhichprovidesthecapacitytocombine

87IRTwithmulti-levelmodelsthatincludecovariates.Inadditiontogeneratinggame-play

88levelparameters,theIRTmodelalsogeneratesalatentfactorforeachlevelofthemodel.

89Wefixthevarianceofourbetween-studentfactortooneandallowthewithin-student

90factortovaryfreely.Wealsoincludeeightcovariatesinourmodel,sevenofwhich

91predictthewithin-student(modellevelI)factorofourmodelandonewhichpredictsthe

92between-student(modellevelII)factor.

93

94TheprimaryresearchquestioninvolvestheimmediateeffectoffourBTgamesonthe

95within-studentCCgameperformancevariation.Thesefourtreatment-relatedcovariates

96arecodedasdummyvariablesandtheconditionwithnoBTgamesservesasareference

97group.Thebetween-studentlatentvariablevariancewasfixedatoneandhasameanof

980,sotheestimateoftheBTgamedummyvariableisintheformatofaneffectsize.Three

99othercovariatesareincludedinthemodeltopredictthewithin-studentfactor(model

100

levelI):1)asimplelinearindicatorofthesessionday,whichrepresentsthecurrent

101

sessiondayforastudent;2)thelengthoftimeofeachsession;and3),avariablethat

102

indicatesifthestudenthadpreviouslyexperiencedanyofthelevelstheyengagedina

103

currentsession.Finally,wealsoincludegenderasacovariatetopredictthebetween-

104

studentfactor(modellevelII).

105

1.Kim,S.Camilli,G.Anitemresponsetheoryapproachtolongitudinalanalysis

106

withapplicationtosummersetbackinpreschoollanguage/literacy.Large-scale

107

assessmentsinEducation.2,doi:10.1186/2196-0739-2-1(2014).

108

2.Samejima,F.EstimationofLatentAbilityUsingaResponsePatternofGraded

109

Scores.PsychometricMonographNo.17,PsychometricSociety,Richmond,VA.

110

(1969).

111

3.Houts,C.R.,Cai,L.flexMIRT:FlexibleMultilevelItemFactorAnalysisand

112

TestScoringUser’sManual(2012).

113

4.Cai,L.High-dimensionalexploratoryitemfactoranalysisbyaMetropolis-

114

HastingsRobbins-Monroalgorithm.Psychometrika,75,33–57(2010).

115

5.Cai,L.Metropolis-HastingsRobbins-Monroalgorithmforconfirmatoryitem

116

factoranalysis.JournalofEducationalandBehavioralStatistics,35,307–335

117

118

(2010).