Gender Gaps in Mathematics M. Chatterji, PhD
Does A Teacher-Mediated Model of “Proximal Assessment and Learner Diagnosis” (PALD) Help Erase Learner Gaps in Mathematics? Early Results in Gender Groups
Madhabi Chatterji
Abstract
This study is a component of a larger research and development effort supported by the National Science Foundation that focuses on a dynamic model of classroom assessment, currently being implemented by teachers in the East Ramapo school district in upstate New York, U.S.A. In all, 14 grade 5-6 classrooms are participating in the project for two years with a focus on mathematics domains. The approach is titled: “Proximal Assessment for Learner Diagnosis” (PALD). This presentation provides the context for the study; the conceptual and theoretical bases of the PALD approach; the logical assumptions underlying the PALD model; patterns in gender gaps found in the nation, New York state and the district; and early results and gap patterns in the sample studied before and after the project was implemented. Both participating and non-participating classrooms in grades 5-6 were used in the study.
------
Associate Professor of Measurement, Evaluation, and Education Co-Director, Assessment and Evaluation Research Initiative (AERI) of the Campaign for Educational Equity Teachers College, Columbia University
Preliminary results show that on pre-test measures in both grades, males were about 1/10 to 2/10 of a standard deviation units above females in both PALD and non-PALD classes. Following a year, we see no differences between males and females on geometry and prime factorization domains in PALD students, although males do continue with an advantage on long division. In a couple of areas, females are doing better than males. Changes on gender gaps are similar in PALD and non-PALD classrooms, but achievement differences in long division and prime factorization favor PALD students when their teachers report high self-efficacy in PALD practices (statistically significant at p<.10 based on two level random coefficients models run with students at level 1 and classroom at level 2). That student achievement is mediated by teacher attitudes and beliefs is a finding consistent with the logic model for the PALD intervention. Gender slopes, when built in models as random variables, are significantly different with Long Division as the outcome measure--indicating that how males and females perform depends on the classroom in which they belong. This last finding, which is not detectable in gross breakdowns of achievement test scores, has implications for classroom teaching and particular school actions. It should be noted that these are preliminary results.
Acknowledgements: My sincere thanks to Jill Goodman, who compiled the U.S. and New York state data for the present study within a very tight timeline.
U.S. Mean Scale Scores and Discrete Achievement Level Percentages for Grade 4 Mathematics on 2004-2005 National Assessment of Educational Progress (NAEP) Reports including Gender Distributions
Participants / Mean Scale Scores / Percent Discrete Achievement LevelBelow Basic / At or Above Basic / At or Above Proficient
All students / 238 / 20% / 80% / 36%
Male / 239 / 19% / 81% / 38%
Female / 237 / 20% / 80% / 34%
Note. Scale scores are reported on a 0-500 scale with cut scores of Basic=214; Proficient=249; Advanced=282. Proficient represents solid academic performance with demonstrated competency over challenging subject matter.
Areas covered on the Grade 4 NAEP Mathematics assessment include: 1) Number properties and operations-40%; 2) Measurement-20%; 3) Geometry-15%; 4) Data analysis and probability-10%; and 5) Algebra-15%.
New York State Mean Scale Scores and Discrete Achievement Level Percentages for Grade 4 Mathematics on 2004-2005 NAEP Reports including Gender Distributions
Participants / Mean Scale Scores / Percent Discrete Achievement LevelBelow Basic / At or Above Basic / At or Above Proficient
All students / 238 / 19% / 81% / 36%
Male / 240 / 18% / 82% / 39%
Female / 237 / 19% / 81% / 33%
Retrieved January 9, 2007 from http://www.nationsreportcard.gov.
East Ramapo Central School District: Elementary-Level Mathematics
Results by Gender / 2004 - 2005
Total Tested / Percentages of Tested Students Scoring at Levels*
Below Basic
1 / Basic
2 / Proficient
3 / Advanced
4
Female / 287 / 5% / 16% / 48% / 31%
Male / 296 / 11% / 18% / 42% / 29%
Total / 583 / 8% / 17% / 45% / 30%
*Elementary-Level Mathematics Levels – Knowledge, Reasoning, and Problem-Solving Standards
Level 1 = Serious academic deficiencies
Level 2 = Need extra help to meet the standards and pass the Regents examination
Level 3 = Meet the standards and, with continued steady growth, should pass the Regents examination.
Level 4 = Exceed the standards and are moving toward high performance on the Regents examination
East Ramapo Central School District: Demographics
2003-2004 District-wide Total Expenditure per Pupil[1] / $18,766
2003-2004 NYS Public Schools Total Expenditure per Pupil[2] / $13,826
2004-2005 Core Classes Taught by Highly Qualified Teachers[3] / 96%
2004-2005 Teachers with No Valid Teaching Certificate[4] / 1%
2004-2005 Total K-12 Enrollment / 8566
District Need to Resource Capacity (N/RC) / 3[5]
Student Ethnic Distribution: 2004-2005
Race/Ethnicity / Number of Students / % of Enrollment
American Indian, Alaskan, Asian, or Pacific Islander / 793 / 9.3%
Black (Not Hispanic) / 5201 / 60.7%
Hispanic / 1480 / 17.3%
White (Not Hispanic) / 1092 / 12.7%
Student Socioeconomic and Stability Indicators: 2004-2005
Demographic / Number of Students / % of Enrollment
Limited English Proficient / 931 / 10.6%
Eligible for Free Lunch / 3492 / 43.6%
Reduced Lunch / NA / 13.6%
Public Assistance / NA / NA
Student Stability / NA / NA
The Proximal Assessment and Learner Diagnosis (PALD) Model
Context
Persistent Learner Gaps in Mathematics
Federal and State Pressures on Schools to Raise Student Achievement Levels and Reduce Gaps Continues….( under NCLB)
Do teachers, working in close proximity of learners, have the supports, tools, and capacities to detect and address gaps as they arise?
Recognizing Learner Gaps in Terms of Alterable Learner Characteristics
National legislative mandates, such as the NCLB Act of 2001, tend to focus on “Unalterable Characteristics” of Learners
Gender gaps, Ethnic gaps, gaps by Poverty level-Useful for Monitoring Scholastic Outcomes at the Institutional Level
Not Useful for fostering Learner Growth in Classrooms
PALD Focus on Learner Gaps in Domain-specific Concepts and Skills—Alterable Characteristics that give Teachers chances for Instructional Intervention and Mediation
Example: Gaps in those who are proficient versus those who are not on Place Value concepts.
What in the history of the learner caused the gaps to arise? What can we do now to change that status through mediation and practice opportunities?
Project Purposes
Research and Development project supported by National Science Foundation (Chatterji & Gordon, 2004)
Duration of formative phase: 2 ½ years (2005-2007)
AIM I: Develop tools, materials, and professional development model to build capacities in teachers and school personnel in PALD
AIM II: Implement and field-test PALD model in selected schools at East Ramapo (4 schools, 16 teachers/classrooms)
AIM III: Monitor and conduct formative evaluation of project
Theoretical Ideas
Learner cognitions improve, with timely diagnosis and mediation by caring adults, even in the cognitively-impaired
Need to know the “proximal etiology” of learners—the causes and origins for gaps in particular domains--to alter outcomes
Cognitive and affective attributes are inter-twined with mastery attainment in domains
Dynamic cycles of formative assessment, diagnosis and timely mediation by teachers in classrooms can lead to mastery and gap reduction
(Sources: Bloom, 1977; Feuerstein, 1979; Gagne, 1994; Nitko, 1989; Nichols, 1998; Gordon et al, 2006; and many others who have written on student-centered classroom assessment).
Proximal Assessment and Learner Diagnosis (PALD): What Teachers Do in Classrooms
I. Goal-Setting
PALD Study: Composition of Mixed-Grade Sample in Year 1
Student Characteristics
/ N / Mean / SD / Min / MaxAge (In Years) / 714 / 11.11 / .70 / 9.00 / 14.00
Gender (Proportion; Male=1) / 714 / .46 / .50 / 0.00 / 1.00
PALD Participation by Child (Proportion; PALD=1) / 714 / .37 / .48 / 0.00 / 1.00
Low Socioeconomic Status (Proportion; Free lunch=1)
Grade 5 (Proportion; G5=1) / 714 / .57 / .25 / 0.00 / 1.00
Grade 6 (Proportion; G6=1) / 714 / .43 / .24 / 0.00 / 1.00
*Previous Math Achievement (Standardized using grade level M, SD) / 714 / .00 / 1.00 / -2.57 / 4.38
PALD / 265 / +.06 / 1.00
Not in PALD / 450 / -.03 / .99
Level 2
Classroom Characteristics
/ N / Mean / SD / Min / MaxSchool Size / 39 / 387.85 / 24.67 / 350.00 / 419.00
School’s Past Math Record (Percent Students at Levels 3-4 in Math) / 39 / 77.00 / 4.26 / 71.00 / 81.00
School’s Ethnic Composition (Percent Non-white) / 39 / 89.91 / 3.50 / 85.70 / 96.30
PALD Participation by Classroom (Proportion; PALD=1) / 39 / .39 / .36 / 0.00 / 1.00
PALD: Teacher Self-efficacy Score / 39 / 53.42 / 7.23 / 32.00 / 65.00
PALD: Teacher Self-efficacy Z-Score / 39 / 0.07 / .82 / -2.36 / +1.38
*Grade 5: Previous math achievement scores were grade 4 Terra Nova scale scores; Grade 6: Previous math achievement scores were grade 5 New York State tests scale scores.
Results after One Year of PALD Implementation
East Ramapo Gender Gaps in Mathematics Achievement Following 1-Year of PALD Intervention: Grade 6 Results
Achievement Measure / PALD Status / Gender Group / M / SD / N / Difference (in pooled SD units)Pre-test
(Grade 5 Standardized Test – Scale Score) / PALD / Female / 663.63 / 35.19 / 68
Male / 669.34 / 33.31 / 65 / +.185
Non-PALD / Female / 659.52 / 24.71 / 89
Male / 659.42 / 30.06 / 81 / +.003
Post-test (PALD Developmental Assessments)
Geometry / PALD / Female / 19.64 / 8.79 / 68
Male / 19.03 / 9.32 / 65 / -0.06
Non-PALD / Female / 14.06 / 8.70 / 89
Male / 14.39 / 9.98 / 81 / +0.03
Prime Factorization / PALD / Female / 23.14 / 9.69 / 68
Male / 22.03 / 9.40 / 65 / -0.11
Non-PALD / Female / 20.96 / 9.94 / 89
Male / 21.36 / 10.27 / 81 / +0.10
Long Division / PALD / Female / 25.16 / 10.88
Male / 25.72 / 11.85 / +0.05
Non-PALD / Female / 23.24 / 10.83
Male / 24.90 / 8.80 / +0.17
Results after One Year of PALD Implementation
East Ramapo Gender Gaps in Mathematics Achievement Following 1-Year of PALD Intervention: Grade 5 Results
Achievement Measure / PALD Status / Gender Group / M / SD / N / Difference (in pooled SD units)Pre-test
(Grade 4 Standardized Test – Scale Score) / PALD / Female / 667.19 / 30.06 / 71
Male / 670.94 / 35.68 / 54 / +.12
Non-PALD / Female / 666.27 / 32.25 / 145
Male / 671.61 / 39.69 / 121 / +.15
Post-test (PALD Developmental Assessments)
Geometry / PALD / Female / 12.46 / 8.79 / 71
Male / 12.39 / 8.43 / 54 / -.008
Non-PALD / Female / 15.68 / 8.27 / 145
Male / 14.97 / 8.19 / 121 / -.084
Prime Factorization / PALD / Female / 17.23 / 8.17 / 71
Male / 15.59 / 8.70 / 54 / -.199
Non-PALD / Female / 20.27 / 7.89 / 145
Male / 19.55 / 7.98 / 121 / -.087
Long Division / PALD / Female / 21.94 / 10.58 / 71
Male / 23.33 / 8.22 / 54 / +.147
Non-PALD / Female / 25.27 / 9.41 / 145
Male / 25.32 / 9.10 / 121 / +.005
Results after One Year of PALD Implementation
Multilevel Analysis of PALD Effects: Long Division Scores as Outcomes
Controlling for prior math achievement at the student-level, did the PALD intervention and reported self-efficacy of teachers in PALD practices improve achievement outcomes at the classroom-level?
Classroom Mean / SD / Standard Error / t / df / P24.41 / 4.67 / 4.08 / 4.20 / 36 / 0.00**
Level 2
Fixed Effects at Classroom Level / Coefficient / SE / t / df / p
PALD / -1.278 / 1.577 / -.81 / 36 / 0.423
PALD Teacher Self-efficacy / +0.146 / 0.08 / 1.86 / 36 / 0.071*
Level 1
Effects (Controls) at Student Level / Coefficient / SE / t / df / p
Gender / .146 / .71 / 0.20 / 709 / 0.84
Prior Math Achievement / .129 / .01 / 9.118 / 709 / 0.00**
Note. Reliability of classroom mean estimates (intercepts): 0.85
*Statistically significant at 10% error level.
**Statistically significant at <1% error level.
Results after One Year of PALD Implementation
Gender Interactions with Classroom and PALD Effects: Long Division Scores as Outcomes
Did boys and girls perform differently in different classrooms and did the PALD intervention help erase those differences in Year 1?
Estimate of Classroom Mean / SD / Standard Error / t / df / pSource of Variance / Variance / df / X2 / p
Classroom Achievement / 21.90 / 35 / 260.412 / .00
Gender Slopes / 6.39 / 37 / 56.32 / .02
Student Achievement within Classrooms / 60.39
16
[1] General Education Expenditures Per Pupil=$11,132; Special Education Expenditures Per Pupil=$20,868
[2] General Education Expenditures Per Pupil=$8,177; Special Education Expenditures Per Pupil=$17,667
[3] To meet the federal definition of “highly qualified,” public school teachers of core academic subjects must have at least a bachelor’s degree and be State certified for and demonstrate subject matter competency in the core academic subject(s) they teach.
[4] Includes teachers with a modified temporary license. 1% in East Ramapo equates to 26 teachers out of 732.
[5] This is an urban or suburban school district with high student needs in relation to district resource capacity.