Mississippi Statewide Accountability System

School Level Accountability Model Based on Achievement and Growth

For Approval by the Mississippi State Board

of Education for Use in Fall 2005

November 18, 2004

The following specifications describe the school level accountability model recommended for use in assigning School Performance Classifications in Fall 2005. The model is the same as that approved by the Mississippi State Board of Education in October 2003 for use in Fall 2004 except for the addition of one secondary school variable. The proposed revisions to the model are indicated in this document using highlighted text.

Students Included for Accountability

A student is included for accountability if (this is the definition for "full academic year")

· End of Month 8 School = Same School on 6 of the 7 Earlier End of Month Records (Month 1 through Month 7)

· End of Month 7 School = Same School on all 6 of the Earlier End of Month Records (Months 1 through 6)

Achievement Model Calculations

· Calculate % Basic and Above and % Proficient and Above at each grade (3-8) and content area for MCT administered at grade level.

· Calculate % Basic and Above and % Proficient and Above at each test level (12-18) and content area for MCT instructional level tests.

· Calculate % Pass and % above an established higher cut point for each subject area test in specific grades -- Algebra I taken in grades 8-12, Biology I taken in grades 9-12, English Reading/Language taken in grades 9-11, and U.S. History from 1877 taken in grades 10-12.

· Calculate % Approaching Proficiency and Above and % Proficient for all students participating in a low stakes alternate assessment.

· Convert the school’s values on the (up to 55) variables above to z-score equivalents using state mean and standard deviation from the base year. This standardizes the variables.

· Weight each variable based on the student n-count on that variable compared to the total full academic year n-count for the school.

· Calculate the Basic Achievement Index (BAI)

· Sum all the weighted z-scores based on the basic achievement measures (% Basic and Above on MCT, % Pass on SATP, % Approaching Proficiency or above on alternate assessments).

· Apply the two established index cut scores to determine whether basic achievement will place the school in achievement level 1 or 2.

· Calculate the Higher Achievement Index (HAI)

· Sum all the weighted z-scores based on the higher achievement measures (% Proficient and Above on MCT, % meeting higher criterion on SATP, % Proficient on alternate assessments).

· Apply the two established index cut scores to determine whether higher achievement will place the school in achievement level 4 or 5.

· Assign school to an achievement level based on BAI and HAI. The achievement level is NOT the School Performance Classification.

Growth Model -- Inclusion Criteria

· Apply additional criteria for student inclusion -- MCT.

· Must have MCT (on grade) data in at least one content area for the school year just ended (call this Year 2).

· Must have MCT (on grade) data in all three content areas for school year before that (call this Year 1).

· MCT scale scores in all three content areas for Year 1 and in the predicted content area for Year 2 must not fall in the “chance score” range.

· MCT level in Year 2 must be one level greater than the level in Year 1.

· Apply additional criteria for student inclusion -- SATP.

· SATP for Year 2 must be Grade 8, Grade 9 or Grade 10 Algebra I; Grade 9 or Grade 10 Biology I; Grade 10 English II; or Grade 11 US History.

· Must have MCT (on grade) data in all three content areas at next lower grade level for Year 1 (for predicting Grade 8 or 9 SATP scores), Grade 8 MCT data from two years earlier (for predicting Grade 10 SATP scores), or Grade 8 MCT data from three years earlier (for predicting Grade 11 SATP score).

· MCT scale scores in the earlier years must not fall in the “chance score” range.

Growth Model -- Assumptions

· Assumptions Underlying the MCT in the Growth Model

· A student with a higher overall achievement level might make more gain than a student with lower overall achievement.

· A student whose score in one content area was high at the beginning of the school year might gain less than a student whose initial score was lower (due to regression to the mean).

· A student’s gain on the MCT can be predicted given that student’s initial achievement on the MCT.

· Assumptions Underlying the SATP in the Growth Model

· A student with a higher overall MCT achievement level might score higher on the SATP than a student with lower overall achievement.

· A student’s score on the SATP (Algebra I, Biology I, English II, US History) can be predicted given that student’s MCT achievement one, two, or three years earlier.

Growth Model -- Predictions

· Predict MCT gain from Year 1 MCT in each content area for each student in each cohort (e.g., Cohort = Grade 2 to Grade 3).

· Apply regression coefficient for overall achievement (based on the sum of the student’s Year 1 scale scores in reading, language, and mathematics).

· Apply regression coefficient for regression to the mean (based on the student’s Year 1 scale score in the content area being predicted).

· Add the constant (y-intercept) value from the regression equation.

· The result is the expected scale score gain from Year 1 to Year 2.

· Subtract the predicted gain from the student’s actual gain from Year 1 to Year 2.

· The result is the degree to which the student met expected gain, stated in scale score points. This value is called the student’s raw growth residual.

· Predict Algebra I, Biology I, English II, and US History Year 2 Scale Score from MCT (grade 8 and 9 SATP scores are predicted from Year 1 data; grade 10 SATP scores are predicted from MCT data two years earlier; grade 11 SATP scores are predicted from MCT data three years earlier).

· Apply regression coefficient to the student’s MCT reading scale score.

· Apply regression coefficient to the student’s MCT language scale score.

· Apply regression coefficient to the student’s MCT math scale score.

· Add the constant (y-intercept) value from the regression equation.

· The result is the expected SATP scale score.

· Subtract the predicted score from the student’s actual score.

· The result is the degree to which the student met the expected score, stated in scale score points. This value is called the student’s raw growth residual.

Growth Model Calculations -- "Met Growth"

· Average the student level raw growth residuals within each prediction cohort (i.e., MCT Reading Grade 2 to Grade 3, … English II Grade 10). This produces a set of mean raw residual values for the school.

· Convert the school’s mean raw residual value for each prediction cohort to a z-score equivalent using the standard deviation from the base year. This step standardizes the residual means.

· Calculate a total student n-count by summing the n-counts for each prediction cohort containing at least 10 students.

· Weight each standardized growth residual based on the student n-count in that prediction cohort compared to the total accountability n-count for the school.

· Sum all the weighted z-scores for prediction cohorts containing at least 10 students. This is the school level standardized/weighted growth composite.

· Apply a 95% confidence interval to the school’s growth composite. The confidence interval is two standard errors of the mean for the distribution of over 800 school growth composites.

· Determine whether the school’s growth composite falls below the lower bound of the confidence interval.

· If the growth composite is at or above the lower bound value, the school met its growth expectation.

· If the growth composite falls below the lower bound value, the school did not meet its growth expectation.

Growth Model Calculations -- "Exceeded Growth"

· Determine whether the school exceeded its growth expectation by at least 10%. The procedure is the same as that used to determine whether the school met its growth expectation (described above), except that the constant/y-intercept value in each prediction equation is increased by 10% before using the equation to predict a student’s MCT gain or SATP score.

· Apply an appropriate confidence interval to the school’s “exceeded” growth composite.

· Determine whether the school’s growth composite falls below the lower bound of the confidence interval.

· If the growth composite is at or above the lower bound value, the school “exceeded” its growth expectation.

· If the growth composite falls below the lower bound value, the school did not “exceed” its growth expectation.

School Performance Classifications

· A school’s achievement level (1-5) and growth status (not met, met, or exceeded) are combined to determine the School Performance Classification. No school meeting its growth expectation is assigned a classification below 3 (Successful).

· School Performance Classifications:

· Level 5 (Superior-Performing) = School is in achievement level 5.

· Level 4 (Exemplary) = School is in achievement level 4 -OR- school exceeded its growth expectation and is in achievement level 3.

· Level 3 (Successful) = School met its growth expectation -OR- school failed to meet its growth expectation and is in achievement level 3.

· Level 2 (Under-Performing) = School failed to meet its growth expectation and is in achievement level 2.

· Level 1 (Low-Performing) = School failed to meet its growth expectation and is in achievement level 1. Some of the schools in School Performance Classification Level 1 may be designated Priority Schools.

Schools that Cannot be Included in the Achievement and/or Growth Models

· Only schools that can be included in the achievement and growth models are assigned a School Performance Classification.

· Schools that can be included only in the achievement model will not be assigned a School Performance Classification, but the achievement level will be reported.