Kent School District Assessment Office

Kent School District Assessment Office

i-Ready and Smarter Balanced Regression Study

We wanted to let you know that there will be a linking study of i-Ready and Smarter Balanced Assessment (SBA) test score results conducted by with Educational Research Institute of America (ERIA), an outside independent evaluator. This study will also include school districts similar to our own across the country. Results will be available sometime this fall, according to the i-Ready vendor.

In the meantime, we have conducted our own analysis comparing i-Ready and Smarter Balanced results for students in grades 3 through 8 in the Kent School District. We matched student scale scores from the winter window of i-Ready in Reading and in Math, with student scale scores in Smarter Balanced English Language Arts/Literacy (ELA) and Smarter Balanced Mathematics. All scores are from school year 2014-15, and only students who had taken both types of assessments, i-Ready and SBA, were included in the study. The data showed a strong, positive relationship. Correlations between i-Ready and Smarter Balanced were in the range of 0.80 to .90, out of 1.0 possible.

What is a regression study?

Researchers are often interested predicting an average value for Y for given values of X.

This linear relationship between independent and dependent variable can be displayed on a scatter graph as the regression line. The dots on the graph represent student scores on both variables. For this regression study, the independent variable is i-Ready score and the dependent variable is the Smarter Balanced score. We are looking for a linear pattern in the scatter graph display and a moderate to strong correlation coefficient, r, at least 0.50 or –0.50 in magnitude.

We worked with the data in a number of ways, including:

1)  Preparing graphs that show the relationship of i-Ready and SBA by grade level, for Reading/ELA and Math,

2)  Looking at the relationship between i-Ready proficiency levels and SBA achievement levels in cross tabulation format,

3)  Creating an Excel template that allows you to input a score for i-Ready and instantly get a prediction of SBA score by grade level and subject.

1)  Scatter graphs

Scatter graphs shown on the next page display the strong relationship between i-Ready and SBA scores. Examples show grade 3 Reading/ELA and grade 7 Math, along with the regression formula. Similar patterns of data were found for the other grades levels in both Reading/ELA and Math.

2)  Cross Tabulation

Cross tabulation is another way to show the relationship between two sets of assessments, in this case by proficiency level. The cells with yellow highlights display the number and percentages of students who achieve the same levels on both assessments. For example, 89.3% of students scoring well below standard on the Grade 3 i-Ready in window 2 also scored well below standard on the Grade 3 Smarter Balanced ELA summative assessment.

Grade 3 SBA ELA
1 Well Below / 2 Below / 3 Proficient / 4 Advanced / Total
Grade 3 i-Ready Reading W2 / 1 Well Below / 326 / 36 / 2 / 1 / 365
2 Below Level / 174 / 160 / 57 / 4 / 395
3 On Level / 45 / 218 / 412 / 447 / 1122
4 Above Level / 0 / 0 / 1 / 69 / 70
Total / 545 / 414 / 472 / 521 / 1952
Grade 3 SBA ELA
1 Well Below / 2 Below / 3 Proficient / 4 Advanced
Grade 3 i-Ready Reading W2 / 1 Well Below / 89.3% / 9.9% / .5% / .3%
2 Below Level / 44.1% / 40.5% / 14.4% / 1.0%
3 On Level / 4.0% / 19.4% / 36.7% / 39.8%
4 Above Level / 1.4% / 98.6%
Total / 27.9% / 21.2% / 24.2% / 26.7%

Over 90% of students not meeting grade level standard on the Grade 3 i-Ready in window 2 also did not meet standard on the Grade 3 Smarter Balanced ELA. Almost 80% of students meeting standard on the Grade 3 i-Ready, window 2, also met standard on the Grade 3 SBA ELA. Results were similar for the other grade levels, and indicate that i-Ready would be useful in predicting SBA scores.

Grade 3 SBA ELA
No / Yes
Grade 3 i-Ready Reading W2 / No / 91.6% / 8.4%
Yes / 22.1% / 77.9%
Total / 49.1% / 50.9%

3)  Creating an Excel Template to Predict SBA from i-Ready Scores

Methodology

Data

Scores used in creating the SBA Predictor Tool include Smarter Balanced Assessment (SBA) scale scores and i-Ready scale scores. SBA scores were pulled from the Online Reporting System on 7-27-2015 while winter i-Ready Scores were pulled from Skyward on 7-28-2015. The nature of this analysis only allows us to use data for students having scores for both the SBA and the winter, i-Ready assessment. Table 1 contains the number of students in each subject and grade having both scores.

Table 1:
Students Included in Analysis
Math / Reading
Grade / Sample Size / Sample Size
3 / 1,951 / 1,927
4 / 1,859 / 1,841
5 / 2,013 / 2,001
6 / 1,908 / 1,918
7 / 1,542 / 1,524
8 / 1,522 / 1,462
11 / 8 / 44

Due to the low number of students taking both the SBA and i-Ready assessments in 11th grade, we decided not to include this grade level in our analysis.

Predictive Model

A univariate regression model was chosen to estimate students’ SBA scale scores using only the winter i-Ready scale score. By grade and subject, we estimate:

ySBAi= β0+ β1xiReadyi+ εi

Where ySBA denotes the SBA scale score of student i and xiReady denotes his/her winter, i-Ready scale score. Since no model can perfectly predict what a student will score on the SBA, the error term ε is added which represents unobserved, random factors contributing to the imperfect relationship between the two scale scores. The winter i-Ready scale score was selected because it is the last assessment result before students sit for the SBA, giving teachers the most up-to-date academic achievement data with which to inform their instruction for the remaining classroom time before the SBA.

Results

Table 2 provides the estimated intercepts and slopes of the best-fit lines used to predict the SBA scale score by subject and grade. The results demonstrate that i-Ready scores are strongly, statistically significant in predicting SBA scores.

Table 2:
Intercept ( β0) and Coefficient (β1) Estimates
Math / Reading
Grade / β0 / β1 / β0 / β1
3 / 1,433 / 2.27 / 1,670 / 1.45
4 / 1,497 / 2.11 / 1,644 / 1.52
5 / 1,435 / 2.23 / 1,714 / 1.39
6 / 1,264 / 2.58 / 1,729 / 1.39
7 / 1,273 / 2.54 / 1,702 / 1.42
8 / 1,176 / 2.72 / 1,691 / 1.45

All coefficients are significant at the p<0.01 level

The above estimates are then applied in the predictor tool. For example, if a 3rd grade student scored 450 on the winter mathematics i-Ready assessment, we would use the corresponding β0 and β1 to estimate the student’s score:

Predicted SBA Score=ySBAi = β0 +( β1 ×xiReadyi)

Predicted SBA Score=2,455=1,433 +(2.27× 450 )

The predictor tool takes the predicted score and references a table of SBA thresholds published by OSPI to determine the corresponding performance level (Level 1, Level 2, Level 3, or Level 4); this table may be found in table A.1 of the appendix. In this particular example, this 3rd grader would have achieved a performance level of Level 3.

Snapshots of the Excel template

Appendix

Correlation Tables

Grade 3 i-Ready Reading and Smarter Balanced ELA Correlations
i-Ready Reading W1 / i-Ready Reading W2 / i-Ready Reading W3 / SBA ELA Score
i-Ready Reading W1 / Pearson Correlation / 1 / .881** / .860** / .833**
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 2058 / 1959 / 1934 / 1933
i-Ready Reading W2 / Pearson Correlation / .881** / 1 / .893** / .846**
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 1959 / 2026 / 1963 / 1952
i-Ready Reading W3 / Pearson Correlation / .860** / .893** / 1 / .846**
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 1934 / 1963 / 2038 / 1993
SBA ELA Score / Pearson Correlation / .833** / .846** / .846** / 1
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 1933 / 1952 / 1993 / 2026
**. Correlation is significant at the 0.01 level (2-tailed).
Grade 4 i-Ready Reading and Smarter Balanced ELA Correlations
i-Ready Reading W1 / i-Ready Reading W2 / i-Ready Reading W3 / SBA ELA Score
i-Ready Reading W1 / Pearson Correlation / 1 / .916** / .903** / .841**
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 1937 / 1859 / 1827 / 1814
i-Ready Reading W2 / Pearson Correlation / .916** / 1 / .922** / .854**
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 1859 / 1946 / 1882 / 1860
i-Ready Reading W3 / Pearson Correlation / .903** / .922** / 1 / .872**
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 1827 / 1882 / 1952 / 1902
SBA ELA Score / Pearson Correlation / .841** / .854** / .872** / 1
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 1814 / 1860 / 1902 / 1923
**. Correlation is significant at the 0.01 level (2-tailed).
Grade 5 i-Ready Reading and Smarter Balanced ELA Correlations
i-Ready Reading W1 / i-Ready Reading W2 / i-Ready Reading W3 / SBA ELA Score
i-Ready Reading W1 / Pearson Correlation / 1 / .894** / .875** / .841**
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 2082 / 2008 / 1985 / 1985
i-Ready Reading W2 / Pearson Correlation / .894** / 1 / .895** / .852**
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 2008 / 2088 / 2032 / 2023
i-Ready Reading W3 / Pearson Correlation / .875** / .895** / 1 / .860**
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 1985 / 2032 / 2091 / 2051
SBA ELA Score / Pearson Correlation / .841** / .852** / .860** / 1
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 1985 / 2023 / 2051 / 2077
**. Correlation is significant at the 0.01 level (2-tailed).
Grade 6 i-Ready Reading and Smarter Balanced ELA Correlations
i-Ready Reading W1 / i-Ready Reading W2 / i-Ready Reading W3 / SBA ELA Score
i-Ready Reading W1 / Pearson Correlation / 1 / .893** / .889** / .821**
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 2003 / 1933 / 1909 / 1910
i-Ready Reading W2 / Pearson Correlation / .893** / 1 / .907** / .824**
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 1933 / 2010 / 1955 / 1946
i-Ready Reading W3 / Pearson Correlation / .889** / .907** / 1 / .844**
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 1909 / 1955 / 2011 / 1970
SBA ELA Score / Pearson Correlation / .821** / .824** / .844** / 1
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 1910 / 1946 / 1970 / 2003
**. Correlation is significant at the 0.01 level (2-tailed).
Grade 3 i-Ready Math and Smarter Balanced Math Correlations
i-Ready Math W1 / i-Ready Math W2 / i-Ready Math W3 / SBA Math Score
i-Ready Math W1 / Pearson Correlation / 1 / .829** / .815** / .806**
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 2050 / 1964 / 1920 / 1933
i-Ready Math W2 / Pearson Correlation / .829** / 1 / .850** / .840**
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 1964 / 2039 / 1969 / 1982
i-Ready Math W3 / Pearson Correlation / .815** / .850** / 1 / .882**
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 1920 / 1969 / 2030 / 2001
SBA Math Score / Pearson Correlation / .806** / .840** / .882** / 1
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 1933 / 1982 / 2001 / 2043
**. Correlation is significant at the 0.01 level (2-tailed).
Grade 4 i-Ready Math and Smarter Balanced Math Correlations
i-Ready Math W1 / i-Ready Math W2 / i-Ready Math W3 / SBA Math Score
i-Ready Math W1 / Pearson Correlation / 1 / .867** / .846** / .835**
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 1943 / 1865 / 1823 / 1829
i-Ready Math W2 / Pearson Correlation / .867** / 1 / .880** / .867**
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 1865 / 1950 / 1877 / 1887
i-Ready Math W3 / Pearson Correlation / .846** / .880** / 1 / .881**
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 1823 / 1877 / 1946 / 1922
SBA Math Score / Pearson Correlation / .835** / .867** / .881** / 1
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 1829 / 1887 / 1922 / 1952
**. Correlation is significant at the 0.01 level (2-tailed).
Grade 5 i-Ready Math and Smarter Balanced Math Correlations
i-Ready Math W1 / i-Ready Math W2 / i-Ready Math W3 / SBA Math Score
i-Ready Math W1 / Pearson Correlation / 1 / .866** / .856** / .834**
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 2078 / 1994 / 1986 / 1986
i-Ready Math W2 / Pearson Correlation / .866** / 1 / .901** / .865**
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 1994 / 2077 / 2027 / 2027
i-Ready Math W3 / Pearson Correlation / .856** / .901** / 1 / .886**
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 1986 / 2027 / 2096 / 2080
SBA Math Score / Pearson Correlation / .834** / .865** / .886** / 1
Sig. (2-tailed) / 0.000 / 0.000 / 0.000
N / 1986 / 2027 / 2080 / 2100
**. Correlation is significant at the 0.01 level (2-tailed).
Grade 6 i-Ready Math, Orleans-Hanna, and Smarter Balanced Math Correlations
i-Ready Math W1 / i-Ready Math W2 / i-Ready Math W3 / Orleans Hanna Gr6 Score / SBA Math Score
i-Ready Math W1 / Pearson Correlation / 1 / .901** / .880** / .745** / .870**