COLORADO READING FIRST
EXTERNAL EVALUATION REPORT
July 14, 2006 to July 15, 2007
Reported by:
Rose Shaw, Ph.D.
Metrica
1703 36th Avenue Court
Greeley, CO 80634-2807
Table of ContentsCOHORT 1: CRF SUSTAINABILITY / Page 1
COHORT 2: INTERMEDIATE OUTCOME EVALUATION / Page 6
THE SPECIAL EDUCATION OBJECTIVE / Page 7
CRF STRENGTHS IN 2006-07 / Page 10
FORMATIVE EVALUATION / Page 10
Feedback from Principals
Feedback from Regional Consultants
Progress Monitoring Assessment Information
Coaches Feedback on Sopris Trainings
FORMATIVE EVALUATION: QUARTERLY REPORTS / Page 15
July-August-September 2006
October-November-December 2006
January-February-March 2007
April-May-June 2007
COHORT 1: CRF SUSTAINABILITY
The ultimate goal of Colorado Reading First is that all children will read at or above grade level by the end of third grade (as measured by proficient or above on the English CSAP). Cohort 1 was funded for three years: 2004-07. Four LEAs (11 schools) were provided fourth year funding. The following data pertain to sustainability in Cohort 1 schools.
Cohort 1 CSAP data were provided by the Colorado Department of Education (CDE) Assessment Department in 2003 (baseline), 2004 (year 1) and 2005 (year 2). The External Evaluator summarized the 2006 (year 3) and 2007 (post-CRF) CSAP results using data posted on the CDE website. The following summative statements refer to Table 1 data. The state proficiency levels (percentage of students proficient or advanced) in 2003 (pre-CRF) and 2007 (4th year) were 74% and 71%, respectively.
· Among the eight 4th year funded schools for the five years (2003-07)
6 (75%) of the 8 schools were at or above the state proficiency level in 2003
These schools were: Fountain, Holly (Shanner), Spann, Liberty, Byers, Bennett
5 (63%) of the 8 schools were at or above the state proficiency level in 2007
These schools were: Wiley, Spann, Byers, Bennett, and Holly (Shanner)
· Among the 18 schools with three years of funding for the five years displayed in Table 1:
2 (11%) of the schools were at or above the state proficiency level in 2003
These schools were: Merino and Nisley
4 (22%) of the schools were at or above the state proficiency level in 2007
These schools were: Merino, Northside, Mesa View, and Burlington
· Four schools that were below the state proficiency level in 2003 and above the state proficiency level in 2007 were:
School / 2003 (Pre-CRF)Proficient/Above / 2007 (4th Year)
Proficient/Above
Northside Elementary / 50% / 82%
Mesa View Elementary / 66% / 81%
Wiley Elementary / 67% / 88%
Burlington Elementary / 73% / 87%
The percentages of students scoring at or above third grade level as measured by the English CSAP are displayed below for Colorado and each Cohort 1 school. The two groups of schools (4th year funded and not 4th year funded) are displayed in descending order of the change in proficiency levels prior to CRF (2003) and in year 4 (2007).
Table 1: Colorado and CRF Cohort 1 percentages of third grade students scoring proficient or advanced from 2003 (pre-CRF) to 2007 (year 4)
Students / 4th Year Funded? / 2003Pre-CRF / 2004 / 2005 / 2006 / 2007
Year 4 / Change Pre to Yr 4 /
Colorado / N/A / 74 / 74 / 71 / 70 / 71 / -3
Wiley / Yes / 67 / 74 / 90 / 73 / 88 / +13
Spann / Yes / 77 / 81 / 84 / 69 / 87 / +10
Bennett / Yes / 75 / 76 / 73 / 90 / 75 / 0
Byers / Yes / 76 / 64 / 81 / 91 / 76 / 0
Liberty / Yes / 74 / 75 / 82 / 75 / 70 / -4
Irving / Yes / 69 / 69 / 67 / 75 / 64 / -5
Holly (Shanner) / Yes / 80 / 75 / 65 / 69 / 72 / -12
Fountain / Yes / 81 / 66 / 89 / 81 / 67 / -13
Granada / Yes / 52 / 64 / 76 / 61 / No data / N/A
Karval / Yes / Too few students to calculate percentage.
Northside / No / 50 / 70 / 58 / 65 / 82 / +32
Mesa View / No / 66 / 82 / 69 / 65 / 81 / +15
Burlington / No / 73 / 78 / 58 / 82 / 87 / +14
Bea Underwood / No / 49 / 63 / 48 / 38 / 59 / +10
Edgewater / No / 53 / 49 / 61 / 60 / 62 / +9
Merino / No / 87 / 80 / 89 / 70 / 90 / +3
Centennial (SD6) / No / 53 / 43 / 50 / 56 / 56 / +3
Monterey / No / 54 / 50 / 54 / 63 / 56 / +2
Alice Terry / No / 39 / 64 / 73 / No third grade
Miami-Yoder / No / 60 / 40 / 71 / 50 / 59 / -1
Haskin / No / 54 / 50 / 58 / 53 / 53 / -1
Centennial (Harr) / No / 49 / 41 / 48 / 56 / 46 / -3
Twombly / No / No data / 40 / 47 / 36 / 36 / ~ -4
Eiber / No / 60 / 57 / 55 / 47 / 55 / -5
Green Acres / No / 56 / 83 / 68 / 64 / 48 / -8
Lumberg / No / 59 / 56 / 39 / 50 / 49 / -10
Butler / No / 63 / 59 / 52 / 47 / 47 / -16
Ft. Logan / No / 45 / 62 / 69 / 56 / 64 / -19
Nisley / No / 74 / 71 / 71 / 72 / 51 / -23
Alice Terry / No / 39 / 64 / 73 / No third grade
Sierra Grande / No / 74 / 65 / 52 / 75 / No data / N/A
Since the ultimate goal of RF is for all students to be proficient readers at the end of third grade, it is important to know not only the percentage of students scoring proficient or above but also the percentages of students scoring “unsatisfactory” and “advanced” on the third grade CSAP. Ideally, the percentages of students scoring unsatisfactory on the third grade CSAP should DECREASE and the percentages of students scoring advanced on the third grade CSAP should INCREASE. Ideally RF schools are called to serve the high risk students as well as the advanced students. One possible pitfall in RF schools as they concentrate on improving reading in students at-risk for reading is that the students who excel at reading will not be challenged to the degree that best serves them.
Table 2 displays the percentages of students scoring “unsatisfactory” from 2003 to 2007. Ideally from 2003 the percentage of “unsatisfactory” scores decreased (change is a negative number) and the percentage scoring “advanced” increased (change is a positive number). The schools in the table are separated into four categories:
Category 1: Schools where the percentages of students that scored “unsatisfactory” decreased and students scoring “advanced” increased.
Category 2: Schools where the percentages of students that scored “unsatisfactory” decreased and students scoring “advanced” decreased.
Category 3: Schools where the percentages of students that scored “unsatisfactory” increased and students scoring “advanced” increased.
Category 4: Schools where the percentages of students that scored “unsatisfactory” increased and students scoring “advanced” decreased.
Ideally, the change in “unsatisfactory” scores was a negative number and the change in “advance” scores was a positive number. The least desirable pair of numbers was an increase in “unsatisfactory” scores (a positive number) and a decrease in “advanced” scores (a negative number).
From 2003 (pre-CRF) to 2007 (year 4):
· For Colorado, the percentage of students that scored “unsatisfactory” increased 3-points, and the percentage of students that scored “advanced” decreased 3-points (Category 4).
· For 4th year funded CRF schools:
Category 1: Decreased “unsatisfactory” and increased “advanced” occurred for one school: Irving
Category 2: Decreased “unsatisfactory” and decreased “advanced” occurred for one school: Holly (Shanner).
Category 3: Increased “unsatisfactory” and increased “advanced” occurred for 4 schools: Wiley, Spann, Bennett and Byers.
Category 4: Increased unsatisfactory and decreased advanced occurred for two schools: Fountain and Liberty
· For schools that were funded for three years:
Category 1: Decreased “unsatisfactory” and increased “advanced” occurred for 6 schools: Northside, Mesa View, Monterey, Miami-Yoder, Butler and Ft. Logan.
Category 2: Decreased “unsatisfactory” and decreased “advanced” occurred for three schools: Edgewater, Merino and Centennial (Harrison SD).
Category 3: Increased “unsatisfactory” and increased “advanced” occurred for one school: Nisley.
Category 4: Increased unsatisfactory and decreased advanced occurred for 7 schools: Burlington, Bea Underwood, Centennial (Greeley 6 SD), Haskin, Eiber, Green Acres and Lumberg.
Table 2: Pre-CRF (2003) and 4th year (2007) percentages of students scoring “unsatisfactory” and the percentages scoring “advanced” grouped by defined category
Students / Category / Percentage Unsatisfactory / Percentage Advanced / Change /2003
Pre-CRF / 2007
Year 4 / 2003
Pre-CRF / 2007
Year 4 / Un-satisfactory / Advanced /
Colorado / 4 / 7 / 10 / 10 / 7 / +3 / -3
Irving / 1 / 5 / 2 / 0 / 2 / -3 / +2
Holly (Shanner) / 2 / 12 / 6 / 8 / 0 / -6 / -8
Wiley / 3 / 0 / 6 / 0 / 12 / +6 / +12
Spann / 3 / 3 / 10 / 0 / 3 / +7 / +3
Bennett / 3 / 4 / 13 / 2 / 3 / +9 / +1
Byers / 3 / 3 / 11 / 0 / 3 / +8 / +3
Liberty / 4 / 1 / 11 / 8 / 0 / +10 / -8
Fountain / 4 / 2 / 7 / 8 / 5 / +5 / -3
Northside / 1 / 17 / 8 / 3 / 12 / -9 / +9
Mesa View / 1 / 15 / 10 / 3 / 8 / -5 / +5
Monterey / 1 / 14 / 9 / 2 / 5 / -5 / +3
Miami-Yoder / 1 / 15 / 6 / 4 / 6 / -9 / +2
Butler / 1 / 13 / 10 / 2 / 3 / -3 / +1
Ft. Logan / 1 / 20 / 18 / 1 / 2 / -2 / +1
Edgewater / 2 / 21 / 13 / 4 / 2 / -8 / -2
Merino / 2 / 9 / 0 / 9 / 7 / -9 / -2
Centennial (Harr) / 2 / 19 / 15 / 4 / 1 / -4 / -3
Nisley / 3 / 4 / 18 / 1 / 3 / +14 / +2
Burlington / 4 / 3 / 6 / 19 / 13 / +3 / -6
Bea Underwood / 4 / 17 / 18 / 5 / 2 / +1 / -3
Centennial (SD6) / 4 / 20 / 23 / 3 / 0 / +3 / -3
Haskin / 4 / 22 / 26 / 5 / 0 / +4 / -5
Eiber / 4 / 12 / 22 / 4 / 2 / +10 / -2
Green Acres / 4 / 15 / 22 / 3 / 1 / +7 / -2
Lumberg / 4 / 7 / 9 / 5 / 0 / +2 / -5
Two Cohort 1 Schools Stand Out in this Sustainability Analysis
From 2003 (pre-CRF) to 2007 (4th year) two Cohort 1 schools increased the percentages of students proficient and the percentages of student scoring advanced while decreasing the percentage of students scoring “satisfactory” with year 4 proficiency levels at 82% and 81%: Northside Elementary (Montrose RE-1J) and Mesa View (Mesa County Valley SD 51), free-reduced lunch 64% and 37%, respectively.
From the beginning of year 1to the end of year 3 of CRF funding Northside retained 63% of its K-3 teachers and Mesa View retained 10%.
COHORT 2 INTERMEDIATE OUTCOME EVALUATION
The emphasis in Cohort 2 schools during the first year was on implementing the core therefore substantial increases in CSAP scores were not expected.
Cohort 2 schools are in the third and final year of funding. Disaggregated data are maintained and reported by the Colorado Department of Education so they are not included in this report. Proficiency levels for Cohort 2 schools are displayed in Table 3.
· Thirteen Cohort 2 schools recorded proficiency levels (proficient or above) at or above the state in 2007; 10 of these schools were at or above the state proficiency level in 2005 (pre-CRF).
· Eighteen Cohort 2 schools were below the state proficiency level (71% proficient or advanced) in 2007; three of these schools were above the state proficiency level (71%) pre-CRF.
Table 3: Percentages proficient or above for Colorado and Cohort 2 schools from pre-CRF (2005) to end of second year (2007)
Students / 2005 (Pre-CRF) / 2006 (Year 1) / 2007 (Year 2) / Change from Pre to Yr 2 /Colorado / 71 / 70 / 71 / 0
Las Animas / 85 / 87 / 94 / +9
Mancos / 75 / 70 / 89 / +14
Sargent / 93 / 82 / 89 / -4
Guadalupe / 96 / 80 / 88 / -8
Washington / 76 / 84 / 87 / +11
Pagosa Springs / 78 / 78 / 83 / +5
Baca / 78 / 84 / 82 / +4
Webster / 87 / 62 / 80 / -7
Wiggins / 81 / 82 / 76 / -5
West Park / 45 / 62 / 74 / +29
Columbian / 86 / 78 / 74 / -12
Park View / 70 / 67 / 72 / +2
Fremont / 60 / 60 / 71 / +11
Underwood / 63 / 59 / 70 / +7
Holyoke / 74 / 78 / 69 / -5
Dupont / 38 / 67 / 68 / +30
Cedaredge / 75 / 89 / 67 / -8
Pioneer / 71 / 58 / 67 / -4
North / 52 / 54 / 66 / +14
Johnson / 57 / 62 / 65 / +8
Columbine / 66 / 74 / 65 / -1
Longfellow / 68 / 71 / 63 / -5
Scott / 68 / 66 / 63 / -5
Stratton Meadows / 69 / 66 / 60 / -9
Queen Palmer / 34 / 32 / 56 / +22
Hanover / 60 / 77 / 56 / -4
Bill Metz / 58 / 70 / 56 / -2
Ellicott / 44 / 50 / 55 / +11
Kemp / 52 / 42 / 49 / -3
Stratmoor Hills / 52 / 59 / 48 / -4
Olathe / 53 / 41 / 43 / -10
The Special Education Objective
Reading First Objective 8.2 of 3: To decrease the percentage of kindergarten through third grade students in schools participating in Reading First who are referred for special education services based on their difficulties learning to read.
Indicator 8.2.1 of 1: Referrals to Special Education: percentage of RF K-3 students referred for special education services based on their difficulties learning to read.
The data in this section pertain to the National Reading First Objective 8.2 of 3 and Indicator 8.2.1.
The percentages of students referred because of reading difficulties to Special Education during the 2004-05, 2005-06 and 2006-07 school years are displayed (Table 4) for all Cohort 2 schools. The average referral rate for reading difficulties was 1.5% (Std. Dev. = 1.9) and the third quartile was 2.3%. Five schools with K-3 enrollment greater than 100 with referral rates for reading difficulties that exceeded 3% were:
Mesa and Underwood (6.8%)
Queen Palmer (6.0%)
Stratton Meadows (3.4%)
Dupont (3.1%)
Table 4: Percentage of Cohort 2 K-3 students referred to SPED based on reading difficulties