NCLB Assessment Peer Review Briefing for State Board

Office of Assessment and Information Services

June 15, 2006

Background:

The status of Oregon’s peer review was listed as “Final Review Pending”. Oregon Department of Education (ODE) staff has been working diligently to address our findings.Access to the full Oregonreport and other state reports are at

Findings of the peers identifies where Oregondid not supply adequate evidence to demonstrate compliance with federal requirements. This briefing will highlight those activities planned or being performed to address these and othersystemic issues identified by ODE staff, stakeholders and peer reviewers.

Systemic Issues:

The following table outlines work the Department is doing to maintain and mature our system. These issues did not have a direct impact on Oregon’s peer review status.

Issue / Recap / Board Role
Technical Advisory Committee / Oregon has not had a committee of national assessment experts to advise the ODE since 2002. We held our first meeting in April and are confident the group of experts we chose will provide much need advice regarding best practices in assessments
  • First meeting covered history of Oregon Assessment, results of Peer Review
  • 11/17/06 meeting will cover Standards Setting and Comparability Study results
/ Information only
Technical Documentation / Issued contract to convert Oregon’s body of evidence regarding the technical adequacy of its system to a technical manual. Reviewers indicated they had difficulty in locating information easily and recommended this strategy. This manual will benefit our stakeholders as well. / Information only
Test Administration / The following contracts are being released this year for the following activities:
  • TESA – Web based testing piloting in 2006-07 and live in 2007-08
  • Paper and Pencil Test Administration beginning in 2006-07
  • ESD Test Administration and Writing Scoring Contract beginning in 2006-07
  • Test Security study beginning in 2006-07
  • English Language Proficiency Assessment beginning in 2006-07
/ Information only

Additional Evidence Required:

USED technical assistance was provided to assist states in achieving full approval as quickly as possible. There were four sessions identified based on common deficiencies across all states: Form Comparability, Alternative Assessments, Alignment and Performance Descriptors. The tableon the reverse side of this document summarizes areas that Oregonmust address to bring the Assessment system into compliance with the Federal Requirements of Title I and NCLB.

Evidence / Recap / Board Role
Form Comparability / Oregon has a number of forms that must have either an equating or linking study performed, i.e. Juried Assessment, TESA, Paper/Pencil, Extended, Russian and Spanish versions. There are two types of studies which need to be addressed depending on Form type:
  • Equating – has a higher standard and relates to correlation between test forms i.e. TESA and P/P
  • Linking – forms are related, but not the same such as Spanish Reading at Grade 3
Studies will validate comparability within a year between forms and across time. Due to the number of forms Oregon allows, this will be difficult and time-consuming. / Information only
Alternate Assessments / Assessments for students with disabilities (SWD) based on alternate achievement standards are the biggest challenge for states. USED is look for a new way to instruct and assess SWD. 70% of students with disabilities can reach grade level standards. Cannot undercut students with disabilities so that the full range of student ability can be measured. Essential Elements of an Alternate Assessment include:
  • Yield results separately in reading and math.
  • Clear guidelines for the administration of the assessment
  • Assessments should be designed to support the AYP determinations (i.e. aligned with state content standards, comparable across schools, test admin and training manual, test specifications.
  • Evidence of validity and reliability with national recognized technical standards.
  • Results must be reported to parents and teachers.
  • Multiple alternative achievement standards must show relationship among assessments.
To address this new assessment, the ODE has applied for a federal grant with the assistance of the University of Oregon to develop an Enhanced Assessment system. / Information only
Alignment / Alignment between Oregon’s Assessments and Content Standards will be reviewed as part of a joint project funded by Gates. While this project will take a look at the alignment between the two, it will be important for us to think about the full scope of how content standards relate to our item pool, to what extent our students are exposed to the full breadth and depth of items through the design of our tests and finally how the ODE plans to implement and maintain the recommendations from the Study. / Information only
AchievementStandards Setting and Performance Descriptors / Allows States to report out and explain to readers what the results mean. Achievement standards must include a list of grade specific content competencies. If done well, it can support the rest of the assessment system. The development of performance descriptors will be part of the work done in conjunction with the creation of content standards. Following are major activities Board will want to follow:
  • Summer 2006 – Performance Standards Setting planning and logistics
  • Winter 2006-07 – Performance Standards Setting conducted
  • Spring 2007 – State Board hearings and adoption of standards
  • Fall 2007 – Districts held accountable for new standards – pending USDE approval
This project has been delayed due to resource issues at DAS. / Board must adopt new descriptors and Performance Standards