STATE BOARD OF EDUCATION – TOPIC SUMMARY

Topic: Diploma Implementation Update

Date: March 13, 2008

Staff/Office:Salam Noor, Educational Improvement and Innovation, ODE

Doug Kosty, Assessment and Information Services, ODE

Action Requested: Information only Policy Adoption Policy Adoption/Consent Calendar

ISSUE BEFORE THE BOARD:

Update on diploma implementation and reports from the Essential Skills Task Force on Essential Skills definitions and from the Standards and Assessment Task Force on assessment of the Essential Skills.

BACKGROUND:

The State Board of Education adopted new graduation requirements in January 2007. The new diploma requirements define what students must know and be able to do to earn a high school diploma. Rollout and implementation of the new diploma requirements involves partners and stakeholders from K-12, community colleges, Oregon University System, Oregon Legislature, business sector, students, parents, and community organizations. Five taskforces, made up of key stakeholders, have been formed to further define the requirements and work on issues related to implementation.

Rollout and implementation of the diploma requirements is ODE’s top priority. ODE is working with partners to provide the field with information, tools, resources, and appropriate supports to successfully implement the diploma requirements. Tools such as Questions and Answers documents, Guidance documents, Self Assessment/Readiness criteria, Implementation Tool Kit for administrators and teachers, Successful and Promising practices, Research and Evidence-Based practices, and Effective Instructional Strategies are being developed. ODE implementation team has also begun identifying products that will be designed at the state level including a quality assurance process and a statewide professional development plan.

In addition to tools and resources, ODE continues to staff and support the various implementation task forces. The Essential Skills Task Force met in August, September, October, November, and December, 2007 to define the Essential Skills. Cross-sector teams collaborated on defining the essential skills and indicators. In December 2007, staff presented Essential Skills definitions Draft 5.0 to the Board. This draft was disseminated for public review via a web-based survey from January 29th through February 22nd and over 500 Oregonians responded from across the state. The Essential Skills Task Force will meet on March 11th to review input and refine the Essential Skills definitions. A revised draft will be presented at the March Board meeting.

The Standards and Assessment Task Force met in November 2007, January and February 2008. They have focused on issues related to assessment of the essential skills, including recommendations on assessment options, proficiency levels, an implementation timeline and a phase-in plan, and resource and professional development needs. Staff will present a draft framework for assessment of the essential skills and recommended assessment options for implementation in Fall 2008.

.


State Board of Education, March 13, 2008

Diploma Implementation Update

DIPLOMA PROJECT COORDINATION

  • Diploma Implementation Project Plan developed to coordinate and manage all task force work, communications, and ODE diploma-related activities.
  • Stakeholder-Engagement Plan developed to ensure involvement all critical audiences, track outreach efforts, and coordinate a feedback process.

COMMUNICATIONS

  • Presentation toolkit finalized; PowerPoint and brochure to deliver common message about the Oregon Diploma to be disseminated.
  • Oregon Diploma video contest completed generating nearly 60 entries from across the state.
  • Editorial Board meetings held with Mail Tribune, Oregonian, News-Register, and Statesman Journal.
  • Student roundtables with middle school students held at Blachly, McMinnville, Lebanon.

TASK FORCES

Implementation Advisory Task Force

  • Met February 7, 2008; next meeting scheduled for May 22nd
  • Received updates from the all task forces and provided input on key issues.
  • Members are communicating about the diploma requirements with their various constituents.
  • Previewed a presentation toolkit in production targeted for various audiences and provided feedback.
  • Members:

Cost Capacity Task Force

  • First meeting on March 3, 2008
  • Meeting will focus on creating a common understanding of the task force charge and its relationship with the other task forces.
  • Task force will develop strategies to estimate type and level of required resources, determine current capacity, evaluate barriers to successful implementation, and develop a costing model to estimate costs for districts to acquire the resources they currently lack.
  • Members:

Credit for Proficiency Task Force

  • First meeting February 5, 2008; next meeting scheduled for March 19th
  • Meeting focused on creating a common foundation and understanding of the task force charge – to create criteria and guidelines for the new Oregon Diploma regarding credit for proficiency.
  • Members reviewed the development of current credit options rules adopted in 2002 and four school districts implementing credit for proficiency programs presented to the group.
  • Members discussed opportunities and key issues and questions concerning implementation to be addressed.
  • Members:

Essential Skills Task Force

  • Met December 11th and developed Draft 5.0 Essential Skills definitions for public review.
  • A survey was disseminated and posted on the ODE website January 29th -February 22nd, and over 500 Oregonians responded from across the state; a summary of results is being drafted and will be posted on the website and presented to the task force.
  • Task force will meet on March 11th to review input and refine the Essential Skills definitions.
  • Members:

Standards and Assessment Task Force

  • Met February 19, 2008; next meeting scheduled March 20th
  • Task force members were joined by the Essential Skills Task Force policy group.
  • Members gave input on a draft framework for assessment of the Essential Skills and identified assessment options for phase-in and implementation beginning Fall 2008.
  • Members:


State Board of Education, March 13, 2008

Standards & Assessment Task Force Report

state board Questions and Staff Summary of Stakeholder Recommendations:

What essential skills will be assessed?

It is our expectation that eventually, all essential skills will be assessed. However, we anticipate that a phase-in process will be required and that students will be accountable for only those essential skills for which an assessment is available by May 1st of their 8th grade year.

What will be assessed at the state level?

Assessments will be scored centrally, regionally, or locally as described below:

  • Centrally Scored Assessments – Administered locally but shipped to a single location for scanning and scoring. Consistency is enforced by administration rules and by virtue of using a single site with a single set of scoring procedures. While Oregon does administer Centrally Scored Assessments (i.e. its multiple-choice tests), 3rd party corporations such as SAT or ACT may also have examples of centrally scored assessments that may be eligible for use by students to demonstrate their mastery of the essential skills.
  • Regionally Scored Assessments – Administered locally, but shipped to a regional center to be scored. Consistency is enforced by administration rules, common training and process, anchoring procedures and monitoring of reliability. The Oregon Statewide Writing Assessment is an example of a regionally scored assessment
  • Locally Scored Assessments – Administered locally and scored locally. Consistency is difficult to enforce and the degree to which consistency is required creates cost and resource implications.

Stakeholders have indicated comfort that Oregon can at some point have centrally or Regionally Scored Assessments of:

  • Apply Math, Reading, Writing

Generally, stakeholders have indicated that the following skills should be assessed locally:

  • Speaking/Listening, Global Literacy, Civic and Community Engagement

Some stakeholders have suggested the following skills should not be assessed separately but could be better evaluated in the context of the other essential skills:

  • Critical Thinking, Use Technology, Personal Management and Teamwork

However, it is important to note, that the conversations with Stakeholders about this topic are currently continuing and that the system will change over time.

Who will assess them?

  • Districts will be responsible for ensuring that students have the opportunity to demonstrate their mastery of the Essential Skills.
  • Districts will administer assessments of the essential skills in a manner consistent with administration and process guidelines provided by the state
  • Stakeholders have expressed concern that state-provided training is a critical factor for implementation

What proficiency levels will be required?

  • Proficiency level for Assessments of Essential Skills will initially be adopted by the State Board.
  • Staff recommendation will likely be that an Essential Skills Advisory Panel (ESAP) facilitated by ODE should be developed to establish guidelines and process for appropriate achievement levels for graduation that will be used as the basis for inclusion of additional assessments of essential skills,
  • Although, the ESAP will strive for consistency, the board has requested that students have access to multiple pathways through which students can demonstrate their mastery of the essential skills. The consequence of this flexibility is that the exact set of skills demonstrated by a student will differ based on the assessment administered.

How will assessments be aligned to post secondary level?

  • Students will be critical consumers of assessments. As such, students will determine which assessments, from list approved by ESAP, to use for their demonstration of essential skills based on their knowledge regarding how they can leverage those data for other purposes.
  • Representatives from post-secondary institutions (e.g. Business, Public and PrivateUniversities and Colleges) will be asked to participate on the ESAP to ensure assessments are meaningful to students and can be leveraged for other purposes.

Will there be a compensatory model in place for students to demonstrate proficiency in multiple ways?

  • There will be multiple pathways for students to demonstrate mastery of the essential skills.
  • The ESAP will attempt to increase over time the options that are available to students
  • As the number of options increase, the degree of consistency will decrease.

What can we do now to preserve the CIM elements?

  • The critical CIM elements of content standards and assessments will be retained according to state and federal law.
  • Districts have been asked not to dismantle their systems built to support the CIM
  • The Board will be asked to adopt an OAR to require districts to have in place a system of monitoring student progress in grades 3 through 8 using performance assessments, consistent with current work samples, embedded throughout curriculum and instruction.
  • Districts will have the option of using state scoring guides or an alternate system of their choosing.

What is not doable at all or for now and why/what can be phased in over time?

  • At the most recent task force meeting, participants suggested that the essential skills of Reading and Writing can be assessed as of June 30th, 2008. However, even these assessments have critical work that needs to be completed quickly prior to being available.
  • Although WestEd determined that the state mathematics multiple choice assessment is aligned to the Math essential skill, thetask force did not achieve consensus regarding supporting its use for this purpose without augmenting the test with additional application and problem solving related items
  • There was overwhelming support for ODE to include constructed response items with its multiple choice assessments. Resource availability will determine the extent to which this is possible.

Standards & Assessment Task Force Report

Key Decisions Made

  • Current Reading and Writing assessments were generally perceived as aligned to reading and writing essential skills.
  • Concern expressed that current math assessment doesn’t align to essential skill of applying mathematics in a variety of settings.
  • Recommendation that ODE add Constructed Response components to the current OAKS tests for assessment of Essential Skills.
  • Professional development was a key concern among representatives.

On Going Discussions

  • Whether Locally Scored Performance Assessments should be options for students to demonstrate mastery of the essential skills.
  • Participants who were in favor of locally scored performance assessments placed teacher professional development and criteria for technical adequacy as pre-requisites.
  • Analysis of cohort data on students who have recently met achievement standards with interest in looking at the cut score that would allow a smooth transition of successful graduates as the new diploma requirements are implemented.
  • Cohort data indicate that if the existing accountability cut scores are used for diploma purposes that the current graduation rates could drop. Although critical points of consensus have been achieved, representative teachers, K-12 administrators, assessment directors, college/university professors and administrators, and ODE disagree on some remaining critical issues despite progress in negotiation and compromise.

Additional Information

  • WestEd has completed an analysis of the relationship among current assessments and the essential skills. They found that the OAKS assessments are generally good measures for the essential skills. The full report is available on the ODE website
  • Core standards have been developed in grades 3 – 8 mathematics. A substantial alignment study is being initiated to determine the alignment of the current assessments with the mathematics content standards. This summer Oregon teachers will write test items for the revised mathematics content standards and begin developing constructed response items to enhance assessment of essential skills.

1

I:\Board\2008\2008 march\2008march13diplomaupdate.doc

This framework for the assessment of Essential Skills outlines future work. ODE staff recently received valuable input from members of the Standards and Assessment Task Force and the Essential Skills Policy Group. Through survey questions and group discussions at the February task force meeting, ODE staff captured the perspectives of various stakeholders primarily K-12 teachers and administrators, and post secondary faculty and administration). Overwhelmingly these stakeholders agreed that extensive professional development is paramount to this work. Subsequent discussions within the ODE Management Team further informed this framework update.

This framework outlines the outcomes from ODE and school districts that will be needed to help ensure the assessment of students in the Essential Skillsso that they are ready to meet new diploma requirements.

  1. Ensure Opportunities to Learn: For purposes of informing instruction and ensuring that students have opportunities to practice their application of knowledge, districts will be required to implement local performance assessments embedded into instruction.
  2. Build on Existing Systems
  3. Assessments: To the extent appropriate,OAKS (Oregon Assessment of Knowledge and Skills) assessments will be eligible for students to use in order to demonstrate proficiency of the Essential Skills for the purposes of earning a high school diploma (See Table 1).
  4. Expertise: ODE will establish an Assessment of Essential Skills Review Panel (AESRP) comprised of members from K-12, post-secondary, and business/industry to make recommendations regarding the assessment of the Essential Skills (for the purposes of earning a diploma). All panel members will have the requisite expertise for making these recommendations.
  5. Additional Pathways: To ensure that students have multiple pathways to demonstrate proficiency of the Essential Skills for purposes of earning a high school diploma, other assessments (e.g., SAT, ACT, AP) will be eligible in addition to OAKS. These assessments and proficiency levels will be evaluated by the AERSP for alignment and technical quality prior to being approved as eligible assessments.
  6. Establish a Phase-In: Students currently in 8th grade (i.e., graduating class of 2012) will be responsible for demonstrating proficiency in the Essential Skills of Reading, Writing, Speaking, and Mathematics prior to receiving a high school diploma. Later cohorts will be accountable for additional Essential Skills as ODE approves more assessments. Notice of any additional essential skill requirements and/or assessments of Essential Skills will be issued by May 1st of a student’s eighth grade year. 8th graders may also use assessments adopted in their 9th – 12th grade.
  7. Provide ClearState Guidance: By May 1st of each year, the Department will issue an Administration Manual for Assessing Essential Skills.Districts will use this document to administer, score, and manage district assessments of students’ proficiency in the Essential Skills required for the high school diploma.

5. Connect with Post-Secondary Opportunities and their Use of Assessment Data: To allow students to use the Assessments of Essential Skills data to achieve their post-secondary goals (e.g., admission, placement), ODE will annually disseminate information regarding how these data are used by post-secondary institutions. This information will be used, in part, to inform K -12 continuous improvement.

The following pages provide:

  • Proposed assessments that this year’s 8th graders (the class of 2011-2012) would use to demonstrate proficiencyin the Essential Skills (Table 1, p. 2 - 5).
  • An outline of proposed Oregon Administrative Rules (Table 2).
  • A decision matrix with questions for the board along with staff recommendations (Table 3).

Assessment of Essential Skills:
 Ensure Opportunities to Learn  Build on Existing Systems — Assessments, Expertise, Pathways

 Establish a Phase-In  Provide Clear State Guidance  Connect with Post-Secondary

Table 1: Proposed assessments of Essential Skills for purpose of earning a high school diploma (shaded). Initially, students will demonstrate proficiency of Essential Skills through either OAKS (Oregon Assessment of Knowledge and Skills) or Work Samples. With the approval of additional assessments, students will have more ways to demonstrate proficiency of the Essential Skills (clear).The proposed assessments included in this table give the board a preview of those that staff anticipates being phased in as additional pathways for students to demonstrate proficiency in Essential Skills.