Using Data to Improve Instruction:

Building on Models that Work

A CCSSO Education Leaders Training Conference

September 12-14, 2007

St. Louis, Missouri

Summary Report

Rolf K. Blank

Director of Education Indicators

Council of ChiefStateSchool Officers

One Massachusetts Avenue, NW

Washington, DC20001

Using Data to Improve Instruction:

Building on Models that Work

Executive Summary

CCSSO hosted a unique conference for state and district leaders in St. Louis, MO, September 12–14, focused on models for using data to improve instruction. A total of 190 leaders from 30 states across the country participated in the event. CCSSO planned the conference with the primary goal of assisting leaders in moving beyond collection and reporting of data, toward greater focus on identifying effective strategies and tools for using data to improve instruction and to provide training on how to implement database approaches to improvement in schools. Several highlights of the conference were

  • An opening panel featuring Chiefs Gerald Zahorchak (PA) and Susan Castillo (OR), and deputies from Minnesota and Rhode Island focused on how state policies can be formulated to merge education improvement initiatives with leadership by states on improved use of data within schools, and how integrated data systems can be designed to meet educator needs.
  • Keynote speakers for the conference were Dr. Mike Schmoker, nationally-known author and consultant on organizing schools to use data in instructional improvement, and Dr. Margaret Heritage,UCLA/CRESST expert on teachers’ use of formative assessment data in classrooms.
  • In breakout sessions, 33 state and local leader teams shared their experience and evidence from use of data-based strategies and tools for raising the quality of instruction
  • Chief Douglas Christensen (NE) led a panel of Nebraska educators in discussing results from a statewide approach for using data for purposes of accountability as well as school-based decisions for increasing effectiveness of instruction.

For our second annual Leader Training Conference on Using Data to Improve Instruction, CCSSO was very pleased with thehigh quality of sessions and presentations focusing on effective strategies and models for using data. Evaluation feedback from participants following the conference was very positive. The conference was hosted and organized by the CCSSO Surveys of Enacted Curriculum state collaborative project. To review the conference sessions and materials, go to the CCSSO webpage

Rationale for Conference

As states and school districts increase their capacity to collect, manage, and report a variety of data, educators are seeking to use the data to inform decisions about how to increase student achievement and improve instruction in classrooms. CCSSO is seeking ways to provide leadership with states to share their models and experiences for using data across states. The St. Louisconference focused on Using Data to Improve Instructionis an important step for moving state leadership beyond design, collection, and reporting of data to greater leadership on uses of data to focus efforts of educators. The Conference hadthree primary objectives:

  • Present and discuss strategies, designs, and tools for using data to improve instruction that are currently being applied with schools across the states
  • Analyze findings and evidence about the effectiveness of the variety of models and tools, and discuss their application as models for professional development in other locations
  • Provide leadership training on how to implement and use data-based approaches to instructional improvement.

This conference series was conceived in 2006 based on the recent experience of state leaders and consultants who are members of the Surveys of Enacted Curriculum (SEC) Collaborative. The Collaborative was moving into more focused use of the SEC data for providing professional development leadership with district and school staff on how to combine instructional practices and content data with achievement data to focus improved student learning. The group recognized the need for leader training on the uses of data for improving instruction. Our state members wanted to gain skills in identifying needs of students and teachers from analyses of achievement data as well as instructional data, and then be able to use the data analyses to focus on change in instructional strategies, priorities on curriculum topics and expectations for learning. At the same time, we recognized that many states, districts, and professional development providers have been working on other specific approaches to using data to drive instructional improvement strategies.

The Collaborative suggested to CCSSO that a conference that included presentations and discussionson a range of current models and strategies for data-driven instructional improvement would be very attractive to state leaders and many local districts. Such a conference, would provide an excellent way to help states review the available models, to consider research and evaluation evidence about effectiveness, and identify ways to link state data systems development to instructional improvement needs.

The 2007 Using Data Conference in St. Louis was planned to build on and incorporate several programs and initiatives with state leaders that are currently operating at CCSSO:

  • The Secondary School Redesign project funded by U.S. Department of Education has been working with state leaders over the past year to share the experiences and progress that states have made through federal grants for secondary school redesign. The Using Data Conference was scheduled to be held in the same hotel immediately after the fall meeting of this project.
  • The Using Data Conference was planned by CCSSO and its planning committee as a complementary event in relation to the CCSSO National Conference on Large Scale Assessment (held in June each year) – which focuses on issues, progress, and research on development and implementation of student assessment programs.
  • The Council has been part of the Consortium for School Improvement for several years and provides a monthly newsletter aimed toward state leaders involved with School Improvement Initiatives to share information, research, and state models for assisting low-performing schools.
  • The Surveys of Enacted Curriculum (SEC) State Collaborative has been working with state education leaders since 1998 to develop, test, and implement a survey instrument that provides comparable, objective data on the curriculum taught in classrooms and the degree of alignment between state standards, assessments, and current instruction.

Conference Highlights

CCSSO issued a call for proposals in spring 2007 inviting sessions to be planned and submitted for the Using Data Conference. Proposals submitted from across the U.S. were reviewed by a planning committee comprised of state leaders, expert consultants, and CCSSO staff, and a total of 33 sessions were selected to comprise the majority of the conference agenda. The sessions were planned for 75 to 90 minutes. Each session provided participants with details about a model or strategy for use of data in improving instruction and evidence about the degree of effectiveness of the model/strategy with schools and teachers. Additionally, the planning committee recommended invited speakers and panels that would be featured in plenary sessions and highlight central issues regarding how data-driven improvement initiatives can be moved forward. Registration was opened to education leaders from state and local levels as well as providers of professional development, vendors, and researchers. In total, 190 persons registered for the 2007 Conference. Major themes, issues, and results presented in the plenary sessions and the focused breakout sessions can be highlighted.

State Policies for Improvement Initiatives and Use of Data.

In the Conference opening plenary session, two chief state school officers and two deputy chiefs described systematic, state-level efforts to incorporate training and leadership with use of data as part of broad state efforts to raise the performance of schools and districts. The efforts of Pennsylvania, Oregon, Minnesota, and Rhode Islandwere featured in the session, and each provided an example of how state-level leadership can integrate policy initiatives for advancing curriculum and instruction and school-based uses of data.

Presentation by Mike Schmoker, consultant on school improvement and Instruction.

Citing a range of research and practical experience, Schmoker focused the attention of conference participants on what is known about effective actions and organizing principles within schools to set learning goals, focus on the what and how of instruction that works, involve staff in learning from each other, and continually assess student performance.

Presentation by Margaret Heritage, expert on the use of formative assessment.

From her research in schools and classrooms, Heritage provided a detailed description and analysis of how one effective teacher develops and uses a range of data from classroom-based assessments, direct work with students, and external system-wide assessments and data resources to systemically advance the achievement levels of all her students.

Statewide approach to use of data featuring Nebraska Educators.

Commissioner of Education Douglas Christensen led a panel of Nebraska educators to explain the state’s approach to student assessment that starts with its purpose for district and school decisions about improving instruction, includes extensive professional development for educators on data analysis, and then provides aggregation and analysis for purposes of accountability at the state level.

Cross-state Roundtable Discussions—Recommendations

The conference participants were involved in roundtable discussions that focused on findings and evidence generated across the sessions and presentations. The topics focused on key questions for moving data-driven improvement forward.

A) What kind of data do educators/teachers need and how should data beprovided?

Student learning data that is close to the classroom:

  • Item analysis
  • Common assessments
  • Student work
  • Cohort data for multiple years
  • “Value-added” data that provides information about student progress under that particular teacher’s “watch”
  • Predictive data that will show the degree to which learning in “on track”

Data about best practices:

  • Data that can dispel myths
  • Data that can assist teachers in locating schools and/or districts where practices are potentially more successful, e.g., in addressing the needs of specific populations, in teaching particular concepts and content, etc. (student learning data from comparable schools)

Multiple sources of data

  • State, district, classroom

What kind of access would best facilitate their use of data?

  • Paper copies of really useful data that is provided to groups of teachers that help them understand how their students are doing
  • Low technology options that teachers develop to track their students progress

Teachers using data need leadership that is:

  • Supportive
  • Weeds out the un-useful data
  • Provides data literacy professional data
  • Provides the time for teachers to work together to analyze and make sense of the data

B)What should be the role of states in organizing and providing educators

access to important data?

  • Implementing a statewide Student ID –enables access to longitudinal student data across districts. This is important with high levels of mobility.
  • Providing a big picture context of what’s happening in education across the states. Which districts are performing well, despite challenging demographics and limited resources? Where should districts look for best practice models?
  • The SEA has a role in identifying the key data that is needed for instruction
  • Largerole in providing professional development to LEAs:

-- How to use data

-- Data quality

-- Building awareness of available tools for improving data use

-- Maintaining a best practices library for districts

  • SEA can be helpful in providing a standard, user-friendly format for district staff and teachers to view their data
  • SEA plays a critical role in certification requirements for teachers and administrators. Can require coursework in data use.
  • SEA needs to support the building of a technology infrastructure statewide—promote common standards, help manage vendors, enable common software development/interoperability
  • SEA can get legislative support for needed resources
  • SEA can help provide common metrics and guidance for connecting data points into meaningful indicators.

C)How can we help teachers use data to improve practice – particularly in

relation to standards?

  • Leadership support is key–and leaders need to know how to use data—PALS example
  • Teach about data use in preparation—pre-service
  • Need a structure for data use in the building, e.g. PLC
  • Educators involved in all aspects of assessment activities
  • Begin small and build (zone of proximal development)
  • Demonstrate value and usefulness
  • Must be continuous and ongoing
  • Use teachers within the state to score assessments
  • All faculty within a buildingmust be trained and involved
  • And all stakeholders must be trained with same process
  • Both summative and formative data training
  • Demystify the terms and concepts used
  • Teachers be allowed to ask the questions
  • Time is a barrier
  • Data must be meaningful
  • Don’t play the blame game
  • 3 steps: collection, analysis, use results

Breakout Sessions: State and Local Models and Tools

The 33 breakout sessions included in the conference can be placed under one of several categories or themes. For most sessions, the model presented likely cuts across several themes but we have listed the title, presenters and abstract (see below) for each session under one category as a way to consider what was included in the 2007 conference. The session categories or themes are:

  • Next generation of school improvement models
  • High school redesign especially for high risk students
  • Applying instructional and curriculum data
  • Educators access to assessment data through technology
  • System-wide models for data organization and use

You can access the presentations from 2006 & 2007 conferences on our webpage at

The webpage includes the Conference program indicating presenter’s institutional affiliation and contact information.

1) Next Generation of School Improvement Models

Michael Hickey, Ronald Thomas, A Team Dialogue Model for Data-based Instructional Decision

Making

The “old model” of data analysis, currently used in many schools, is largely dysfunctional. It attempts to use the essential accountability data for school improvement purposes which does not work. A Classroom-Focused Improvement Process (CFIP), based on a team dialogue protocol, is proposed to address these concerns.

Maren Harris, Brad Oliver,School Reform for Student Success: One School’s Restructuring Story

During the 2006-2007 school year, Washington-CarverElementary school, located in Muncie, IN entered its first year of full restructuring under NCLB. This session will detail the school's restructuring plan and focus on the successes and lessons learned pertaining to the school's curriculum, instruction, policies, professional development, and resource allocation after one year in restructuring.

Beata Thorstensen, Jan Sheinker-- Helping Educational Leaders use Accountability Data For School Improvement: New Mexico’s Principal Support Network

In 2005, as part of the WallaceFoundation-fundedState Action for Education Leadership Project, New Mexico started the Principal Support Network. The purpose of this network is to provide professional development to principals in the use of assessment data to create comprehensive school improvement plans using pivot tables to analyze data and CCSSO’s Data-Based Decision Making model. Attendees will learn about data analysis tools and techniques, and state-level policy changes that have occurred because of this initiative.

Deborah Swensen, Aaron Brough--MISSION IMPOSSIBLE: Creating a Culture that Cares –

about DATA!

The mission impossible is to initiate and sustain change while creating an educational culture that utilizes data within a high school. Educators need assistance in recognizing and utilizing data effectively. This session focuses on changing data perceptions and attitudes. Examples of information and meaningful reports will be provided. We will show innovative programs in Utah designed to create a dynamic culture that actively uses data in an effective way for the benefit of students.

Aaron Butler, Michael Jordan, Deborah Williams-- Tennessee Exemplary Educators: An External

Change Agent Approach to Using Data in State-Identified Schools

The Tennessee Department of Education assigns external change agents called Exemplary Educators to provide technical assistance to schools identified by the state for improvement. This session highlights the ways in which these Exemplary Educators use data from formative and summative assessments, perceptual surveys, projection growth modeling, and other sources to facilitate improvement in state-identified schools. The session also presents strategies and tools that Exemplary Educators have used to build the internal capacity of schools to collect and use assessment data to improve school outcomes.

Sheila Engle-Rinderknecht, DianaBourisaw--School Improvement Planning: Getting Results

This session focuses on the SLPS school improvement planning process. The process, finishing its first year of implementation, is a data-driven school improvement model that requires all schools to engage in the practice of first collecting and then analyzing trend data on student assessments, attendance, discipline, graduation and stakeholder involvement. An essential component of the plan is root cause analysis (RCA) to determine possible reasons for chronically low performance.

Sharnell Jackson, Gerald Biemler-- Linking Data and Learning Teams with Practices that Improve Achievement

Promoting student achievement begins with the implementation of well-defined student instruction units, and teacher team meetings for periodic review and assessment of student performance. Chicago Public Schools uses SchoolNet software to support: 1) Automating ways to review student data to allow teachers to have timely access to data to help inform instruction 2) Providing relevant information about students to systematically monitor progress and personalize learning 3) Giving teachers and administrative leaders access to high quality Web 2.0 instructional resources to help support mass customization 4) Using the Learning Teams model to support professional learning communities practices, data-driven decision making, and the school improvement planning process to monitor progress