Comments for session on using data for school improvement

Papers by Mason (Milwaukee), Heisted & Spicuzza (Minneapolis), Murnane & Sharkey (Boston)

AERA 2003

Bill Clune, 5/7/03

I will use the chart (see below) to organize my comments, and I want to accomplish two things: (1) locate the papers on the chart, and (2) ask the authors some questions

The chart uses typology from Mason paper of knowledge management (KM): data, information, and knowledge. All three papers report progress and problems in all three KM boxes. In the data and information boxes, we see issues of reliability, data collection, and analysis, including tailoring of data for various kinds of school improvement purposes. In the knowledge box, progress where it occurs is invariably due to expertise and sustained leadership. This includes the expertise of Minneapolis central office, where stability of leadership over many years has produced to an unusually high level of capacity.

Moving to the third column of the chart (or table), all three papers report lots of barriers, in fact, more barriers than successes, as is typical with systemic reform. Overcoming these barriers is the key to productivity gains in each KM box. Because organizational barriers are “negative feedback loops” in Senge’s (1992) terminology, overcoming the barriers – rather than simply working harder -- provides the greatest leverage for change. All three papers also report dealing with assets and liabilities coming from the policy environment. In sum, as the chart suggests, the systemic reform represented by progress down the vertical KM vector requires the kind of organizational change represented by the horizontal vector, overcoming barriers and dealing with the policy environment. Roughly speaking, that is the model of systemic reform that I developed in a recent paper for the Hewlett Foundation.

Out of this, I have some questions for the authors. For all three papers, a key question is what is the next step for expanding the critical zone of expertise and leadership that leads to gains in productivity? In other words, what is the strategic plan for institutionalization and sustainability? For the Heisted paper, how much can sophistication in data and information at the central office help the leadership at the school level? For Mason and the Boston team, how do we go to scale with these school success stories? For all authors, would it help to show actual results, namely increases in student achievement at a reasonable cost? Mason’s model of continuous improvement talks about being “result-oriented.” At what point does maintaining an orientation toward results require actual results as reinforcement?

Using Data for Improved Productivity

/ LEVEL OF PRODUCTIVITY /

CHARACTERISTICS

/ BARRIERS/ ORG. INERTIA / SYSTEM/ POLICY SUPPORTS

Data

/ Trustworthy
(reliable & valid) / • / Alignment of:
- Assessments
- Mandates
- Incentives
- Resources
- Leadership
- Central office
coordination
Accessible & timely / •
Information / Analytically powerful (patterns) / •
Tailored for a purpose / •
Knowledge / Pushed by leadership vision / •
Organizationally supported & integrated / •
Not too costly or difficult / •
Highest Productivity for outcomes / •
Systemic
Reform /

Bill Clune, 4/22/03