Scenario A: Multirater Principal Evaluation

Three Scenarios for Principal Evaluation

Scenario A: Multirater Principal Evaluation

What if…new district leadership comes on board and decides to implement a 360-degree, multirater evaluation system that is designed to assess principals’ performance as school leaders? District leadership chooses to design the 360-degree system to include feedback from the superintendent, teachers and other staff, and the principal’s assigned mentor (if available); the principal’s self-evaluation; and a portfolio developed by the principal. In addition, parents and students are solicited for input about the school as a whole, and their ratings also are included in the principal’s evaluation. The purpose of this comprehensiveapproach is to allow both the principal and his or her evaluator to obtain a complete picture of potentialareas for school and personal improvement as well as evidence of good leadership practices that may not be ascertained through a single type of assessment (e.g., observation or walk-through).

Frequency

The principal is formally evaluated annually, with one formative and one summative assessment by the superintendent or other trained district personnel.
Feedback is formally collected annually from the mentor, teachers and other staff, parents, and students, but additional information from these groups also can be included in the principal’s evaluation.
Feedback and evaluation findings are incorporated into the principal’s professional development plan.

Measures

Principals are evaluated on a four-point rating system (e.g., distinguished, proficient, needs improvement, unsatisfactory) that is tied to leadership standards, such as the Interstate School Leaders Licensure Consortium (ISLLC) standards (Council of Chief State School Officers, 2008). The summative evaluations rely on the following:

Review of the principal-submitted portfolio that chronicles the principal’s growth and any formal professional development
Review of data about teachers (e.g., retention and attendance)
Student achievement scores
Assessments of a variety of leadership areas: communication, use and analysis of student achievement data to inform school policy, working conditions, teacher engagement and support, management of budget, and related areas

The informal or more formative evaluations consist of the following:

Drop-in observations by the superintendent or other trained district personnel
Annual surveys and informal interviews of parents, teachers, and students
Review of key school documents (e.g., parent newsletters)
Evaluators also must be careful to take into account the context of the school. If a principal
is dealing with high poverty, student mobility, teacher attrition, low parental involvement,
or other challenges, these factors need to be taken into consideration.

Consequences

Having captured an accurate and complete picture of the principal’s strengths and weaknesses, the evaluation results are given to the coach, who meets with the principal twice monthly, sometimes over the phone.
There are few consequences if a principal is rated poorly on the four-point scale or fails to improve over time; it is hoped that with the intensive support provided by the coach, the enactment of consequences will not be required.

Moderator Questions for Scenario A: Multirater Principal Evaluation

What do you think about ScenarioA? What do you like about it? What do you dislike about it?
Some say that it is important that multiple stakeholder groups have an opportunity to provide feedback on the principal because such groups are the primary constituents that the principal serves. On the other hand, there are concerns that students, teachers, and parents cannot really be unbiased, and the most vocal (or unhappy) individuals from each group may impact or skew the type of feedback received. How, if at all, should the school or district go about ensuring that all voices (positive and negative) are heard? Should equal weight be given to all stakeholder groups?
Some say that the superintendent, as the leader of the district, ought to conduct all aspects
of each principal’s evaluation. Others worry that the superintendent may not have the time to evaluate each principal or may not be adequately informed about what principals at certain types of schools (e.g., high school and elementary school) do to be effective. What do you think?

Who else should be responsible for evaluations?
If more than one person in a district is conducting evaluations, how can you make sure that each person is using the same criteria for evaluation?
What kind of training would ensure that the superintendent or other individual uses the evaluation tools and rubrics appropriately?

Do you like the idea of having a four-point scale tied to leadership standards?

How would that scale make the evaluation different from what you have now?
Should the scale be adjusted for newer principals, who would otherwise likely score low
on the scale due to their lack of experience?

Some say that having principals create portfolios gives them an opportunity to highlight their work and professional development; others say that portfolios are very subjective, do not capture the magnitude of all the principal’s responsibilities, and take up precious time that would be better spent running the school. What do you think?
If portfolios are included in the principal evaluation process, what types of data or information should be covered?
What is the best role for a principal’smentor or coach in addressing issues that come up in the evaluation? Is it realistic for principals to have a mentor or coach,as described in this scenario?

Should a principal’s evaluation data be given to a mentor or coach?

Now that we’ve discussed Scenario A, what do you think?
Does anyone have a different opinion? Please share.

Scenario B: Principal Evaluation Based on Observable Practices

What if…the district decides to revise its principal evaluation protocol and align it to the teacher evaluation protocol? The task force leading the change is intent on assessing the multiple hats that principals wear in the multiple contexts in which they work: serving as instructional leaders inside the school; navigating district systems and resources outside the school; and engaging with a wide range
of stakeholders, including teachers, resource and support staff, students, parents, district administrators, business and community members, and sometimes independent contractors providing professional development or other services. At the same time, the superintendent also recognizes the need to give principals the time, space, and autonomy to focus on their jobs without being unduly burdened by responsibilities related to their evaluation.

Frequency

Observational data from faculty meetings, leadership team meetings, and whole-group professional development meetings are collected throughout the school year. The observations are announced and arranged ahead of time.
Observational data from scheduled parent-teacher association meetings and other events that engage parents or community members also are collected throughout the school year. These observations are announced and arranged ahead of time.
Walk-throughs by the superintendent or other district personnel take place approximately twice each year and are announced and arranged ahead of time.
Formal evaluations are conducted once every three years and involve a meeting between the principal and the superintendent.

Measures

The observations cover the following observed practices:

Ability to run an effective meeting
Ability to communicate with district leaders, school staff, parents, and other community members
Inclusive decision making
Ability to access and secure targeted support from the district as needed
Ability to set goals, plan, use data, and monitor a meeting to support the meeting’s objectives (e.g., curriculum planning, teacher scheduling, and analysis of student test data)
Solid instructional coaching and the creation of a positive school culture

Consequences

Principals who receive strong evaluations are publicly praised for their good work. They also are asked to provide minimal coaching to struggling principals in the district.
Principals who receive poor evaluations are placed on a growth plan that involves attending several courses and being evaluated every year instead of once every three years. Principals
who receive three consecutive poor ratings are not renewed for the next academic year.

Moderator Questions for Scenario B: Principal Evaluation Based on Observable Practices for Principal Evaluations

What do you think about Scenario B? What do you like about it? What do you dislike about it?
In what ways is this scenario similar to your current evaluation system? In what ways is it different?
Is one evaluation every three years enough? If not, how often should principals be evaluated?
Does this approach adequately measure those principal practices (e.g., conducting meetings, communicating with all staff, making inclusive decisions) that contribute most to student achievement?
What observable practices are the most relevant to principal evaluation? Are they addressed
in this scenario? What are some additional observable practices of a principal that should
be captured?
What important principal practices are not captured in this evaluation system?
How important is it for a principal to be recognized by district leadership for a job well done?
What are the best ways to recognize principals for their successes?
Which would you prefer: a more hands-off approach (such as this one) or a more intensive evaluation process?
Now that we’ve discussed Scenario B, what do you think?
Does anyone have a different opinion? Please share.

Scenario C: Turnaround Principal Evaluation

What if…the district determines that a new principal evaluation system is needed to take into account
its vision of the school principal as a leader of drastic, urgent improvement? Helping teachers to raise student achievement levels is central to this new system, and “quick wins” for the school earn high marksfor principals. Principals are heavily rewarded, or otherwise penalized, based on student test-score growth. To create the extensive, lasting change that is needed, the superintendent believes that there must be distributed leadership throughout the school. Principals are evaluated jointly with their leadership teams, in part on their ability to effectively share leadership and responsibilities among staff.

Frequency

All principals are evaluated three times each year to allow for continuous feedback and opportunities for improvement and tailored support. Two formative evaluations are conducted
to provide feedback, and an end-of-year summative evaluation is conducted. The summative evaluation results are entered into the principal’s file.
School walk-throughs and observations by district staff for evaluative purposes occur regularly and are unannounced.

Who Conducts the Evaluation?

The district superintendent (or other district-assigned designee) conducts the evaluation. This person is required to complete intensive training on the evaluation protocol and ensure interrater reliability because of the high-stakes nature of the evaluations.
Evaluators also are required to complete training in providing critical yet tactful and constructive feedback, which is shared during required pre- and post-evaluation debriefing sessions. After each evaluation, the principal is required to refine his or her professional development goals.

Measures

Diverse amounts of school-level data are collected—including student achievement and growth data, teacher and student retention data, attendance data, teacher working conditions survey results, teacher evaluation results, and other information—to capture key leadership practices, such as evidence of the following:

Student learning gains
Shared leadership
Effective interactions with district leadership (e.g., providing timely updates and securing targeted resources)
Instructional leadership, the use of data, ability to work with parents, and creation of a culture of continuous improvement

Consequences

Principals who receive two poor evaluation ratings (out of three) in a given year lose their jobs.
Principals whose schools show large test-score gains are given a significant financial bonus.
All principals receive intensive, ongoing feedback through face-to-face meetings with their supervisors to discuss strengths and weaknesses and suggestions for improvement.

Moderator Questions for Scenario C: Turnaround Principal Evaluation

What do you think about Scenario C? What do you like about it? What do you dislike about it?
Some people think that using test scores as part (but not all) of the principal’s evaluation is essential for determining if students are really learning or not. Others think that test score data
are less relevant. What weight, if any, do you think student test scores should have in a principal’s evaluation?
Should principals be evaluated based on student achievement (e.g., raw student test scores), student growth (e.g., the change in test scores from the beginning to the end of the year), or both? Should the evaluator use value-added models of student growth (that is, complex statistical models that take into account students’ test score growth over the past several years as well as additional demographic factors that typically influence student achievement)?
Are there other ways to tell if principals are really helping to improve student learning? What are those ways?
What percentage—70 percent, 50percent, 30percent, or some other percentage—of a principal’s evaluation should be determined by student test scores? Why did you choose the percentage that you did?
How should the evaluation be modified to take into account that several months or a year may be necessary for improvements to be seen after some of the significant changes that occur within a turnaround school?
Should a principal be evaluated on the degree to which leadership is shared?

If so, what is the best way to evaluate for the presence of shared leadership?
If shared leadership should be included, should the principal be the only person evaluated according to ISLLC or other leadership standards in the school?

Do you think it is important to include personal goal setting by principals as part of the evaluation?
In this scenario, the principal is constantly provided with a lot of critical but constructive feedback. Is there ever too much feedback?
Is it right to give significant financial bonuses to principals who receive strong evaluations?
Now that we’ve discussed Scenario C, what do you think?
Does anyone have a different opinion? Please share.

Reference

Council of Chief State School Officers. (2008). Educational leadership policy standards: ISLLC 2008 (as adopted
by the National Policy Board for Educational Administration). Washington, DC: Author. Retrieved from