Figure 1: State/District Dual-Support Approach Method

Executive Summary

Our national trend toward using Value-Added Modeling (VAM) to measure educator effectiveness has provided administrators with hard data to use as they evaluate teachers, analyze professional development needs, and support teacher professional growth. Unfortunately, although many states have made great strides using the data provided to evaluate teachers, there are still gaps in the level of professional development site-based administrators receive to ensure they understand high-quality instruction, evaluate teachers appropriately, and provide clear, specific instructional feedback that leads to improved teacher practice. Modern technology allows for an analysis of student test scores that provides detailed information about how much growth a student achieves during one school year. This data has revealed a distinct discrepancy in teacher evaluation ratings versus their corresponding value-added scores in North Carolina that indicates a need to provide deeper, sustained professional development. One strategy to support principals as they evaluate teachers fairly and effectively and gain the pedagogical understandings to provide accurate feedback about classroom instruction that will lead to improved student learning outcomes is to develop a partnership between the state education agency and district leadership. State-level development of stronger inter-rater reliability and calibration systems for evaluations is needed for accurate and meaningful evaluations. Furthermore, district-level support is necessary to provide principals with sustained, differentiated professional development targeted toward specific recommendations for teacher growth. This dual-support approach will create a ripple effect that will impact not only principal instructional leadership but also teacher effectiveness and ultimately student learning outcomes.

Figure 1: State/District Dual-Support Approach Method

State / District

• Develop a calibration system to ensure inter-rater reliability
• Create an evaluation certification program to identify principals who demonstrate proficiency in teacher evaluation
• Facilitate on-going regional Principals’ PLCs

/

• Identify trends in evaluation data
• Facilitate monthly data meetings with principals and develop SMART goals
• Provide time for collegial conversations and evaluation coaching sessions using collaborative walkthroughs with emphasis on instructional feedback

Introduction

Tony Flach (2014), National Practice Director at Houghton Mifflin-Harcourt, contends that instructional leadership should be defined as “the ability to guide adults to improve instruction through the creation of favorable learning environments, building of adult content and pedagogical knowledge, and explicit monitoring of the learning of both adults and students.” The principal as 'instructional leader' is a somewhat new concept that began to materialize during the early 1980's. Prior to this shift in responsibilities, most principals functioned as managers and operational leaders. This movement toward academic and instructional leadership was influenced by current research during the time which indicated that effective schools most often had principals who understood and articulated the importance of instructional leadership (Brookover and Lezotte, 1982). Another major shift in education was the ability to access data that clearly and accurately revealed, without bias, student achievement, discipline, demographic disparities, and retention rates. Suddenly, public education became much more public with widespread access to data. This movement even further perpetuated the need for an instructional leader in the principalship.

The high demands of the principalship are difficult to prioritize, and historically principals have overlooked the importance of the evaluation process. Now, in the second decade of the 21st century, educational leaders have access to even more data that highlights the alignment and misalignment between evaluation and rating. Teacher evaluations are more important than ever with high-stakes decisions, such as tenure and performance pay, connected to evaluation results. Principals, assistant principals, and other educational leaders designated to evaluate teachers are responsible for accurate and reliable evaluations (NGA Center for Best Practices, 2011). The benefits to teacher evaluation are not usually immediate. Too often, principals are handling problems that need immediate attention and an immediate solution, thus creating urgency. Numerous research sources provide evidence that a combination of rigorous classroom observations combined with additional data measures will provide an accurate evaluation of teacher effectiveness (Bill & Melinda Gates Foundation, 2013; Ho & Kane, 2013; Taylor & Tyler, 2012).

A focus on accurate teacher evaluation is necessary. Ongoing professional development provided through a dual-support model between the state education agency and local district level leadership is crucial to improving the quality of instructional feedback administrators can provide teachers. This model, if implemented successfully, will result in improved teacher effectiveness. A conceptual understanding of how a principal’s leadership impacts teaching, learning, and student learning outcomes is the first step to understanding the need for change in the preparation and development of principal preparation programs. Furthermore, developing a framework for support for principals from the state and district level will ensure principals hone the skills needed to improve and inform instruction. Ongoing, aligned, and monitored professional development will also lead to more reliable and valid evaluations. Principals will also develop a deeper understanding of content standards, pedagogy, and instructional design to provide clear, specific, and constructive feedback to teachers that will lead to improved student learning outcomes. One-shot professional development is not enough to inform principals’ understanding of instructional leadership.

Research clearly supports the need for teacher evaluation, but little attention has been given to ensure that evaluators are trained and certified to make subjective decisions regarding a teacher’s performance. During the 2009-10 school year, the North Carolina Department of Public Instruction conducted state-wide, two-day train-the-trainer professional development on the new North Carolina Professional Teaching Standards, the evaluation process, and look fors during classroom observations. Since that time, training of new administrators and review sessions for experienced administrators has become the responsibility of each school district. Policy TCP-C-004: Policy establishing the Teacher Performance Appraisal process states that Component 1: Training “Before participating in the evaluation process, all teachers, principals and peer evaluators must complete training on the evaluation process.” The consistency, quality, and fidelity of these trainings is unknown.

The evaluation of teachers must be purposeful, reliable and valid. The comprehensive study by Yoon et al. (2007) indicates that not only the duration of the professional development plays an important role in the success of the initiative but that follow up and on-going, job-embedded opportunities for discussion, feedback, and continued emphasis on the professional development are all an integral part of a successful implementation. Creating experiences where principals can learn from reflecting on their experiences is an important part of the learning process. Standards one, two, and three of the North Carolina Educator Evaluation Standards for North Carolina School Executives requires principals to reflect on their practice in the areas of strategic leadership, instructional leadership, and cultural leadership (North Carolina School Executive: Principal and Assistant Principal Evaluation Process Manual, 2012).

One integral way district leaders can ensure principals have the opportunity to support teachers and grow as instructional and cultural leaders is to provide ongoing professional learning opportunities aligned with state goals - either as part of their regularly scheduled professional learning communities (PLCs) or in follow-up sessions designed around reflection, sharing, and feedback.

The goals of the state/district partnership would be to develop strong inter-rater reliability and instructional leadership between and among site-based administrators across the state. Currently, in North Carolina, there are major discrepancies between evaluation ratings and teachers’ VAM. To ensure principals have a deep understanding of the standards, how to rate teachers, and how to provide strong instructional feedback, changes must be made in the current model of support.

Background

North Carolina principals are currently faced with a variety of new challenges that are previously unknown. In 2010, North Carolina adopted new standards for every content area and grade level,Common Core State Standards (CCSS) in English Language Arts and Mathematics as well as the Common Core State Standards for Literacyin History, Social Studies, Science, and Technical Subjects. In addition to these standards, North Carolina also adopted the North Carolina Essential Standards (NCES) for all other grade levels and all content areas (In the States, 2012). According to the North Carolina Department of Public Instruction (NCDPI) (2011), the new standards are based on a philosophy of teaching and learning consistent with current research, best practices, and new national standards. The NCDPI contends that the new North CarolinaStandard Course of Study (NCSCoS) is designed to support the state’s educators as they provide the most challenging education possible for North Carolina students. The ultimate goal of these new standards is to prepare all students for a career and/or college. Not only do principals have the important task of providing teachers with high-quality instructional feedback to improve their performance, but they also must take time to learn the new content standards across the board. Now, more than ever, principals need sustained and focused professional development to support them as they evaluate the quality of teaching and provide instructional feedback that improves student learning outcome

For almost two decades, quality teaching has been consistently identified by researchers as the most important school-based factor in student achievement (McCaffrey, Lockwood, Koretz, & Hamilton, 2003; Rivkin, Hanushek, & Kain, 2000; Rowan, Correnti & Miller, 2002; Wright, Horn, & Sanders, 1997). Instructional guidance, support, and feedback that teachers receive from principals is imperative in improving their practice. Research has proven that evaluation is more effective when the evaluators are trained (Darling-Hammond et al., 2011). Trainings should include resources that support the evaluation process (McGuinn, 2012). A high-quality professional development partnership is the key to successful implementation of any teacher evaluation system. States must rethink the way evaluators have been trained in the past and develop a new model designed to grow instructional leaders through ongoing training, modeling, collaboration, and support.

The Widget Effect, reported by Weisburg et.al. (2009) describes the school district’s assumption that teacher effectiveness is the same from teacher to teacher. Teachers are not viewed as individual professionals but rather as “interchangeable parts.” This report suggests that better evaluation will not only improve teaching to benefit students, but it will benefit teachers by treating them as professionals. Characteristics of the Widget Effect in Teacher Evaluation:

• All teachers are rated good or great.
• Excellence goes unrecognized.
• Inadequate professional development is provided. No special attention is given to novice teachers.
• Poor performance goes unaddressed

The Widget Effect is simply another indicator that principal instructional leadership has taken a back seat to managerial and organizational components of the principal’s role. Without a clear emphasis on evaluation from the state and the ongoing opportunities for discourse, professional development, and support, the quality and accuracy of teacher evaluations is not likely to improve.

As more and more states are turning to measuring student growth data as using VAM such as the Education Value Added Assessment System (EVAAS) from the SAS Institute, The Widget Effect is more prominent than ever. Through VAM data, notable discrepancies between evaluation and student learning outcomes have been revealed. Value-added assessment systems, such as EVAAS, provide individual teacher, school, district and state growth data. The North Carolina State Board of Education implemented EVAAS data as part of teacher and principal evaluations during the 2011-12 school year. EVAAS determines the effectiveness of teachers, schools, and districts with regards to student achievement and provides multiple aimed at analyzing student and teacher performance on standardized assessments. A multifactorial correlation study was conducted to analyze the data surrounding the correlation between teacher performance evaluation ratings and the EVAAS student achievement data. The dataset included 11,430 North Carolina teachers in 35 local education agencies (LEAs) having both Educator Value Added Assessment System (EVAAS) scores and performance evaluation ratings assigned in 2010-11 school year. Although 46,000 teachers had evaluation data for 2010-11, only around 11,000 of those also gave an end of grade (EOG) or End of course (EOC) assessment. These 11,000 received an EVAAS data score. Research found that there was a small distribution of evaluation ratings in the study. Out of 11,000 teachers, the 100 teachers with the best student achievement data received the same ratings as the 100 teachers with the worst achievement data. The study did not find a correlation between performance evaluation data and EVAAS data (Batton, Britt, DeNeal, & Hales, 2012). This finding alone demonstrates a deep need to better prepare evaluators to provide instructional feedback on content, pedagogy, and instructional design. A comprehensive system of support from district and state agencies is mandatory.

Conceptual Frameworks

In 2012, The American Institutes for Research published a report titled: “The Ripple Effect” to examine principals’ influence on student achievement.The report provided a conceptual framework for understanding the role of the principal in terms of instruction, the direct effects of the principal’s practice on both teachers and the school as a whole, and the indirect effects that a principal’s impact has on classroom instruction and learning. A current modification and personalized iteration of “The Ripple Effect” framework helps toidentify thesignificanceof a state-district

Figure 2: Adaptation of "The Ripple Effect"

partnership to improve principals’ practice, improve instructional quality, and impact student learning outcomes (Clifford et al., 2012). This iteration of the framework suggests that principals need strong, focused professional learning opportunities and support to directly impact teacher quality, which ultimately can have an indirect effect on student achievement (Figure 2).

To ensure that the professional development and support provided by state and local agencies will result in improved student learning outcomes, both agencies must make a concerted effort to address the components of planning, implementation, and evaluation (PIE) in the professional development cycle (Figure 2).

Figure 3: The PIE Cycle of Professional Development

One-shot, fragmented workshops lasting 14 hours or less show no statistically significant effect on student learning (Darling-Hammond, Wei, Andree, Richardson, and Orphanos, 2009). Effective professional-development programs are job-embedded and provide participants with five critical elements:

• Collaborative learning: Opportunities to learn in supportive groups where content is organized
• Clear evident links among curriculum, assessment, and professional-learning decisions as related to specific teaching contexts: Emphasis on the importance of developing content knowledge, specifically in math and science, as well as pedagogical understandings specific to content areas (Blank, de las Alas, and Smith, 2008; Blank and de las Alas, 2009; Heller, Daehler, Wong, Shinohara, and Miratrix, 2012).
• Active learning: Application of new knowledge and opportunities to receive feedback, use of ongoing data to reflect how teaching practices influence student learning over time
• Emphasis on deep content knowledge and pedagogy: Direct ties to how to teach content through new techniques and strategies
• Sustained learning, over multiple days and weeks: Engagement in 30 to 100 hours of learning over six months to twelve months to increase student achievement

Every educational institution in our country plans professional development in hopes to improve the quality of teaching and student learning outcomes. However, research has shown that very specific and deliberate steps must be followed in the PIE cycle to ensure that not only do participants benefit from professional development but also that student learning outcomes are improved. Several important components that have traditionally been overlooked must be addressed. These include the allocation of funding, time, and follow-up to ensure participants have rich instruction and receive the appropriate on-going support necessary to ensure overarching positive changes in teacher practice.

Little research in the field measures the impact of professional development on student learning outcomes. However, a great deal of research has been conducted around creating and evaluating high-quality professional development. An examination of the three meta-analyses provides a great deal of compelling empirical evidence regarding the qualities of professional development that led to student gains. Those qualities can be represented in the three distinct elements of the PIE cycle. The outer ring of the PIE cycle represents the ongoing components of high-quality professional development that increases student achievement, and the three inner components represent the phases that districts or schools must adhere to in order to ensure success.

No longer is it an acceptable practice to make assumptions about the impact of professional development. Advances in technology include more online assessment opportunities for students, an increase in state and federal student testing mandates, and VAM models that provide a clear and undeniable data regarding the impact of instruction on student growth. These data points provide the opportunity to assess the impact of professional development more accurately than ever before. This deep understanding uncovers a need for a monumental change in training, support systems and structures, and follow up for practicing principals. According to the Learning Forward Center for Results, “policies, resources, calendars, daily schedules, coaches, and budgets influence the quality and results of collaborative professional learning and may need to be discussed and altered for alignment” (Standards Assessment Inventory, n.d.).State and district leaders must think creatively and collaboratively to address the need of developing instructional leadership capacity in principals as well as create structures to ensure success.