A Brief Overview of the PACT Assessment System

In fall 1998, SB 2042was enacted in California to maintain multiple pathways to a teaching credential, but ensure that regardless of the pathway (e.g., student teaching, district internships, university internships), candidates meet a uniform set of standards. Among other provisions, it established a requirement for all California candidates for a preliminary teaching credential to pass a state-approved teaching performance assessment with demonstrated validity and reliability to supplement training, course assignments and supervisor evaluations. The California Commission on Teacher Credentialing contracted with the Educational Testing Service to develop such an assessment. SB 2042 explicitly allowed for the development of alternative assessments that were as rigorous as the state-developed assessment. Over the next four years, a job analysis of teaching was conducted and a set of standards for prospective teachers, called Teaching Performance Expectations or TPEs, was developed. Assessment Design Standards for the Teaching Performance Assessments or TPAs were also developed.

Twelve institutions (8 University of California institutions, San Diego State, San Jose State, Stanford, and Mills) formed the PACT consortium in 2001 to develop an alternative assessment. The design was to consist of a common standardized assessment plus a set of other assessments that varied between programs. (See the diagram of the PACT Assessment system) The group committed to developing a common assessment that still enabled unique program values and emphases, with the varying assessments affording programs a chance to enhance the representation of their unique values and emphases in the PACT assessment system.

Teaching Event

The design of the common assessment, called the Teaching Event, was modeled after the portfolio assessments of the Connecticut State Department of Education, INTASC (the Interstate New Teacher Assessment and Support Consortium) and the National Board. The common assessment was to draw from artifacts created while teaching, accompanied by commentaries that provide context and rationales needed to understand and interpret the artifacts. The common assessment was also to place student learning at the center, with special attention to subject-specific pedagogy and the teaching of English Learners. The assessment design chosen was that of a portfolio assessment, with Context, Planning, Instruction, Assessment, and Reflection tasks documenting a brief segment of learning. Anintegrated task design was chosen to prompt candidates to make connections between these different teaching tasks, and to provide evidence to understand a candidate’s teaching of a brief learning segment in some depth through the distinct lenses provided by the different tasks.

A central design team was established at Stanford University through foundation support. This team worked in collaboration with subject-specific development teams composed of faculty/supervisors from PACT member institutions to develop Teaching Events for piloting in the 2002-03 academic year. Feedback and suggestions for improvement were solicited from faculty/supervisors, trainers, and scorers the first year and used to direct revisions for the 2003-04 academic year. This established the practice of viewing the Teaching Event as a dynamic assessment that responded to demonstrated needs for improvement identified by the field. The combination of the assessment expertise of the design team, the expertise of faculty/supervisors in subject-specific pedagogy, and reflection on candidate performance has resulted in constant improvement of the Teaching Events.

Subject-specific pedagogy is represented in the Teaching Event through the foci for the learning segment documented by the Teaching Event, the foci for the video clips, the use of scorers with subject-specific expertise, and subject-specific benchmarks and training. The foci for the learning segment and the clips were selected to represent critical teaching and learning tasks in the credential area that occur with great frequency (to allow student teachers options in selecting a learning segment to document). For example, the focus element for Elementary Literacy is on teaching students to comprehend and/or compose text. For Single Subject mathematics, the foci are conceptual understanding, proceduralfluency, and mathematical reasoning. The prompts for the commentary and directions for constructing the Teaching Event were revised over the years to more clearly direct the candidates to plan, teach, and reflect on their teaching in light of what they know or learn about their students and their learning. The questions were selected to not only gather data, but to represent enduring questions that candidates would be expected to continue to ask themselves throughout their teaching career, with increasingly complex responses.

The emphasis on English Learners also shifted over the pilot years. Initially, there was a rubric in each category (Planning, Instruction, Assessment, Reflection, known as PIAR) that focused on English Learners. However, the evidence generated was insufficient to support that number of rubrics, and it was judged that the student teacher knowledge base also lacked sufficient complexity. Also, the feedback from some candidates and their supervisors suggested that formal, academic language was an issue for more students than just English Learners, e.g., students who spoke African-American Vernacular English, students from parents with low levels of education, secondary students who were not accustomed to using disciplinary terminology or to writing complex texts. Focusing the discussion of language issues narrowly on English learners was frustrating for these candidates, especially when the number of English learners relative to other students struggling with academic language was small. The concept of English Language Development was broadened to that of Academic Language, and an expectation for language development as well as content development for each learning segment was established. We are continuing to explore how to represent developing the academic language of English Learners along with that of other students, while avoiding a loss of representation of English Language Development for English Learners.

Academic Language was added as a separate scoring category drawing from evidence across tasks, resulting in a Planning, Instruction, Assessment, Reflection, Academic Language or PIARL scoring structure. The constructs measured in the Teaching Event and the scoring levels are described in an Implementation Resource called Thinking Behind the Rubrics, which appears in the section on Training of Trainers and Scorers.

Embedded Signature Assessments (ESAs)

Development of the Teaching Event has outpaced development of the campus-specific tasks, which have come to be known as Embedded Signature Assessments (ESAs). Data for the Teaching Event alone is sufficient to meet the standards for high-stakes assessments, including the Commission on Teacher Credentialing’s Assessment Quality Standards. Therefore the Teaching Event is being submitted as the sole component of PACT’s Teaching Performance Assessment at this time. However, development work on the ESAs continues. Many programs have identified or developed assignments intended to be ESAs, and have also developed more formalized scoring criteria or rubrics to enable standardized scoring by more than one person. They are now piloting these ESAs, collecting data, and analyzing the results in a process similar to that of the development of the Teaching Event.

Use of scores for program improvement

Many programs have used scoring data as well as focused conversations stemming from scoring similar student work across the entire program to identify program strengths and targets for program improvement. The eleven rubrics from the Teaching Event, 2-3 for each category scored (PIARL) across five scoring categories, provide feedback for both individual students and for the program as a whole. Taken together, the 2-3 rubrics within a scoring category providefeedback that is targeted at different constructs, providing amulti-dimensional picture of performance in each category, and providing
a complex picture of candidate strengths and areas for growth.

Teacher certification programs have used a variety of strategies to strengthen their PACT program, many of which will be shared later in this Handbook. As a result, a strengthening of candidate performance on the Teaching Event can be seen across the years. Yet, at the same time, it does not appear that the standardized Teaching Event has unduly narrowed candidate performance, as a variety of program values and emphases continue to be found throughout the PACT institutions in addition to a variation in effective approaches to instruction.

PACT today

Despite the suspension of the Teaching Performance Assessment (TPA) requirement in 2003, the members of the PACT consortium not only renewed their commitment to continued development but actually grew in numbers. With the reinstatement of the TPA requirement by SB 1209 in fall 2006, PACT has grown to 31 members. The Teaching Event was formally submitted to the California Commission on Teacher Credentialing for approval as a TPA in March, 2007 and was approved. The consortium plans to continue to work on the ESAs to complete the PACT assessment design, and to resubmit the Teaching Event/ESA package when the ESAs meet the required standards for technical quality.