UCLA S Working Plan for the Scoring of the Teaching Events

iv. Sample Scoring Implementation Complete Plan (UCLA)

This example from UCLA is a comprehensive plan for Scoring including a timeline, rubrics for documenting scores, score report, and summary of Teaching Events.

UCLA’s Working Plan for the Scoring of the Teaching Events

• Identify scoring dates and when Teaching Events will be due. In your discussion, take into consideration the schedule and coursework of your students, when scorers are available, and how you can provide feedback in a timely manner to your students regarding their performance.

• Plan for training needs of scorers, determine dates for training, who will need to participate, what information people need access to, materials needed and who will lead the training. Take into consideration the following:

1) Tech training: Be sure all potential scorers have access to the UCOP system and receive a password to log onto the site if students have completed electronic versions of the Teaching Event.

2) Brand new scorers

3) Lead scorers

4) Initial meeting of the scorers from each scoring team

• At UCLA all advisors were given legal-sized envelopes with labels attached containing each candidate’s ID #, content area of Teaching Event, and BLCAD option to distribute to their students. All students were instructed to make sure that their Teaching Events and videotapes/CDs could fit in these envelopes. For elementary candidates, we had candidates circle either Literacy or Math and also whether the Teaching Event was completed in a basic to waiver class (BCLAD). When the Teaching Events are collected, sort them according to their respective content areas/levels.

NOTE: Teaching Events are now submitted through an electronic platform at UCLA; This process collected paper copies, but could also be modified to collect CDs with files.

• Determine the number of Teaching Events by category that needs to be scored. In your tally, consider the number of Teaching Events which may need to be scored by a Spanish-speaking person. The areas we needed to tally included:

1. Elementary Literacy
2. Elementary Mathematics
3. English-Language Arts
4. Social Studies
5. History/Social Science
6. Mathematics

• Based on your count, be sure to duplicate enough copies of scoring rubrics for each content area. When making copies of rubrics, be sure to duplicate extra copies for Teaching Events which will be used for calibration and ones which may need to be scored twice.

• Identify all of your scoring resources, including scorers who can score events for BCLAD candidates. (At UCLA, all TEP faculty, university field supervisors, and the intern program faculty participate in the scoring. To enlarge our pool of scorers, we have considered soliciting the assistance of our guiding teachers, coaches in schools and our off-track alumni to help with the scoring; however, we have not been successful with this.) Then, determine the number of scorers you need for each category to ensure that all the Teaching Events in that category get scored. Divide your scorers into teams and identify who can serve as the lead scorer. Be sure there are sufficient scorers on each team who can score Teaching vents of BCLAD candidates. Create a spreadsheet of this information. Inform all scorers of the dates and times for the scoring of the Teaching Events and to which scoring team they have been assigned. Make sure they understand that they need to show up and participate.

• Take an inventory of the resources you have available such as computers/laptops which have QuickTime Player, TV/VCRs, headphones.

• Based on the size of your scoring teams, locate rooms which will accommodate each team for the scoring dates. You might also want to reserve a computer lab for on-line scoring.

• Meet with your lead scorers to review the scoring procedures you want all scoring teams to observe and follow.

• Ask each advisor to recommend at least 2 Teaching Events from students whose performance would likely approximate a solid “2”. These will be used at the beginning of your first scoring session to develop a “shared understanding” of how to score using the rubrics. Make sure each lead scorer reviews these events.

• On the day of the initial scoring, be sure that each lead scorer informs the scoring team that it is ESSENTIAL for all of the Teaching Events to be scored during the dates which have been set aside for this purpose. Also, the lead scorer should point out the total number of Teaching Events which need to be scored and the number of Teaching Events each scorer is responsible for scoring. In each team, the lead scorer should pair each new scorer with someone who had at least one prior year of scoring experience to provide an apprenticeship model of training. Have each scoring team score in pairs beginning with one of the recommended benchmark events. When discussing the scoring, try to have scorers reach consensus for the scores that are assigned. Make sure that the comments given match the rubrics. Come to an agreement about the types of comments to write and how much feedback each Candidate should receive. Review the Primary Sources of Information for Each Rubric and the Thinking Behind the Rubrics documents. I believe that these two documents are especially useful for the scoring process.

• We have developed a Teaching Event Score Sheet for our scorers to complete to determine PASS or FAIL.

• As the Teaching Events are scored, each lead scorer needs to keep track of events which have passed and those which need to be scored twice.

• Once all of the events have been scored, prepare a spread sheet of each task for each of the different Teaching Event content areas. The data from these spreadsheets can then be used to prepare a summary of the overall performance of your candidates in each of the task areas. This information can then be analyzed for programmatic purposes to determine how well your program prepares candidates to plan, instruct, assess, reflect, and develop students’ academic language.

UCLA TEP/Teach LA

PACT 2005

Important PACT-Teaching Event Dates

June 7th:Completed Teaching Events due to faculty advisor

Novice Faculty advisors submit one copy of each candidate’s Teaching Event to Matt Dingman (hard copies for those submitting paper copies; for those who submit via e-Portfolio, candidates must formally submit event to be scored by this date).

Note: For electronically submitted events, candidates must ensure that all attachments and videos are properly uploaded and that candidate formally submits event to be scored by marking event for scoring submission (completed) through the e-portfolio system.

June 8th:9:30-12:00 – Lead Trainer Meeting (for each TE subject) – Conference Room

June 8th:Each faculty advisor submits a nomination of a representative “satisfactory” event (and a complete hard copy if not previously submitted) to Matt Dingman (for use in calibration the following week).

June 10th9:00- 2:00—Mandatory Teaching Event training for all new scorers or those wishing a refresher on scoring. Email Anthony Rosilez at to reserve your spot by June 1st. Room TBD.

Week of June 13th: Teaching Event Scoring

June 13th9:00 AM – All scorers must be present for participation in scoring calibrations and assignment of Teaching Events to be scored. Room assignments will be posted on Center X doors. Computer lab in GSE&IS building will be available after calibration for those scoring electronic events.

June 13th – 16thLead trainers will assign Teaching Events for scoring to each scorer. Return events to Lead. Lead trainer will also arrange for a final calibration event with all scorers at end of scoring period, as needed.

Source: UCLA

Created: Spring 2005

Multiple Subject Teaching Events Count

Source: UCLA

Created: Spring 2005

2005 Teaching Events

Scoring Dates: June 13-17

Scoring Teams: [Italicized/Underlined Names denotes E-Format scorers]

Science- Freking*, Ning*, Nava, Swanson (19)

Math- Rosiliez*, Hasan, Abajian, Park, Ho, Hernandez, Priselac, CSUDH: Kamal Hamdan, Deandrea Newton (26)

English/LA- Williams*, Peitzman, Camangian, Montes, Reynolds, Richardson, Norma Willson, CSUDH: Cheryl Trujillo, Shirley Lal, Richard Gordon (22)

Social Studies-Hipolito, Kloes, Gelbwachs, Hoffman-Kipp, Butler, Kent Lewis (30)

Elementary Math-Lane, Francois, Perez, Powell, Shultice, Kapner, Martinez, Oswald, Oved, Sields-Smith, Y. Smith, Torres, Olsen, Ralls, CSUDH: Carol Nakayama, Dennis Dulyea, Gwen Brockman (55)

Elementary Literacy-Suzuki*, Battenburg, Dwyer, Staake, Banks, Small, Sato-Tenorio, Weber, Chang 35)

Breakdown by Teams and Content Areas

Elementary:

Dwyer: 25 Paper; 0 E; Literacy (1BCAD); 18 Math

Francois – 20 Paper; 4 E; 14 Literacy (2 BCLAD); 10 Math (4 BCLAD); 1 Unreported

Powell- 21 Ppaer, 1-3 Possibly E; 1 unreported; 11 Literacy; 12 Math (1 BCLAD?)

Shields-Smith-TeachLA 18E

Secondary

Hasan- 22 Math ESwanson-15 Science: 5 Paper; 10EShields-Smith TeachLA-14 E

Hipolito-16 SS: 15 paper; 1E; 8 English (2 BCLAD); 7 Paper; 1EPeitzman- 12 SS: 10 Paper; 2 E; 10 English: 9 Paper; 1 E

Source: UCLA

Created: Spring 2005

PACT Teaching Events Record Sheet

Elementary Literacy/Math

Source: UCLA

Created: Spring 2005

UCLA TEACHER EDUCATION PROGRAM

2005-2006 PACT Teaching Event Score Report

Scorer: Please complete the requested information below and calculate mean scores for each Teaching Event task. Thank You.

Candidate ID: ______

Scorer: ______

Insert 2 scanned pages here.

Source: UCLA

Created: Spring 2005

2005 Teaching Events Summary

UCLA Teacher Education Program

(2) P-1 Balanced Instructional Focus: 2.62(9) R-1 Monitoring of Student Progress: 2.32

(1) P-2 Accessible Content: 2.66(7) R-2 Reflection on Learning: 2.39

(3) P-3 Assessment Design: 2.58

(6) I-1 Engagement in Learning: 2.43(8) L-1 Understanding Language Demands: 2.33

(10) I-2 Monitoring Learning: 2.29(5) L-2 Supporting Academic Language Development: 2.46

(4) A-1 Analysis of Student Work: 2.48

(11) A-2 Assessment Informing Teaching: 2.15

Source: UCLA

Created: Spring 2005