Murrieta Valley Unified School District BTSA Induction Program
Section A, Part I:
Contextual Information
Biennial Report Contextual InformationProgram Information
Local Educational Agency
CD Code: 33-75200 / Number of Schools / Type of BTSA Induction Program / Support Provider Model(s) Used / Formative Assessment System
K-12 / X / Elementary / 11 / Single District / X / Classroom-based / X / FACT / X
Elementary / Middle / 4 / Consortium / Full-time Released / NTC FAS
High School / High / 4 / Multi-District / Part-time Released / X / Locally Designed
COE / Other / 2 / Other / Retired / X
Participant Information
08-09 09-10 08-09 09-10
Number of candidates (public schools) / 45 / 21 / Total Number of candidates assigned to School Improvement or SAIT-identified settings / Year 1 / 0 / 0
Number of candidates (private schools) / 0 / 0
Number of active Support Providers / 7 / 5 / Year 2 / 0 / 0
Candidate : Support Provider ratio / 6.4:1 / 4.2:1 / Total Number of candidates assigned to a school in Program Improvement / 0 / 0
Total number of candidates recommendedfor Clear MS or SS Credential / 25 / 13 / Number of Verification of Unavailability of a Commission-Approved Induction Program (CL-855) notices issued to eligible candidates / 0 / 0
Number of candidates recommended for Clear MS or SS Credential via Early Completion Option / 1 / 1
Program Changes
Significant changes made since the last Biennial Report or Program Assessment Review
Program Standard(s) / Explanation of Change
MVUSD(616)-Biennial Report-August 2010
Murrieta Valley Unified School District BTSA Induction Program
Section A, Part II:
Candidate Assessment/Performance and Program Effectiveness Information
Assessment #1: IIP Rubric
Description: One of the most important components of an effective inquiry is a close connection between the inquiry question established by the participating teacher and the action research plan carried out in order to explore it. We wanted to examine how effectively our PTs were able to do this and how effective a job our program did in supporting PTs in achieving this tight connection and focus. In order to assess this, we chose to examine the results of our IIP rubric scores and focus in on the rubric descriptor that measures the overall effectiveness of the inquiry.
Results: Each IIP was studied and assessed as to the overall quality of the focus question and action plan. The following rubric was used:
Beginning = 1
Developing = 2
Maturing = 3
Accomplished = 4
The chart that follows shows the frequency and overall percentage with which the overall focus question and action plan were assessed at these levels:
# Respondents / 1 / 2 / 3 / 4 / MeanPT / 19 / 0% / 5% / 10% / 85% / 3.77
PT Y1 / 6 / 0% / 0% / 17% / 83% / 3.71
PT Y2 / 13 / 0% / 8% / 8% / 84% / 3.76
Assessment #2: PT State Survey Results
Description: In addition to examining the results of the actual IIP evaluation, we wanted to examine how the PTs felt about the connection they were able to make between their inquiry question and action plan. Because a major purpose of our induction program is to foster reflective practices that our PTs will continue throughout their careers, it is important to ensure that they recognize the value of inquiry and action research. We chose to examine the results of state survey question D18F in order measure this. State survey question D18F asks the following: How much impact did the development of your Individual Induction Plan have on your classroom practice? If the PT believes the development of his or her IIP has a strong impact on classroom practices, then it is likely they will value the process and continue to implement it throughout their careers.
Results: The following descriptors are used to measure the results:
No Impact = 1
Some Impact = 2
Moderate Impact = 3
Very Strong Impact = 4
Did Not Participate in This Activity = 0
The following chart represents the frequency with which responses were given at each of the above levels:
# of Respondents / 0 / 1 / 2 / 3 / 4 / 5 / Mean18 / 0% / 0% / 0% / 40% / 60% / 0% / 3.7
Assessment #3: SP and PT State Survey Results
Description We also wanted to measure the connections both PTs and SPs felt between their work together and the development of their IIP, specifically focusing on the connection between the goals of their inquiry and the action plan developed. To measure this, we chose PT State Survey questions C15D and SP State Survey Question 25D. PT State Survey question C15D asks PTs the following question: How strong were the connections between the goals and research activities you developed for your Individual Learning Plan and your work with your BTSA Induction Support Provider? SP State Survey question 25D asks the following: In thinking about the Individual Induction Plan that you and your participating teacher developed together: How strong were the connections between the goals and research activities of your Participating Teacher’(s) Individual Induction Plan and his/her work with you, the PT’s BTSA Induction Support Provider? By comparing the results of these two similar questions, we could examine the correlation between how the PTs felt about the effectiveness of their collaboration with their SP with how the SPs felt about this collaboration. We measured this by comparing the rating given by the PT with the rating of the SP. We then identified the difference between the ratings.
Results: The following descriptors are used to measure the results:
PTs:
No Connections = 1
Some Connections = 2
Moderate Connections = 3
Very Strong Connections = 4
SPs:
Very Weak Connections = 1
Weak Connections = 2
Strong Connections = 3
Very Strong Connections = 3
The following chart represents the frequency with which responses were given at each of the above levels as well as the difference (+/-) between the PT response and the SP response:
# of Respondents / 1 / 2 / 3 / 4 / MeanPT / 18 / 0% / 0% / 40% / 60% / 3.6
SP / 5 / 0% / 0% / 33.3% / 66.7% / 3.7
+/- / -6.7 / +6.7 / +.1
Assessment #4: Work Session Evaluation and Overall IIP Rubric Scores
Description: Finally, we wanted to examine how effective our monthly work sessions were in fostering effective inquiry practices. In order to do this, we chose take the results of the evaluation of the work session that specifically concentrated on developing a tight focus between the inquiry question and action plan and correlate these with the overall IIP rubric scores identified in Assessment 1 above. The work session evaluation measured how helpful PTs found the information and support provided at this session in meeting their needs for developing a specific, relevant action plan.
The work session evaluation asked the PTs the following questions:
1. Overall, how helpful was today’s work session in meeting your needs for developing a specific, meaningful focus question for your inquiry?
2. Overall, how helpful was today’s work session in meeting your needs for developing a specific, relevant action plan for you IIP?
We wanted to see if the overall responses to this question correlated with the final results in order to determine if the support provided during these sessions actually led to measurable results.
Results: The following descriptors were used to measure the results:
Not at All = 1
Somewhat = 2
Fairly = 3
Very = 4
Question / # of Respondents / 1 / 2 / 3 / 4 / Mean1 / 14 / 0% / 0% / 14.29% / 85.71% / 3.86
2 / 14 / 0% / 0% / 14.29% / 85.71% / 3.86
MVUSD(616)-Biennial Report-August 2010
Murrieta Valley Unified School District BTSA Induction Program
Part III and IV
Analysis of Candidate Assessment Data/Use of Assessment Results to Improve Candidate and Program Performance
Focus Area #1:
Common / ProgramStandards / Data Source / Areas of Strength / Area for Growth / Proposed Changes
PS 5 and 6
CS 2 and 9 /
- IIP Rubric Overall Score
- Participating Teacher Statewide Survey Results for Question D18F
Data sources indicate that both Year 1 and 2 PTs earned high marks for the quality of their inquiry when it came to this tight focus. This is an important quality of effective inquiry that promotes the development of instructional practice and impacts student learning. When this is then analyzed against results of Assessment 2, it suggests that the tight focus of the IIP may have contributed to the PT’s attitudes regarding the impact their inquiry had on student learning. The 60% felt that the inquiry had a “Very Strong Impact,” and the rest felt it had at least a “moderate impact.” While this is obviously a subjective assessment, it is important in that teachers who see value in the inquiry process are more likely to practice it throughout their careers. As a result, Assessments 1 and 2 lead us to positive conclusions with regard to candidate competence related to the quality of their inquiries, the effectiveness of their inquiry process, and the development of reflective skills and tendencies. / Need to increase the number of candidates scoring 4 on the IIP rubric and limit scores below 3.
While the majority received strong IIP rubric scores, our goal is to have all PTs scoring 3 or higher on the rubric. Having a year 2 PT score a 2 is unacceptable, and we would like to have even more PTs scoring at the 4 level. / Utilize our IIP rubric as a learning tool for candidates to support them in the development of their IIPs.
We adopted the rubric we would use for evaluating our IIPs late in the year, and therefore the PTs did not have an opportunity to use it as a guide to developing their IIPs. Our plan is to give the PTs the rubric at the start of the year so that they have a clear understanding of what they must know and be able to do in order to be successful with their IIP. They will be able to refer to this as they develop their IIPs or while giving and receiving feedback during work sessions to ensure that they are on the right track and providing accurate feedback to fellow candidates. It is our belief that this will lead to improved results on IIP scores reflective of greater candidate competence.
Focus Area #2:
Common / ProgramStandards / Data Source / Areas of Strength / Area for Growth / Proposed Changes
PS 5 and 6
CS 2 and 9 /
- IIP Rubric Overall Score
- Work Session Evaluation
Data sources indicate that the Work Sessions we conduct are also effective in supporting PTs through the inquiry process. The focus of this Work Session was specifically designed to help PTs establish a focus question for their inquiry and develop a clearly relevant action plan with measurable results in response to that question. It included model IIPs, practice critiquing sample IIPs in small groups, and an opportunity to collaborate with other PTs and their SP on their own IIP. Each PT had the opportunity to give and receive feedback on the quality of their focus question and proposed action plan, and the time to use this feedback to further refine their IIP. The evaluation results suggest that the majority of PTs (85.71%) felt that the session met their needs in developing a “specific, meaningful focus question” and in developing a “specific, relevant action plan” for their IIPs. The other 14.9% felt that the session was “fairly relevant” in both regards. While it is important that PTs responded positively to the session, more importantly, the positive response correlates to the strong overall IIP rubric scores. This suggests that the effectiveness of the work session may have contributed to quality of the IIPs produced by our PTs.
. / Provide more and improved professional development and collaboration opportunities for the development of IIPs.
Want to improve evaluation results to reduce percentage only finding them “fairly relevant.” Also need to increase the number of sessions focusing on the IIP as we were not able to address all components of the IIP process. / Implement a year-long focus in work sessions/seminars on reflective practice and inquiry.
We began this shift a mid-year last year following a regional cluster meeting in which ideas for this type of professional development for PTs were presented. At the time, or work sessions had focused on imparting instructional strategies or classroom management techniques. These were essentially one-size-fits-all mini-workshops that helped those who found them relevant but did not necessarily meet the individual needs of our PTs. We determined that it was essential to focus on fostering effective reflective practices among our PTs, which is why we made the shift. Mini-workshops would still be provided via a menu of professional development options available to PTs via our online staff development program. While the change was well-received and effective, we believe it was too late, and that implementing this focus from the start of the new school year will help ensure all PTs conduct effective inquiry. These sessions will be designed to ensure candidates have the tools and essential understandings to develop an IIP, and that we can do a better job fostering a commitment to reflective practice that will remain with PTs throughout their careers.
Focus Area #3:
Common / ProgramStandards / Data Source / Areas of Strength / Area for Growth / Proposed Changes
PS 5 and 6
CS 2 and 9 /
- IIP Rubric Overall Score
- Participating Teacher Statewide Survey Results for Question D18F
- Work Session Evaluation
Data sources indicate that collaboration with SPs was a major factor in the positive results identified in Assessments 1 and 2. Assessment 3 indicates that both PTs and SPs believe that their collaborations had a significant impact on the quality of the IIPs completed by the PTs. Sixty percent of PTs felt there were a “Very Strong Connections” between their work with their SP and the goals and research activities established and completed in the IIPs. Forty percent felt there were at least “Moderate Connections” in this collaboration. Similarly, the majority of SPs (66.7%) found “Very Strong Connections” between their work with PTs and the resulting goals and research activities completed in the IIPs. The rest (33.3%) found there were “Strong Connections.” Considered against the strong candidate competencies demonstrated in Assessments 1 & 2, the results of Assessment 3 suggest the support provided by our SPs had a strong impact on the quality of the inquiries completed by our PTs.
Additionally, work session evaluations suggest support provider do an effective job providing activities and resources to support candidates during these sessions in the development of their IIPs. / Increase support provider skills in supporting candidates in IIP development individually and through professional development.
We want to increase the number of PTs and SPs who find “very strong connections” between their collaborative efforts and the development of their IIPs. We also need to be well prepared to provide activities and resources during work sessions that will shift in focus to supporting inquiry and action research for the development of IIPs (see focus area 2). This will also lead to improved IIP rubric scores. / Support providers will engage in professional development designed to improve effectiveness of the support they provide candidates during the IIP process and to prepare them to lead work sessions with a new focus on inquiry and action research (see focus area 2).
In order to increase program effectiveness, our SPs are going to engage in reading The Reflective Educator’s Guide to Classroom Research: Learning to Teach and Teaching to Learn Through Practioner Inquiry by Mancy Fichtman Dana and Diane Yendol-Hoppey. This book was shared by our County Regional Directors and read by the program coordinator. Excerpts have been shared with SPs and PTs. We believe that reading this book will increase the skills of our SPs in individually supporting PTs through the inquiry process. It will also provide ideas and resources for our work sessions. We also plan to consider it as a possible text for PTs the following year. We believe engaging in this reading together will enhance program effectiveness.
Focus Area #4:
Common / ProgramStandards / Data Source / Areas of Strength / Area for Growth / Proposed Changes
PS 5 and 6
CS 2 and 9 /
- IIP Rubric Overall Score
- Participating Teacher Statewide Survey Results for Question D18F
- Participating Teacher Statewide Survey Results for Question C15D
- Support Provider Statewide Survey Results for Question 25D
- Work Session Evaluation
Overall data analysis reflects scores on the positive side of all measurement scales. / Utilize evaluation tools more consistently and includes tools that reflect growth over time.
We want to improve the data we collect for evaluating program effectiveness. We considered using our Support Logs to help evaluate the program by examining the average number of hours SPs collaborated with PTs on their IIPs. We then wanted to correlate the hours spent to the final rubric results to determine if there was a clear connection between time spent and the quality of the inquiry. However, when we analyzed the logs, we discovered that there was inconsistency with how they were completed from SP to SP. Upon further investigation, we realized that it was partly due to redundancies in the log choices and a lack of training for SPs in the completion of the log.
Another area related to improved data collection was the realization that we really don’t have sufficient data to accurately assess the effectiveness of the inquiry process in terms of its impact on student learning. We only have the PT question (Assessment 2) in which PTs self-report this impact. Therefore, while the above data gives us some indication of the level of candidate competence fostered through the inquiry process, we need to ensure we have better means to measure this. We need to have a pre-assessment against which we can measure the final results. While we have the final PT ratings on the Continuum of Teaching Practice (COTP), we did not have entry level data to compare these ratings to in order to make stronger correlations to the other data we were examining. Because of our use of the COTP instead of the DOP, we did not make use of Form E in FACT. However, we believe that this was a program shortcoming and that using the DOP in conjunction with Form E will provide us with better data to assess candidate competence. / Revise and utilize support logs more consistently and implement evaluation tools that will demonstrate candidate growth over time.
We plan to revise our Support Logs and then ensure through SP training that these are used more consistently and completed so as to provide useful data that can assist us in assessing program effectiveness.
A seccond part of our action plan is to ensure that every PT complete FACT Form E-2.1. Form E-2.1 is a CSTP self-assessment that requires PTs to rate themselves on a 1-4 rubric by each CSTP and element prior to the beginning of their inquiry process. This can then be used as an entry-level assessment in conjunction with the DOP(new CTP) following the inquiry process. While the value to the PT as a reflective tool is obvious, it will also provide us with powerful evidence of candidate competence that we can use in the evaluation of our program. So we will be implementing the required use of forms E1, E2, and E3, and we plan to consider using the new CTP instead of our current COTP.
MVUSD(616)-Biennial Report-August 2010