1.4.D Key Assessments and Scoring Guides for Programs Not Already Available in AIMS

Exhibit 1.4.d1

1.4.d Key assessments and scoring guides for programs not already available in AIMS

Two major assessments and scoring guides are not available in aims. These are the Candidate Dispositions Progress Report and the Mentor Rating of Candidates’ Use of Technology.

Dispositions

Précis:

• The Candidate Dispositions Progress Report was piloted in 2002-03 and has been in continuous use since that time.
• A study conducted 2006 provided evidence of strong inter-rater reliability for the assessment.
• Mean ratings on the Candidate Dispositions Progress Report document positive dispositions of both initial and advanced candidates. There are strong mean ratings of dispositions targeted by our Transformation Initiative Proposal.
• Weekly monitoring of the Candidate Dispositions Progress Report and Brief Reports provides programs with reports of exemplary dispositions and concerns. Concerns are addressed at the discretion of the program coordinator, and may involve the development of an action plan.
• Action plans are monitored and may result in these decisions: (a) Concern has been resolved/goals have been met; (b) Progress toward goal(s) noted – continue with revised Action Plan; (c) Advised to research other career options; (d) Dismissed from Licensure Program; candidates may develop a rebuttal for the report and decision.
• Due to unit-wide interest in going beyond the summative nature of the Candidate Dispositions Progress Report, a more detailed instrument is being piloted with various observers, including candidate peers, mentors, instructors, and university supervisors.
• Scholarly efforts related to our documentation of dispositions has resulted in a publication: Laine, C., Bauer, A. , Johnson, H. Kroeger, S., Troup, K., & Meyer, H. (2010). Moving from reaction to reflection. In P. Murrell, M. Diez, S. Feiman-Nemser, & D. Schussler, (Eds.), Teaching as a moral practice: Defining, developing, and assessing dispositions in teacher education, (pp. 73-94). Cambridge, MA: Harvard Education Press

______

Our unit dispositions were collaboratively identified by faculty members representing teacher education, art and music education, school psychology, educational foundations, and arts and sciences reviewed and provided input into these dispositions, as did elementary, secondary, middle school, and special education public school teachers and other professional school personnel. Consensus was reached on candidate dispositions for both initial and advanced programs in May, 2002, with pilots of our assessment during the 2002-2003 academic year. During the 2003-2004, following analysis of pilot data and review of the assessment, a revised assessment was implemented. Changes were, for the most part, positioning of the items and the rubric, and data are provided from 2002 to 2011.

As we entered into our Transformation Initiative, it became apparent that we all needed a stronger mutual understanding of our dispositions. Did we really want to begin by asking our candidates to be present, on time, and appropriately dressed, or were we more interested in our candidates demonstrating agency in working in complex environments? Feiman-Nemser (1991) describes the “critical orientation” as the educators’ obligations to students and society. This orientation challenges us to help educators develop educational opportunities that reflect democratic principles and teach them to questing assumptions about teaching, learning, knowledge, and students.

Variations among our programs reflect Feiman-Nemser’s “practical orientation,” which focuses on the aspects of teaching and learning to each licensure area. The practical orientation supports the notion that our candidates must learn to be flexible, and, at times, invent responses in the face of unsolvable problems. Schon (1983) describes this engaging in on-the-spot reflection and experimentation, drawing on a range of images, theories, and actions to construct an appropriate response. Feiman-Nemser contends that this practical knowledge is gained through a combination of clinical experiences, interaction with peers and mentors, and general exposure into the community of practitioners and practice.

Dispositions: As educators transforming the conversation of learning, and creating a third space for instruction we prepare candidates who:

• demonstrate initiative on behalf of all learners.
• demonstrate responsibility to promote effort and excellence in all learners
• demonstrate rapport with students, peers, and others.
• demonstrate a commitment to reflection, assessment, and learning as an ongoing process grounded in inquiry,
• are willing to work collaboratively with other professionals to improve the overall learning environment for students.
• appreciate that “knowledge” includes multiple perspectives and that development of knowledge is influenced by the perspective of the knower.
• are dedicated to teaching the subject matter and in keeping informed and are competent in the discipline and its pedagogy.
• appreciate both the content of the subject are and the diverse needs, assets, and interests of the students and value both short and long term planning.
• are committed to fairness and the expression and use of democratic values in the classroom.
• take responsibility for making the classroom and the school a “safe harbor” for learning, in other words, a place that is protected, predictable, and has a positive climate.
• recognize the fundamental need of students to develop and maintain a sense of self-worth, and that student misbehavior may be attempts to protect self-esteem.
• believe that all children can learn and persist in helping every student achieve success.
• value all students for their potential and people and help them value each other.
• are committed to high ethical and professional standards.

The assessment that has been used with slight modifications since 2003 is available at https://survey.education.uc.edu/websurvey/rws3.pl?FORM=Cand_Dispositions_Progress_Report.

In addition to this assessment,completed by all mentors a minimum of one time during a candidate’s field experience., we have used an “incident” reporting system since 2002. This “brief report” can be used by members of the professional community, faculty members, mentors, or university supervisors to document a concern or an exemplary disposition. This form is available at https://survey.education.uc.edu/websurvey/rws3.pl?FORM=Dispositions_Brief_Report. Instructors, concerned that assessment in the field by mentors and this brief report did not reflect the classroom environment, initiated an instructor disposition assessment in 2009. This assessment was implemented to monitor the professional behavior of candidates on campus and is available at https://survey.education.uc.edu/websurvey/rws3.pl?FORM=Candidate_Classroom_Dispositions_Report_Form . This form also provides an opportunity for faculty members to observe dispositions of advanced candidates.

Candidate Dispositions Progress Report Since its inception in 2002, the Candidate Dispositions Progress Report has followed essentially the same process for monitoring candidate performance. In addition to the mentor’s completion of the report, A university supervisor may complete additional disposition forms for a candidate if they see a need to document exceptional behavior, positive or negative. Completed disposition forms are be printed and shared with the candidate. The candidate must be aware of the report. A candidate may file a rebuttal to a disposition. Since 2006, an electronic version of the completed dispositions form is submitted to the Office of Assessment and Continuous Improvement. The Office of Assessment and Continuous Improvement monitors the forms weekly and provides any evidence of potential issues to the Program Chair. The chair then acts on the information depending on the level of concern. If the concern is significant, and may have an impact on the student continuing in the placement or entering a subsequent placement, an action plan is written. This action plan is an agreement between the candidate and program faculty, describing objectives for improvement, the ways in which the plan will be monitored, and a review date for the plan. Failure to successfully complete an action plan may result in termination of the placement, a change in placement, or leaving the program without licensure recommendation. The action plan format is provided at the end of this document.

The Survey and Aggregated Data Data are presented for the unit, initial programs, and advanced programs. In addition, ratings of dispositions demonstrated by advanced candidates in the university classroom are presented for the two years of implementation. Disposition data are presented as aligned with dispositions, institutional standards, and general education outcomes.

In 2006, 5% of teaching associates were randomly selected as participants in a reliability study. Ratings by university supervisors were compared to those of mentors, generating the following results:

This strong inter-rater reliability contributes to our confidence in the clarity of the rubric and items.

Analysis The following charts provide mean ratings of candidate dispositions disaggregated by initial and advanced programs. Examples of individual program ratings are provided following the analyses. Means reported are the averages of ratings on the rubrics provided for individual disposition items. Analyses are by disposition alignment with the items.

Mean ratings suggest that both initial and advanced program candidates demonstrate strong attendance and punctuality. With a rating of 4 being exemplary (the rubric indicates perfect attendance and never late as the “4” rating for these items) candidates appear professional in attendance and punctuality. Though the scale is different for the classroom disposition report (3 indicates that the candidate meets expectations) there are high mean ratings of attendance and punctuality.

Both initial and advanced candidates’ mean ratings for commitment to high ethical and professional standards are consistently high. Raters are asked to indicate if candidates demonstrate these standards as yes, no, or unable to rate. The ratings are mean percent of yes responses. Advanced candidates’ mean ratings are 100% across the board; we would anticipate ratings such as these from candidates who are practicing professionals.

Consistently high ratings are reported for both initial and advanced programs. No true trend line was generated in the initial program; there has been a positive trend in the mean ratings of advanced candidate reflection, assessment, and recognition of learning as an ongoing process.

Mean ratings of initiative and responsibility are also consistently high across initial and advanced programs.

Mean ratings on these items, essential dispositions in our transformation initial and our conceptual framework, are consistently high. With our program emphases, we anticipate that these ratings will become even higher.

Advanced programs do not assess the general education baccalaureate competencies. However, oral and written communications are assessed on the classroom disposition form. Mean ratings indicated that the outcomes are met, approaching strongly met.