Recommendations from the
Task Force on Evaluation of Teaching
April 24, 2012
Executive Summary
Provost Liz Grobsmith charged the Task Force on Evaluation of Teaching to explore current NAU practices in relation to the evaluation of teaching and best practices in other institutions and to make recommendations for Northern Arizona University. The recommendations are summarized below.
At its core, the evaluation of teaching is conducted to ensure a positive and productive learning experience for our students, while helping to develop the effectiveness of those who teach. To establish a teaching evaluation system that preserves this perspective, the Task Force recommends a framework that invites units to explore reasonable means of gathering relevant data from multiple sources for effective evaluations.
The Task Force further recommends the following practices in relation to student opinion surveys (questionnaires):
· Clarify the purpose and uses of course questionnaires for students;
· Administer strategic mid-term evaluations (as determined by units) that are more comprehensive, in conjunction with brief end-of-course questionnaires;
· Adopt a web-based tool (we recommend SmarterSurveys be piloted as the most promising option) for student opinion surveys for the following reasons:
o Enhanced reliability
o Meaningful comparisons (within and beyond NAU)
o Ease of administration and analysis
o Integration with the Faculty Activity and Achievement Reporting (FAAR) system
The Task Force recommends the following in relation to the role of student success. At the unit level (department or program):
· Articulate clear expectations for student success (represented, in part, by the proportion of students achieving better than a D, F, or W in a course);
· Use data related to student success to determine the appropriateness of the course design and curricular placement;
· Use data related to student success to determine the appropriateness of the course assignments to instructors and the development of faculty;
· Incorporate context-sensitive judgments about teaching quality vis-à-vis student success data for individual faculty members.
Recommendations from the
Task Force on Evaluation of Teaching
April 24, 2012
Introduction
Provost Liz Grobsmith charged the Task Force on Evaluation of Teaching to explore current NAU practices in relation to the evaluation of teaching and best practices in other institutions, in order to make recommendations for Northern Arizona University. This document provides the requested recommendations.
The charge from the Provost (September, 2011), summarized briefly, included the following issues:
· Explore alternatives to the current course evaluation practices at NAU, and
· Propose revisions to the system, procedures and instruments in place at NAU.
Task Force membership:
Kathy Bohan, ACC, COE
David Boyce, GSG
Wendy Campione, Teaching Academy, FCB
Ryan Ellis Lee, ASNAU
Gae Johnson, Faculty Senate, COE
Dan Kain, Provost’s Office and Convener
Karen Mueller, CHHS
Mary Reid, President’s Distinguished Teaching Fellows, CEFNS
Michael Vincent, Deans and PALC, CAL
Andy Walters, SBS
Eric Yordy, AADR, FCB
The Task Force met monthly throughout the AY2011-12 year, through April. In addition, three subcommittees (Current Practices, Best Practices, and Common Procedures, Practices and Instruments) met regularly.
Recommendations of the Task Force on Evaluation of Teaching
April 24, 2012
Purpose and Goals of Evaluation of Teaching
The Task Force agrees that a system for evaluating teaching must address multiple purposes and goals. At its core, the evaluation of teaching is conducted to ensure a positive and productive learning experience for our students, while helping to develop the effectiveness of those who teach. However, within that overarching purpose, an effective evaluation system should also accomplish the following goals:
· To provide faculty members with direct and helpful feedback about the effects of their efforts to help students learn and to be successful;
· To provide information that will assist faculty members with opportunities for developing their skills as teachers;
· To provide faculty reviewers with valid and reliable information about teaching performance for decision-making in formal review processes;
· To ensure that students are active participants in the enhancement of teaching at NAU;
· To include faculty members as partners in their own evaluation process.
General Principles for Evaluation of Teaching
In recognition of the emphasis on excellent teaching at Northern Arizona University, the Task Force endorses a perspective on evaluation of teaching that is supportive, formative, and developmental. The Task Force encourages a system of evaluation that provides faculty members with direct feedback on the effects of their instructional/teaching efforts as well as opportunities to continue to grow as teachers throughout their careers. The act of teaching centers on an interaction that enables students to be successful in acquiring knowledge, skills, and dispositions (e.g., openness to diversity). The Task Force endorses a perspective on the evaluation of teaching that derives from sound principles that have emerged in the field of higher education. These principles are articulated, in brief, below:
· Effective evaluation is embedded in a process that is supportive of the growth and development of faculty members. While it is appropriate to make judgments about teaching quality, the starting point of effective evaluation is the recognition that evaluation is part of a system of support and opportunities for growth.
· Evaluation of teaching must recognize the contextual variation inherent in teaching at a large and multi-faceted university. Thus, any evaluation process must adapt to variations in that context.
· Principles of learning should inform evaluation processes. (See Appendix A for sample principles).
· Effective evaluation of teaching draws on multiple forms of evidence. Thus, it is appropriate for the evaluation of teaching to incorporate different forms and sources of evidence (see below).
· Effective evaluation of teaching can occur any time in the second half of a course, and the blend of mid-term and end-of-term student opinion surveys can provide information needed by different participants in the process.
· To the extent possible, effective evaluation should honor the time commitments and perspectives of those called on to provide information. Thus, for example, consideration should be given to the demands placed on students in evaluating multiple instructors.
· Effective evaluation involves a systematic process of informing faculty, their peers, and administrators of the areas for growth, as well as areas of strength.
The Role of Student Success in Evaluation of Teaching
The Task Force considered the role of student success (represented, in part, by the rate of students achieving better than a D, F, or W in a course) in evaluating teaching. The Task Force does not endorse a specific percent of students achieving above the DFW level as a uniform measure of student success. We recognize that variables related to the course content, the decisions of students about taking particular courses, and the requirement status of courses (i.e., required vs. elective) all affect this issue. However, the Task Force does endorse unit-level consideration of student success as one component of the evaluation process. The following considerations are offered as questions for the units to consider in making judgments:
· What are appropriate indicators of student success for a particular program?
· In relation to course grades, what level of success does a department or program expect in the various courses (e.g., is 80% better than DFW acceptable?)? This should be articulated.
For cases where the proportion of students succeeding is below the unit expectations:
· Is the course properly situated in the curriculum so that students have necessary prerequisite knowledge and skills to be successful? (Should the curriculum be revised?)
· Are students enrolling inappropriately so that they are not academically and/or developmentally prepared for the course?
· Is the low student success a part of a pattern (for the particular students, the course, the instructor, something else)?
· Does the instructor’s teaching assignment match his or her teaching strengths and interests?
· What explanation does the instructor provide for levels of success that are under unit expectations?
· What actions has the instructor taken to enhance the success of students?
Ultimately, the Task Force recommends that units establish clear expectations both for acceptable student success and also for inclusion of this factor in making judgments about teaching.
Framework for Data Gathering
Assuming the value of multiple forms and sources of evidence, the Task Force recommends the following framework, based on Arreola (2000) and Felder and Brent (2004)[1], for gathering data from appropriate sources in order to provide a balanced view of evaluation. Additional information about the sources of data and data-gathering processes is included in Appendix B.
14
Task Force on Evaluation of Teaching, April 2012
framework for evaluating teaching (After Arreola, 2000, and Felder Brent, 2004)
Summary of salient criteria for teaching evaluation, the appropriateness of each evaluator group (faculty member, students, peers, chair/director) for evaluating these criteria (N/A = not appropriate), and the technique for evaluation. Main evaluator for each criterion shown in bold. Activities, questions, and conditions of use for each evaluator addressed in Task Force Report Appendix B.
Criteria / Sub-Criteria / EvaluatorFaculty Member / Students / Peers / Chair/Director
Content Expertise / Analysis of content expertise / Assesses ongoing development of content expertise / N/A / Review course materials / Reviews course materials
Perceptions of instructor’s content expertise / Team teaching, peer assessment, teaching scholarship / Evaluate instructor via questionnaire / N/A / N/A
Instructional Delivery / Effectiveness of delivery / Describes philosophy and methods; (video self-assessment possible) / Evaluate instructor via questionnaire / (Review videotape if available) / N/A
Student success / Describes efforts to ensure student success / Evaluate instructor via questionnaire / Review DFW rates and explanation in concert with departmental indicators / Reviews DFW rates and explanation in concert with departmental indicators
Instructional Design Skill / Analysis of instructional design / Describes philosophy and approach / N/A / Review course materials / Reviews course materials
Perceptions about course design / Describes course objectives, outcomes and measures / Evaluate instructor via questionnaire / N/A / N/A
Course Management / Describes philosophy and methods / Evaluate instructor via questionnaire / Review course materials / Reviews course administration materials and student feedback
14
Task Force on Evaluation of Teaching, April 2012
The Task Force recommends that units explore reasonable means of gathering relevant data from multiple sources for effective evaluations. However, the Task Force was specifically charged with determining whether to recommend any common procedures and/or items for student opinion surveys. To that end, the following recommendations are offered:
Questionnaires for Student Evaluations (Student Opinion Surveys)
A key element in the matrix presented above involves questionnaires or student opinion surveys. The Task Force recommends the following university-wide procedures:
Clarify the Purpose of Evaluations
Research demonstrates that the nature of student responses varies according to their understanding of the purposes and uses of course evaluations. Therefore, it is incumbent upon units and faculty members to communicate the value and uses of the information gathered through the evaluation process.
Midterm and End-of-Term Questionnaires
Using both midterm and end-of-term questionnaires is a promising practice for several reasons, as enumerated below. The use of a two-stage process
· may diminish the time demands placed on students at the end of the term;
· provides faculty an opportunity to incorporate valid suggestions into their courses while the providers of that feedback can experience the changes;
· offers the possibility of more individualized and substantive feedback for faculty members;
· forms a reasonable approach to distinguish the uses of evaluation material (for example, a midterm questionnaire might be used in the year of comprehensive reviews for tenured faculty members, but not in expedited review years); and
· allows for streamlining the final course evaluation process, potentially increasing the likelihood of student participation.
Therefore, the Task Force recommends a unit-determined approach of blending more comprehensive midterm evaluation forms with brief end-of-term questionnaires.
Commercial vs. “Home-grown” Surveys
The Task Force recommends NAU use a commercial provider to administer the student opinion surveys. This approach will enable several advantages: comparability among units, uniformity in the administration of surveys, enhanced reliability of survey data, reduction of staffing demands within NAU, flexibility in the development of midterm survey items and integration with the Faculty Activity and Achievement Report (FAAR) system. The collection of reliable data with opportunities for comparisons provides departments with a tool to enhance unit effectiveness. We recommend the use of SmarterSurveys. The advantage of the services offered by SmarterSurveys include the following: single price for the year, regardless of the number of surveys administered; flexibility in constructing instruments to meet the needs of various units; and integration with the Faculty Activity and Achievement Reporting (FAAR) system. The Task Force recommends the tool be piloted in AY2012-13. Results of the pilot experience will be incorporated into the decision-making process.
Common Items in End of Term Questionnaires
The Task Force endorses a small set of common items for end-of-term evaluation forms. These items are the components of the SETE (Student Evaluation of Teaching Effectiveness), provided through SmarterSurveys (www.smartersurveys.com).
Administering Surveys
The Task Force endorses the administering of surveys via a web protocol. We recognize that there are legitimate concerns about the response rates for student questionnaires. Several practices tend to lead to higher response rates, and we encourage units to explore how best to incorporate the following:
· Students are more likely to respond if they are convinced the information is put to use (both by faculty members and by administrators). Therefore, communication about the uses of course evaluations must be proactive.
· Students are more likely to respond if the end-of-term evaluations are succinct and less demanding of their time.
· Students are more likely to respond if they are assured of anonymity. [Note: it is crucial that students understand that midterm evaluations will be read by instructors during the course.]
· The Task Force urges units to consider carefully the appropriateness of incentives (e.g., extra credit) that faculty members offer students for completing evaluations.