Improving the Evaluation of College Teaching

Chapter 7:

IMPROVING THE EVALUATION OF COLLEGE TEACHING

L Dee Fink[1]

The question of how to evaluate teaching is critical in institutions of higher education for several reasons. Individual professors, in order to work on improving their teaching, must have some way of knowing whether this way of teaching is better or worse than that way. The institution, if it wants to encourage, recognize, and reward excellence in teaching, must have some reliable means of distinguishing between more effective and less effective teachers. Yet, despite the importance of this question, most colleges and universities continue to struggle with the question of how to find a satisfactory system of evaluating teaching.

At my university, student evaluations of all courses have been required for several years. This has given administrators a numerical basis for assessing the teaching activities of the faculty in annual performance evaluations. But many professors are bothered by the idea of having their teaching measured by one number or a set of numbers from student questionnaires. Periodically pressure builds up to find a better solution to the problem. When this happens, a department or college committee often requests that I come in as a consultant on educational evaluation.

In these situations, I share the ideas that have emerged in the published literature on this topic and that have taken the form of the recommendations in this essay. In 1979, Centra published an influential summary of research and analysis on faculty evaluation, including the evaluation of teaching. Even at this relatively early date, he had separate chapters with comments on the pros and cons of several different modes of evaluation: student ratings, self-assessment, evaluation by colleagues, and assessment of student learning. Later Seldin (1984) conducted a nationwide survey of prevalent and changing practices in the evaluation of teaching. He noted that, from 1978 to 1983, colleges had significantly increased their use of systematic student ratings, assessment of course materials, and self-evaluation (pp. 44-55). Fink (1995) offered a description of how faculty could do an in-depth assessment of their own teaching, using five distinct sources of information. Braskamp and Ory (1994) took a broader view of assessing faculty work in general, by suggesting the faculty members and departments work together to set expectations and then to periodically assess progress. Within this framework, they noted that efforts to jointly assess faculty teaching could use multiple types of evidence: descriptive, outcomes (student learning), judgments, eminence indicators, and self-reflection. More recently, several chapters in another book edited by Seldin (1999) on evaluating teaching focused on different sources of information and assessments: student ratings, peer classroom observations, and self-evaluation including the creation of teaching portfolios.

The general conclusion I draw from the published literature is that teaching needs to be evaluated using more than student ratings. Hence, whenever I am called on to advise a special committee charged with finding a new and better way of evaluating teaching, I try to persuade the group to change their guiding question. Usually they start with the question of "How can the student questionnaire be improved?" I try to get their attention focused on the more fundamental question of "How should teaching be evaluated?"

When one focuses on the latter question, then it is fairly easy to work to the conclusion that any improvement in the present system requires two more fundamental adjustments. The first is the need to examine multiple dimensions of teaching, something more than just what the teacher does in the classroom. The second is the need for multiple sources of information, something more than an exclusive reliance on student evaluations of their teachers.

This chapter describes the reasons for believing that these two principles are fundamental to effective evaluation. It also presents some guidelines for academic units that wish to establish evaluation procedures incorporating multiple dimensions of teaching and multiple sources of information.

The Nature of Teaching

Before evaluating teaching, one must develop a clear concept of that which is to be evaluated. For purposes of evaluation, teaching can be defined as helping someone else learn something. To advance this one step further, good teaching can be defined as being effective in the process of helping someone else learn something significant. The two added elements of effectiveness and significance both seem necessary to warrant the label of "good teaching."

The act of teaching can also be viewed as an interactive process that involves a teacher and students. This interaction occurs within a context or environment that can influence the success of that interaction. This definition of good teaching and the interactive character of teaching have a number of implications for evaluation.

$The primary purpose of teaching is to generate as much significant learning as possible. Students and teachers may bring additional purposes to the classroom; but, for evaluative purposes , the main concern is the amount of significant learning generated.

$The teacher is an important but indirect factor in the process of learning. This is simply a recognition of the fact that it is the student who does the learning; the teacher's role is to help the student in whatever ways possible.

$In higher education, the teacher has primary responsibility for key decisions about a course. These decisions include such things as determining the scope of a course, identifying the educational goals, selecting reading materials, constructing tests, and assigning grades.

$The quality of the teachers' classroom behaviors also have a major effect on the students' reaction to the course on a day-to-day basis. This refers to characteristics such as the clarity of the teachers' explanations, the enthusiasm they show for the subject, the rapport they develop with students, and the degree to which they are organized and prepared for class on a regular basis.

$Teaching takes place within several kinds of contexts, all of which can have a significant influence on the quality of the teaching and the learning. Examples of such contexts include the following:

B Physical: the characteristics of a classroom and the time at which a course is scheduled.

B Social: the relationship between the teacher and the students is an interactive one; students can inspire or discourage the teacher (and vice versa).

B Institutional: the attitude and actions of the department and the larger institution; do these attitudes and actions encourage or discourage good teaching?

B Personal: The situation of the teacher's nonprofessional life; has there been an illness, divorce, or financial problems?

In summary, teaching can be viewed as an interactive process that takes place within several types of contexts for the purpose of generating as much significant learning as possible.

The Nature of Evaluation

The type of evaluation appropriate for use in higher education is four dimensional. It calls for an examination of the input, the process, the product, and the context of an event or action. When this general framework is applied to the specific situation of college courses, it results in the six items identified in Figure 1.

The first dimension of college teaching is the "input", and it consists of two factors B student characteristics and teacher characteristics. Individual students vary considerably in the knowledge, values, and beliefs they bring to the learning situation. In addition, the mix of student personalities in a particular class can also be a major factor in the success of a given course. Similarly teachers vary in their readiness to teach any given course. Sometimes the subject matter is a topic that has been of interest to a faculty member for many years. At another time a faculty member may have to teach a course for which she or he has limited background knowledge and limited motivation. Another important variable for teachers is the degree to which they have learned how to teach in different situations: lower division as well as graduate courses, large classes as well as small classes, or courses that require active learning procedures as well as lecture courses. Teachers who are up-to-date in their fields and have undertaken the required research and preparation for a class provides the input necessary for significant learning. It follows that any breakdown in either factor of this input component diminishes the learning process.

Figure 1

MULTIPLE DIMENSIONS OF TEACHING

General Dimensions Specific Aspects of College

of Evaluation Teaching to be Examined__

INPUT: Student Teacher

Characteristics Characteristics

------

PROCESS: Course Classroom

Decisions Behaviors

------

PRODUCT: Student

Learning

------

Multiple Contexts:

-Physical

CONTEXT: -Social

-Institutional

-Personal

The second dimension of evaluation, the "process", involves two separate activities in college teaching: course decisions and classroom behavior. When professors teach courses, they make decisions about the scope of the subject matter to be covered, the teaching strategy to be used, the grading system, and course policies. To make such decisions, teachers need to consider such factors as the nature of the curriculum and the characteristics of the students and then design the course accordingly. Also part of the "process" of teaching, but quite different in nature, is what professors do in the classroom. Once the basic course decisions have been made and professors step into the classroom, they must engage whatever communication and interaction skills they have to deliver lectures, lead discussions, ask questions, motivate students, and generate interest.

The third dimension is concerned with the "product." of the teaching, which is the amount and type of learning that occurs in a given course. In almost every course some students are going to learn something. In good courses a large percentage of the students learn a lot, and they learn things that are significant rather than trite. To draw an example from my own discipline of geography, I would find only limited value in students learning the capitals and products of all the countries of the world. I would feel much better if the majority of the students understood such things as how the human geography of places affects the physical geography and vice versa. Another important part of learning is the students' interest in further learning. Did the course promote such interest or not? If students learn all about the physical and human geography of Europe but also learn that the subject is boring, I have "won the battle but lost the war."

The fourth dimension is context. In college teaching, there are several contexts that affect the quality of a given case of teaching: physical, (e.g., the characteristics of the classroom), social (e.g., the nature of the students), institutional (e.g., the support given to teaching), and personal (e.g., other events in the life of the teacher).

What then are the questions that have to be answered in order to make confident and valid judgements about the quality of teaching? The five general questions and related sub-points shown below are applicable to all classroom teaching in a higher education setting. The manner in which answers are found to these questions will vary from department to department and from college to college, but the questions themselves are inherent in the nature of teaching and in the nature of evaluation. These important questions then are as follows.

$Does the teacher have adequate and up-to-date knowledge of the subject matter, including academic and/or practical experience and efforts to improve?

$How good were the teacher's decisions about the course, inclusive of goals, teaching strategies, reading/laboratory/homework assignments , testing, and grading?

$How well did the teacher's classroom behavior promote good learning, inclusive of organization and clarity, enthusiasm, interaction with the class as a whole, and relationships with individual students?

$How good were the educational results of the course, inclusive of the amount of learning, the significance of what was learned, and attitude towards learning more about the subject?

$How much was the quality of the teaching and learning influenced by contextual factors, inclusive of physical context, social context, institutional context, and personal context?

Evaluating the Quality of Teaching

The quality of teaching, therefore, can be conceptualized as consisting of six components: student characteristics, teacher characteristics, the teacher's course decisions, the teacher's classroom behavior, the amount of significant learning, and the influence of contextual factors. In order to effectively evaluate any particular instance of teaching, one must engage in the task of collecting and analyzing information about each one of these components.

No single source of information, however, is adequate for assessing all six components of teaching. This means that multiple sources of information are not only advisable but are in fact necessary. Therefore different information sources need to be assessed to determine their relative value for answering questions about each of the six components. A number of different sources of information are available for this particular task, and these are:

$course materials,

$students (present students, seniors, and alumni),

$the teacher's own comments.

$peers (i.e., other faculty members),

$administrators,

$and observations of an instructional consultant.

One further distinction has to be made concerning the three types of evaluation situations common in academic settings: annual personnel decisions, periodic personnel decisions (e.g., tenure, promotion, teaching award), and diagnostic self-improvement. These three situations have some degree of similarity, but the differences are sufficient to warrant separate consideration for purposes of evaluation. The primary difference among the three situations lies in the nature of the basic question being asked.

When teaching is being assessed as part of an annual performance review, the question is: "How well did this person teach this year, compared to others in this academic unit?" This means that the only relevant information is the information pertaining to the teacher's performance this year. Hence, information from seniors or former students is inherently irrelevant. In the second situation, periodic personnel decisions, a different question is being posed. "Does this person generally teach well enough to be worthy of tenure, promotion, or a teaching award?" In this case, information from former students is not only relevant but essential. Finally, in the situation of diagnostic self-evaluation, the question becomes "what aspects of my teaching can most productively be improved?" At this time, all sources of information are relevant to some extent. The major difference is that an instructional consultant is ready to be an important source of information here, something most consultants do not want to be in administrative evaluation situations.

Before proceeding further, mention should be made of one special evaluation procedure that has gained widespread national and international interest in recent years: teaching portfolios. Seldin (1991, 1997) has provided the leadership in showing the many uses of teaching portfolios and describing how to create one. Basically a teaching portfolio consists of a brief narrative (e.g., 6-9 pp.) describing and assessing one's own teaching, plus an appendix containing various supporting documents. Although teaching portfolios are especially valuable for self-evaluation, faculty members can in fact use them in any of the three evaluation situations described above.

Recommended Procedures

If one accepts the two principles already explained B multiple dimensions of teaching and multiple sources of information B and also the difference in the three evaluation situations, then two principles of good evaluation become evident.

$Use different sources of information to assess different aspects of teaching.

$Use different sources in different evaluation situations.

Figure 2 summarizes the sources of information recommended for each of the six dimensions of teaching in each of the three evaluation situations. In the following three sections of this essay I will comment further on the recommended sources of information in each of the three situations.

Figure 2

Recommended Sources of Information for Evaluating Teaching

S T U D E N T S:

Factors Affecting the Quality
of Teaching: / Course
Mat’ls / Current Semester / Seniors / Alumni / Teacher’s
Comments / Peers / Admin. / Instruc.
Consult.

Student

Characteristics / A / P / A P D / a p / a d / d
2. Teacher’s
Knowledge / a / A P D
3. Course
Decisions / A D / a d / p / P / a P D / a d / a / D
4. Classroom
Behavior / A D / p / P / a p D / a p d / a p / D
5. Learning / A / p / P / a P D / a p d / a p / D
6. Context / A D / A P D / a d / a / D

Evaluating for Self-Improvement

All faculty members, as professionals, should be interested in knowing what they can do to improve their teaching. In recent times, however, another factor has increased the importance of this activity. As institutions hold faculty members responsible for ever greater levels of performance and accountability, the institutions acquire a parallel obligation to provide resources and information for faculty who are ready to improve their professional performance.

In the area of teaching, the college or university can probably contribute most by supporting a faculty and instructional development program. The academic units, through the office of the chairperson, need to inform their faculty of the availability of support services and to encourage their use.

Faculty members themselves need to use whatever resources are available to understand their teaching better and to improve it. Possible resources include present students, peers, administrators, and instructional consultants. Of these, the consultant if available can be a very important resource by providing informed, personalized feedback as well as general in-formation about teaching and learning. The instructional consultation process is explained further in the ensuing chapters of this section.

The utilization of an instructional consultant is the primary difference between evaluation for self-improvement and the other two types, both of which relate to personnel decisions. Most consultants work very hard to separate themselves clearly and completely from becoming involved in any evaluation connected with administrative decision making. They believe that any such involvement would interfere with faculty readiness to contact them for diagnostic evaluation intended for self-improvement, which is the consultant's primary raison d'etre. While administrators may and can recommend that particular faculty members with teaching problems should consider using the services of an instructional consultant, it is a generally accepted principle of consultation that the consultant cannot feed privileged information back to administrators. Such information is only for the faculty member.