Developing Portfolio Assessment to Replace Comprehensive Exams

PROPOSAL

Developing Portfolio Assessment to Replace Comprehensive Exams

Deborah Bainer Jenkins

Marie Holbein

State University of West Georgia

Carrollton, GA 30118

This paper recounts the process of developing an innovative portfolio assessment to replace comprehensive exams in a doctoral program. It examines the barriers to the process which were encountered, and if they were overcome or persisted as barriers to change.

PURPOSE:

This paper recounts the process of developing an innovative portfolio assessment strategy to replace comprehensive exams in a doctoral program. Further, it explores the barriers encountered during the process, how these barriers were overcome, and which persisted and blocked effective change.

PERSPECTIVE:

The process of educational change is not easy. People resist change. Common barriers to educational change in elementary and secondary schools are time, organizational structure, and school culture (Tye, 1993).

An innovative doctoral program in School Improvement was developed by a southern regional university. It aims to prepare graduates to overcome these barriers and to act as change agents in their schools and districts. Because of the nature of the knowledge, skills, and dispositions graduates are expected to develop throughout the program, assessment using a traditional comprehensive exam seemed inappropriate.

A subcommittee was appointed to explore portfolio assessment as an evaluation alternative. The National Board for Professional Teacher Standards (NBPTS) use of portfolio assessment followed by an applied writing experience provided a rigorous, valid assessment model. Based on the six program Core Competencies, five integrated portfolio entries were developed. Detailed instructions, examples, and scoring rubrics and worksheets were also developed. The subcommittee was assisted by evaluation experts at the National Board, teachers currently going through the portfolio experience, and teachers who had successfully attained National Board certification.

METHOD:

The process of developing and presenting an innovative evaluation method met with some resistance from the faculty committee and graduate student representatives. Because the final adoption is still being discussed by the committee, we continue to collect data about the process of change.

Data are being collected following the “insider’s approach” (Cantrell, 1993). Data which will be submitted to the final content analysis includes the following:

a)committee agendas and minutes,

b)notes taken by two researchers during committee discussions to capture specific quotes and general moods and topics of discussion,

c)notes from debriefing sessions following the committee meetings involving the two researchers,

d)notes from conversations with the committee chair, many which provide background information about the early development of the program and attitudes/positions of committee members,

e)notes from conversations with the former dean who spear headed the development of the program, and

f)notes from conversations with individual committee members to explore their reasons for support or nonsupport of the portfolio, their background in assessment, and their assumptions and concerns.

(Note that due to the collegial nature of the committee, tape recording is not a data collection option.)

RESULTS:

Although data collection will not be completed until May 2001, preliminary data suggest some interesting challenges to the change process.

1)Committee members bring to the meetings very different ideas about what a portfolio is and what it can do. Most perceptions regard portfolios as “scrapbooks” rather than assessment tools.

2)One objection to developing a portfolio assessment strategy was time: nobody has time to develop it, nobody has time to be trained to score it, nobody has time to score it, how will this count toward a faculty member’s promotion and tenure.

3)Organizational structure, more specifically the structure of graduate programs from which committee members graduated, also serves as a barrier. Committee members tend to hearken back to their own graduate experience to justify or diminish discussions of assessment.

4)University culture also provides a barrier. Four full professors were hired to work with the new program at salaries higher than regular faculty members. These professors brought with them a vision for a new graduate program. The more enthusiastic these “outsiders” are, the more entrenched the perspectives of the rest of the committee members become. Discussion of developing an innovative program is often met with references back 1-2 years to thinking when the program was originally conceived.

IMPLICATIONS:

When the data is all collected, analyzed, and triangulated, it is expected that many observations about how change takes place in higher education will be evident. An investigation of the barriers and resistance to change by a College of Education which purports to prepare change agents for schools should hold many implications for higher education, elementary and secondary education, and tour understanding of the change process itself.