Integrating Alternative Assessment in a Project-Based Learning Course for Pre-Service Science

Integrating Alternative Assessment in a Project-Based Learning Course for Pre-service Science and Technology Teachers.

MOTI FRANK, Technion – Israel Institute of Technology

ABIGAIL BARZILAI, Technion – Israel Institute of Technology

Paper presented at the Learning Communities and Assessment Cultures Conference organised by the EARLI Special Interest Group on Assessment and Evaluation, University of Northumbria, 28-30 August 2002

ABSTRACT This paper deals with a required methods course, based on the national curriculum of science and technology for junior high schools. The course participants are pre-service teachers who study towards a B.Sc. degree in Education in Science and Technology parallel to their studies in one of the faculties of Sciences or Engineering. Working in small teams the students carry out a project in a Project-Based Learning (PBL) environment. The final outcomes of the project are group and individual written reports, a portfolio, a multimedia presentation and a physical model. The research question was: what implementation issues and processes do higher education students encounter in a Project-Based Learning environment which involves an alternative assessment approach? Qualitative and quantitative tools for collecting data included ‘the participant as observer’ observations at the classroom, semi-structured interviews with students, questionnaire, and analysis of students’ reports and products. The findings relate to the benefits of PBL from the students’ point of view, the challenges students experienced and perceived while conducting their projects, the benefits students may gain from formative assessment, and the difficulties located by the course teachers throughout the assessment meetings.

Introduction

A required methods course, based on the national curriculum of science and technology in junior high schools, has been developed in the Department of Education in Technology and Science. The course participants are pre-service teachers who study towards a B.Sc. degree in Education in Science and Technology parallel to their studies in one of the faculties of Sciences or Engineering. The learning environment of the course is based on a constructivist approach, and designed to enable the students to construct their knowledge through active learning, and through interaction with staff and colleagues. The teachers act as facilitators and mentors. The assessment in the course is also derived from the constructivist approach and is based on formative assessment of students’ performance in microteaching, active learning in the National Museum of Science and Technology, group assignments, and interdisciplinary team projects.

This paper focuses on an approach that is implemented to assess the students while carrying out the project in a Project-Based Learning (PBL) environment. Students are required to design a system based on scientific, technological, social and environmental principles. To emphasize technological, and not merely scientific literacy, the unique quality of the project is that the starting point is the technological requirements and needs rather than a research question. The students first define what would be required of the technological system, including the tasks to be performed, and determination of the system outcomes. They investigate alternatives for implementation, collect and analyze data through a process of investigation and collaboration, and conduct a feasibility study, after which they designed the system, using a top-down approach.

During the course students learn how to use project-based learning in their classrooms. The course covers the theoretical background of PBL and some field research examples which describe the use of PBL as well as the strategy for conducting a student-project: where to begin, how to coach students through a project, and how to assess student products.

The final outcomes of the project are group and individual written reports, a portfolio, a multimedia presentation in classroom in front of the course colleagues and staff and a physical artifact, which can assist the secondary school teacher while demonstrating a scientific and/or technological principle underlying the system.

We believe that alternative assessment should take into consideration intangible parts of the project and not emphasize the final products or presentations only. The real learning is often in the doing or in the process leading up to the product. Students presenting a product that fails to work may still have learned the central principles, especially if they can describe why their product does not work and what is needed to correct it (Buck, 1999). Therefore, in our course, the project grade is based on the assessment of tangible (about one third out of the project grade) and intangible outcomes.

Following are examples of projects performed by students: a car driven by solar energy, a water desalination system, a remote cardiologic testing system, an automated watering system, a hot air balloon system, and an automated purification system for aquarium water.

Project-Based Learning

What is the project-based learning approach? According to Krajcik, Czerniak and Berger (1999), this approach engages learners in exploring important and meaningful questions through a process of investigation and collaboration. Students ask questions, make predictions, design investigations, collect and analyze data, use technology, make products, and share ideas. According to Buck Institute of Education (1999), PBL is an innovative model for teaching and learning. It focuses on the central concepts and principles of a discipline, involves students in problem-solving investigations and other meaningful tasks, allows students to work autonomously to construct their own knowledge, and culminates in realistic products.

Thomas (2000) also tries to define this approach, and emphasizes that in the PBL environment students are, in fact, investigating solutions to a problem. They build their own knowledge by active learning, interacting with the environment as suggested by the constructivist approach, working independently or collaborating in teams, while the teacher directs and guides, and they make a real product.

We shall now discuss the advantages of the PBL approach for teaching and/or learning, firstly from the students’ point of view, and then as concerns the instructor. Green (1998) notes that learning by means of a project is likely to increase motivation and give the students a sense of satisfaction. It is also helps to develop long-term learning skills. Krajcik, Czerniak and Berger (1999) suggest four benefits for the student. Firstly, learners develop deep, integrated understanding of content and process. Secondly, students learn to work together to solve problems. Collaboration involves sharing ideas to find answers to questions. In order to succeed in the real world, students need to know how to work with people from different backgrounds. Thirdly, this approach promotes responsibility and independent learning. As a final benefit, this approach actively engages students in various types of tasks, thereby meeting the learning needs of many different students.

Orevi and Danon (1999) also list the advantages of PBL from the students’ point of view - it develops collecting and presenting data skills, develops thinking skills, suits personal learning styles, enhances motivation, and develops independent learners.

So far we have discussed the advantages for the student. Krajcik, Czerniak and Berger (1999) also suggest three possible advantages for the teacher. Firstly, the teacher may find the work enjoyable, interesting and motivating, since teaching will vary every year as he/she will be exploring new projects with each new group of students. Secondly, in project-based teaching, the teacher continually receives new ideas, thus becoming a ‘lifelong learner’. Thirdly, classroom management is simplified because when students are interested and involved, they are likely to cause fewer disciplinary problems.

What has research discovered about the usefulness of the PBL approach? In a research described by Shepherd (1998), it was found that students who applied the PBL approach statistically received significantly higher grades on the Critical Thinking Test than did those in the comparative group who had studied in the traditional manner. The PBL students also demonstrated greater self-confidence and improved learning abilities.

Interestingly, Rosenfeld and Rosenfeld (1999) found that students with low academic records who studied in the conventional framework did better in courses based on PBL, whereas those with higher grades in regular studies had lower achievement when PBL methods were applied. Based on their findings, the researchers suggest that teaching styles and learning environments be adapted to the student’s learning mode. Low academic grades do not necessarily demonstrate lack of ability, but rather the unsuitability of the pedagogic system. They advise that students be exposed to PBL in order to give those who fail a chance of doing better, and to encourage those with high academic achievements in subjects taught traditionally to develop additional expertise.

The teaching role in PBL is changed. The teacher is no longer the expert lecturer, facts provider, and director of instruction but rather a resource provider, learning environment shaper, how-to-learn teacher, advisor, tutor and colleague (Buck, 1999).

Teaching by means of PBL presents several challenges. These include: teachers’ content knowledge, students’ lack of experience in this new approach and their preference for traditional-structured approach; their preference for learning environment which require less effort on their part; and problems arising from time stress. Students struggling with ambiguity, complexity, and unpredictability and are liable to sense frustration in an environment of uncertainty, where they have no notion of how to begin or in which manner to proceed.

The system is rather time-consuming and requires that the teacher make a lot of efforts over a long period of time. Certain teachers may feel that they don’t have sufficient time to meet the curriculum guidelines. Class management, in which the students have the freedom to talk together, is often more difficult.

Teachers often feel a need to direct lessons to insure that students get the “right” information. Teachers frequently give students too much independence without structuring the situation, or providing feedback. Sometimes, teachers have difficulty incorporating technology into the classroom. Other problems are discussed by Krajcik, Czerniak and Berger (1999, pp. 322-328) and by Buck Institute of Education (1999, Potential Problems section).

To sum up, we will present the main advantages of the PBL method (PBL that based on identifying needs and designing system to meet that needs rather than PBL that based on answering research question). Students are engaged in active learning and gain interdisciplinary knowledge while working in a real-world context. Students develop deep and integrated understanding of content and process and focus in depth on central ideas and salient issues. Since students deal with relevant issues, their motivation is increased. PBL promotes responsibility and independent learning, offers multiple ways for students to participate and to demonstrate their knowledge, and can be matched to the various learning styles of students.

Through PBL the students are familiarized with the synthesis processes (not just the analysis processes), experiment in teamwork, become familiar with the design procedures and the underlying principles of systems engineering. The students learn to apply the important engineering principle - how to design to an optimum (not necessarily to the maximum).

The learning environment promotes the development of inquiry skills, problem solving skills, and information skills. Students develop a capacity for systems thinking, acquire lab experience, and gain a higher level of cognitive skills (such as asking questions) and affective outcomes (such as curiosity and skepticism). Finally, the awareness of students to Scientific-Technological-Environmental-Social aspects is increased, academic achievement may be improved and students learn to simulate the professional work of the scholar, researcher, engineer, reporter, planner, manager, and other practitioners.

Alternative Assessment

The traditional assessment in most courses of the higher education is mainly based on paper and pencil tests. The grade in a given subject is mainly based on a final test. Usually these tests measure how well the student can repeat the declarative knowledge transferred from the teacher to the passive student. In our course the traditional assessment methods were found less appropriate for measuring the kind of understanding the students acquired in PBL.

According to Krajcik, Czerniak and Berger (1999, p. 207), traditional questions would fail to assess the multimode of ideas students learned. They would not measure students’ ability to work as a team. They would not show how students could apply their knowledge and skills to everyday life. They would not show that they could design and plan investigations.

Therefore we decided to apply an alternative assessment in our course. Assessment of students’ performance in the project was based on meetings and discussions between the course staff and each team during the semester, observations of the students’ work in classroom, group report and portfolio, personal reflective reports, and an exhibition at the end of the course - posters, multimedia presentations, and the artifacts. A rubric was also developed and validated for this assessment.

The following table presents the characteristics of alternative assessment (Buck, 1999; Krajcik, Czerniak and Berger, 1999) and the assessment strategies implemented by us in the course:

TABLE 1. The characteristics of alternative assessment, and the strategy of the assessment implemented in the course.

In light of the above characteristics, Krajcik, Czerniak and Berger (1999) claim that alternative assessment is more valid and reliable, more closely matches today’s educational goals, better accommodates cultural diversity, is consistent with cognitive learning theory, and measures deep understanding.

Method

Research Design

This paper deals with a required methods course, based on the national curriculum of science and technology for junior high schools. The research question was: what implementation issues and processes do higher education students encounter in a PBL environment which involves an alternative assessment approach? The purpose of the current study was to learn about ideas (cognitive aspects), emotions (affective aspects), difficulties, and behavior (behavioral aspect) of the students in the above course.

The study is based, mainly, on qualitative data analysis. The merit of a qualitative study lies in its internal validity, not in its generalization. Therefore, we do not recommend making generalizations on the basis of the case study described here. However, the findings and implementation tips presented in this paper may serve as a basis for a follow up empirical/quantitative study based on a random sample and inferential data analysis.

The strategies adopted in the study served to reinforce reliability, inner validity, and external validity. This was carried out by cross-referencing sources and triangulation. Denzin (1988) outlines four levels of triangulation. The first is data triangulation that includes time, space and persons. The second level is investigator triangulation - using several observers. The third is methodological triangulation - using more than one research method, and the fourth level is theory triangulation - creating a broad theoretical framework.