Teacher Change in High School Science: Findings from the First Three Years of the Vertically Integrated Partnerships (VIP) K-16
September 2005
Kimberley Raue, Westat
Joy Frechtling, Westat
Xiaodong Zhang, Westat
Gary Hedges, Montgomery County Public Schools, MD
Introduction
In 2002, the University System of Maryland (USM), in partnership with Montgomery County Public Schools (MCPS), Montgomery College, and seven USM institutions (Towson University, UM Baltimore County, UM College Park, UM Biotechnology Institute, UM Center for Environmental Science, Maryland Sea Grant College, and Universities at Shady Grove), developed the Vertically Integrated Partnerships (VIP) K-16, a sustainable professional development model aimed at high school science teachers, college faculty, and pre-service science teaching candidates. Through this professional development model, VIP aims to achieve the following five goals: 1) improve MCPS science student learning outcomes, 2) improve MCPS teacher content knowledge, which includes both concepts and process skills, 3) improve college faculty teaching skills, 4) enhance graduate student engagement in teaching, and 5) increase undergraduate student retention in science.
Evaluation duties for this project are shared by an internal evaluation staff of MCPS, the Office of Shared Accountability (OSA) and the external evaluator, Westat. OSA is responsible for providing data about students and supporting the collection of data from teachers. Westat is responsible for general evaluation development, coordination, analysis, and reporting, including data provided by OSA.
External evaluation activities have focused primarily on project goals 1 and 2, which address the activities directed at MCPS high school science teachers and students. To date, Westat has had only a small role in the evaluation of goals 3–5, which concern the higher education partner. Westat has functioned more as a coach and gadfly than as an external evaluator for the institution of higher education (IHE), partly because a coherent set of goal-related activities has been slower to emerge at the IHE level and partly because faculty have wished to be engaged in their own data collection. Increased interest in external evaluation at the IHE level has been expressed, and Westat has begun to engage more fully in addressing critical questions regarding the IHE.
As a result, this paper reports on findings from the first three years of implementation within MCPS, specifically changes in teachers’ instructional practices and attitudes, as well factors that have facilitated or hindered teacher change. Additionally, VIP has strongly promoted science as inquiry, the ability of students to use “scientific reasoning and critical thinking to develop their understanding of science,” which is a reflection of national standards (National Science Teachers Association, 2004; National Research Council, 1996; American Association for the Advancement of Science, 1993). An emphasis on inquiry-based instruction has been infused throughout VIP-supported activities, and given its importance within VIP, as well as in standards-based educational reforms more generally, particular attention is paid to its implementation and differing conceptions of inquiry-based teaching within the VIP community.
Data are drawn primarily from two sources: 1) pre- and post-program surveys of VIP’s Biology Cohort and 2) classroom observations of a sample of VIP’s Matter and Energy/Earth Space Systems Cohort. Data collected through other means, such as teacher evaluations of VIP-sponsored conferences and summer institutes, a focus group of Biology Cohort teachers, and a survey of Master Science Teachers (MSTs) reinforce findings from the survey and observations. The discussion of inquiry and factors influencing teacher change has broad applicability to other projects embarking on educational reform.
Survey of Teachers’ Instructional Practices and Attitudes, Biology Cohort
Instrument
Westat adapted a survey previously developed for its Evaluation of Scientific Work Experiences for Teachers (SWEPT), a study funded by the National Science Foundation (NSF) under a grant to Columbia University. The surveys were designed to provide a wide range of information about teachers and their practices. Although primarily intended to inform goal 2 of the VIP—use of inquiry-based practices and high-quality instructional activities—the survey also addresses other areas of teacher practices and behavior. Specifically,
· Practices, including
- how the material is taught,
- how learning is assessed,
- time and resources used for lesson preparation, and
- the kinds of behaviors and activities encouraged in students.
· Other areas, such as
- teachers’ confidence in their teaching skills,
- how they see themselves as teachers, and
- their assessment of the textbooks they used.
· Professional development, including
- the areas addressed, and
- the areas in which skills have been improved.
Methodology
This paper presents descriptive analysis of the data obtained from the survey of the Biology Cohort. Later analyses will examine the findings for this cohort in greater depth and, starting next year, look at relationships between the survey responses and student outcome data.[1]
Successive cohorts of MCPS teachers have been phased into VIP beginning with MSTs in 2002, Biology teachers in 2003, Matter and Energy and Earth Space Systems teachers in 2004, and Chemistry and Physics teachers in 2005. In December of 2003, a pre-program survey was administered to all high school science teachers expected to participate in VIP. Over 400 surveys were printed and sent to science resource teachers in each high school. A memorandum explaining the process for completing and submitting the surveys was included, along with timesheets for each teacher. Science teachers completed the survey on their own time and signed and dated their timesheets. The surveys were returned to the MCPS Office of Shared Accountability (OSA); the timesheets, to the NSF project manager for processing. Each teacher was compensated the MCPS standard rate of $20 for the one hour required to complete the survey. The MCPS OSA compiled the data and sent the files to Westat for analysis. The overall response rate was 90 percent.
The post-program survey was administered to biology teachers in April 2005, as they completed their second year of participation in VIP. The post-survey was sent via MCPS internal email, and respondents received two reminders to complete them. The final response of 113 teachers represents 66 percent of the MCPS teachers currently teaching biology in the high schools. Item response rates are uniformly high, with each question answered by more than 95 percent of the respondents. MCPS was responsible for modifying the pre-program survey and collecting, cleaning, and preparing the survey data. Westat did the analysis.
Changes in teacher responses from pre- to post-program were examined in two ways. First, Westat looked at changes for all teachers who responded (131 to the pre-test; 113 to the post-test) at each of the two data collection points (cross-sectional samples); second, Westat looked at changes for a subsample of 72 teachers who responded to both the pre-program and post-program surveys (longitudinal sample). Table 1 shows selected characteristics of these samples. Generally, they are quite similar; however, there is a higher percentage of biology teachers in the 2004–05 cohort holding a Maryland teaching certificate (93.7), and especially in biology (88.5), than in the 2003–04 cohort (87.7 and 76.3, respectively). A larger percentage of teachers hold an advanced certificate in 2004–05 (63.0 percent) than the previous year (54.3 percent). Both samples show a large percentage of teachers having a master’s degree (over 70 percent). More than 70 percent of the biology teachers are female, and over 80 percent are white.
Table 1.—MCPS biology teacher characteristics
Pre / Post / sample
III. Your background and experience
Q17. Including this school year, how many years have you been employed as a science teacher?
In total / 9.7 / 11.0 / 11.3
In high school / 8.4 / 8.7 / 8.9
In MCPS / 8.1 / 8.9 / 9.6
At this school / 6.1 / 5.9 / 6.3
Q18. Do you hold a Maryland Teaching Certificate? / 87.7% / 93.7% / 95.7%
Q19. Please circle in the fields in which you hold a teaching certification and/or endorsement.
Biology / 76.3 / 88.5 / 88.9
General science / 33.6 / 33.6 / 31.9
Chemistry / 7.6 / 7.1 / 5.6
Physics / 0.0 / 0.9 / 1.4
Earth science / 4.6 / 2.7 / 1.4
Special education / 11.5 / 7.1 / 6.9
Other / 5.3 / 3.5 / 2.8
Q20. What type of teaching certificate do you hold?
Regular or standard / 41.4 / 29.6 / 33.8
Advanced / 54.3 / 63.0 / 63.2
Provisional / 3.4 / 4.6 / 1.5
Probationary / 0.0 / 0.9 / 1.5
Temporary / 0.0 / 0.9 / 0.0
Q23. Circle your highest postsecondary degree
Doctorate / 0.0 / 2.7 / 1.4
Master’s / 72.3 / 73.0 / 72.2
Bachelor’s / 27.7 / 24.3 / 26.4
Q24. Are you currently working toward an advanced degree?
Yes / 22.1 / 22.4 / 20.0
Q27. What is your gender?
Male / 29.2 / 25.9 / 29.6
Female / 70.8 / 74.1 / 70.4
Q28. Which best describes you?
American Indian / 0.8 / 0.9 / 1.4
Asian American / 3.1 / 2.7 / 4.2
African American / 11.6 / 11.6 / 11.3
Hispanic / 3.1 / 0.9 / 1.4
White / 79.8 / 81.3 / 80.3
Other / 1.6 / 2.7 / 1.4
NOTE: Percentages may not add to 100 because of rounding or because respondents could chose more than one answer to some questions.
Most of the post-program survey items are identical to the pre-program survey. For these items, Westat also conducted t-tests to examine the change over time and its statistical significance, results of which are represented in Table 2.[2]
Table 2.–Pre-post survey comparison of VIP biology teachers’ survey responses
Survey item / Cohort 2004–05 mean (N=113) / Cohort 2003–04 mean (N=131) / P value / Post-survey mean (N=72) / Pre-survey mean (N=72) / P valueQ2. In this course, how much emphasis did you give to each of the following goals or objectives? (scale of 1-4)
a. Integrating the course curriculum with other subjects / 2.37 / 2.29 / .41 / 2.39 / 2.28 / .17
b. Teaching facts, rules, or vocabulary / 3.23 / 3.29 / .50 / 3.29 / 3.42 / .12
c. Showing the importance of the subject in everyday life / 3.43 / 3.50 / .42 / 3.44 / 3.53 / .32
d. Increasing students’ interest in taking advanced courses / 3.05 / 2.80 / .02* / 3.06 / 2.82 / .04*
e. Encouraging students to explore alternative explanations or methods for solving problems / 2.86 / 2.99 / .14 / 3.27 / 2.99 / .52
f. Using the 5E’s instructional format / 3.09 / 3.10 / .94 / 3.07 / 3.03 / .66
g. Teaching the essential indicators in the curriculum as prescribed by MCPS / 3.83 / 3.86 / .55 / 3.85 / 3.86 / .81
h. Providing an in-depth study of selected topics or issues, as opposed to exposure to a broad range of topics / 2.71 / 2.56 / .14 / 2.69 / 2.63 / .49
i. Understanding the theoretical concepts and ideas underlying scientific or mathematical applications / 2.96 / 3.06 / .32 / 3.03 / 3.00 / .81
j. Using components of the Well-Designed Investigation / 3.24 / 3.34 / .32 / 3.18 / 3.24 / .58
k. Providing opportunities for students to ask and answer their own testable questions. / 2.78 / 2.83 / .58 / 2.69 / 2.74 / .68
l. Utilizing pre-, during, and post-reading strategies / 2.82 / 2.74 / .48 / 2.86 / 2.63 / .02*
m. Providing opportunities for writing (hypothesis, observations, procedures, communicating results, etc.) / 3.29 / 3.37 / .43 / 3.33 / 3.28 / .44
Q3. How often did you use each of the following teaching methods in this course? (scale of 1-5)
a. Lecture or talk to the whole class / 3.39 / 3.50 / .33 / 3.44 / 3.53 / .38
b. Teacher-led whole class discussions / 3.45 / 3.51 / .54 / 3.47 / 3.47 / 1.00
c. Student responding orally to questions on subject matter covered in class or homework / 3.71 / 3.98 / .01** / 3.76 / 4.10 / .00**
d. Student-led whole-group discussions or presentations / 2.63 / 2.63 / .93 / 2.54 / 2.56 / .90
e. Students working individually on inquiry projects / 2.50 / 2.51 / .98 / 2.44 / 2.42 / .91
f. Students working together in cooperative groups / 3.77 / 3.75 / .82 / 3.75 / 3.79 / .59
g. Reviewing homework or other assignments / 3.51 / 3.74 / .03* / 3.42 / 3.82 / .00**
Table 2.—Pre-post survey comparison of VIP biology teachers’ survey responses (continued)
Q4. How often did you have students engage in the following learning activities in this course? (scale of 1-5)
a. Conduct inquiry activities (e.g., doing lab activities or using manipulatives) / 3.74 / 3.77 / .64 / 3.77 / 3.83 / .40
b. Reflect on course material by writing in a notebook or journal / 2.38 / 2.44 / .69 / 2.38 / 2.42 / .77
c. Write responses to BCR items / 3.59 / 3.45 / .08 / 3.58 / 3.39 / .02*
d. Respond to selected response items / 3.50 / 3.57 / .38 / 3.58 / 3.61 / .74
e. Use calculators or computers for learning, practicing skills, or solving problems. / 2.62 / 2.58 / .74 / 2.63 / 2.69 / .58
f. Work individually on written work or assignments in a workbook or textbook / 2.85 / 3.05 / .05* / 2.81 / 3.00 / .08
g. Critique/evaluate their own or other students’ class work or homework / 2.84 / 2.65 / .06 / 2.89 / 2.64 / .03*
h. Consider a real-world problem relevant to the course and develop a plan to address it / 2.85 / 2.69 / .22 / 2.87 / 2.69 / .11
i. Use technical passages (from news or science journals) to investigate current issues / 2.92 / 2.59 / .00** / 2.90 / 2.62 / .01**
j. Listen to guest speakers or go on field trips relevant to the curriculum / 1.67 / 1.48 / .04* / 1.58 / 1.46 / .13
k. Investigate possible career opportunities in mathematics, science, or technology / 2.22 / 2.08 / .21 / 2.19 / 2.07 / .28
l. Design and implement their own scientific investigation / 2.71 / 2.59 / .30 / 2.61 / 2.46 / .25
m. Use “state-of-the-art” equipment or technologies / 2.87 / 2.76 / .35 / 2.90 / 2.81 / .39
Q5. What portion of your planning and preparation time for this course did you spend on each of the following activities? (scale of 1-4)
a. Revising current lessons/curriculum units / 3.19 / 3.18 / .91 / 3.21 / 3.14 / .53
b. Creating new lessons/curriculum units / 2.99 / 3.05 / .57 / 2.86 / 3.06 / .08
c. Contacting community resources, including making arrangements for speakers, tours, etc / 1.51 / 1.43 / .37 / 1.44 / 1.43 / .87
d. Using the Internet to access materials. 22.3 / 2.96 / 2.85 / .26 / 2.93 / 2.82 / .31
e. Using the Internet to network with colleagues / 2.33 / 2.18 / .21 / 2.38 / 2.22 / .38
f. Consulting with USM/MC faculty of professional scientists/ mathematicians / 1.37 / 1.40 / .76 / 1.24 / 1.34 / .24
g. Using a reflective teaching journal / 1.60 / 1.51 / .37 / 1.54 / 1.51 / .70
h. Learning to use science equipment / 2.08 / 2.19 / .27 / 1.96 / 2.18 / .04*
i. Improving computer and/or software skills / 2.21 / 2.31 / .38 / 2.14 / 2.26 / .34
j. Writing grants to secure funding for programs and/or equipment / 1.35 / 1.13 / .00** / 1.34 / 1.13 / .02*
k. Interacting with other teachers at your school to coordinate lessons/activities / 3.29 / 3.31 / .91 / 3.25 / 3.28 / .74
l. Participating in a professional learning community / 3.04 / 2.97 / .48 / 3.03 / 2.87 / .13
m. Responding to e-mail you receive from students / 2.97 / 2.95 / .87 / 2.90 / 2.93 / .78
n. Conducting practitioner research / 2.08 / 1.85 / .03* / 2.07 / 1.76 / .01**
o. Interacting with teachers from other schools to coordinate lessons/activities / 1.82 / 1.82 / .98 / 1.80 / 1.80 / 1.00
Table 2.—Pre-post survey comparison of VIP biology teachers’ survey responses (continued)