Summary Report:

BIOL 3201 Writing Assessment Project

This report summarizes the findings of a comparative study of BIOL 3201 term project papers written before and after the introduction of targeted writing support into the course curriculum. The study was conducted by the UH Writing Center at the request of the College of Natural Sciences and Mathematics. As shown below, papers written subsequent to the curriculum changes were found to exhibit better overall writing quality as well as significantly better abstract, results, and discussion sections.

Project Description

The papers rated for the study were a formal write-up of a genetics laboratory term project in which students applied scientific experimentation and analysis to identify specific mutations in Drosophila melanogaster (fruit fly). Guidelines for the papers were provided in the students’ laboratory manual and specified that the paper should include abstract, introduction, materials and methods, discussion, and results sections. During the spring and fall semesters of 2005, a rubric specifying writing objectives was introduced, students were given a workshop on writing scientific reports, and course teaching assistants were provided with a workshop on teaching and grading the papers.

This study compared a random sampling of papers written prior to the Spring 2005 semester with a random sampling of papers written in the Fall 2005 semester. Originally 40 papers from each group were selected. One paper from the Fall 2005 semester was subsequently excluded from the study when it was discovered that some of the original pages were missing. Papers from both sampling periods were photocopied and any identifying marks such as names, dates, and grades were blacked out.

Papers were rated by Writing Center staff members, all of whom were working on or had completed graduate degrees and who had some familiarity with scientific writing. The raters were given a rubric developed by the Writing Center in consultation with faculty from the Department of Biology (see attached) and trained using sample papers for each of the levels in the rubric. The rubric asked raters to assign separate level scores for the abstract, introduction, materials and methods, discussion, and results section as well as for overall writing quality (4=fully and completely meets all objectives . . . 1= serious deficiencies with respect to objectives). Each paper was read by two raters. The final score for each section was the average of the two raters’ scores (e.g., Rater 1=3, Rater 2=4, Final Score=3.5). In cases where the raters disagreed by more than one point, the paper was read by a third rater, and the two scores that were in closest agreement were used to compute the average.

Results

Table 1 provides mean scores for each of the sections as well as the overall writing quality. As indicated, prior to the curriculum changes the mean score for each area – except materials and methods – was below two (2=addresses some objectives but not all), with the Results section producing the lowest mean. Subsequent to instruction the results section showed improvement but was still below two. All other scores were above two, indicating on average at least partial mastery of the objectives for the assignment.

Table 1: Descriptive Statistics

Sample / Mean / Std. Deviation / N
Abstract / Pre / 1.96 / .80 / 40
Post / 2.55 / .84 / 39
Introduction / Pre / 1.96 / .82 / 40
Post / 2.21 / .78 / 39
Materials and Methods / Pre / 2.33 / .81 / 40
Post / 2.36 / .74 / 39
Results / Pre / 1.46 / .52 / 40
Post / 1.77 / .74 / 39
Discussion / Pre / 1.74 / .53 / 40
Post / 2.19 / .67 / 39
Overall / Pre / 1.88 / .56 / 40
Post / 2.32 / .62 / 39

A multivariate analysis of variance procedure indicated that the effect of the sampling period on the writing scores was significant (F=3.60, p=.004, η2 = .231), suggesting that the changes in the curriculum benefited the students.

Table 2 shows the results of individual analyses of variance run to test the effect on the individual sections. As indicated, the improvement shown in the ratings for the abstracts, results, discussions, and overall writing quality were significant at p<.05 or greater.

Table 2: Individual ANOVAs for the effect of the sample period on each score

Dependent Variable / F / Sig. / η2
Abstract / 10.22 / .00 / .12
Introduction / 1.81 / .18 / .02
Materials and Methods / .04 / .85 / .00
Results / 4.52 / .04 / .06
Discussion / 11.31 / .00 / .13
Overall / 11.13 / .00 / .13

These findings suggest that the joint efforts of the Writing Center and the Biology department to target student writing in BIO 3201 during 2005 benefited students in that they are better able to meet the expectations for the term project paper. Analysis of the individual means for fall 2005, however, indicates that there is room for continued progress in each of the areas, which further collaboration will more than likely facilitate.

BIOL 3201 Assessment - 2