1
Written Feedback and Scoring of Sixth-Grade Girls’ and Boys’ Narrative and Persuasive Writing
Abstract
This study examined the possible gender differences in teachers' scoring and written feedback on two narrative and two persuasive writing samples sent to 108 grade six teachers throughout one Canadian province. Participating teachers read a narrative and a persuasive piece of writing from one boy, and a narrative and persuasive piece written by one girl. The four papers were credited to a male author for some teachers and to a female author for others. The teachers evaluated the writing using the provincial scoring guides for narrative and persuasive writing. They also wrote comments and/or indicated needed edits and revisions on the piece of writing, providing the same kind of feedback to the student writers that they would provide to their own students. A two-way ANOVA was used to compare the scores by the teacher’s gender and the identified gender of the writer for each of the four papers.
There were significant differences between scores assigned to female and male writers on particular papers within specific scoring categories. Nevertheless, teachers’ ratings of the writing showed no consistent patterns privileging female or male writers. Additionally, female and male teachers’ scores were not significantly different for three of the four writing samples. The persuasive papers overall were scored higher than the narrative papers. With one exception, the highest scores within each of the four scoring categories were assigned to papers whose writers were identified as boys. Teachers scored two papers higher when the student writer’s perceived gender matched their own (same-sex appreciation). Their scoring demonstrated the opposite effect (same-sex depreciation) for the other two papers.
Teachers wrote extensive comments to the student writers. The greatest percentage of their comments to the narrative writers were editive and the greatest percentage of comments about the persuasive papers were revisional in nature. Girls received approximately the same quantity of comments as boys did by female and male teachers. Of the comments directed to male student authors, the greatest percentage were editive in nature. Female and male student authors received relatively equal numbers of verdictive (praising) and revisional comments.
Key Words: assessment, elementary, writing, gender, narrative, persuasive
In Canada, Great Britain and the United States, girls on overage outperform boys in writing on large-scale writing tests (Council of Ministers of Education, Canada, 1998; Education Quality and Accountability Office, 2000; Office of Educational Research and Improvement, 1995; Ohio Department of Education, 2000; Stobart, Elwood & Quinlan, 1992). In an effort to understand factors contributing to the gender disparity in large-scale writing assessments, this study examined possible gender differences in grade six teachers’ evaluation of students’ narrative and persuasive writing and in their evaluative feedback on the two types of writing. The design of the study and the examination of the results were guided by two key research questions:
(a) What are the possible influences of the teacher’s gender, the student writer’s gender, and the discourse mode (narrative or persuasive writing) on the teacher’s scoring of sixth-grade writing?
(b) What types of evaluative feedback do female and male teachers provide to girls and boys on their narrative and persuasive writing?
Review of the Literature
Previous research examined gender characteristics within elementary children’s narrative and persuasive writing and gender differences in the scoring of their writing. Research examining gender characteristics of student writing provided a framework for selecting the writing samples. Studies examining teachers’ gender perceptions and the influence of those perceptions on the evaluation of cross-gendered student writing (writing that has some stylistic characteristics traditionally attributed to writers of the opposite gender to that of the writer and some characteristics that are expected of writers of the same gender) provide a point of comparison for the results of this study.
Gender Differences in Student Writing
In studies of gender differences in students’ writing, gender is defined as a set of social and cultural expectations for talking, thinking and acting that are associated with being a girl or a boy. Researchers who assessed gender differences in elementary students’ narrative writingfound in boys’ writing a limited offering of roles for female characters. They also found that student writers positioned male characters in powerful roles that required independent problem solving to overcome obstacles, often in violent ways (Gray-Schlegel & Gray-Schlegel, 1995-96; Trepanier-Street & Romatowski, 1991; Tuck, Bayliss & Bell, 1985). Female characters in girls’ narratives tended to play powerful roles. Although violence was an element in some girls’ stories, characters were more likely to resolve conflicts through the creation of alliances with others than through independent, aggressive action.
In Knudson’s (1992a) study of students’ persuasive writing at grades three, five, ten, and twelve, no gender differences were found in terms of norm invocations (appeals to rules, fair play and reason), positive sanctions (offers of gifts, bargaining, and politeness), requests, and assertions.
Teachers’ Perceptions of Gender and their Evaluation of Writing
When asked to identify the gender of writers of cross-gendered papers, sixth-grade teachers in a Canadian study (Peterson, 1998), and secondary and post secondary English teachers in American studies (Barnes, 1990; Earl-Novell, 2001; Graham, 1996; Haswell & Haswell, 1995; Roulis, 1995) were not always accurate. Their perception of the quality of the writing was often influenced by their perception of the writer’s gender.
Peterson (1998) found that sixth-grade teachers viewed girls’ writing as being organized, detailed, descriptive, and showing a sophisticated use of sentence structure and vocabulary. In contrast, they identified writing that lacked in detail and was poorly developed, in terms of characters, plots and the use of language, as boys’ writing. Although teachers’ descriptions of girls’ and boys’ writing indicated their perceptions of girls as better writers than boys, the scores that they assigned to four ofthe five papers were not significantly higher if the teachers perceived the papers to have been written by girls. There was a significant difference in the scoring of a fifth paper, however. Teachers who felt that a boy had written the paper assigned significantly lower scores to the paper than did teachers who identified the writer as a girl.
The results of research examining teachers’ gender perceptions of cross-gendered writing at the secondary and postsecondary levels showed a bias against female writers (Barnes, 1990; Earl-Novell, 2001; Graham, 1996; Haswell & Haswell, 1995; Roulis, 1995). Participating teachers privileged the linear, impersonal style traditionally attributed to men’s persuasive writing over the contextual style typically attributed to women’s persuasive writing. Barnes (1990) found that male teachers were critical of emotional writing, particularly that written by female writers. Female teachers in her study were more concerned than male teachers about the form of the writing and the author’s use of writing conventions.
Haswell and Haswell (1995) found that male teachers tended to rate male writers’ papers lower than female teachers did, and that female teachers tended to rate female writers’ papers lower than their male counterparts did. Haswell and Haswell termed this phenomenon “same-sex depreciation” and explained that their findings corroborated those of Etaugh, Houtler, and Ptasnik (1988), but did not support Roen’s (1992) findings. In Roen’s study, same-sex appreciation was found, as male high school teachers favored the writing of male students and female teachers privileged girls’ writing.
Methods
Materials: Writing Samples
Two narrative papers and two persuasive papers served as the sample materials for this study. We selected papers from the assigned writing of sixth-grade students in one urban elementary classroom of 22 students. The narratives were about a dream and the persuasive papers considered the ongoing value of the Reserve system for First Nations people in Canada. The four selected papers exhibited characteristics of both boys’ and girls’ writing, as observed in research studies (Gray-Schlegel & Gray-Schlegel, 1995-96; Knudson, 1992a; Trepanier-Street & Romatowski, 1991; Tuck, Bayliss & Bell, 1985). We selected a paper from each mode of discourse written by one boy and one girl in order to compare the scores and evaluative comments given to two types of writing composed by one student. Although we did not edit the papers, we did have them type-written to control for the influence of girls’ and boys’ handwriting styles and the neatness of the writing on teachers’ assessment.
To verify the cross-gendered nature of the writing samples, 11 classroom teachers from a graduate course on literacy read and "identified" the gender of the writers of five narrative and persuasive papers from the selected grade six class. In addition, these same teachers identified gender markers within the narrative writing. The two narrative papers we selected to include in the study were the ones for which most teachers showed great uncertainty in identifying the writer’s gender and/or for which there were very mixed perceptions of the writer’s gender. We then added the persuasive writing samples from the same two students to our research sample. Teachers found it difficult to determine the gender of the writers of all of the persuasive papers.
The girl’s untitled narrative is about a dream in which the first person narrator plays on a school soccer team in a metal uniform. The inevitable injuries that result convince the coach that new uniforms are in order. Her persuasive paper is entitled, “Home Sweet Home First Nations.” The boy’s narrative, entitled “The Dream,” is about a group of children who win environmental awards on the last day of school. The next day they clean up a polluted creek, restoring the habitat of a family of ducks. His persuasive paper is entitled, “Native Case.”
Procedure
We recruited sixth-grade teachers in schools from 17 public and Catholic school districts in one Canadian province. After contacting principals to explain the project and to request the names of grade six teachers in their school, we telephoned teachers from each school, inviting their participation in the study. We sent packages containing the two narrative and two persuasive papers, the provincial scoring guides for narrative and persuasive writing (Ontario Ministry of Education and Training, 1999) and a feedback form that included a space for a numerical evaluation of the writing samples as well as space for further evaluative comments to the students. The form also requested demographic information on the gender of the teachers and the number of years they had taught sixth-grade writing. Teachers were asked to score all four papers using the provincial scoring guide appropriate to the mode of discourse of each paper and to mark or comment on them as they would had a student in their own class submitted the writing to them. They were not told whether the students would actually receive the feedback, however. The overall response rate was 52%. Of the 108 participants, 76 were female (70.4%) and 32 were male (29.6%). Almost a third (29.6%) had taught for less than five years, 39.8% had taught between five and 15 years, and 30.6% had taught for 15 years or more.
We assigned pseudonyms to the writers of each paper. Half of the participating teachers received a set of papers in which Melissa had written the soccer story and Andrew had written the environmental story. The other half of the teachers received surveys in which the two names had been reversed. Similarly, half of the teachers received surveys in which Jeremy had written “Home Sweet Home First Nations” and Kathryn had written “Native Case” and vice versa.
There were four conditions in this study. In Condition 1, for example, the gender of the name assigned to each piece of writing matched the gender of the student who actually wrote the piece: that is, the pieces were identified as female-male-female-male. Condition 2 alternated the genders of the persuasive papers while Condition 3 also switched the narrative papers so that none of the papers were identified by their correct gender. Finally, in Condition 4 the persuasive papers were changed back so that both the assigned and actual gender of the persuasive papes matched. This condition and the number of participants in each condition are summarized in Table 1.
______
Insert Table 1 about here
______
We used provincial scoring guides for narrative and persuasive writing (Ontario Ministry of Education and Training, 1999) because teachers have access to them and are familiar with their use. Within the two scoring guides, there are four separate categories for consideration : (1) reasoning (complexity of ideas and number and relevance of supporting details), (2) communication (establishing and achieving the purpose of the writing, vocabulary, sentence variety, and evidence of the writer’s voice), (3) organization (overall structure and paragraph structure), and (4) conventions (refers to mechanics, such as spelling, grammar and punctuation). Each scoring guide describes four levels of performance applicable to each of the four categories. Level 1 represents a low, but passing performance. Level 3 is performance at grade level.
Data Analysis
Means, standard deviations, and correlations were computed for teachers' ratings. We used two-way analyses of variance (ANOVA) for each of the writing samples to compare the scores by the identified gender of the writer and the teacher’s gender.
Two raters analyzed the teachers’ written feedback to the student-writers using categories from a previous study (Barnes, 1990, p. 146) of college instructors’ evaluative comments on students’ expository writing as a framework:
(1)editive: used when the teacher’s intent is to have the student make lower-order revisions—mechanical, grammatical, lexical, or syntactic changes,
(2)revisional: used when the teacher’s intent is to have the student make higher-order revisions—informational, organizational, or holistic alterations,”
(3)verdictive: comments used to praise.
Throughout the analysis process, the raters worked independently and then compared their analyses, clarifying rationales for particular categorizations when disagreements arose until they reached consensus. They coded the comments in terms of the identified gender of the writers to which the comments were directed and the mode of discourse of the writing, calculating percentages of comments directed to male and female writers for each of the pieces of writing.
Results
Teacher’s Gender
Table 2 presents the average marks in each category for each writing piece by teachers’ gender. It also presents the correlations among the marks. Only the third writing piece, the persuasive paper “Home Sweet Home”, received significantly different scores from male teachers than it did from female teachers. Male teachers scored the paper significantly higher in Reasoning, F(1,102) = 4.28, p < .05, and Communication, F(1,102) = 4.14, p < .05 .
______
Insert Table 2 about here
______
Gender and Type of Writing
As shown in Table 3, both male and female teachers tended to give higher marks if they believed that a girl had written narrative #1. These differences were significant for the female teachers in the Conventions category and for male teachers in the Organization category. This paper contains a great number of conventions/mechanics errors. The opposite pattern was apparent for the second narrative. If teachers believed that the narrative was written by a boy, they gave it higher marks.
For each writing sample, a series of ANOVAs were conducted to determine whether teacher gender and perceived student gender affected the marks assigned to the paper. Across both male and female teachers, the following scores were higher if the student was perceived to be a girl: Communication, F(1,102) = 5.71, p < .05, and Organization, F(1,102) = 5.36, p < .05. None of the interaction terms were significant. (Note: All of these statistics must be interpreted cautiously, as many statistical tests were performed, increasing the chance of labeling a difference significant at the .05 level when it is not.)
For narrative 2 (environment), two of the interactions were significant. For Conventions, female teachers gave similar marks to both “Andrew” and “Melissa,” but male teachers gave “Andrew” significantly higher marks than they gave Melissa, F(1,102) = 4.27, p < .05. Similarly, “Andrew” received significantly higher Communication ratings from male teachers than did Melissa, F(1,102) = 8.44, p < .01.
For the third and fourth writing samples, no significant differences were found.
Both writing sample 1 (narrative-Soccer) and writing sample 3 (persuasive-Home Sweet Home) were written by the female student. Significantly higher scores were assigned to this student’s persuasive piece in all four score categories: Reasoning (2.15 vs. 2.92; t(104) = -7.91, p < .001), Communication (2.20 vs. 2.92; t(104) = -7.85, p < .001), Organization (2.14 vs. 2.76; t(105) = -7.29, p < .001), and Conventions (1.88 vs. 3.04; t(105) = -16.86, p < .001) regardless of the perceived writer. Writing Samples 2 (narrative - Environment) and 4 (persuasive - Native Case) were, similarly, written by the same male student. The only significant difference for Writing Samples 2 and 4 was for Conventions (2.60 vs. 3.19; t(104) = -7.86, p < .001), where significantly higher scores were assigned for the persuasive piece regardless of the assigned name.