Module 4: Data Integration Plan

MODULE 4

Sara Mills

George Mason University

EDRS 797

December 9, 2009

Module 4: Data Integration Plan

The purpose of my proposed study is to explore how teachers implement self-regulated strategy development (SRSD) for writing in their classrooms. Research questions include: (a) What does it take to train classroom teachers to implement SRSD writing instruction in their intact classrooms? To me, this is more of a practical goal than a research question, since it’s not directly answerable by your study. Answerable research questions could be “What was the effect of the training on classroom teachers' use of SRSD instruction?”and “Why was the training effective (or ineffective) in facilitating the use of SRSD?” (b) What types of modifications do classroom teachers make to SRSD instruction when implementing it in the classroom? (c) How and why do they make those modifications? (d) Does students’ writing improve as a result of SRSD instruction? and (e) How do teachers and students perceive of the usefulness and effectiveness of SRSD instruction?

The study will take place in an elementary school, working with teachers at the upper elementary level (grade 4, 5, or 6). Ideally, the study will be conducted with a grade-level team of teachers, including special education teachers working in inclusive classrooms. Teachers will be trained in SRSD instruction. Training will begin with one, long training session, followed by on-going coaching (“booster sessions”) as the instruction is implemented into the classroom. Student essays will be collected prior to, during, and after the intervention.

Qualitative data sources will include field notes of training sessions, observations of teacher planning and writing instruction, and interviews with teachers following implementation. Field notes and notes from observations will be analyzed on an ongoing basis as they are collected. Traditional qualitative data analysis will be conducted, comparing and connecting experiences over time and across teachers. Interviews will be similarly analyzed as they are conducted following instruction. Playing Jeopardy with these methods suggests the second research question I recommended above.

Quantitative data sources include student essays and a social validity this term could use a definition; I had to look it up on the Web, and it seems that it has a range of meanings questionnaire for students. Students will write essays prior to instruction, throughout instruction, and post-instruction. All pre- and post-essays will be scored, and a sample of essays completed during instruction will be scored. (It is anticipated that students will complete 3-5 essays during instruction). Essays will be scored in several ways: (a) total number of words; (b) number of essay parts (e.g., topic sentence, reasons); and (c) a holistic quality score. The student social validity questionnaire will ask the student to rate items on a 5-point Likert-type scale. Mean scores for each item will be computed, and breakdowns of mean ratings by different groups of students will be done (e.g., regular vs. special ed; teacher 1 vs. teacher 2).

While qualitative and quantitative data sources will be analyzed and scored in traditional ways, they will be used in an interactive way throughout the research study. Observations of teachers’ instruction will be used to determine when additional booster training sessions are needed. Data from student essays collected throughout instruction will be used to determine whether any changes need to be made to instruction, which could also trigger the need for booster training sessions. Essay data can also let teachers know when student are ready to move forward to the next stage of instruction, or when they are ready to complete the intervention. Pre- and post-test data can provide a sense of the overall effect of the intervention. This makes sense, but I don’t see any interaction between qualitative and quantitative data sources so far. Similarly, teacher interview data and the social validity questionnaire can help gain an overall sense of usefulness, ease of implementation, and likeliness of future use. How will these be integrated?

Given the separate, traditional data collection and analysis approaches, integrating the two into a coherent, easy-to-follow manuscript will be a challenge. True, but you have Milgram as a model. I see the results portion of the manuscript having the following subparts:

Teacher training: This section will give a chronological account of the teacher training sessions (initial and booster sessions), including detailed information about the content and structure of the training, from field notes.  Because both observational data and student essay data will dictate the need for and type of training for the booster sessions, information obtained from those data sources will be presented throughout the description of the training sessions. I can see that the student essays would be used in planning the training, but studying the actual implementation seems to require qualitative data, unless you survey the teachers. The student essay data presented in this section will be from the sample of essays written by students as they learn the strategy (i.e., not pre- and post-essays).
Instructional modifications: This portion of the results will be qualitative in nature, describing the types of modifications teachers made to the original SRSD instruction. The main data sources will be observational data, as well as data from teacher interviews that explains why teachers made the modification they did. Data from the sample of student essays may be referenced to explain the need for modifications.
Student essays: Pre- and post-test student essays will be analyzed quantitatively in this portion of the results section. Essays will be scored for length, number of essay parts, and quality. These are the measures included in other SRSD writing studies, and will allow for comparisons to be made across studies. This analysis will allow me to answer the research question about the effectiveness of the strategy.
Social validity: Information presented in this section will be a mix of qualitative and quantitative information. Themes that emerge from teacher interviews about the effectiveness and usefulness of the training and strategy will be presented. Student survey results will also be included, which will be analyzed using quantitative methods.

The discussion section, then, will further pull the various forms of data together to answer the research questions. Some of the research questions require data of only one type. For example, “What does it take to train classroom teachers?” will be answered with the qualitative field note data. “Does student writing improve as a result of SRSD instruction?” on the other hand, will be answered using quantitative data. Two questions will rely on a combination of qualitative and quantitative data to provide the answer. First, “How and why do teachers make modifications to instruction?” will likely be answered by a combination of observation, interview, and student essay data. I’m not clear how the essay scores are useful in answering this question. Second, “How do teachers and students perceive of SRSD instruction?” will tap into qualitative teacher interview data and quantitative student survey data. OK.

Sounds easy, right? 

Sara:

As a plan for the study, this makes sense, but I don’t see much explicit discussion of how you’ll integrate the qualitative and quantitative data, or why you chose this approach (which was a key part of the assignment). It might help to think about how your study relates to Greene’s purposes for mixing methods and types of mixed method designs.

Your study seems to me to strongly resemble what’s called design research; have you used this model?

Grade for assignment: A-

Course grade: A