Doctoral Study Summary

Doctoral Study Findings Summary

Dr. Donna Spangler / December 2009

Title of Study: Effects of Two Foreign Language Methodologies, Communicative Language Teaching (CLT) and Teaching Proficiency through Reading and Storytelling (TPRS), on Beginning-Level Students’ Achievement, Fluency, and Anxiety

Availability of Full Doctoral Study: Currently being processed through copyright. Process takes between eight to twelve weeks. Availability is approximately beginning of March 2010.

Introduction: CLT and TPRS are currently two dominant methodological frameworks that many foreign language practitioners use to teach second language (L2) in the United States. While both methodologies strive for language acquisition as their goal, these two methodologies have striking difference. CLT is a communication-based method of instruction encouraging both L2 input and L2 output while TPRS is a content-based method of instruction emphasizing L2 input. To date, no empirical studies exist comparing the effectiveness of these two methodologies with regard to students’ L2 achievement, fluency, and anxiety.

Purpose: The purpose of this quantitative quasi-experimental research study was to provide empirical research comparing the achievement, fluency, and anxiety levels of beginning-level CLT students and TPRS students learning Spanish in both a middle-school and in a high-school classroom setting. Because teachers in the CLT and TPRS methodologies provide different types and expectations for language input, processing time, and language output, this study tested the hypotheses that CLT and TPRS methodologies have different effects on student’s second language achievement and fluency and on students’ anxiety levels.

Research Approach and Rationale: For the past decade, teacher practitioners have held strong opinions about whether the CLT or TPRS methodology is more effective. Until this study, teachers have aligned with one or the other methodology based upon personal opinions, individual perceptions, and teachers’ deeply-rooted, personal belief systems about how second language is acquired. While teachers’ opinions and perceptions are important in education, they cannot be the only points upon which a teacher bases his or her instruction. In order for student education to be effective, it is critical to examine objectively if a strategy, a technique, or a methodology is making a positive difference in students’ learning. Sometimes, what we as teachers personally believe to be the way students should learn may fit into our own belief paradigm, but it may not be the most effective learning strategy, technique, or methodology in order for the student to learn.

Participants and Data Collection Sites: Teacher selection for the study needed to occur through criterion sampling. Because this study required teachers who are familiar with both CLT and TPRS methodologies, I needed a systematic approach to teacher selection in order to find appropriate teachers for this study.

The sites for the study depended upon the teacher participants’ school locations. A total of 162 public school students and two public school teachers participated in this study. Part of this study took place in a public high school in California involving 129 students in five Spanish classes and one teacher. Fifty-one beginning-level high school students experienced 14 weeks of language learning using the CLT methodology in Spanish classrooms while the other 78 students had 14 weeks of language learning using the TPRS methodology in Spanish classrooms. The same teacher in California taught both the CLT and the TPRS classrooms of students. The other part of this study ran concurrently with the study in California but took place in a public middle school in Rhode Island involving 33 students in two Spanish classes and one teacher. Fourteen beginning-level middle school students experienced 14 weeks of language learning using the CLT methodology in one Spanish classroom, while 19 students had 14 weeks of language learning using the TPRS methodology in a second Spanish classroom. The same teacher in Rhode Island taught both the CLT and the TPRS classrooms of students. In sum, two public school teachers instructed a total of seven public school Spanish classes with 162 participants: 51 high-school students using the CLT methodology, 78 high-school students using the TPRS methodology, 14 middle-school students using the CLT methodology, and 19 middle-school students using the TPRS methodology.

Data Collection Procedures: To start the data collection, interested student participants from the classes of the two selected teachers in California and Rhode Island were given a Parental Consent Form and a Student Assent Form. Interested participants then completed a Background Questionnaire to ensure they met the research study criterion of being beginning-level second language learners without significant prior second language experience. At week 12 of the 14 week study, all student participants took Horwitz et al. (1986) Foreign Language Classroom Anxiety Scale (FLCAS) to evaluate student anxiety. At week 14 of the study, all student participants took the Standards-based Measurement of Proficiency (STAMP) test.

Treatment Phase: For 14 weeks, both the high-school Spanish teacher and middle-school Spanish teacher instructed one set of Spanish classrooms using the CLT methodology and another set of Spanish classrooms using the TPRS methodology. In this study, the same teacher taught both methodologies in each school. Although both teachers were fully knowledgeable about both the CLT and TPRS methodologies, I reviewed with both teachers the key techniques and strategies common to each methodology before the actual treatment phase of the study began. Collaboratively, the two teachers and I identified strategies to be used in the CLT classroom and the TPRS classroom throughout the treatment phase of the study.

Summary CLT strategies (emphasizing BOTH student L2 input and student L2 output) used included: real-life situations that necessitate communication, information-gap activities, structured language output activities, verbal and written role-play activities, eavesdropping activities, gist activities, student performance, communicative classroom activities, and communicative games.

Summary TPRS strategies (emphasizing student L2 input) used included: kindergarten day, chapter stories, personalized mini-stories, personalized question and answer, free voluntary reading, gestures from TPR, student actors as story is told in the target language, use of BEP (bizarre, exaggerated or personalized) information, circling story question techniques, use of pop-up grammar, and songs or chants.

Full treatment strategies outlined in full study appendix.

Recording Tools: In this study, the two types of data were: (a) the Foreign Language Classroom Anxiety Survey (FLCAS) instrument, and (b) the Standards-based Measurement of Proficiency (STAMP) test.

At week 12 of the 14 week study, all student participants took Horwitz et al. (1986) Foreign Language Classroom Anxiety Scale (FLCAS) to evaluate student anxiety. The FLCAS is a 33-item self-report instrument that uses 5-point Likert scales with choices from “strongly agree” to “strongly disagree” to measure the level of anxiety student experience in the foreign language classroom. It was originally developed from “student self-reports, clinical experiences, and a review of related instruments” (Horwitz et al., 1986, p. 37). The high internal consistency of the FLCAS using Cronbach’s alpha was .93 with a sample of 108 students and had a test-retest reliability over eight weeks was r = .83 (p < .001) with a sample of 78 students (Horwitz et al., 1986).

At week 14 of the study, all student participants took the Standards-based Measurement of Proficiency (STAMP) test. STAMP is a web-based assessment tool characterizing proficiency levels that are tied to the American Council of the Teaching of Foreign Languages (ACTFL) Performance Guidelines. Unlike discrete vocabulary and grammar-type tests, the STAMP test is designed to measure student achievement and proficiency with regard to reading, writing, and speaking. STAMP tests items are developed from authentic and semi-authentic texts and are realia-based. In this way, students interact with the text in using a real world context. These types of test items require that the student demonstrate that he or she is proficient in understanding and using the target language, and the test measures the level of the student’s proficiency in the language. Avant Assessment developed the STAMP web-based assessment in conjunction with its research partner, The Center for Applied Second Language Studies at the University of Oregon (CASLS). This test measures benchmarks that are descriptions of proficiency tied to the International Language Roundtable (ILR) and the ACTFL Guidelines. The STAMP measures students’ achievement and proficiency from benchmark levels of Novice-Low through Intermediate-Mid. In the Avant Assessment STAMP system, these benchmarks are identified as levels 1-5

The STAMP test is age appropriate for grades seven through college senior and these tests measure students’ reading, writing, and speaking levels. Multiple topics are covered in each benchmark level so individual students are not only measured by achievement and proficiency levels, but also data on their strengths and weaknesses by topic are also provided in the assessment reporting. In addition, STAMP is a textbook- independent test so it does not favor a particular textbook or methodology of instruction.

The STAMP test is a statistically-validated test available through Avant Assessment. Foreign language professionals at the University of Oregon Center for Applied Second Language Studies (CASLS) developed the original STAMP test items which were then piloted and statistically verified to consistently measure student achievement at specific levels. Over 30,000 middle school, high school, and college students were involved in the piloting of STAMP. In addition, the scoring of the STAMP test is externally scored to language benchmarks with high levels of inter-rater reliability at Avant Assessment. Twenty percent of all student scores from Avant are rated by a second rater to assist in tracking grader accuracy and reliability.

In addition, unlike most foreign language assessments, this test is a computer adaptive test. As a result, the delivery of the STAMP test items is unique for each student tested assuring that each student receives a test that customizes itself to each student’s ability level. Questions are initially delivered with randomly ordered answer sets. However, as the student correctly or incorrectly answers the questions, the computer generates higher or lower test items to ensure the appropriate level suited to each student’s individual achievement and proficiency capacity. Then, additional test items are presented to the student at slightly higher levels to probe for individual areas of strengths and weaknesses.

Research Results:

Research Question 1: Reading Achievement

The first research question is based on the difference between the mean reading scores: Specifically, “Does the CLT or TPRS methodology of instruction result in higher levels of students’ achievement in L2?” Achievement in this study meant the students’ understanding (or comprehension of language input) in Spanish as measured by the accuracy of students’ reading comprehension of unfamiliar text. The statistical tests for Research Question 1 were descriptive statistics and the independent-samples t test.

The first section all students took on the STAMP test was the reading section. Initially, students were asked mid-level, multiple-choice questions in a random format until the computer could establish an appropriate level of questioning using realia-based texts for each student. Because the test was computer adaptive in nature, each student received different multiple-choice questions resulting in individual, customized tests from a test bank containing of hundreds of reading test questions. After a time frame of approximately 45 minutes for most students, the computer tabulated each student’s responses and individually assigned a numerical benchmark level to each student based on ACTFL standards.

The median score for the students receiving the CLT instruction was 1.00 (M = 1.65, SD = .84). The second group of students receiving the TPRS instruction had a median score of 1.00 (M = 1.63, SD = .93). This meant there was a .02 difference; a very small difference which did not appear to be statistically significant. In assessing the null and alternative hypotheses, the Levene’s test for heterogeneity of variance was non-significant: F(1,160) = 1.13, p > 0.05. Consequently, the independent-samples t test with equal variances assumed was done. The t statistic was non-significant: t(160) = .121, p > .05. There was not a significant difference between the mean reading scores for students whom received CLT instruction (M = 1.65, SD = .84) and those whom received the TPRS instruction (M = 1.63, SD = .93). Therefore, although the CLT and TPRS methodologies provide different types of and expectations for language input, processing time, and language output, there was no statistically significant relationship between the achievement in reading of the CLT students and the achievement in reading of the TPRS students in this study.

Research Question 2: Writing Fluency

The second research question addressed student fluency with regard to both writing and speaking results from the STAMP test. This data section will review students’ writing results; the following data section will examine students’ speaking results. The second research question is based on the mean difference between the mean writing scores: Specifically, “Does the CLT or TPRS methodology of instruction result in higher levels of students’ fluency in L2?” In this study, fluency was defined as the students’ production (or language output) in students’ writing and speaking samples in Spanish. In this study, L2 fluency was ability of the students to express ideas in a comprehensible manner in Spanish without excessive hesitancy or difficulty. However, it did not refer to grammatical correctness of native-like pronunciation (Ray & Seely, 2004). The statistical tests for Research Question 2 included descriptive statistics and the independent-samples t test.

To answer this question, after the 162 students completed the reading section of the STAMP test and the computer adaptive test had established an individual reading benchmark for each student, all students moved on to the writing section of the test. Students began by reading instructions and receiving one sample items dependent upon the benchmark level the student earned in the reading section. Students then received three random picture prompts that used a culturally authentic context at the student’s level of proficiency to respond to in writing. As students wrote, students had access to the rubrics that AVANT assessors would use to grade their response so students were aware of the assessor’s grading criteria.

The median score for the students receiving the CLT instruction was 3.00 (M = 2.99, SD = .53). The group of students receiving the TPRS instruction was 3.00 (M = 2.86, SD = .76). There was a .13 difference; a small difference which did not appear to be statistically significant. In assessing the null and alternative hypotheses, the Levene’s test for heterogeneity of variance was significant: F(1, 160) = 6.44, p < 0.05. Consequently, the independent-samples t test with no equal variances assumed was done. The t statistic was non-significant: t(159.79) = 1.30, p > .05. There was not a significant difference between the mean writing scores for students whom received CLT instruction (M = 2.99, SD = .53) and those whom received TPRS instruction (M = 2.86, SD = .76). Therefore, although the CLT and TPRS methodologies provide different types of and expectations for language input, processing time, and language output, there was no significant relationship between the writing of the CLT students and the writing of the TPRS students in this study.

Research Question 2: Speaking Fluency

The second research question addressed student fluency with regard to both writing and speaking results from the STAMP test. This data section will examine the students’ speaking results. The second research question is based on the difference between the mean speaking scores: Specifically, “Does the CLT or TPRS methodology of instruction result in higher levels of students’ fluency in L2?” In this study, fluency was defined as the students’ production (or language output) in students’ writing and speaking samples in Spanish. In this study, L2 fluency was ability of the students to express ideas in a comprehensible manner in Spanish without excessive hesitancy or difficulty. However, it did not refer to grammatical correctness of native-like pronunciation (Ray & Seely, 2004). In this data section, the statistical tests for Research Question 2 incorporated descriptive statistics and the independent-samples t test.

Based upon the individual benchmark level attained by each student during the reading section of the STAMP test, each student received three random picture prompts

to respond to by recording a speaking sample. Using a computer microphone and speaker headset, students recorded a speaking sample in Spanish that related to the visual picture prompts.