A Review of Research on Literacy and Technology: Replicating and Extending the NRP Subcommittee

A Review of Research on Literacy and Technology: Replicating and Extending the NRP Subcommittee Report on Computer Technology and Reading Instruction

Julie Coiro, University of Connecticut

Donald J. Leu, University of Connecticut

Charles K. Kinzer, Teachers College, Columbia University

Linda Labbo, University of Georgia

William Teale, University of Illinois, Chicago

Lisa Bergman, James Sulzen, and Dongping Sheng, University of Connecticut

, ,

It is clear to many (Coiro, 2003; International Reading Association, 2001; Lankshear & Knobel, 2003; Luke, 2000) that new information and communication technologies (ICT) are entering the classroom, redefining reading, literacy, and instruction. Thus, a central issue today has become: As teachers seek to integrate technology within their literacy classrooms, which research-based principles should inform classroom instruction? Unfortunately, the issue has been almost completely neglected in recent national reports, policy initiatives, and state assessments (Leu, Kinzer, Coiro, & Cammack, in press). The single national report on reading that explored this issue (National Reading Panel, 2000) could only locate 21 studies that met their section criteria, determining that, “It is extremely difficult to make any specific instructional conclusions based on the small sample of experimental research available.” (p. 6-2). Several aspects of this review, however, call into question its conclusions:

1.  A careful analysis suggests that a number of studies were included that violated the selection criteria.

2.  While the report was published in 2000, the most recent study in the NRP review was published in 1996, suggesting that the review had been completed before the committee even began to meet, missing much recent research.

3.  Despite the increasingly diverse nature of classrooms, for some reason the NRP subcommittee excluded studies that included special populations and non-native speakers of English.

Thus, there are a number of reasons to reevaluate the issues studied by the NRP subcommittee on computers and reading.

The Overall Purpose

The purpose of this project was to review, replicate, and extend the National Reading Panel’s Subcommittee Report on Computer Technology and Reading Instruction (National Reading Panel, 2000). This report takes place within a larger project to systematically replicate the subcommittee’s work, initially following the same procedures and selection criteria and then expanding their selection criteria to include more diverse populations, more contemporary research, and newer technologies. The results will provide important direction for schools and teachers who seek to integrate technology into literacy instruction based on conclusions drawn from the research literature.

Methods: Replication and Extension of the NRP Report

Our methodology initially replicated the NRP procedures and then extended them. Using identical search procedures, we conducted our analyses in three waves:

1.  We conducted a strict replication (reviewing studies up to 1996) using all criteria and procedures from the original report;

2.  We conducted a strict replication, using all criteria and procedures from the original report, but updated to include studies appearing through 2002;

3.  We conducted an expanded replication through 2002 broadening the search terms to include special populations, English Language Learners, writing, and new technologies such as the Internet, all of which were excluded in the initial study.

Results and Conclusions

The results of this replication indicate that, even when strict criteria were established, more than twice as many experimental or quasi-experimental studies, published in peer-reviewed journals, meet the original NRP criteria (n=43). When expanded criteria were used, including more diverse populations, new technologies, writing, and more contemporary research, nearly ten times (n=191) the number of original studies appeared. The most common issue explored in these studies was comprehension (41.8%). Other frequently appearing topics included: writing (26.2%), word recognition (24.6%), decoding (17.3%), motivation (16.8%), vocabulary (15.2%), spelling (10.9%), and fluency (10.5%).

Our analysis of this replication leads us to several conclusions:

1.  The National Reading Panel Subcommittee Report on Technology and Reading (National Reading Panel, 2000) inexplicably failed to include many studies that had been published during the period from 1996-2000.

2.  The National Reading Panel Subcommittee Report on Technology and Reading (National Reading Panel, 2000) inexplicably failed to review studies with the diverse populations that appear in today’s classrooms.

3.  There are a sufficient number of studies, meeting NRP criteria, to draw reasonable conclusions about the use of technology to support reading instruction.

Table 1.

Number of Studies Identified During the Replication and Extension of the Search Procedures Used by the NRP Subcommittee on Computer Technology and Reading Instruction

NRP Procedures
1980-1996 / NRP Procedures
1997-2002 / NRP Procedures Plus Extension / Total
1980-2002
Number of
studies Number of studies / 43a / 12 / 136 / 191

a21 studies appeared in the original report

Table 2.

Results within NRP Extended Category (Number of Research Questions)

NRP
1980-1996 / NRP
1997-2002 / NRP Extended / Total
1980-2002

Special Populations

/ 1 / 0 / 78 / 79
Comparisons / 7 / 3 / 65 / 75
Design Issues / 10 / 1 / 31 / 42
Writing / 0 / 0 / 41 / 41
New Technologies / 0 / 0 / 6 / 6
Eng. Lang. Learners / 0 / 0 / 3 / 3

Table 3. Table 4.

Studies Sorted by Literacy Dimension (n=191) Studies Sorted by Technology (n=191)

Total # / % / Total # / %
Comprehension / 80 / 41.8 / CAI / 58 / 30.4
Writing / 50 / 26.2 / Word Processing / 34 / 17.8
Word Recognition / 47 / 24.6 / Speech Feedback / 33 / 17.3
Decoding / Letter ID / 33 / 17.3 / Comp. Mediated Text / 29 / 15.2
Motivation / 32 / 16.8 / Hypermedia Envir. / 11 / 5.7
Vocabulary / 29 / 15.2 / Electronic Storybooks / 7 / 3.7
Spelling / 21 / 10.9 / Keyboarding / 6 / 3.1
Fluency / 20 / 10.5 / Multimedia Envir. / 4 / 2.1
Phonological Aware. / 19 / 9.9 / Simple Screen Display / 4 / 2.1
Grammar / Syntax / 4 / 2.1 / Internet / 4 / 2.1
Expressive Language / 4 / 2.1 / Word Prediction Soft. / 2 / 1.0
Other / 5 / 2.6

Table 5.

Studies Sorted by Population Type (n=191)

Total # / %
Typical Students / 126 / 66.0
Learning Disabled / 70 / 36.6
Learning Handicapped / MR / 10 / 5.2
Dyslexic / 5 / 2.6
Hearing Impaired / 5 / 2.6
English Language Learners / 3 / 1.6
Autistic / 2 / 1.0
Attention Deficit Disorder / 1 / 0.5
Emotionally Disturbed / 1 / 0.5
Gifted / 1 / 0.5
Visually Impaired / 0 / 0.0

Methods: Comprehension Studies

This paper also presents the results of our analysis of studies that evaluated comprehension issues (n=80), the largest single category of studies in our review and a critical area for research (RAND Reading Study Group, 2002). We took a content analytic approach to this work. Our analysis looked at the major research questions in every study and then identified the related tests of statistical significance that were used to test each question. Patterns from the results of these tests emerged using inductive analysis. This approach sought to derive principles of effective practice that might be useful to teachers and educational leaders who often have to make decisions about classroom technology acquisition and use.

Results: Comprehension Studies

Table 6.

Number of Comprehension Studies Identified During the Replication and Extension of the Search Procedures Used by the NRP Subcommittee on Computer Technology and Reading Instruction

NRP Procedures
1980-1996 / NRP Procedures
1997-2002 / NRP Procedures Plus Extension / Total
1980-2002
Number of studies / 24 / 5 / 51 / 80

Table 7. Table 8.

Comprehension Studies (n=80) by Technology Comprehension Studies (n=80) by Population

Technology / No. / Population / No.
CAI / 25 / Typical Students / 64
Computer Mediated Text / 23 / Learning Disabled / 26
Speech Feedback / 10 / English Language Learners / 4
Hypermedia Envir. / 6 / Dyslexic / 3
Integrated Lrng. Sys (ILS) / 5 / Learn. Handicapped / MR / 3
Simple Screen Display / 4 / Emotionally Disturbed / 1
Electronic Storybook / 4 / Hearing Impaired / 1
Other / 4 / Gifted / 1
Internet / 3 / Autistic / 0
Attention Deficit Disorder / 0

Table 9.

Comprehension Studies (n=80) by Age Group

Grades / No.
PK-2 / 32
3-5 / 47
6-8 / 34
9-12 / 20

Two major patterns emerged from this content analysis of statistically significant effects in studies that explored issues of technology use and reading comprehension:

Principle 1.  Many different computer technologies appear to contain the potential for supporting the development of reading comprehension. These include:

·  ILS Systems

·  CAI

·  Multiply Mediated (Supportive) Texts

·  Speech Supported Texts

·  Internet Technologies

·  Word Prediction Software

Principle 2.  The potential of any technology to support the development of reading comprehension may only be realized when teachers make appropriate decisions about its use. Several elements need to be considered by teachers as they make decisions about the use of technology to support the development of reading comprehension:

·  the design of the technology that is used;

·  the ways in which teachers and students collaborate and interact with each other while using technology;

·  the nature of individual differences among learners;

·  the nature of how comprehension outcomes are defined and assessed.

Discussion

Our results suggest that there is sufficient research to begin to draw conclusions about the use of technology to support literacy instruction in school classrooms. An analysis of studies that investigated comprehension issues allows us to identify two important principles of effective practice. These principles are consistent with the emphasis placed in the RAND report (2002) on contextual variables such as the task, text, reader that take place within a particularized social context.

The results help us to understand that the central question in the use of technology in literacy classrooms is not just whether or not we should use technology to support the development of reading comprehension. Instead, the most important questions we need to ask are contextualized into the particular spaces of particular classrooms: How is a technology to be used? For whom is it most effective? and How are comprehension outcomes to be measured? The results suggest that classroom teachers will become much more important, not less important, as new technologies for literacy and learning enter classrooms. The decisions that each teacher makes about how literacy software is used will be critical to any results he/she might expect to see.

The results also suggest that teacher education and professional development are critically important if we hope to realize the potential that technology possesses for improving reading comprehension performance. If teachers’ decisions are central to the effective use of technology and if only 20% of teachers feel comfortable with integrating technology into the classroom (CEO Forum, 1999) enormous effort is required in both teacher education and professional development.

Finally, these results suggest several important implications for research:

1. Research needs to begin to study how and why some teachers use technology and realize important comprehension gains in their classrooms, while others do not.
2. Research on technology use in classrooms needs to include richer designs which explore not just whether a technology helps students to learn but which students benefit most/least from a particular technology experience and which contextual variables seem to assist this.

References

Coiro, J. (2003). Reading comprehension on the Internet: Expanding our understanding of reading comprehension to encompass new literacies. The Reading Teacher, 56, 458-464. Available at Reading Online. [Online Serial]: http://www.readingonline.org/electronic/elec_index.asp?HREF=/electronic/rt/2-03_Column/index.html

International Reading Association, 2001. Integrating literacy and technology in the curriculum. Newark, DE: International Reading Association.

Lankshear, C. & Knobel (2003). New literacies: Changing knowledge and classroom learning. Buckingham: Open University Press.

Leu., D.J., Jr., Kinzer, C.K., Coiro, J., Cammack, D. (in press). Towards a theory of new literacies emerging from the Internet and other ICT. In R.B. Ruddell & N. Unrau (Eds.), Theoretical Models and Processes of Reading, Fifth Edition. International Reading Association: Newark, DE.

Luke, C. (2000). Cyber-schooling and technological change: Multiliteracies for new times. In B. Cope & M. Kalantzis (Eds.) Multiliteracies: Literacy learning and the design of social futures (pp. 69-91). London: Routledge.

National Center for Education Statistics (2002). Internet access in public schools and classrooms: 1994-2000. Available: http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2002018

National Reading Panel, 2000. Teaching children to read: Report of the subgroups. Bethesda, MD: National Institute of Child Health and Human Development.

RAND Reading Study Group. (2002). Reading for understanding: Toward an R&D program in reading comprehension. Santa Monica, CA: Rand Education. Also available at http://www.rand.org/multi/achievementforall/reading/.

The New London Group. (2000). Multiliteracies: Literacy learning and the design of social futures. London: Routledge.

APPENDIX A

Reading Comprehension Studies Used in This Study

(Sorted by Technology Dimension)

Computer Assisted Instruction (CAI) (25 studies)

Bass, G., Ries, R., & Sharpe, W. (1986). Teaching basic skills through microcomputer assisted instruction. Journal of Educational Computing Research, 2(2), 207-219.

Blank, D., Murphy, P. A., & Shneiderman, B. (1986). A comparison of children's reading comprehension and reading rates at three text presentation speeds on a CRT. Journal of Computer Based Instruction, 13(3), 84-87.

Bond, C. L., & Clariana, R. B. (1989). Prescriptions in reading CAI: A study considering the effects of three instructional combinations with either computer-adaptive or controlled placement on reading achievement levels. Journal of Computing in Childhood Education, 1, 59-72.

Bornas, X., & Llabraes, J. (2001). Helping students build knowledge: What computers should do. Information Technology in Childhood Education Annual, 13, 267-280.

Breznitz, Z. (1990). Vocalization and pauses in fast-paced reading. Journal of General Psychology, 117(2), 153-159.

Breznitz, Z. (1997). Enhancing the reading of dyslexic children by reading acceleration and auditory masking. Journal of Educational Psychology, 89(1), 103-113.