Reliability and Validity of the Reynolds Intellectual Assessment Scale

Reliability and Validity of the Reynolds Intellectual Assessment Scale:

A Brief Literature Review

Guy M. McBride, Ph.D.

School Psychologist

May, 2006

Description:

"The RIAS is an individually administered test of intelligence, which has been normed and standardized on an age population from 3 to 94 years. Utilizing the Cattell-Horn Model of Intelligence (Horn and Cattell, 1966), the authors have developed a reliable measure of general intelligence and its two primary components, fluid and crystallized intelligence. The RIAS includes a measure of verbal and nonverbal intelligence and a resultant Composite Intelligence Index (CIX). Verbal intelligence is assessed with tasks involving verbal problem solving and verbal reasoning tasks. Nonverbal intelligence is assessed by visual and spatial ability tasks. the authors have eliminated reliance on motor coordination and visual-motor speed for the measurement of intelligence. A Composite Memory Index (CMX) can be derived from two supplementary subtests which assess verbal and nonverbal memory. The basic RIAS can be administered by a trained examiner in 20 to 25 minutes. The CMX adds another 10 minutes of administration time." (Elliot, 2004)

Some Advantages of RIAS:

1. The RIAS provides a "nice option as compared to the majority of extant language loaded intelligence tests that seem to have grown longer every year." (Bracken, 2006)

2. The RIAS employs "well tested and proven measures of intelligence." (Bracken, 2006.)

3. The RIAS presents strong evidence for reliability. (Bracken, 2006).

4. "Concurrent validity studies with the WISC III and WAIS III yielded comparable mean scores and overall positive correlations." (Bracken, 2006)

5. "Correlations ranged from .61 to .79 using the VIX, NIX, and CMX composite scores from the RIAS and three composite scores from the WAIS III." (Schraw, 2006) Correlations between CIX and FSIQ on the WISC III ranged from .60 to .78 (Edwards, 2006)

6 The RIAS substantially reduces or eliminates "dependence on motor coordination and visual motor speed in the measurement of intelligence." (Reynolds and Kamphaus, 2003)

7. The RIAS provides a "practical measurement device in terms of efficacies of time, direct costs, and information needed from a measure of intelligence." (Reynolds and Kamphaus, 2003).

8. The RIAS allows for continuity of measurement across all developmental levels from ages 3 through 94 years for both clinical and research purposes. (Reynolds and Kamphaus, 2003) (The Wechsler series employs three separate tests across those age ranges, making comparisons of performance on different tests impractical; and even the DAS uses substantially different subtests for different age ranges, making comparisons of performance from the preschool age range to school age age range more problematic. GMM)

9. The RIAS accurately predicts basic academic achievement comparable to IQ tests twice its length. (Reynolds and Kamphaus, 2003).

10. "The RIAS substantially lessens the time to assess intelligence without compromising statistical integrity." (Elliott, 2004)

11. "Reliability and validity are comparable, if not stronger, than reliability and validity measures on similar measures of intelligence that are substantially longer and more complex." (Elliott, 2004).

12. "It is likely that the RIAS will become a very popular measure of intelligence for school districts that are under serious pressure to provide measures of intelligence for more rapid fashion. The RIAS can satsitify that need and should become part of every school psychologist's battery of tests." (Elliott, 2004)

Discussion:

Although test reviewers talk about construct, concurrent, and predictive validity, ultimately ALL forms of validity studies reflect on construct validity . . . "Does the test do what it is supposed to do?" (Joint Standards; Office for Civil Rights, 2000).

In validating the inferences of the test results, it is important to consider the consequences of the test's interpretation and use.

With respect to the RIAS, the mean and distribution of scores are reportedly similar for both the RIAS and Wechsler series. Nevertheless, different tests WILL give different scores. (A child will not score the same on a Woodcock Johnson Tests of Achievement that he would on the End of Grade test; some children will score higher, some lower.)

A test must be validated for the purpose for which it is being used in order to be in compliance with the federal regulations. When the Wechsler was last revised, it doubled the loading on working memory and processing speed, both psychological processing skills are typically deficient in children with learning disabilities. In essence, this meant that children who were having difficulty with achievement because of deficits in these areas were also being penalized on some tests of ability (such as the Wechsler), in essence creating a case of double jeopardy.

"Do children with learning disabilities score lower on WMI and PSI?

Studies reported in the [WISC IV] Manual suggest that children with learning and/or attention disorders tend to perform lower on tasks that measure working memory and processing speed."

(Frequently Asked Questions, Harcourt Brace website)

The Mark Penalty occurs in psychological testing for LD based on a discrepancy model when the examiner uses a test that is biased by the same psychological processing deficits that formed the basis, in whole or part, for the child's academic problems.

Psychological Corporation responded to these criticisms by issuing Technical Report No. 4, which allows the psychologist, after conducting a post hoc analysis, to calculate a General Abilities Index to be used in lieu of FSIQ. The GAI, which calculates a cognitive score based on a combination of the six Verbal and Performance subtests (excluding WM and PS), will result in a higher score for children who have been found, after the testing, to have significant problems in either area.

Indeed, one limited (N=34) study of referred children tested using the WISC IV and the RIAS confirmed that trend. The children tested had a RIAS CIX that was on average about 8 to 9 points higher than the FSIQ; but when GAIs were calculated, that difference fell to three to four points.

Others, like John Willis and Ron Dumont, have argued that it is more appropriate to eliminate the "contamination" of working memory, processing speed, and psychomotor skills before the fact, not (as with the Wechsler) after the fact when testing children suspected of learning disabilities or mental impairment.

"If a student has weaknesses in basic sensory, motor, or psychological processes (e.g., visual impairment, hearing loss, cerebral palsy, oral language disorder, word-finding impairment, auditory perception, visual perception, processing speed, working memory, etc.), we think it is only reasonable to seek a measure of intelligence that is not contaminated by weaknesses that have been documented by testing (other than the cognitive ability test itself), observation, classroom performance, and other sources of information. In fact, Ron Dumont has said that the intelligence test for a child should not be selected until other assessment is completed. If you are trying to assess thinking, reasoning, and problem-solving ability, you do not want to use intelligence tests that require a blind child to copy block designs or a child with an oral language disorder to define words. You want an intelligence measure that is not contaminated by the documented weaknesses, even – or especially – if those weaknesses are in abilities important to intellectual functioning."

Since the RIAS eliminates many of those potentially contaminating factors up front, however, the amount of analysis required to determine the tasks' appropriateness is clearly reduced. (For example, the RIAS would in most cases not be the best test to use with a blind or deaf child or an ESL child.)

Reliability: A brief comparison

CIX reliability is reported to be about .97; the reliability for the FSIQ on the WISC IV was reported to be about .98; reliability for GAI on the WISC IV was reported to be .96. The DAS reports reliabilities of .94 to .95 for preschool children, and .95 to .96 for school age children.

Validity: A brief review

The DAS reports predictive reliabilities (correlations with achievement tests) ranging from .52 (spelling) to .60 (Basic Number Skills, Word Reading) when correlated with scores obtained by children in the normative sample of the DAS Achievement tests (Colin Elliott, 1983) When compared with other tests, correlations ranged from .46 (spelling at age 7 and reading age at 11) to .66 (reading at age 7.)

Reported correlations between CIX and WIAT Composites ranged from .60 to .69. (Reynolds and Kamphaus, 2003). [Verbal Intelligence Index reportedly correlated .73 (Elliott, 2003).]

Brief Summary of Correlations with Other Cognitive Measures

By way of the comparison, the Differential Ability Scales, the GCA on the DAS reportedly correlated .75 with the Kaufman Assessment Battery for Children 5-7 years old. It correlated .88 with the Stanford Binet IV for 9 and 10 year olds. It correlated .84 with the WISC III. For 8 to 10 year olds. (Colin Elliott, 1983)

The RIAS (based on a study of 54 children) correlated .76 with FSIQ on the old WISC III.

In his study, Edwards (2006) found a correlation of .86 between CIX and FSIQ on the WISC IV, despite differences between the mean scores.

Criticisms: No review would be complete without criticisms. The most frequent criticism levied at the RIAS in 2006 was that in the best of all possible worlds, there would be more studies of predictive validity than were included in the test manual. No one, however, has suggested that the predictive validity studies done by Reynolds, et. al, were flawed or that the data reported was less than satisfactory. Test validation is an on going process, not a finished task once the first technical manual is issued. See some of the references cited above for more current opinions.

Summary: The RIAS offers significant advantages over other measures of cognitive ability when assessing children who are suspected of being SLD and one reviewer for Buros thought it best for assessing children with cognitive impairments.

Overall, comparable scores for the general population would be expected. However, the RIAS may provide higher scores for children who have significant psychomotor problems or specific problems with working memory or processing speed (underlying psychological processing deficits oft times associated with SLD children.) This is not a flaw.

Working memory, processing speed, and psychomotor skill deficits may all impede academic progress. (The Bender, for example, has a .6 correlation with some measures of achievement.) They are not unimportant, and they should be documented. We just shouldn't call them "IQ" or "intelligence."

Guy’s Conclusion: While the RIAS may not be the best tool for assessing children who are blind, deaf, or who have limited English proficiency, the RIAS is an appropriate assessment tool for use both in initial and reevaluations. As Elliot suggested in his review, this reviewer also believes the RIAS should be in every school psychologist’s assessment toolbox.

References:

Bracken, Bruce, and Gregory Shaw. Review of the Reynolds Intellectual Assessment Scales. Buros Mental Measurements Yearbook. 16th Edition.

Edwards, Oliver. Referred Students' Scores on the RIAS and WISC IV: Implications for Test Selection. PowerPoint Presentation, NASP, 2006.

Elliott, Colin. Differential Ability Scales. Introductory and Technical Handbook. Harcourt Brace. 1983

Elliott, Robert. Reynolds Intellectual Assessment Scales. Archives of Clinical Neuro-Psychology. 2003.

Reynolds, Cecil. Re: [Nasp Listserv] Rias v. Wechsler. Personal Communication by email published on NASP Listserv. April, 2006.

Reynolds, Cecil, and Randy Kamphaus. Reynolds Intellectual Assessment Scales. Professional Manual. Psychological Assessment Resources, Inc. 2003.

Standards for Educational and Psychological Testing. American Educational Research Association, American Psychological Association, Nationa Council onf Measurement in Education. 1999.

The Use of Tests as Part of High Stakes Decision Making for Students. A Resource Guide for Educators and Policy Makers. "Chapter One. Test Measurement Principles." United States Department of Education. Office for Civil Rights. 2000.

Willis, John. Using the DWI or the GAI. Internet web page. Undated.