SPED 512: Standardized

Instrument Critique and Demonstration

(30 points for paper; 20 points for presentation)

Purpose

The purpose of this assignment is to help you understand how to evaluate a standardized achievement assessment in terms of the quality of its psychometric characteristics in the context of its intended use. The second purpose is for you to model the administration of components of the instrument and scoring procedures in class.

Test Selection

You and your partner are to choose an assessment from the following list (available for checkout):

  1. Clinical Evaluation of Language Fundamentals (CELF 4)
  2. Comprehensive Assessment of Spoken Language (CASL)
  3. Peabody Picture Vocabulary Test (PPVT 4)
  4. Process Assessment of the Learner: Math (PAL-II: Math)
  5. Process Assessment of the Learner: Reading/Writing (PAL-II: Reading/Writing)
  6. Wechsler Individual Achievement Test (WIAT 3)
  7. Test of Reading Comprehension (TORC 4)
  8. Gray Oral Reading Test (GORT 4)
  9. Expressive Vocabulary Test (EVT 2)
  10. Comprehensive Test of Phonological Processing (CTOPP)
  11. Key Math 3
  12. Wide Range Achievement Test 4 (WRAT 4)
  13. Kaufman Test of Educational Achievement – 2nd Ed.
  14. Test of Orthographic Competence (TOC)
  15. Test of Silent Word Reading Fluency

Assignment

Student pairs will:

v  Learn one commercially available standardized achievement assessment (NOT the WJ-III) from the available tests above.

v  Critique the assessment using the electronic template provided. Be sure you completely answer questions and provide a thorough critique.

v  Present an oral discussion/demonstration of the instrument covering the elements of the written critique in a 10 minute (or less) presentation. Make it interesting! During the presentation, demonstrate relevant portions of the assessment (including a discussion on scoring) to the class with one person playing the role of the student and the other the assessor. (Total presentation time not to exceed 10 minutes.)

Critical Evaluation of an Assessment

(30 points for paper)

Students’ Names: Date of Presentation:

Assessment (2 pts)

Author: Year: Price:

Publisher (Name and Address):

Stated Uses (3 pts)

What are the stated uses of this instrument?

Target Populations (7 pts)

To which population does this assessment apply (i.e., carefully examine the normative sample). To which other populations does it claim to generalize?

Questions (8 pts)

Is the cost reasonable and justifiable? Yes No

Is the assessment theoretically grounded? Yes No

Would the procedures be appropriate for ELLs? Yes No

Are the outcomes clearly stated? Yes No

Is there peer reviewed research that supports or contradicts

the stated purposes? Yes No

Has the author published other assessments? Yes No

Is the publishing company reputable? Yes No

If it is in Spanish, is the language accessible to the population in our

region? Yes No

Overall Critique (10 pts)

Discuss the strengths and weaknesses of the instrument including at least one published review and/or supporting research and, if possible, information from someone who has used it. How well does it measure the domains it purports to? Is it a good match to today’s curriculum? How clear are the scoring procedures? What types of scores are available? Would you want to purchase and use this test? Why or why not?


Standardized Instrument Demonstration (20 points)

Names:

Date:

4 / 3 / 2 / 1
Knowledge of the assessment / The speaker(s) demonstrated thorough knowledge of the test’s purpose and structure. Answered all questions posed by audience accurately and demonstrate expertise. / The speaker(s) demonstrated good knowledge of the test’s purpose and structure. Answered most questions posed by audience. / The speaker(s) demonstrated some knowledge of the test’s purpose and structure. Had difficulty answering questions posed by audience. / The speaker(s) demonstrated limited knowledge of the test’s purpose and structure. Cannot answer questions posed by audience.
Quality of Oral Critique / The speaker(s) addressed all items listed on the critique handout and were able to answer all questions. / The speaker(s) addressed most items listed on the critique handout and were able to answer some questions. / The speaker(s) addressed some items listed on the critique handout but were only able to answer some questions. / The speaker(s) addressed few items on the critique handout and were not able to answer questions.
Presentation Style/Audience Engagement / The speaker(s) handled unexpected questions/problems with humor, grace, and persistence, without losing focus. The audience was engaged and interested throughout the presentation. / The speaker(s) handled unexpected questions/problems with humor, grace, and persistence, for the most part. The audience was mostly engaged and interested during the presentation. / The speaker(s) were flustered by unexpected questions/problems and had difficulty getting back on track. The audience was sometimes engaged. / The speaker(s) were overcome by unexpected questions/problems. The audience was bored.
Demonstration and Scoring Procedures / The speaker administers exemplar items from the test with accuracy and fluency. Scoring thoroughly discussed and examples provided. / The speaker(s) administered exemplar items from the test but with some uncertainty. Scoring thoroughly discussed. / The speaker(s) administered exemplar items from the test with some errors and lacked fluency. Scoring discussed briefly. / The speakers did not model the assessment nor demonstrated knowledge of the test. Scoring was not discussed.
Organization of Presentation / The individual presentation was well organized with an introduction, body and conclusion. Appropriate props were used to highlight the organization. / The individual presentation was mostly well organized with only minor problems with the introduction, body and conclusion. Appropriate props were used that mostly helped the organization. / The individual presentation was somewhat organized but contained problems with the introduction, body or conclusion. Appropriate props were used to highlight the organization but they were ineffective. / The presentation rambled with little evidence of an introduction, body, or conclusions. The props used did not assist audience in following the organization of the material.

Comments:

1

Dr. Julie Esparza Brown

SPED 512/PSU