Review and Evaluation Study of the California Community Colleges

University of Georgia:

Review and Evaluation Study of the California Community Colleges

Learning Disabilities Eligibility Model

Final Report

November 2009

Principal Investigator:

Noel Gregg, Ph.D.

Research Partners:

University of Georgia

College of Education

Research Office

University of Georgia

Educational Policy and Evaluation Center

College of Education

University of Georgia

Regents’ Center for Learning Disorders

Psychology Department

Franklin College of Arts & Sciences

Executive Summary

The primary purpose of this study was to investigate the validity of the California Community College (CCC) Learning Disabilities Eligibility Model (LDEM) in the context of accountability. Investigation of both the products available and the processes used for identifying individuals suspected of learning disabilities (LD) within the CCC System was the focus of the study. The product evidence was drawn from an investigation of the LDEM eligibility components, and the process evidence was provided by integrating data pertaining to the entire context of the LDEM program. Five individual studies were generated to supplement the information provided by the Chancellor’s Office specific to the Learning Disabilities Eligibility Model (LDEM).

Results: The LDEM provides a valid and reliable means of documenting LD services for the CCC system as it provides a standardized process for LD identification. Both internal and external validity evidence supports this claim. Despite strong reliability data for the process of identification, and internal and external validity evidence, several recommendations are provided for enhancing the LDEM.

Recommendations

· A standardized process for collecting and storing intake data be established

· Evaluators be trained in and practice more selectivity for the instruments chosen to identify if an individual is eligible for LD services.

· The discrepancy formula used in either the processing or ability/achievement component of the LDEM should be investigated in greater depth.

· A written report be provided to a student at the conclusion of the assessment process.

· Current best practice related to psychological assessments for individuals with disabilities, not just LD, be incorporated into on-line training modules for all LD specialists. A more comprehensive understanding of assessment issues pertaining to other disabilities might help professionals struggling with issues of co-morbidity during the assessment process.

· Greater flexibility be built into the LD certification for licensed psychologists.

· Increased training related to best practice for the assessment and accommodation of English Language Learners (ELL) and culturally diverse populations be provided.

· Greater information on best practice for documenting accommodations, particularly alternative media formats and assistive technologies be provided.

· A more standardized curriculum for the LD Assessment Classes be developed.

· Further analysis of the cost benefits should be explored from an empirical basis.

Table of Contents

Executive Summary 2

Introduction 7

Goals of Evaluation Project

Timeline

Data Sources

Summary of Evaluation Goals 8

Goal One: Policy 8

Definition 8

Postsecondary Documentation Guidelines 10

Survey and Structured Interview Data 11

Stakeholders 13

Table 1: Stakeholders 13

Profile of Professional Participants 14

Table 2: Professional Participants 14

Profile of Students 15

Table 3: Demographics of Student Respondents 16

Figure 1: Initial diagnosis of LD 18

Faculty Policy Survey Questions 18

Table 4: Equitable services 18

Table 5: Faculty Awareness of Policy Pertaining to LD 19

Goal Two: Validity of LDEM 19

CARS-W Outcome Data (Internal Validity) 20

Decision-Making in Component One (Intake) 20

Decision-Making in Component Two (Measured Achievement) 20

Decision-Making in Component Three (Ability Level) 21

Decision-Making in Component Four (Processing Deficit) 21

Decision-Making in Component Five (Aptitude-Achievement

Discrepancy) 22

Decision-Making in Component Six (Eligibility Recommendation)

CARS-W Investigation of Three Eligibility Models (External Validity) 23

Questionnaire and Structured Interview Data 25

Over and Under Diagnosis of LD 25

Figure 2: Over and Under Diagnosis 26

Validity of LDEM Based on Constituent Feedback 26

Table 6: Weights Assigned to each Component for

Eligibility Decisions 27

Table 7: LD Specialists and DSPS Weights 28

Perceptions across Eligibility Models 29

Table 8 – Selection Of Valid LD Eligibility Models 30

Goal Three: Constituent Perceptions 31

Perception of the LDEM’s Sensitivity to Diversity 31

Figure 3: LDEM and Sensitivity to Diversity 31

Quality and Usefulness of the Evaluation Report 33

Table 9: Perceptions of the Written Report 32

Table 10: Usefulness of the Evaluation Report to Access

Accommodations 33

Table 11: Usefulness of the Evaluation Report to Access Services

Table 12: Student Perceptions of the Evaluation Report 34

Figure 4: Time between Enrolling in CCC and Accessing Evaluation

Goal Four: Training 36

Figure 5: Professional Training for LDEM 37

Goal Five: Benefits of the LDEM 38

Validity of the Model 38

Transfer from CCC to CSU and UC

Transitioning Secondary Students 38

Benefits for Students Accessing Accommodations 40

Figure 6: LDEM and Accommodations 40

Table 13: Students Use of Accommodations 40

Table 14: Students Most Frequently Used Accommodations 41

Table 15: Acceptance and Use of Accommodations 42

Cost Benefit of LDEM (Structured Interviews) 43

Proposed Changes to LDEM (Structure Interviews) 45

LD Specialists 45

DSPS 47

Administration 49

Summary

Recommendations

References

Introduction

The following five goals provided the framework for this study. Specific objectives, data sources, and methodologies to evaluate each goal are included in Appendix A.

GOAL 1.0:

Investigate current policy governing the provision of services to students with LD transitioning to and /or attending the CCC System.

GOAL 2.0:

Investigate the validity evidence for the LDEM.

GOAL 3.0:

Investigate constituent perceptions as to the effectiveness of the LDEM in serving postsecondary students with LD within CCC system.

GOAL 4.0:

Investigate the effectiveness of the training program for LD specialists

participating in the LDEM process.

GOAL 5.0:

Investigate the benefits of the LDEM to the CCC System

Timeline

The project contract began on January 1, 2009 and ended on June 30, 2009. However, funds for the project operations were not released until April 2009. The first draft of the final report to the California Community College Chancellor’s Office (CCCCO) was delivered September 15, 2009. Feedback was provided to the Learning Disabilities Advisory Committee September 2009, and at the California Association of Postsecondary Education (CAPED) in October 2009. The final report was delivered November, 2009 to the CCCCO.

Data Sources

Five individual studies were generated to supplement the information provided by the Chancellor’s Office specific to the Learning Disabilities Eligibility Model (LDEM). Therefore, the following data sources were used to evaluate the validity of the LDEM:

· Policy from the California State Universities (CSU) and the University of California (UC) websites related to disability documentation criteria

· Policy from the California Department of Education governing students in grades K-12 related to disability criteria

· On-line questionnaire study (see Appendix A)

· Structured interview study (see Appendix A)

· Internal Validity Study (see Appendix B)

· External Validity Study (see Appendix C)

· Learning Disabilities Eligibility Model: Introduction and Overview (2002 Revision)

· Learning Disabilities Model Development Study: Technical Report (June, 1993)

· Listings from the LD Specialist, DSP, and Technology list serves

· History of LDEM (Mellard & Byrne, 1993; Mellard, 1990)

SUMMARY OF EVALUATION GOALS

GOAL One: Investigate policy governing the provision of services to students with LD transitioning to and /or attending the CCC system.

Definition of Learning Disabilities

The definition of disability in the Americans with Disabilities Act of1990 (ADA) was imported directly from Section 504 of the Rehabilitation Act of 1973. Entitlement to accommodations requires individuals to demonstrate proof that they are qualified and sufficiently limited to be considered "disabled." Under ADA, a disability is defined as "...a substantial limitation to one or more major life activities of that person." In turn, "substantial limitation" is defined as an inability or a significant restriction in the condition, manner, or duration in which one performs a major life activity "as compared to most people" (28 C.F.R. Pt. 35, App. A § 35.104; italics added for emphasis). In addition, to this broad definition of disability, the CCC system and the CSU include a definition of LD in their policies. The University of California appears to provide such a definition on individual university web sites.

The CCC system definition of LD includes the following significant criteria: (1) average to above average intellectual ability; (2) severe processing deficits (s); (3) severe aptitude-achievement discrepancy (ies); and (4) measured achievement in an instructional or employment setting (see Appendix C). The California State Universities adhere to the National Joint Committee on Learning Disabilities (2001) which includes the following criteria: (1) significant difficulties in achievement; (2) processing deficits; (3) lifespan condition; and (4) concomitant disabilities can occur.

A variety of government codes and regulations, both state and federal, govern whether a child or adolescent is eligible for special education in California. The federal Individuals with Disabilities Education Act (IDEA) allows qualified disabled children access to “free appropriate public education,” including an individualized instruction plan for each child’s needs (20 U.S.C. §§ 1400(c).). To qualify for this protection under the California Education Code, an “individual with exceptional needs” must have an impairment that “requires instruction, services, or both which cannot be provided with modification of the regular school program” (Calif.Educ.Code 56026(b).).

Generally, a child has “exceptional needs” if: 1) the child has a disability identified in Section 1401(3)(A) of Title 20 of the United States Code; 2) the impairment requires instruction and services that cannot be provided with modification of the regular school program; 3) the child falls within certain age categories (see 56026(b) – which includes those younger than 5 with certain restrictions, those between five and 18 years inclusive, and those between 19 and 21 enrolled in an eligible program prior to their 19th birthday who have not completed a prescribed course of study or met proficiency standards or graduated from high school.); and 4) the child meets eligibility criteria set forth in the State Board of Education regulations (Calif.Educ.Code 56026.). Students are not eligible if their needs are due only to limited English, lack of instruction, cultural or economic factors, or temporary physical disabilities, among other things. (Calif.Educ.Code 56026; see also, Calif.Educ.Code 56026.1 [“an individual with exceptional needs who graduates from high school with a regular high school diploma is no longer eligible for special education and related services.”]).

In the case of LD, both 20 U.S.C. 1401(3)(A) and California Education Code section 56337 define a child with a disability as one who has a “specific learning disability,” which means “a disorder in one or more of the basic psychological processes involved in understanding or in using language, spoken or written, which may manifest itself in the imperfect ability to listen, think, speak, read, write, spell, or perform mathematical calculations.” (Calif.Educ.Code 56337, 20 U.S.C. 1401(3)(A), 20 U.S.C. 1401(30)(A).). It includes conditions such as brain injury, dyslexia, and developmental aphasia. But it does not include conditions that are the result of visual or hearing disabilities, mental retardation, or environmental factors, among other things. Calif.Educ.Code 56337, 20 U.S.C. 1401(30)(B), 20 U.S.C. 1401(30)(C).

The California Department of Education regulations spell out the specific requirements for determining eligibility for state programs. (see, eg. 5 CCR § 3030.). California Administrative Code 3030 states, among other things, that a learning-disabled pupil is eligible for special education 1) if he or she has a disorder in basic psychological processing involved in using or understanding language, and 2) there exists a severe discrepancy between intellectual ability and achievement in at least one of the academic areas specified in California Education Code 56337(a). (5 CCR § 3030(j).). Intellectual ability is defined as both acquired learning and learning potential, and level of achievement shall be measured by standardized achievement tests. (Id.) The regulation also provides that an individualized education team including personnel under section 56341(d) shall make the decision that a student’s impairment requires special education (5 CCR § 3030).

California Education Code 56337.5 deals with dyslexia specifically. It provides that a pupil is entitled to special education and related services if he or she is assessed as being dyslexic, meets the criteria specified in section 56337, supra, and meets the eligibility criteria of the state regulations mentioned above (5 CCR § 3030.). (Calif.Educ.Code 56337.5.).

Other related definitions and information related to children with disabilities can be found at: Cal.Educ.Code 56031 (definition of special education); Cal.Educ.Code 56032 (definition of individualized education program); California Education Code 8208 (Childcare Development and Services Act Definitions); Cal.Educ.Code 56441.11 (Special Education Eligibility Criteria for Preschool Children).

IDEA (2004) states that discrepancy alone cannot define LD (see Gregg, 2009). In addition, IDEA 04 requires states to provide a procedure within the evaluation process that is based on a response to intervention (RTI: 300.307 (a) (2). In the state of California, the Response to Instruction and Intervention (RtI²) policy provides the basis for understanding a systematic, data-driven approach to instruction. With the significant cuts in the California state budgets, it is difficult to determine at this time what resources will be available for RtI² and other assessment policy governing secondary students. Secondary school personnel in California are not required to provide a transition evaluation. Therefore, many students transitioning from secondary to postsecondary institutions will not have evaluation documentation sufficient to meet the standards set by any of the California institutions of higher education (i.e., CCC, CSU, or UC).

Postsecondary Documentation Guidelines

In an attempt to provide a valid and reliable means by which to operationalize the definition of LD at postsecondary institutions, Gregg (2009), citing best practice, suggests that the eight questions provided in Appendix C be addressed. These eight questions address the three constructs underlying the Americans with Disabilities Act (ADA) definition of disability: duration; condition; and manner in which one performs the activity of learning. The CCC LDEM, the CSU, and UC policies for LD eligibility suggest that they utilize the majority of these eight points when documenting LD services or accommodations.