Kidder-Ashley, P., Deni, J. R., Azar, K. R. & Anderton, J. B. (1999). How 41 education agencies identify students with emotional problems. Education, 119(4), 598-609.
How 41 education agencies identify students with emotional problems
Kidder-Ashley, Pamela
Definitions and category labels for serious emotional disability (SED) were obtained from 41 state education agencies and examined to determine their congruence with the federal definition. State eligibility criteria and procedures also were obtained to determine which particular assessment procedures and criteria are prescribed and whether states attempt to operationalize terminology. The correlation between number of criteria/procedures specified and percentage of students identified also was examined. Most states' terminology and definitions are consistent with the federal definition, but assessment procedures and eligibility criteria vary considerably across states. There was a significant negative correlation between number of criteria/procedures specified and percentage of students found eligible for services, suggesting that the more flexibility allowed for in the identification process the higher the number of students identified. Implications of these findings are discussed.
The literature indicates that the category of serious emotional disabilities (SED) is the most underrepresented of all the categories of educational disability (Forness & Knitzer, 1992). This underidentification is often attributed to variations in definitions and criteria across states, as well as to the vague terminology of the federal definition, which is difficult to operationalize. The present research was designed to examine (1) the degree to which state terminology and definitions are consistent with the federal definition, (2) the range of eligibility criteria and assessment procedures most commonly used, (3) the extent to which states attempt to operationalize the terminology, and (4) the relationship between criteria and procedures employed and rate of SED identification.
The Education of the Handicapped Act of 1977 (Public Law 94-142), amended in 1981, and reauthorized in 1997 as the Individuals with Disabilities Education Act (IDEA), included the following definition of the category of Serious Emotional Disturbance (SED):
The term means a condition exhibiting one or more of the following characteristics over a long period of time and to a marked degree, which adversely affects educational performance:
A. An inability to learn which cannot be explained by intellectual, sensory, or other health factors;
B. An inability to build or maintain satisfactory interpersonal relationships with peers and teachers;
C. Inappropriate types of behavior or feelings under normal circumstances;
D. A general pervasive mood of unhappiness or depression; or
E. A tendency to develop physical symptoms or fears associated with personal or school problems.
The term includes children who are schizophrenic. The term does not include children who are socially maladjusted unless it is determined that they have a serious emotional disturbance (as cited in McConaughy & Ritter, 1995.)
Concern over the inadequacy of the federal definition of SED has been voiced virtually since the definition first came into use, due to the definition's vague language and undefined terms. Interpretation of these terms, as well as the development of specific procedures for identifying children with SED, has been left largely up to individual states. For example, phrases referring to duration ("over a long period of time") and severity ("to a marked degree") must be operationalized at a state or local level. The social maladjustment exclusionary clause has engendered an ongoing debate over how the clause should be interpreted (Shapiro, 1992.) These shortcomings, according to various experts in the field, have created "at best a great potential for inconsistency across referred children and, at worst, conditions allowing unchecked bias, inequity, and prejudice" (Knoff, 1995, p. 852).
Alternative definitions and category labels for SED have been proposed, most notably by the National Mental Health and Special Education Coalition (NMHSEC), which is comprised of about 30 professional mental health and educational associations. Based on an earlier draft by the Council for Children with Behavior Disorders, the Coalition's proposed definition is as follows:
i. The term emotional or behavioral disorder means a disability characterized by behavioral or emotional responses in school so different from appropriate age, cultural, or ethnic norms that they adversely affect educational performance. Educational performance includes academic, social, vocational, and personal skills. Such a disability
A. is more than a temporary, expected response to stressful events in the environment.
B. is consistently exhibited in two different settings, at least one of which is school-related; and
C. is unresponsive to direct intervention in general education or the child's condition is such that general education interventions would be insufficient.
ii. Emotional and behavioral disorders can co-exists with other disabilities.
iii. This category may include children or youth with schizophrenic disorders, affective disorders, anxiety disorder, or other sustained disorder of conduct or adjustment when they adversely affect educational performance in accordance with section (i) (as cited in Forness & Knitzer, 1992, p. 13.)
Another identification model was developed for use in Iowa's public schools. Iowa's approach to identifying students with "behavior disorders" is an empirically based model that was developed in part through factor analysis of data examining common characteristics of children with emotional and behavioral disturbances (Knoff, 1995). Iowa's definition reads as follows:
1. Behaviorally disordered is the inclusive term for patterns of situationally inappropriate behavior which deviate substantially from behavior appropriate to one's age and significantly interfere with the learning process, interpersonal relationships, or personal adjustment of the pupil to such an extent as to constitute a behavioral disorder.
2. Clusters of behavior characteristics of pupils who are behaviorally disordered include: Cluster I-Significantly deviant disruptive, aggressive or impulsive behaviors; Cluster Ill-Significantly deviant withdrawn or anxious behaviors; Cluster III-Significantly deviant thought processes manifested with unusual communication or behavioral patterns or both; and Cluster IV - Significantly deviant behavior patterns characterized by deficits in cognition, communication, sensory processing or social participation or a combination thereof that may be referred to as autistic behavior. A pupil's behavior pattern may fall into more than one of the above clusters.
The determination of significantly deviant behavior is the conclusion that the pupil's characteristic behavior is sufficiently distinct from his or her peer group to qualify the pupil as requiring special education programs or services on the basis of a behavioral disorder. The behavior of concern shall be observed in the school setting for school-age pupils. It must be determined that the behavioral disorder is not maintained by primary intellectual, sensory, cultural or health factors.
4. In addition to those data required within the comprehensive educational evaluation for each pupil requiring special education, the following areas of data collection shall be gathered when identifying a pupil as behaviorally disordered which describe the qualitative nature, frequency, intensity, and duration of the behavior of concern. If it is determined that any of the areas of data collection are not relevant in assessing the behavior of concern, documentation must be provided explaining the rationale for such a decision. (as cited in Knoff, 1995, pp. 854-855.)
Iowa's model stands as a good example of an approach that attempts to use the available empirical literature. It identifies four major clusters of behaviors, and facilitates a link between the assessment process and intervention efforts.
Yet another conceptual model of "behavior disorders" (BD) has been proposed by Gresham (1991). He writes that "resistance of behavior to change is the defining characteristic of BD and therefore represents the heart of a classification decision as BD" (p. 34). Gresham argues that good evidence for the classification of a student as having BD lies in how resistant to intervention that student's behaviors are. He defines resistance to intervention as "the lack of change in target behaviors as a function of intervention." Gresham's approach necessitates the identification of variables maintaining the behavior and, thus, engenders a direct link between assessment and intervention.
One reason for the numerous criticisms aimed at the federal definition is the underidentification of children in the SED category. The national rate for identification of children with SED has consistently hovered around .9% of all school children since national data were first collected for special education categories (Oswald & Coutinho, 1995). This identification rate persists despite prevalence estimates two to three times higher. As Katsiyannis, Landru, Bullock, and Vinton (1997) noted, "schools presently identify and serve less than 1% of the student population under the [SED] category, while professional estimates suggest that this figure should at least double, and may be as high as 6 to 10%" (p.131). The inadequacies of the federal definition are considered at least partly responsible for this under-identification (Forness & Knitzer, 1992.) In a study relevant to this conclusion, Tharinger, Laurent, and Best (1986) compared the percentage of children who would be identified as emotionally or behaviorally disordered under P. L. 94-142 and two other widely used classification systems. They determined that by far the lowest percentage of students would be identified under the PL 94-142 definition. The authors commented that the P. L. 94-142 system was "influenced by a variety of factors, including the local interpretation of the P. L. 94-142 handicapping conditions, acceptance by the parents, and a number of implicit organizational variables and constraints" (p.119). The authors went on to comment that research about the process involved in assigning the SED label was needed, "as it would inform us about the decision-making process and could potentially provide information about the reliability and validity of the handicapping conditions in P. L. 94-142" (p. 119).
Such research has been undertaken, though on a fairly small scale, and it sheds light on another reason for dissatisfaction with the federal SED definition-lack of consistency in SED identification rates across the states. According to U. S. Department of Education data, state identification rates in 1992-1993 ranged from a low of .04% to a high of 2.08% (as cited in Oswald & Coutinho, 1995), suggesting that equal access across states to "free and appropriate" services for SED may not be a reality. In examining this inconsistency, several studies (e.g., McLaughlin & Owings,1992; Oswald & Coutinho,1995) have found that a number of demographic and economic variables are associated with differences in state identification rates. Of particular relevance to the present study, several earlier investigations (Epstein, Cullinan, & Sabatino,1977, Cullinan, Epstein, & McLinden, 1986; Tallmadge, Gamel, Munson, & Hanley,1985, cited in Wright, Pillard, and Cleven,1990) have examined the relationship between state SED terminology and definitions and state identification rates. Reviewed by Wright et al., these studies reveal growing convergence between state and federal definitions and terminology and "increasing definitional influence on prevalence rates" (p. 19.) In their own study, Wright, et al. found that, for the year 1983-1984, 32% of the variance in prevalence rates could be accounted for by differences in states' definitions, which represented an increase of 19 percentage points over the amount accounted for when data from 1976-1977 were examined. However, the association between state SED prevalence rates and how states go about implementing the federal mandate to serve children with SED has not been examined in recent years. The current study sought to re-examine that association but in a somewhat different way from the one employed by Wright, et al. (1990) and the other studies cited above. In addition to examining state terminology and definitions for congruence with P. L. 94-142/IDEA, we chose to examine procedural guidelines as well. We sought first to identify the range of assessment procedures and eligibility criteria employed by the states and then to determine whether a state's prevalence rates are associated with the number of these procedures/criteria that are specified in a state's guidelines. We chose this approach because we thought it likely that states' efforts to operationalize federal regulations might be even more apparent in their procedural guidelines than in their formal definitions and terminology. Furthermore, Wright et al. found that differences in states' definitions alone accounted for only one-third of the variance in state prevalence rates; twothirds of the variance remained unaccounted for. We speculated that at least some of the unaccounted-for variance might stem from differing degrees of specificity in states' assessment procedures and eligibility criteria. Thus, consistent with earlier findings, we predicted that specificity in states' SED identification guidelines would be negatively correlated with the likelihood of students being identified as having SED.
Method
In the fall of 1995, each state's public education agency was contacted and asked to provide their official definition of SED as well as their criteria and procedures for determining eligibility for SED services. After follow-up letters were sent, a total of 41 states responded with the needed definitional information, but only 40 states provided the eligibility criteria and procedures.
The investigators used the P. L. 94142/IDEA terminology as a point of reference in examining the states' categorical labels and definitions. Similar to the approach used by Tallmadge, et al. (1985, as cited in Wright, et al., 1990), categorical labels were classified as focusing (1) on the emotional aspects of the disability (like the federal terminology), (2) on the behavioral aspects (like the terminology proposed by the NMHSEC), or (3) on a combination thereof (similar to the Iowa model). The federal definition was broken down into its five main components as follows: (1) inability to learn; (2) ineffective interpersonal relationships; (3) inappropriate behavior or feelings; (4) unhappiness/depression; and (5) physical symptoms or fears. The individual state definitions were then examined with respect to how many of the five components they shared with the federal definition and were classified as either equivalent (containing all of the five components), modified (omitting one of the components), or different (omitting two or more of the components). Because the federal definition also includes children who have schizophrenia but excludes children who are socially maladjusted (unless they also have an emotional disability), state definitions also were checked for specific reference to those populations.
The investigators also examined the states' procedural guidelines to determine which particular assessment procedures and eligibility criteria are prescribed. Finally, a Pearson product moment correlational analysis was conducted to determine if there is a relationship between the number of procedures or criteria specified and the percentage of all students identified as having SED. An alpha level of .05 was used for this analysis. (Data regarding the percentage of a state's total student population who are identified as having SED were obtained from U.S. Department of Education statistics reported by Ysseldyke and Algozzine in 1995.)
Results
As indicated earlier, 40 states provided all requested information regarding terminology, definitions, and guidelines regarding eligibility criteria and assessment procedures. One state provided only terminology and definitional information. Thus, the percentages reported in this section reflect this discrepancy.
Of the 41 available state definitions of SED, 36 (88%) are equivalent to the federal definition, containing all five components. Two states' definitions (5%) are modified, comprised of four of the components; one of these omits the interpersonal relationships component, one omits the physical symptoms/fears component. Only three states (7%) were classified as different, omitting two or more components. The only component found to be included without exception in all state definitions of SED was the first one-the inability to learn.
Regarding the specific inclusion of children with schizophrenia, 22 (54%) of the states' definitions do specify their inclusion. Similarly, 28 (68%) of the states specifically exclude children who are socially maladjusted. Interestingly, only one of those states provides an operationalized definition of social maladjustment.
The categorical labels assigned to the disability are quite varied, with fifteen different terms used across the 41 responding states. The majority of the states' labels (25, 61%) refer only to the emotional aspect of the disability, with 14 (34%) matching the federal term of SED; three of these states use the term emotionally disturbed; one uses the term socially and emotionally maladjusted. Six states' terms (15%) refer to the disability as primarily behavioral, with behavior disorders being the most common term (employed by 5 of these 6 states). Ten states' terms (24%) refer to both emotional and behavioral problems; one-half of these use the term emotional and behavioral disorder.
The criteria and assessment procedures that are revealed in the states' procedural guidelines are listed in Table 1. As indicated in the table, most states specify that the behavior of concern must have been exhibited "for a long time;" however, only six states (15%) provide guidelines for what constitutes "a long time." For example, three states suggest that the behavior of concern should have been exhibited for at least six months, with one of those states providing an alternative to this recommendation by stating that the behavior "results from a crisis." Of the remaining three states, one recommends a duration of three months, and another specifies that the behavior be evidenced for two to nine months, depending on the age of the child and the type of behavior occurring. Although the remaining state recommends the longest duration-a minimum of one year-this criterion may be waived if there is a risk that the behavior will extend for a long time or there is a pattern of severe behavior of short duration.
Another aspect of the federal definition, "behavior exhibited to a marked degree," was found to be present in most of the states' documents, as noted in Table 1. However, none of the states attempts to operationalize this portion of the definition by specifying how "a marked degree" should be documented.
Two other criteria were observed in state documents-pervasiveness and a minimum age for identification. One-fourth of the states require that the behavior of concern must be evident in more than one setting. As indicated in Table 1, only three states specify a minimum age. Each of those states sets a different minimum age for identification; respectively, they require that a child may not be identified as having SED unless he or she has reached the age of three, four, or five years of age.
The assessment procedures stipulated in the state documents can be categorized into three general assessment domains. The first encompasses evaluation of academic and general classroom functioning, including response to interventions attempted. It was observed that many states require an educational evaluation, which is described by some of the states as comprised of a review of classroom performance data, possible achievement testing, an examination of grades, and/or assessment of academic progress. Most states stipulate in their regulations that such an evaluation must occur, whereas only 13 states require formal standardized achievement testing, and a few consider such testing optional. Incidentally, it was noted that although the majority of states consider educational performance an important factor to consider in determining SED eligibility, none of the responding states reported the use of any sort of eligibility formula such as those used in determining the presence of a learning disability.