1

Comparison of ASQ and PEDS

Comparison of the ASQ and PEDS in Screening for Developmental Delay in Primary Care Settings

Marjolaine M. Limbos, PhD, R. Psych, Department of Family Practice, University of British ColumbiaVancouver, British Columbia & David P. Joyce, MD, CCFP, Department of Family Practice, University of British Columbia, Vancouver, British Columbia.

Abbreviations:

AAP = AmericanAcademy of Pediatrics

ASQ = Ages and Stages Questionnaire

PEDS = Parents’ Evaluation of Developmental Status

Short Title: Comparison of ASQ and PEDS

Key Words: Developmental delay, screening, primary care, well-child visit

Financial disclosure: None

Conflicts of interest: None

Address correspondence to:

Dr. Marjolaine M. Limbos, PhD, R. Psych

4428 West 2nd Ave.

Vancouver, BC

V6R 1K5

Phone: (604) 346-6505

Fax: (604) 221-0826

Email:

Contributor’s Statement Page

Dr. Limbos and Dr. Joyce equally contributed to the conception, design, acquisition of data and data analysis and interpretation. Dr. Joyce made major contributions to the recruitment of primary care providers and participants. Dr. Limbos was most responsible for data analysis. The initial draft of the article was made by Dr. Limbos drawing heavily on prior works (e.g., ethics proposals, etc.) that were drafted by both authors. Dr. Joyce made substantial revision and input on the final manuscript. Both authors approved the final version of the manuscript.

What’s Known, What’s New

What’s Known on this Subject:

While early intervention can improve outcomes in children with developmental delay, many physicians do not screen using standardized tools like the PEDS and ASQ. There has been little research on the PEDS and ASQ in primary care settings.

What’s This Study Adds:

Both the ASQ and PEDS had moderate sensitivity and acceptable specificity for screening for developmental delay in primary care settings. While the accuracy of the ASQ was higher than the PEDS, the choice should depend on the practice setting and patient characteristics.

ABSTRACT

Objectives

The purpose of this study was to investigate the sensitivity and specificity of two brief, parent-completed developmental screening measures, the Ages and Stages Questionnaire (ASQ) and the Parents Evaluation of Developmental Status (PEDS) in children presenting to their primary care provider.

Method

A sample of 334 children aged 12-60 months was recruited. Parents completed the ASQ and the PEDS in their home. The presence of ≥1 predictive concerns or abnormal domains was considered a positive screen. All children underwent evaluation with the criterion measures, administered by a psychologist: the Bayley Scales of Infant Development or the Wechsler Preschool and Primary Scale of Intelligence, the Preschool Language Scale and the Vineland II.

Results

The mean age of children was 32.4 months. Developmental delay was identified in 34 children (10%). The PEDS had moderate sensitivity (74%) and reasonable specificity (64%). In comparison, the ASQ had significantly higher sensitivity (82%) and specificity (78%). Using ≥2 predictive concerns on the PEDS or ≥2 abnormal domains on the ASQ significantly improved specificity of both tests (89% and 94%, respectively) but resulted in very low sensitivity (41% and 47%, respectively).

Conclusions

These findings support the guidelines of the AmericanAcademy of Pediatrics, demonstrating that both the PEDS and ASQ have reasonable test characteristics for developmental screening in primary care settings. Although the ASQ appears to have higher sensitivity and specificity, the choice of which measure to use should be determined by the practice setting, population served and preference of the physician.

INTRODUCTION

Developmental delays are common affecting up to 15% of children.1,2 Recent evidence demonstrating the long term benefits of early intervention has sparked renewed focus on the importance of developmental screening in primary care settings.3-10 This is reflected in recent guidelines by the American Academy of Pediatrics (AAP) and other authorities who recommend that physicians institute a program of regular developmental screening for young children using standardized tools.11-13 Despite these recommendations, there remains a significant disparity between guidelines and actual practice. Indeed, surveys have demonstrated that a minority of physicians perform routine screening using standardized tools and many children with developmental problems are not identified until they reach school age.14-20 While there are many possible contributing factors, the lack of screening by physicians may be in part due to inadequate time or remuneration, as well as the paucity of research that has been conducted in primary care settings.18, 20-23

The Parents’ Evaluation of Developmental Status (PEDS) and Age and Stages Questionnaire (ASQ) are parent-completed developmental screening tools that are increasingly being recommended for use in clinical practice.11,18, 21, 23-26 Their popularity is in large part due to several favorable qualities that make them suitable for incorporation into busy practices including administration by parents, thus requiring little physician time; ease of administration and interpretation; and low cost.27,28 While there is growing research demonstrating the accuracy of both tests in research settings, generalization of the findings to primary care settings is limited by several factors including: (1) a scarcity of studies in primary care settings;29-33 (2) conflicting results for those studies that have used primary care samples;21,23 (3) failure to administer an adequate criterion measure;21,23 and (5) failure to compare various screening tools on the same sample.34,35

The current study set out to compare the accuracy of the ASQ and the PEDS in preschool children presenting to their primary care provider for routine care. The goal was to examine the performance of these tools in the setting for which they are being recommended for use, namely primary care, using a large unselected sample, and administering both screening tools and a clinically relevant criterion measure to all children.

PATIENTS AND METHODS

Study Procedures

A convenience sample of children aged 12-60 months who presented to their primary care provider for routine care were considered eligible for participation. Children were recruited by receptionists using a written script. Parents agreeing to participate were given an appointment to meet in their homes or at the primary care clinic of one of the investigators (DPJ). Recruitment began in December of 2007 and was completed in September of 2008.

On the assessment day, a registered psychologist administered the criterion measures to the child, while parents completed the ASQ and the PEDS in a separate room. Parents and the psychologist were blinded to the results of each others’ testing. The instructions manuals for the ASQ and PEDS were followed for administration and scoring.29,34 Parents were provided with items required to complete the ASQ. Order of administration of the screening tools was alternated in the research packages at the onset of the study.

The PEDS and ASQ were scored by a family physician (DPJ) and the psychologist independently scored all of the criterion measures. Each was blinded to the results of the other’s scoring. A summary of the results of the testing was mailed to the child’s primary care provider with instructions for follow up.

Information on medical and developmental history, ethnic and socioeconomic status were obtained by review of the child’s medical record and a parent-completed questionnaire.

Participants

A total of 462 children were approached to participate in the study. Children were excluded if there was a prior history of: (1) developmental delay or disability; (2) psychiatric disorder; or (3) birth < 36 weeks gestation. All parents were fluent in English.

Children were recruited from the offices of 80 primary care providers in Ontario, Canada. Seventy were family physicians, 7 were nurse practitioners and 3 were pediatricians.

Measures

Developmental Screening Measures

ASQ.30,33 The ASQ is a brief (15 minute) measure, in which parents rate their child on 30 questions covering 5 domains of development including: communication, gross and fine motor, problem-solving, and adaptive skills. A pass/fail score is assigned for each area of development. The form closest to the child’s chronological age was used. The presence of any failed domain of the ASQ was considered a positive screen. The ASQ has been validated in large standardization samples of children.21,27,30,33

PEDS.29,32 The PEDS is a brief, validated, one page developmental screening tool designed for children aged birth to 8 years of age. The tool elicits parent concerns on 10 items across nine domains of development or behavior and takes 2-5 minutes to complete. Response options include yes, no or a little. The presence of one or more predictive/significant concern was considered a positive screen. The PEDS has a reported moderate sensitivity and specificity and has been shown to perform well in comparison with other developmental screening tests.11,21,28,29,37

Criterion Measures

The criterion measures for developmental delay in children consisted of a battery of psychological tests of adaptive, cognitive, developmental and language functioning. This battery of measures is used widely in clinical practice and in research on screening for developmental delay.38, 39 Criterion measures were administered in English or French based on the language the child spoke most often at home.

Bayley Scales of Infant Development – Third Edition (BSID-III).40 The BSID-III is a measure of development for children aged 1 to 42 months. There are 5 subscales: Cognitive, Language, Motor, Social-Emotional, and Adaptive Behavior. To avoid overlap with other tests, language and adaptive behavior scales were not administered. The BSID-III has excellent reliability and validity, and is considered by many to be one of the best measures of infant development.38-41 It will serve as the primary measure of developmental, cognitive, and motor functioning, in children 12 to 30 months of age.

Wechsler Preschool and Primary Scale of Intelligence Third Edition (WPPSI-III).42 The WPPSI-III is one of the most widely used tests of cognitive functioning for children above 2 ½ years. It provides composite scores of specific domains of intellectual functioning: Verbal Intelligence Quotient (IQ) and Performance IQ, as well as a Full Scale IQ which represents a child’s overall intellectual ability. It has several decades of research to support the clinical utility of the scale, and its correlation with a large number of other scales of intelligence and development supports its validity.42-45

Vineland Adaptive Behavior Scale-2nd Edition (VABS-II).46 The VABS-II is a 297-item caregiver interview that assesses the adaptive functioning of children between the ages of birth and 19 years. Adaptive behavior is measured in four domains: Communication, Daily Living Skills, Socialization and Motor Skills. An overall Adaptive Behavior Composite is calculated. The VABS-II was standardized using 3,000 individuals, has demonstrated reliability and validity and has been used repeatedly to validate developmental screening measures.45 - 47

Preschool Language Scale – 4th Edition (PLS-IV).48, 49 The PLS-IV is a diagnostic instrument for receptive and expressive language disorders in children up to 6 years of age. The scale has been standardized and validated, has demonstrated internal consistency and inter-rater reliability and is used extensively in clinical and research settings to examine language development in preschool children.48, 49

Definition of Developmental Delay

A child was classified as having a developmental delay if he/she scored below the 10th percentile on any of the criterion measures (i.e., Cognitive, Motor, or Social-Emotional Scale Composite Scores of the BSID-III; Verbal IQ, Performance IQ or Full Scale IQ scores on the WPPSI-III; or Auditory Comprehension, Expressive Communication or Total Language on the PLS-IV) and had a concurrent score below the 10th percentile on the Adaptive Behavior Composite score of the VABS-II. This criterion is consistent with standards used in research and clinical practice for the identification of developmental and intellectual disability.38, 39,44, 47, 50-52

Data Analysis

Differences in characteristics were examined using t tests for continuous and χ2 tests for categorical variables. Contingency tables were constructed using the number who screened positive or negative on the ASQ and PEDS in the rows and the presence of developmental delay in the columns. From these tables, sensitivity and specificity were calculated, expressed as percentages with 95% confidence intervals. Values below 70% were classified as low, 70-85% as moderate and >85% as high.53 Matched sample tables were made comparing the PEDS and ASQ results for children with and without developmental delay. McNemar’s χ2 test with Yates correction were calculated for each matched sample.54 Cohen’s κ was used to determine level of agreement between screening tests. The DAG_Stat spreadsheet was used for statistical analyses.55 Results were considered significant at p <0.05.

A sample size calculation was performed using a variation of the Log Odds Ratio test. The sample size required to confirm a difference of 10% in sensitivity or specificity was 210 participants.

RESULTS

A total of 334 children aged 12-60 months (mean 32.4 months) participated in the study. This represented 72% of children who were initially approached to participate. Reasons for exclusion included: (1) parents not interested in the study (n=58); (2) inability to attend assessment (n=32); (3) outside of age range for inclusion (n =24); or (4) prior diagnosis of developmental delay (n =14). Complete data sets were available for 331 children for the PEDS and 334 children for the ASQ. Table 1 summarizes the remainder of the demographic variables. Children identified with developmental delay were significantly more like to be male gender and come from lower income families.

Children with Developmental Delay

Thirty-four children (10%) were identified as having a developmental disability using the criterion measure. The primary cause of delay was deemed to be cognitive in 19 children, speech language in 12, and motor 3. More than one developmental delay was present in several children. Of those with cognitive delay, 1 had concurrent speech language delay, and 3 had concurrent motor and speech language delays. One child with motor delay had concurrent speech language delay

Agreement between ASQ and PEDS

To understand the test characteristics of the PEDS and ASQ, the level of agreement in classifying children with and without developmental delays was examined. Two by two matched sample tables for the ASQ and PEDS are shown in Table 2 for children with and without developmentally delay. As can be seen, overall there was agreement in classification of 206 children (63%). More specifically, there was agreement in classification of 25 (74%) of children with developmental delay (κ=0.38, fair agreement) and 181 (62%) of those without developmental delay (κ=0.02, slight agreement).

Sensitivity and Specificity of the ASQ and PEDS

Table 3 summarizes the results on the accuracy of the ASQ and PEDS. The ASQ had moderate sensitivity (82.3%) and specificity (77%) in screening for developmental delay. The PEDS had moderate sensitivity (74%) but low specificity (64%). There was a significant difference in both the sensitivity and specificity of the ASQ and PEDS. Significant differences were found in the sensitivity and specificity of the following test pairs: PEDS (≥1 Predictive Concern) and ASQ (≥1 Failed Domain); PEDS (≥1 Predictive Concern) and PEDS (≥2 Predictive Concerns); ASQ (≥1 Failed Domain) and ASQ (≥2 Failed Domains).

Sensitivity and Specificity Using 2 or More Abnormal Domains or Predictive Concerns

Because some clinicians do not refer children for assessment unless 2 or more abnormal domains or predictive concerns exist, the test characteristics of the PEDS and ASQ using these alternate cutoffs were examined.20 As can be seen in Table 3, use of this cut off significantly improved specificity, but sensitivity fell to below acceptable levels for both tests. There was a significant difference between the sensitivity and specificity using the ≥ 2 compared to the ≥1 abnormal cut-offfor each test. However, there was no significant difference between the sensitivity or the specificity of the ASQ compared to the PEDS using this cut off.

DISCUSSION

This study adds to the growing literature supporting the use of the ASQ and PEDS for developmental screening in primary care. While past research has demonstrated the ease of use, acceptability, and validity of these brief screening measures individually, the current study directly compares the sensitivity and specificity of the ASQ and PEDS in primary care settings.23-33 The main results of this study, that both the ASQ and PEDS have moderate sensitivity (82% and 74%, respectively) accompanied by reasonable specificity (78% and 64%, respectively) for screening for developmental delays, are consistent with findings of past research on standardization samples.11, 27, 29, 30, 332 The findings strongly support the use of either test for systematic developmental screening, in keeping with the AAP’s policy on early detection of children.11

Our results suggest that the ASQ may have superior sensitivity and specificity to the PEDS. Indeed the higher sensitivity and specificity of the ASQ was not only statistically significant, but also appears to be clinically significant. Of particular concern was the finding that the specificity of the PEDS fell below 70%, the usual cut-off for an acceptable screening measure.9, 35, 47, 53 Past research on the accuracy of these measures is variable but consistent with our findings, showing a trend toward higher sensitivity and specificity of the ASQ as compared to the PEDS.11, 21, 29-32 Despite this, it is important to consider other test factors that will affect actual use by physicians in clinical practice. Indeed, some studies have suggested that the PEDS may have some advantages over the ASQ, primarily related to shorter administration time and slightly lower cost.28 Thus, we conclude that while the ASQ appears to have the optimal combination of sensitivity and specificity, both the ASQ and PEDS are acceptable for developmental screening.

To date there has been little research directly comparing the accuracies of developmental screening tools in primary care settings. Rydz and colleagues studied a sample of 248 children presenting for well child visits, and assigned children to receive either the ASQ or the Child Development Inventory (CDI).23 Another developmental inventory was used as the criterion measure and was only administered to those who tested positive on the screening test. Their findings suggested that both measures had low sensitivity and only the CDI had acceptable specificity. The interpretation of these findings is significantly limited by the failure to administer the criterion measure to all children (i.e., making determination sensitivity and specificity problematic), use of a single inventory as the criterion measure, and having a significant lag time between the administration of the screening test and criterion measure.