Norms and Standardization of Child and Parent Report Questionnaires and Researcher’s Report Form
Child Report Questionnaire
Distribution of scores
Analyses were carried out to norm and standardize 311 questionnaires completed by children and adolescents who stutter (CWS). As described in the article, the questionnaires were based on 1 which was modified for assessment of CWS (a copy of the Child Report Questionnaire is included as supplementary material). Reponses were obtained from 252 boys and 59 girls. The range of ages of the CWS was from 8 years 0 months to 19 years 7 months (mean was 13 years 1 month and sd was 2 years 6 months).
The data were approximately normally distributed, as seen in Figure 1 which shows raw scores and a normal distribution fitted to those scores. The distribution of the raw scores was tested for skewness and kurtosis. The skewness statistic provides information about the symmetry of the distribution and the kurtosis statistic provides information about the peakiness of the distribution. If the distribution was perfectly normal a skewness and kurtosis value of 0 would be obtained. The skewness and kurtosis values of the raw scores on the Child Report Questionnaire were converted to z scores and compared against the values that would be expected by chance alone. Values above 1.96 (p < .05) were considered to be significantly different from chance. Neither skewness (z = 1.239) nor kurtosis (z = -1.201) was significant. Statistics about the distribution of the scores are given in Table 1. The statistics show the distribution of the data showed was fitted reasonably well by a normal distribution.
Figure 1. Histogram showing distribution of scores on the Child Report Questionnaire with normal distribution curve fitted.
Table 1. Descriptive statistics for total scores on the Child Report Questionnaire.
Mean / 19.17Median / 19.00
Std. deviation / 5.306
Skewness / 0.171
Std. error of skewness / 0.138
Kurtosis / -0.398
Std. error of kurtosis / 0.276
Construct reliability (internal consistency of the questionnaire)
With regard to questionnaires in general, construct reliability refers to the extent to which the instrument is an effective measure, whilst validity is the extent to which the questionnaire is measuring what it purports to measure. If a questionnaire is unreliable it is impossible for it to be valid; on the other hand it is possible for a questionnaire to be reliable but invalid. Therefore, we report on reliability followed by validity. Cronbach's alpha statistic was used as an indication of how well the set of items (or questions) from the questionnaire measured a single unidimensional latent construct. 2 When the data do not measure a single dimension (they are multidimensional), Cronbach's alpha is low. An alpha value of 0.8 or higher was used here as a measure to see if the psychometric instrument developed was useful. 3 The Cronbach alpha coefficient for the Child Report Questionnaire was 0.81. This indicates high internal consistency of the items, so the Questionnaire is reliable.
Test-retest reliability
Next reliability across test occasions (test-retest reliability) was assessed. Fifty-one of the participants were tested twice (the median test-retest interval was 13 days). The correlation coefficient of the two sets of scores was .74 which indicates test-retest reliability is acceptable. 4 Minimum score, maximum score, mean and standard deviation are given separately for the two test occasions in Table 2.
Table 2. Descriptive statistics for total scores on two test occasions for the Child Report Questionnaire.
N / Minimum / Maximum / Mean / Std. deviationTime 1 / 51 / 10 / 32 / 18.98 / 4.860
Time 2 / 51 / 9 / 28 / 18.76 / 4.348
Internal (construct) validity
Internal (construct) validity was assessed by examining the structure of the test. This was done by looking at the relationship between responses to different test items. An unrotated principal axis factor was used to do this and the analysis supported the interpretation that the test measured two factors. The principal factor accounted for 44.9% of variance and all items loaded at .45 or above on this factor. The second factor accounted for a further 13.6% of the variance.
The identity of the factors was established by rotating the factors. In this analysis, it was apparent that items 1, 2, 7 and 8 loaded heavily on the first factor (.6 and above) and items 3, 4, 5, and 6 loaded heavily on the second factor (again .6 and above). Examination of the Child Report Questionnaires showed that the items loading on the first factor were all related to how the child currently felt (for example, “How would you currently rate your speech”). The items that loaded on the second factor were all related to comparative assessments (for example, “How much are you stuttering/stammering now, compared to before you first saw your speech therapist”). The interpretation of both factors is in terms of judgments either about current stuttering alone or how this related to their stuttering in the past. The questionnaire that measures both factors provides a valid measure of stuttering over time.
External (criterion) validity
Criterion validity is a measure of how well set of variables predict an outcome based on information from other related variables. To assess criterion validity, the raw total score from the Child Report Questionnaire and stuttering severity (SSI-3) scores 5 for the corresponding child for 183 of the children were available. The Child Report Questionnaire showed a high correlation with the SSI-3 scores (r = .66). A linear regression model showed that the raw score from the Child Report Questionnaire was a significant predictor of the child’s SSI-3 score (the regression model accounted for 43.1% of the variance in SSI-3 scores). ANOVA indicated that the amount of variance accounted for was significantly above zero (the null expectation) F(1,180) = 138.11, p <.001).
Standardization
Total raw scores from the 311 questionnaires used for the criterion validity analysis were standardized to t scores with a mean of 50 and an sd of 10. Scores below 21 on the Child Report Questionnaire were considered fluent. This was based on analyses of groups of 35 fluent speakers aged 8-10 and 19 aged 12 plus who had SSI-3 scores up to, but not above 20.6 An SSI-3 score of 21 is at the 41st percentile of SSI-3, and SSI-3 scores of 21 and below represent mild or very mild stuttering. 5 When the 183 questionnaires from respondents where we also had SSI-3 scores were examined, 71 out of 87 (81.6%) low scores on the Child Report Questionnaire related to children whose SSI-3 scores were lower than or equal to 20 on the SSI-3 (at the 40th percentile and below on the SSI-3, which is referred to as mild or very mild stuttering). Eighty-six out of 96 (89.6%) high scores on the Child Report Questionnaire related to children whose SSI-3 scores were greater than or equal to 21. Percentiles for the Child Report Questionnaire raw scores are given in Table 3.
Table 3. Raw Child Report Questionnaire scores and corresponding percentiles based on 311 questionnaires. SSI-3 taxonomic severity ratings and actual SSI-3 values are given for the same percentiles.
Total Raw Score on Child Report Questionnaire / Percentile / SSI-3 severity rating for same percentiles / Raw SSI-3 scores9 – 10 / 1 - 4 / Very mild / 6-8
11 – 12 / 5 - 11 / Very mild / 9-10
13 – 15 / 12 - 23 / Mild / 11-15
16 – 17 / 24 - 40 / Mild / 16-20
18 – 21 / 41 - 60 / Moderate / 21-23
22 – 23 / 61 – 77 / Moderate / 24-27
24 – 25 / 78 – 88 / Severe / 28-31
26 – 28 / 89 – 95 / Severe / 32-35
29 and up / 96 – 99 / Very Severe / 36 and up
Parent Report Questionnaire
Distribution of scores
Analyses were carried out to norm and standardize the 302 questionnaires completed by parents of CWS. The questionnaires were again based on 1 but this time the questions asked about the parent’s view about the CWS (so the questions were altered to refer to the child in the third person). There were 246 boys and 56 girls about whom the parents reported. The ages of the CWS ranged from 8 years 0 months to 19 years 7 months (mean was 12 years 11 months and sd was 2 years 6 months). The parental data were normally distributed, as shown in Figure 2. The skewness and kurtosis values of the raw scores on the Parent Report Questionnaire were converted to z scores and compared against values that would be expected based on chance alone and found not to be significant (Skewness z = 0.857; Kurtosis z = -1.485). Statistics about the distribution are given in Table 4 which, as was found with the child version, showed there was no discrepancy from normality.
Figure 2. Histogram showing distribution of scores on the Parent Report Questionnaire with normal distribution curve fitted.
Table 4. Descriptive statistics for the total scores on the Parent Report Questionnaire.
Mean / 21.64Median / 21.00
Std. deviation / 6.294
Skewness / 0.120
Std. error of Skewness / 0.140
Kurtosis / -0.617
Std. error of kurtosis / 0.280
Construct reliability (internal consistency of the questionnaire)
Cronbach's alpha was again used as an indication of how well the set of items from the questionnaire measured a single unidimensional latent construct. The Cronbach alpha coefficient for the Parent Report Questionnaire was 0.87. This indicates high internal consistency so the questionnaire is reliable. 3
Test-retest reliability
Test-retest reliability was assessed on 51 participants who were tested twice (the median test-retest interval was 13 days). The correlation coefficient for the two sets of scores was .85, which indicates good reliability. 4 Minimum score, maximum score, mean and standard deviation are given separately for the two test occasions in Table 5.
Table 5. Descriptive statistics on two test occasions for the Parent Report Questionnaire.
N / Minimum / Maximum / Mean / Std. deviationTime 1 / 51 / 8.00 / 33.00 / 21.1373 / 6.22903
Time 2 / 51 / 10.00 / 39.00 / 21.6078 / 6.40649
Internal (construct) validity
An unrotated, principal axis factor analysis showed all of the questions in the Parent Report Questionnaire loaded at .6 or above on a single factor. This factor accounted for 54.3% of the variance in the questionnaire. All the questions in the Parent Report Questionnaire refer to stuttering (current and past) by the child. As such, the factor provides an integrated impression of stuttering over time. Recall that current and past stuttering weighted on different factors for the children’s responses. These differences in the way parents and children evaluate stuttering is interesting, but cannot be discussed here. The statistical analysis suggests that the parent questionnaire provides a valid measure of stuttering across time for their child.
External (criterion) validity
Criterion validity was assessed using the raw total scores from the Parent Report Questionnaire and the corresponding SSI-3 scores for the child. These pairs of scores were available for 182 parent-child pairs. The Parent Report scores correlated highly with the SSI-3 score (r = .65). A linear regression analysis showed that the raw score from the Parent Report Questionnaire was a significant predictor of SSI-3 score (the regression model accounted for 44.9% of the variance in SSI-3 scores). ANOVA indicated that the amount of variance accounted for was significantly different above zero F(1,180) = 148.30, p <.001.
Standardization
Total raw scores from the 302 Parent Report Questionnaires used in the criterion validity analysis were standardized to t scores with a mean of 50 and an sd of 10. The same cutoff of 21 as was used with the child form. Eighty-five out of 93 (91.4%) high scores on the Parent Report Questionnaire related to children whose SSI-3 scores were greater than or equal to 21. Seventy-two out of 89 (80.9%) low scores on the Parent Report Questionnaire related to children whose SSI-3 scores were less than or equal to 20. Percentiles for the Parent Report Questionnaire raw scores are given in Table 6. The percentiles are given so the SSI-3 column corresponds to that reported by Riley. 5
Table 6. Raw Parent Report Questionnaire scores and corresponding percentiles based on 302 questionnaires. SSI-3 taxonomic severity ratings and actual SSI-3 values are given for the same percentiles.
Total Raw Score on Parent Report Questionnaire / Percentile / SSI-3 severity rating for same percentiles / Raw SSI-3 scores9 – 10 / 1 – 4 / Very mild / 6-8
11 – 14 / 5 – 11 / Very mild / 9-10
14 -1 6 / 12 – 23 / Mild / 11-15
17 – 19 / 24 – 40 / Mild / 16-20
20 – 23 / 41 - 60 / Moderate / 21-23
24 – 27 / 61 – 77 / Moderate / 24-27
27 – 29 / 78 – 88 / Severe / 28-31
30 – 32 / 89 – 95 / Severe / 32-35
33 and up / 96 – 99 / Very Severe / 36 and up
Researcher’s Report Form
Distribution of scores
Analyses were carried out to norm and standardize the 328 Researcher’s Report Forms about the CWS. The reports were intended for assessment of CWS and to assist designation as persistent or recovered. There were 267 boys and 61 girls and the sample ranged in age from 8 years 1 month to 19 years 11 months (with a mean of 13 years 0 months and an sd of 2 years 7 months).