Through a series of multivariate classification studies, we examined the sensitivity, specificity, and overall accuracy of the Adjustment Scales for Children and Adolescents (ASCA; McDermott, Marston, & Stott, 1993) in detecting emotional disturbance. The ASCA, a teacher rating scale normed on a stratified national sample of 5- through 17-year-old youths, was first applied to a sample of 150 nondisabled subjects and 150 subjects who were socially or emotionally disturbed (SED) matched for age, gender, race, and grade level. Discriminant analyses, including single- and split-sample cross-validations, produced sensitivity, specificity, and overall accuracy ratings of approximately 80%, a level significantly above chance expectancy. The discriminant solution maintained similar accuracy when applied independently to preadolescent, adolescent, male, female, White, and African American children. Accuracy also sustained when the scale was used to differentiate the SED subjects from samples of students who were learning disabled, communication impaired, or gifted/talented, and from a mixture of 1,843 nondisabled subjects and 537 others with learning, mental, and physical disabilities. The scale's performance was evaluated in light of classification accuracy research on other popular instruments.
Get full access to this article
View all access options for this article.
References
1.
Achenbach, T. M. (1988). Integrating assessment and taxonomy. In M. Rutter, A. H. Tuma, & I. S. Lann (Eds.), Assessment and diagnosis in child psychopathology (pp. 300–342). New York: Guilford.
2.
Achenbach, T. M. (1991). Manual for the Teacher's Report Form and 1991 Profile.Burlington: University of Vermont Department of Psychiatry.
3.
Achenbach, T. M., & Edelbrock, C. (1983). Manual for the Child Behavior Checklist and Revised Child Behavior Profile.Burlington: University of Vermont.
4.
American Psychological Association. (1985). Standards for educational and psychological testing.Washington, DC: Author.
5.
Anastasi, A. (1988). Psychological testing (6th ed.). New York: Macmillan.
6.
Anderberg, M. R. (1973). Cluster analysis for applications.New York: Academic Press.
7.
Anderson, T. W. (1984). An introduction to multivariate statistical analysis (2nd ed.). New York: Wiley.
8.
Banez, G. A., & Compas, B. E. (1990). Children's and parents' daily stressful events and psychological symptoms. Journal of Abnormal Child Psychology, 18, 591–605.
9.
Barnett, D. W., & Macmann, F. M. (1992). Decision reliability and validity: Contributions and limitations of alternative assessment strategies. The Journal of Special Education, 25, 431–452.
10.
Bird, H. R., Canino, G., Gould, M. S., Ribera, J., Rubio-Stipec, M., Woodbury, M., Heurtas-Goldman, S., & Sesman, M. (1987). Use of the Child Behavior Checklist as a screening instrument for epide-miological research in child psychiatry: Results of a pilot study. Journal of the American Academy of Child and Adolescent Psychiatry, 26, 207–213.
11.
Brandon, K. A., Kehle, T. J., Jenson, W. R., & Clark, E. (1990). Regression, practice, and expectation effects on the Revised Conners Teacher Rating Scale. Journal of Psychoeducational Assessment, 8, 456–466.
12.
Breen, M. J., & Altepeter, T. S. (1991). Factor structure of the Home Situations Questionnaire and the School Situations Questionnaire. Journal of Pediatric Psychology, 16, 59–67.
13.
Brody, G. H., & Forehand, R. (1986). Maternal perceptions of child maladjustment as a function of the combined influence of child behavior and maternal depression. Journal of Consulting and Clinical Psychology, 54, 237–240.
14.
Campbell, S. B., March, C. L., Pierce, E. W., Ewing, L. J., & Szumowski, W. K. (1991). Hard-to-manage preschool boys: Family context and the stability of externalizing behavior. Journal of Abnormal Child Psychology, 19, 301–313.
15.
Campbell, S. B., & Steinert, Y. (1978). Comparisons of rating scales of child psychopathology in clinic and nonclinic samples. Journal of Consulting and Clinical Psychology, 46, 358–359.
16.
Compas, B. E., Howell, D. C., Phares, V., Williams, R. A., & Giunta, C. T. (1989). Risk factors for emotional/behavioral problems in young adolescents: A prospective analysis of adolescent and parental stress and symptoms. Journal of Consulting and Clinical Psychology, 57, 732–740.
17.
Costenbader, V. K., & Keller, H. R. (1990). Behavioral ratings of emotionally handicapped, learning disabled, and non-referred children: Scale and source consistency. Journal of Psychoeducational Assessment, 8, 485–496.
18.
Dawes, R. M. (1962). A note on base rates and psychometric efficiency. Journal of Consulting Psychology, 26, 422–424.
19.
Edelbrock, C., Costello, A. J., Dulcan, M. K., Kalas, R., & Conover, N. C. (1985). Age differences in the reliability of the psychiatric interview of the child. Child Development, 56, 265–275.
20.
Elliot, C. D. (1990). Differential Ability Scales: Introductory and technical handbook.San Antonio, TX: Psychological Corp.
21.
Fantuzzo, J. W., & McDermott, P. A. (1992). Critical issues in the assessment of family violence involving children. In R. Ammerman & M. Hersen (Eds.), Assessment of family violence: A clinical and legal sourcebook (pp. 11–25). New York: Wiley.
22.
Ferguson, D. M., Lynskey, M. T., & Horwood, J. (1993). The effect of maternal depression on maternal ratings of child behavior. Journal of Abnormal Child Psychology, 21, 245–269.
23.
Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76, 378–382.
24.
Garfield, S. L. (1978). Research problems in clinical diagnosis. Journal of Consulting and Clinical Psychology, 46, 596–607.
25.
Goyette, C. H., Conners, C. K., & Ulrich, R. F. (1978). Normative data on revised Conners Parent and Teacher Rating Scales. Journal of Abnormal Child Psychology, 6, 221–236.
26.
Griner, P. F., Mayewski, R. J., Mushlin, A. I., & Greenlan, P. (1981). Selection and interpretation of diagnostic tests and procedures: Principles and applications. Annals of Internal Medicine, 94, 557–592.
27.
Harman, H. H. (1976). Modern factor analysis (3rd ed.). Chicago: University of Chicago Press.
28.
Hora, S. C., & Wilcox, J. B. (1982). Estimation of error rates in several-population discriminant analysis. Journal of Marketing Research, 19, 57–61.
29.
Horn, W. F., Wagner, A. E., & Ialongo, N. (1989). Sex differences in school-aged children with pervasive attention deficit hyperactivity disorder. Journal of Abnormal Child Psychology, 17, 109–125.
30.
Hosmer, D. W., Jr., & Lemeshow, S. (1989). Applied logistic regression.New York: Wiley-Interscience.
31.
Kazdin, A. E., & Heidish, I. E. (1984). Convergence of clinically derived diagnoses and parent checklists among inpatient children. Journal of Abnormal Child Psychology, 12, 421–435.
32.
Kline, R. B. (1988). Methodological considerations in the evaluation of the convergence of psychiatric diagnoses and parent-informant checklists. Journal of Abnormal Child Psychology, 16, 289–298.
33.
Lachenbruch, P. A., & Mickey, M. A. (1968). Estimation of error rates in discriminant analysis. Technometrics, 10, 1–10.
34.
LeBuffe, P., Hess, R., Jr., Servis, L., & Waschbusch, D. (1992, August). Identifying SED students: The Devereux scales vs. the TRF. Paper presented at the Centennial Convention of the American Psychological Association, Washington, DC.
35.
Loeber, R., Green, S. M., & Lahey, B. B. (1990). Mental health professionals' perceptions of the utility of children, parents, and teachers as informants on childhood psychopathology. Journal of Clinical Child Psychology, 19(2), 136–143.
36.
Mattison, R. E., Bagnato, S. J., Mayes, S. D., & Felix, B. C., Jr. (1990). Reliability and validity of teacher diagnostic ratings for children with behavioral and emotional disorders. Journal of Psychoeducational Assessment, 8, 509–517.
37.
McDermott, P. A. (1981). Sources of error in the psychoeducational diagnosis of children. Journal of School Psychology, 19, 31–44.
38.
McDermott, P. A. (1982). Generality of disordered behavior across populations of normal and deviant school children: Factorial relations analyses. Multivariate Behavioral Research, 17, 69–85.
39.
McDermott, P. A. (1986). The observation and classification of exceptional child behavior. In R. T. Brown & C. R. Reynolds (Eds.), Psychological perspectives on childhood exceptionality: A handbook (pp. 136–180). New York: Wiley-Interscience.
40.
McDermott, P. A. (1988). Agreement among diagnosticians or observers: Its importance and determination. Professional School Psychology, 3, 225–240.
41.
McDermott, P. A. (1993). National standardization of uniform multisituational measures of child and adolescent behavior pathology. Psychological Assessment, 5, 413–424.
42.
McDermott, P. A. (in press). A nationwide study of developmental and gender prevalence for psychopathology in childhood and adolescence. Journal of Abnormal Child Psychology.
43.
McDermott, P. A., Marston, N. C., & Stott, D. H. (1993). Adjustment Scales for Children and Adolescents.Philadelphia, PA: Edumetric and Clinical Science.
44.
Meehl, P. E., & Golden, R. R. (1982). Taxometric methods. In P. C. Kendall & J. N. Butcher (Eds.), Handbook of research methods in clinical psychology (pp. 127–181). New York: Wiley.
45.
Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). New York: American Council on Education, Macmillan.
46.
Milich, R., Widiger, T. A., & Landau, S. (1987). Differential diagnosis of attention deficit and conduct disorders using conditional probabilities. Journal of Consulting and Clinical Psychology, 55, 762–767.
47.
Morrison, D. F. (1976). Multivariate statistical methods.New York: McGraw-Hill.
48.
Naglieri, J., LeBuffe, P., & Pfeiffer, S. (1992). The Devereux School Form.San Antonio, TX: Psychological Corp.
49.
Ollendick, T. H., Greene, R. W., Weust, M. D., & Oswald, D. P. (1990). The predictive validity of teacher nominations: A five-year follow-up of at-risk youth. Journal of Abnormal Child Psychology, 18, 699–713.
50.
Perlman, M. D. (1980). Unbiasedness of the likelihood ratio tests for equality of several covariance matrices and equality of several multivariate normal populations. Annals of Statistics, 8, 247–263.
51.
Phares, V., Compas, B. E., & Howell, D. C. (1989). Perspectives on child behavior problems: Comparisons of children's self-reports with parent and teacher reports. Psychological Assessment: A Journal of Consuiting and Clinical Psychology, 1, 68–71.
52.
Piacentini, J. C., Cohen, P., & Cohen, J. (1992). Combining discrepant diagnostic information from multiple sources: Are complex algorithms better than simple ones?Journal of Abnormal Child Psychology, 20, 51–63.
53.
Quay, H. C. (1986). Classification. In H. C. Quay & J. S. Werry (Eds.), Psychopathological disorders of childhood (3rd ed., pp. 1–42). New York: Wiley.
54.
Richters, J. E. (1992). Depressed mothers as informants about their children: A critical review of the evidence for distortion. Psychological Bulletin, 112, 485–499.
55.
Robins, L. N. (1985). Epidemiology: Reflections on testing the validity of psychiatric interviews. Archives of General Psychiatry, 42, 918–924.
56.
Rutter, M., Tizard, J., & Whitmore, K. (1970). Education, health and behavior.London: Longman.
57.
Schaughency, E. A., & Lahey, B. B. (1985). Mothers' and fathers' perceptions of child deviance: Roles of child behavior, parental depression, and marital satisfaction. Journal of Consulting and Clinical Psychology, 53, 718–723.
58.
Sicoly, F. (1992). Estimating the accuracy of decisions based on cutting scores. Journal of Psychoeducational Assessment, 10, 26–36.
59.
Stein, M. A., & O'Donnell, J. P. (1985). Classification of children's behavior problems: Clinical and quantitative approaches. Journal of Abnormal Child Psychology, 13, 269–280.
60.
Steinhausen, H. C., & Gobel, D. (1987). Convergence of parent checklists and child psychiatric diagnoses. Journal of Abnormal Child Psychology, 15, 147–151.
61.
Strauss, C. C., Lahey, B. B., Frick, P., France, C. L., & Gynd, G. W. (1988). Peer social status of children with anxiety disorders. Journal of Consulting and Clinical Psychology, 56, 137–141.
62.
Swets, J. A. (1988). Measuring the accuracy of diagnostic systems. Science, 240, 1285–1293.
63.
Tharinger, D. J., Laurent, J., & Best, L. R. (1986). Classification of children referred for emotional and behavioral problems: A comparison of PL 94--142 SED criteria, DSM III, and the CBCL system. Journal of School Psychology, 24, 111–121.
64.
Thorndike, R. L. (1982). Applied psychometrics.Boston: Houghton Mifflin.
65.
Trites, R. L., Blouin, A. G. A., & Laprade, K. (1982). Analysis of the Conners Teacher Rating Scale based on a large normative sample. Journal of Consulting and Clinical Psychology, 50, 615–623.
66.
United States Code, 1977, Title 20, Secs. 1401(19), 1414(a)(5).
67.
U.S. Department of Commerce. (1988). Current population survey, March 1988 [Machine-readable computer file]. Washington, DC: Author.
68.
U.S. Department of Commerce. (1990). School enrollment: Social and economic characteristics of students. Current Population Reports (Series P-20, No. 443). Washington, DC: Bureau of the Census.
69.
U.S. Department of Education. (1989). Eleventh annual report to Congress on the implementation of the Education of the Handicapped Act. Washington, DC: Author.
70.
Watkins, M. W., & McDermott, P. A. (1979). A computer program for measuring overall and partial congruence among multiple observers on nominal scales. Educational and Psychological Measurement, 39, 235–239.
71.
Watson, S. M., Henggeler, S. W., & Whelan, J. P. (1990). Family functioning and the social adaptation of hearing-impaired youths. Journal of Abnormal Child Psychology, 18, 143–163.
