This review provides an overview and critique of the design characteristics, technical properties, and validity evidence of behavior ratings scales focusing on measurement of the characteristics of emotional disturbance. Manuals and published research supporting nine parent and teacher rating scales were reviewed. These rating scales included the Behavior Evaluation Scale-2: Home Version (McCarney, 1994a); the Behavior Evaluation Scale-2: School Version (McCarney, 1994b); the Behavior Disorders Identification Scale, Second Edition: Home Version (McCarney & Arthaud, 2000a); the Behavior Disorders Identification Scale, Second Edition: School Version (McCarney & Arthaud, 2000b); the Devereux Behavior Rating Scale-School Form (Naglieri, LeBuffe, & Pfieffer, 1993); the Emotional and Behavior Problem Scale, Second Edition: Home Version (McCarney & Arthaud, 2001a); the Emotional and Behavior Problem Scale, Second Edition: School Version (McCarney & Arthaud, 2001b); the Scale for Assessing Emotional Disturbance (Epstein & Cullinan, 1998); and the Social-Emotional Dimension Scale (Hutton & Roberts, 1986). All instruments demonstrated several limitations in technical adequacy and were supported by generally incomplete and weak collections of validity evidence that limit their usefulness during the assessment of behaviors associated with emotional disturbance.
Get full access to this article
View all access options for this article.
References
1.
Achenbach, T. M. (1991). Integrative guide for the 1991 CBCL/4-18, YSR and TRF profiles. Burlington: University of Vermont, Department of Psychiatry.
2.
Achenbach, T. M., McConaughy, S. H., & Howell, C. T. (1987). Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin, 101, 213-232.
3.
Aiken, L. R. (1997a). Rating scales and checklists: Evaluating behavior, personality and attitudes. New York: Wiley.
4.
Aiken, L. R. (1997b). Psychological testing and assessment (9th ed.). Needham Heights, MA: Allyn & Bacon.
5.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
6.
American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.
7.
American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text revision). Washington, DC: Author.
8.
Anastasi, A., & Urbina, S. (1997). Psychological testing (7th ed.). Upper Saddle River, NJ: Prentice-Hall.
9.
Athanasiou, M. S. (2000). Current nonverbal assessment instruments: A comparison of psychometric integrity and test fairness. Journal of Psychoeducational Assessment, 18, 211-229.
10.
Barnett, D. W., Lentz, F. E., Jr., & Macmann, G. (2000). Psychometric qualities of professional practice. In E. S. Shapiro & T. R. Kratochwill (Eds.), Behavioral assessment in schools (2nd ed., pp. 355-386). New York: Guilford.
11.
Benson, J. (1998). Developing a strong program of construct validation: A test anxiety example. Educational Measurement: Issues and Practice, 17, 10-22.
12.
Bower, E. M. (1959). The emotionally handicapped child and the school. Exceptional Children, 26, 6-11.
13.
Bower, E. M. (1969). Early identification of emotionally handicapped children in school. Springfield, IL: Charles C. Thomas.
14.
Bower, E. M. (1982). Defining emotional disturbance: Public policy and research. Psychology in the Schools19, 55-60.
15.
Bracken, B. A. (1987). Limitations of preschool instruments and standards for minimal levels of technical adequacy. Journal of Psychoeducational Assessment, 5, 313-326.
16.
Bracken, B. A., Keith, L. K., & Walker, K. C. (1998). Assessment of preschool behavior and social-emotional functioning: A review of thirteen third-party instruments. Journal of Psychoeducational Assessment, 16, 153-169.
17.
Braden,J. P., & Kratochwill, T. R. (1997). Treatment utility of assessment: Myths and realities. School Psychology Review, 26, 467-474.
18.
Bradley-Johnson, S. (2001). Cognitive assessment for the youngest children: A critical review of tests. Journal of Psychoeducational Assessment, 19, 1944.
19.
Breen, M. J., & Fiedler, C. R. (Eds.). (1996). Behavioral approach to assessment of youth with emotional/behavioral disorders. Austin, TX: PRO-ED.
20.
Brown, L., & Hammill, D. D. (1978a). Behavior Rating Profile-Parent Rating Scale. Austin, TX: PRO-ED.
21.
Brown, L., & Hammill, D. D. (1978b). Behavior Rating Profile-Teacher Rating Scale. Austin, TX: PRO-ED.
22.
Cone, J. D. (1998). Psychometric considerations: Concepts, contents, and methods. In M. Hersen & A. S. Bellack (Eds.), Behavioral assessment: A practical handbook (4th ed., pp. 24-46). Boston: Allyn and Bacon.
23.
Costello, E. J., Messer, S. C., Bird, H. R., Cohen, P., & Reinherz, H. Z. (1998). The prevalence of serious emotional disturbance: A re-analysis of community studies. Journal of Child and Family Studies, 7. 411-432.
24.
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281-302.
25.
Cullinan, D., Harniss, M., Epstein, M. H., & Ryser, G. (2001). The Scale for Assessing Emotional Disturbance: Concurrent validity. Journal of Child and Family Studies, 10, 449-466.
26.
Epstein, M. H., & Cullinan, D. (1998). Scale for Assessing Emotional Disturbance: Examiner's manual. Austin. TX: PRO-ED.
27.
Epstein, M. H., Cullinan, D., Harniss, M. K., & Rvser, G. (1999). The Scale for Assessing Emotional Disturbance: Test-retest and interrater reliability. Behavioral Disorders, 24, 222-230.
28.
Epstein, M. H., Nordness, P. D., Cullinan, D., & Hertzog, M. (2002). Scale for Assessing Emotional Disturbance: Long-term test-retest reliability and convergent validity with kindergarten and first-grade students. Remedial and Special Education, 23, 141-148.
29.
Flanagan, D. P., & Alfonso, V. C. (1995). A critical review of the technical characteristics of new and recently revised intelligence tests for preschool children. Journal of Psychoeducational Assessment, 13, 66-90.
30.
Forness, S. R., & Kavale, K. A. (1997). Defining emotional or behavioral disorders in school and related services. In J. W. Lloyd, E. J. Kameenui, & D. Chard (Eds.), Issues in educating students with disabilities (pp. 45-61). Mahwah, NJ: Erlbaum.
31.
Forness, S. R., & Kavale, K. A. (2000). Emotional or behavioral disorders: Background and current status of the E/BD terminology and definition. Behavioral Disorders, 25, 364-369.
32.
Forness, S. R., & KnitzerJ. (1992). A new proposed definition and terminology to replace “serious emotional disturbance” in Individuals with Disabilities Education Act. School Psychology Review, 21, 12-20.
33.
Fuchs, L. S., & Fuchs, D. (1998). Treatment validity: A unify4ng concept for reconceptualizing the identification of learning disabilities. Learning Disabilities Research and Practice, 13, 204-219.
34.
Gimpel, G. A., & Nagle, R. J. (1996). Factorial validity of the Devereux Behavior Rating Scale-School Form. Journal of Psychoeducational Assessment, 14, 334-348.
35.
Goh, D. S. (1997). Clinical utility of the Devereux Behavior Rating Scale-School Form among culturally diverse children. Psychology in the Schools, 34, 301-308.
36.
Gresham, F. M. (1999). Noncategorical approaches to K-12 emotional and behavioral difficulties. In D. J. Reschly, W. D. Tilly III, & J. P. Grimes (Eds.). Special education in transition (pp. 107-137). Longmont, CO: Sopris Wlest.
37.
Hammill, D. D., Brown, L., & Bryant, B. R. (1992). A consumer's guide to tests in print (2nd ed.). Austin, TX: PRO-ED.
38.
Hayes, S. C., Nelson, R. O., & Jarrett, R. B. (1987). The treatment utility of assessment: A functional approach to evaluating assessment quality. American Psychologist, 42, 963-503.
39.
Haynes, S. N. (2001). Clinical applications of analogue behavioral observation: Dimensions of psychometric evaluation. Psychological Assessment, 11, 73-85.
40.
Haynes, S. N., Richard, D. C. S., & Kubany, E. S. (1995). Content validity in psychological assessment: A functional approach to concepts and methods. Psychological Assessment, 7, 238-247.
41.
Hutton, J. B., & Roberts, T. G. (1986). Social-Emotional Dimension Scale: A measure of school behavior. Austin, TX: PRO-ED.
42.
Individuals with Disabilities Education Act Amendments of 1997, Pub. L. No. 105-17. 20 USC Chapter 33, Sections 1400 et seq. (Statute)
43.
Kamphaus, R. W., & Frick, P. J. (2002). Clinical assessment of child and adolescent personality and behavior. Boston: Allyn & Bacon.
44.
Kane, M. T. (2001). Current concerns in validity theory. Journal of Educational Measurement, 38, 319-342.
45.
Kolko, D. J., & Kazdin, A. E. (1993). Emotional/behavioral problems in clinic and nonclinic children: Correspondence among child, parent, and teacher reports. Journal of Child Psychology and Psychiatry, 34, 991-1006.
46.
Lambert, N. M., & Bower, E. M. (1961). A process for in-school screening of children with emotional handicaps: Technical report for school administrators and teachers. Sacramento, CA: California State Department of Education.
47.
Mash, E. J., & Barkley, R. A. (Eds.). (1996). Child psychopathology. New York: Guilford.
48.
Mash, E. J., & Wolfe, D. A. (1999). Abnormal child psychology. Belmont, CA: Wadsworth Publishing.
49.
McCarney, S. B. (1989a). Behavior Disorders Identification Scale: Home Version technical manual. Columbia, MO: Hawthorne Educational Services.
50.
McCarney, S. B. (1989b). Behavior Disorders Identification Scale: School Version technical manual. Columbia, MO: Hawthorne Educational Services.
51.
McCarney, S. B. (1989c). Emotional and Behavior Problem Scale IEP and intervention manual. Columbia, MO: Hawthorne Educational Services.
52.
McCarney, S. B. (1994a). The Behavior Evaluation Scale-2: Home Version technical manual. Columbia, MO: Hawthorne Educational Services.
53.
McCarney, S. B. (1994b). The Behavior Evaluation Scale-2: School Version technical manual. Columbia, MO: Hawthorne Educational Services.
54.
McCarney, S. B., & Arthaud, T. J. (2000a). Behavior Disorders Identification Scale: Home Version technical manual (2nd ed.). Columbia, MO: Hawthorne Educational Services.
55.
McCarney, S. B., & Arthaud, T.J. (2000b). Behavior Disorders Identification Scale: School Version technical manual (2nd ed.). Columbia, MO: Hawthorne Educational Services.
56.
McCarney, S. B., & Arthaud, T. J. (2001a). Emotional and Behavior Problem Scale: Home Version (2nd ed.). Columbia, MO: Hawthorne Educational Services.
57.
McCarney, S. B., & Arthaud, T.J. (2001b). Emotional and Behavior Problem Scale: School Version (2nd ed.). Columbia, MO: Hawthorne Educational Services.
58.
McCarney, S. B., & Bauer, A. M. (1990). Parent's guide. Columbia, MO: Hawthorne Educational Services.
59.
McCarney, S. B., Leigh, J., & Cornbleet, J. (1983). Behavior Evaluation Scale. Columbia, MO: Hawthorne Educational Services.
60.
McConaughy, S. H. (1993). Evaluating behavioral and emotional disorders with the CBCL, TRF, and YSR cross-informant scales. Journal of Emotional and Behavioral Disorders, 1 (1), 40-52.
61.
McConaughy, S. H., & Ritter, D. R. (2002). Best practices in multidimensional assessment of emotional or behavioral disorders. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology IV (pp. 1303-1320). Bethesda, MD: The National Association of School Psychologists.
62.
Merrell, K. W. (1999). Behavioral, social, and emotional assessment of children and adolescents. Mahwah, NJ: Erlbaum.
63.
Merrell, K. W. (2000). Informant reports: Conceptual and research in using child rating scales in school settings. In E. S. Shapiro & T. R. Kratochwill (Eds.), Behavioral assessment in schools (2nd ed., pp. 233-256). New York: Guilford.
64.
Messick, S. (1989). Validity. In R. Linn (Ed.), Educational measurement (3rd ed., pp. 13-103). Washington, DC: American Council on Education.
65.
Messick, S. (1995). Validity of psychological assessment: Validity of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-749.
66.
Naglieri, J. A., Bardos, A. N., & LeBuffe, P. A. (1995). Discriminant validity of the Devereux Behavior Rating Scale-School Form for students with serious emotional disturbance. School Psychology Review, 24, 104-111.
67.
Naglieri, J. A., & Gottling, S. H. (1995). Use of the Teacher Report Form and the Devereux Behavior Rating Scale-School Form with learning disordered/emotionally disordered students. Journal of Clinical Child Psychology, 24, 71-76.
68.
Naglieri, J. A., LeBuffe, P. A., & Pfieffer, S. I. (1993). Devereux Behavior Rating Scale-School Form. San Antonio, TX: The Psychological Corporation.
69.
National Mental Health Association. (1986). Severely emotionally disturbed children: Improving services under Education of the Handicapped Act (PL 94-142). Washington, DC: Author.
70.
Nickerson, A. B., & Nagle, R. J. (2001). Interrater reliability of the Devereux Behavior Rating Scale Form-School Form: The influence of teacher frame of reference. Journal of Psychoeducational Assessment, 19, 299-316.
71.
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.
72.
Quay, H. C., & Peterson, D. R. (1996). Revised Behavior Problem Checklist: Professional manual. Odessa, FL: Psychological Assessment Resources.
73.
Ramsey, M. C., Reynolds, C. R., & Kamphaus, R. W. (2002). Essentials of behavioral assessment. New York: Wiley.
74.
Reynolds, C. R., & Kamphaus, R. W. (1992). Behavior Assessment System for Children basic). Circle Pines, MN: American Guidance Service.
75.
Spivack, G., & Spotts, J. (1966). Devereux Child Behavior Rating Scale. Devon, PA: The Devereux Foundation.
76.
Spivack, G., Spotts, J., & Haimes, P. (1967). Devereux Adolescent Behavior Rating Scale. Devon, PA: The Devereux Foundation.
77.
Stamoulis, D. T., & Hauenstein, N. M. A. (1993). Rater training and rating accuracy: Training for dimensional accuracy versus training for rate differentiation. Journal of Applied Psychology, 76, 994-1003.
78.
Sulsky, L., M., & Day, D. V. (1994). Effects of frame-of-reference training on rater accuracy under alternative time delays. Journal of Applied-Psychology, 79, 535-543.
79.
U.S. Department of Education, Office of Special Education Programs. (2001, August). IDEA Part B Child Count Data (2000): Table AA 11. Retrieved June 15, 2002, from http://www.ideadata.org/tables24th/araal1.htm
80.
Walker, H. (1970). Walker Problem Identification Checklist. Los Angeles: Western Psychological Corporation.
81.
Walker, H. M., & Severson, H. H. (1990). Systematic Screening for Behavior Disorders (2nd ed.). Longmont, CO: Sopris West.
82.
Wilson, M. J., & Bullock, L. M. (1989). Psychometric characteristics of behavior rating scales: Definitions, problems and solutions. Behavioral Disorders, 14(3), 186-200.
83.
Witt, J. C., Daly, E. M., & Noell, G. (2000). Functional assessments: A step-by-step guide to solving academic and behavior problems. Longmont, CO: Sopris West.
84.
Woehr, D. J. (1994). Understanding frame of reference training: The impact of training on the recall of performance information. Journal of Applied Psychology, 4, 525-534.
85.
Wunderlich, K. C. (1988). Teacher's guide to behavioral interventions. Columbia, MO: Hawthorne Educational Services.
