Problems in the definition and conceptualization of behavioral disorders have created problems for both researchers and practitioners alike. The implications of these conceptual difficulties for assessment are considered in a quantitative synthesis of correlations between behavioral ratings and classroom observation. The results, based on 89 correlations drawn from 16 studies, suggest significant problems in the areas of construct and criterion-related validity of widely-used measures of social behavior. The mean correlation between teacher ratings and observations of similar behaviors was, 23. Although correlations were somewhat higher when broken down by behavior, analysis of a multitrait, multimethod matrix suggests that minimum requirements for convergent validity may not be met. These results appear to pose serious methodological difficulties for the models of assessment that stress individual pathology. An alternative “components of variance” model is proposed that appears to better account for social and contextual variables present in educational decision making.
Get full access to this article
View all access options for this article.
References
1.
AchenbachT. M., & EdelbrockC. S. (1978). The classification of child psychopathology: A review and analysis of empirical efforts.Psychological Bulletin, 85, 1275–1301.
2.
AdamsP., & AndersonP. (1982). A comparison of teachers' and Mexican-American children's perceptions of the children's competence.Reading Teacher, 36, 8–13.
3.
AlgozzineB. (1976). The disturbing child: What you see is what you get.Alberta Journal of Educational Research, 22, 330–333.
4.
AlgozzineB. (1977). The emotionally disturbed child: Disturbed or disturbing?Journal of Abnormal Child Psychology, 5, 1205–211.
5.
AlgozzineB. (1980). The disturbing child: A matter of opinion.Behavioral Disorders, 5, 112–115.
6.
AlgozzineB., ChristensonS., & YsseldykeJ. (1982). Probabilities associated with the referral to placement process.Teacher Education and Special Education, 5, 19–23.
7.
AlgozzineB., & CurranT. J. (1979). Teachers' predictions of children's school success as a function of their behavioral tolerances.Journal of Educational Research, 72, 344–347.
8.
AlgozzineR., SchmidR., & MercerC. D. (1981). Childhood behavior disorders: Applied research and educational practice.Rockville, MD: Aspen.
9.
BanduraA. (1973). Aggression: A social learning analysis.Englewood Cliffs, NJ: Prentice-Hall.
10.
BlundenD., SpringC., & GreenbergL. M. (1974). Validation of the Classroom Behavior Inventory.Journal of Consulting and Clinical Psychology, 42, 84–88.
11.
BolstadO. D., & JohnsonS. M. (1977). The relationship between teachers' assessment of students and the students' actual behavior in the classroom.Child Development, 48, 570–578.
12.
BowerE. M. (1981). Early identification of emotionally handicapped children in school (3rd ed.). Springfield, IL: Thomas.
13.
BowerE. M. (1982). Defining emotional disturbance: Public policy and research.Psychology in the Schools, 19, 55–60.
14.
BrophyJ. E., & EvertsonC. M. (1981). Student characteristics and teaching.New York: Longman.
15.
BrophyJ. E., & RohrkemperM. M. (1980). Teachers' specific strategies for dealing with hostile, aggressive students(Research Report No. 86). East Lansing: Michigan State University, Institute for Research on Teaching.
16.
CairnsR. B., & GreenJ. A. (1979). How to assess personality and social patterns: Observations or ratings. In CairnsR. B. (Ed.), The analysis of social interactions: Methods, issues, and illustrations (pp. 209–226). Hillsdale, NJ: Erlbaum.
17.
CampbellD. T., & FiskeD. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix.Psychological Bulletin, 56, 81–105.L
18.
CampbellS. B., SchleiferM., & WeissG. (1978). Continuities in maternal reports and child behaviors over time in hyperactive and comparison groups.Journal of Abnormal Child Psychology, 4, 199–207.
19.
ChinnP. C., & HughesS. (1987). Representation of minority students in special education classes.Remedial and Special Education, 8, 41–46.
20.
ChristensonS., YsseldykeJ., & AlgozzineB. (1982). Institutional constraints and external pressures influencing referral decisions.Psychology in the Schools, 19, 341–345.
21.
ClarizioH. F. (1987). Differentiating emotionally impaired from socially maladjusted students.Psychology in the Schools, 24, 237–243.
22.
ConeJ. D. (1979). Confounded comparisons in triple response mode assessment.Behavioral Assessment, 1, 85–96.
23.
CopelandA. P., & WeissbrodC. S. (1978). Behavioral correlates of the hyperactivity factor of the Connors Teacher Questionnaire.Journal of Abnormal Child Psychology, 6, 339–343.
24.
Council for Children with Behavioral Disorders. (1987). Position paper on definition and identification of students with behavioral disorders.Behavioral Disorders, 13, 9–19.
25.
CronbachL. J. (1984). Essentials of psychological testing.New York: Harper & Row.
26.
CullinanD., EpsteinM. H., & McLindenD. (1986). Status and change in state administrative definitions of behavior disorder.School Psychology Review, 15, 383–392.
27.
EpsteinM. H., CullinanD., & SabatinoD. A. (1977). State definitions of behavior disorders.Journal of Special Education, 11, 417–425.
28.
FeldmanD., KinnisonL., JayR., & HarthR. (1983). The effects of differential labeling on professional concepts and attitudes toward the emotionally disturbed/behaviorally disordered.Behavioral Disorders, 8, 191–198.
29.
FosterG. G., & YsseldykeJ. E. (1976). Expectancy and halo effects as a result of artificially induced bias.Contemporary Educational Psychology, 1, 37–45.
30.
FosterS. L., & ConeJ. D. (1986). Design and use of direct observation procedures. In CimineroA. R., CalhounK. S., & AdamsH. E. (Eds.), Handbook of behavioral assessment (2nd ed.; pp. 253–324). New York: Wiley.
31.
GelbS. A., & MizokawaD. T. (1986). Special education and social structure: The commonality of “exceptionality”.American Educational Research Journal, 23, 543–557.
32.
GerstenR., WalkerH., & DarchC. (1988). Relationship between teachers' effectiveness and their tolerance for handicapped students.Exceptional Children, 54, 433–438.
33.
GliedmanJ., & RothW. (1980). The unexpected minority: Handicapped children in America.New York: Harcourt, Brace, Jovanovitch.
34.
GradenJ. L., CaseyA., & BonstromO. (1985). Implementing a prereferral intervention system: Part II. The data.Exceptional Children, 51, 487–496.
35.
GreshamF. M. (1983). Multitrait-multimethod approach to multifactored assessment: Theoretical rationale and practical application.School Psychology Review, 12, 26–33.
36.
GreshamF. M. (1985). Behavior disorder assessment: Conceptual, definitional, and practical considerations.School Psychology Review, 14, 495–509.
37.
GutkinT. B., & TiegerA. G. (1979). Funding patterns for exceptional children: Current approaches and suggested alternatives.Professional Psychology, 10, 670–680.
38.
HogeR. D. (1983). Psychometric properties of teacher judgment measures of pupil aptitudes, classroom behaviors, and achievement levels.Journal of Special Education, 17, 401–429.
39.
HogeR. D. (1985). The validity of direct observation measures of pupil classroom behavior.Review of Educational Research, 55, 469–483.
40.
JohnsonL. J., & BlankenshipC. S. (1984). A comparison of label-induced expectancy bias in two preservice teacher education programs.Behavioral Disorders, 9, 167–174.
41.
JohnsonS. M., & BolstadO. D. (1973). Methodological issues in naturalistic observation: Some problems for field research. In HamerlynckL. A., HandyL. C., & MashE. J. (Eds.), Behavior change: Methodology, concepts, and practice (pp. 7–68). Champaign, IL: Research Press.
42.
KauffmanJ. M. (1985). Characteristics of children's behavior disorders (3rd ed.). Columbus, OH: Merrill.
43.
KaufmanA. S., SwanW. W., & WoodM. M. (1980). Do parents, teachers, and psychoeducational evaluators agree in their perceptions of the problems of black and white emotionally disturbed children?Psychology in the Schools, 17, 185–191.
44.
KazdinA. E., Esveldt-DawsonK., & LoarL. L. (1983). Correspondence of teacher ratings and direct observations of classroom behavior of psychiatric inpatient children.Journal of Abnormal Child Psychology, 11, 549–564.
45.
KendallP.C., & WilcoxL. E. (1977). Self-control in children: Development of a rating scale.Journal of Consulting and Clinical Psychology, 47, 1020–1029.
46.
KennyD. A. (1979). Correlation and causality.New York: Wiley.
47.
MaheadyL., TowneR., AlgozzineB., MercerJ., & YsseldykeJ. (1983). Minority overrepresentation: A case for alternative practices prior to referral.Learning Disability Quarterly, 6, 448–456.
48.
MandellC. J., & StrainP. S. (1978). An analysis of factors related to the attitudes of regular classroom teachers toward mainstreaming mildly handicapped children.Contemporary Educational Psychology, 3, 154–162.
49.
McMahonR. J. (1984). Behavioral checklists and rating scales. In OllendickT. H., & HersenM. (Eds.), Child behavioral assessment: Principals and procecures (pp. 80–105). New York: Pergamon.
50.
NelsonC. M. (1971). Techniques for screening conduct disordered children.Exceptional Children, 37, 501–507.
51.
QuayH. C. (1979). Classification. In QuayH. C., & WerryJ. S. (Eds.), Psychopathological disorders of childhood (pp. 1–42). New York: Wiley.
52.
ReisE. M. (1986). Minority disproportionality in special placements as an assessment and programing issue.Journal of Instructional Psychology, 13, 135–140.
53.
RhodesW. C., & PaulJ. L. (1978). Emotionally disturbed and deviant children: New views and approaches.Englewood Cliffs, NJ: Prentice-Hall.
54.
RosenthalR. (1984). Meta-analytic procedures for social research.Beverly Hills, CA: Sage.
55.
RossM. B., & SalviaJ. (1975). Attractiveness as a biasing factor in teacher judgments.American Journal of Mental Deficiency, 80, 96–98.
56.
Safran.J. S., & SafranS. P. (1987). Teachers' judgments of problem behaviors.Exceptional Children, 54, 240–244.
57.
SalviaJ., & YsseldykeJ. E. (1988). Assessment in special and remedial education (4th ed.). Boston: Houghton Mifflin.
58.
SarasonS. B., & DorisJ. (1979). Educational handicap, public policy, and social history.New York: The Free Press.
59.
SchacharR., SandbergS., & RutterM. (1986). Agreement between teachers' ratings and observations of hyperactivity, inattentiveness, and defiance.Journal of Abnormal Child Psychology, 14, 331–345.
60.
SchlosserL., & AlgozzineB. (1979). The disturbing child: He or she?Alberta Journal of Educational Research, 25, 30–36.
61.
ShapiroE. S., LentzF. E., & SofmanR. (1985). Validity of rating scales in assessing aggressive behavior in classroom settings.Journal of School Psychology, 23, 69–79.
62.
ShepardL. A. (1982). Assessment of learning disabilities (ERIC/TM Report No. 84). Princeton, NJ: Educational Testing Service.
63.
SkibaR. J., & O'SullivanP. J. (1987). Implications of the relationship between observational and rating scale data for the assessment of behavioral disorders. In BraatenS., RutherfordR. B.Jr., & MaagJ. (Eds.), Programming for adolescents with behavioral disorders (Vol. 3, pp. 5–15). Reston, VA: Council for Children with Behavioral Disorders.
64.
SmithC. R., FrankA. R., & SniderB. C. F. (1984). School psychologists' and teachers' perceptions of data used in the identification of behaviorally disordered students.Behavioral Disorders, 10, 27–32.
65.
SwapS. (1978). The ecological model of emotional disturbance in children: A status report and proposed synthesis.Behavioral Disorders, 3, 186–196.
66.
TobiasJ., & ColeC. (1982). Teacher-student ethnicity and recommendations for special education referrals.Journal of Educational Psychology, 74, 72–76.
67.
WoodF. H., & LakinK. C. (1979). Defining emotionally disturbed/behaviorally disordered populations for research purposes. In WoodF. H., & LakinK. C. (Eds.), Disturbed, disordered, or disturbing? Perspectives on the definition of problem behavior in educational settings (pp. 3–16). Minneapolis: Advanced Institute for Trainers of Teachers for Seriously Emotionally Disturbed Children and Youth.
68.
YsseldykeJ. E., ThurlowM., GradenJ., WessonC., AlgozzineB., & DenoS. (1983). Generalizations from five years of research on assessment and decision making: The University of Minnesota Institute.Exceptional Education Quarterly, 4, 75–93.
