Office discipline referrals (ODRs) are widely used by school personnel to evaluate student behavior and the behavioral climate of schools. In this article, the authors report the results of a review of the relevant literature to evaluate the validity of ODR data as indices of school-wide behavioral climate, the effects of school-wide behavioral interventions, and differing behavior support needs across schools. They used Messick's unified approach to validity by focusing on examples of evidence for empirical and ethical foundations of interpretations, uses, and social consequences of ODR measures at the school-wide level. The authors also discuss ongoing issues, study limitations, and related recommendations for interpretations and uses of ODR measures as school-wide indices, based on the existing literature.
Get full access to this article
View all access options for this article.
References
1.
Abbeville County (SC) School District, 28 IDELR 1208, 28 LRP 5013 (OCR, 1998).
2.
Achenbach, T.M., & Edlebrock, C. (1991). Child behavior checklist. Burlington: University of Vermont, Department of Psychiatry .
3.
Bingham, R.D., Haubrich, P.A., & White, S.B. (1993). Explaining teacher/principal differences in evaluating schools. Journal of Educational Administration, 31(1), 70—76.
4.
Bryant, A.L., Schulenberg, J., Bachman, J.G., O'Malley, P.M., & Johnston, L.D. (2000). Understanding the links among school misbehavior, academic achievement, and cigarette use: A national panel study of adolescents. Prevention Science, 1, 71—87.
5.
Carran, D.T., Nemerofsly, A., Rock, E.E., & Kerins, M. (1996). Risk of unsuccessful program completion for students with serious emotional/ behavioral disorders: An epidemiologic risk analysis . Behavioral Disorders, 21, 172—189.
6.
Character Counts! (2002). Does CC! work? Records of disruption. Retrieved February 7, 2003 , from http://www.charactercounts.org/doing/surveyreports.htm
7.
Cheney, D., Stage, S., Walker, B., & Blum, C. (2004). A multi-year, multi-method analysis of school-wide positive behavioral support. Unpublished manuscript.
8.
Colvin, G., Kame'enui, E., & Sugai, G. (1993). Reconceptualizing behavior management and school-wide discipline in general education. Education and Treatment of Children, 16, 361—381.
9.
Cone, J.D. (1997). Issues in functional analysis in behavioral assessment . Behaviour Research & Therapy, 35, 259—275.
10.
Cordori, M. (1987). A computer-based recording system for analyzing disciplinary referrals. NASSP Bulletin, 71(501), 42—43.
11.
Cowen, E.L., Hightower, A.D., Pedro-Carroll, J.L., Work, W.C., Wyman, P.A., & Haffey, W.G. (1996). School-based prevention for children at risk: The Primary Mental Health Project. Washington, DC: American Psychological Association.
12.
Davies, J., & Brember, I. (2001). The closing gap in attitudes between boys and girls: A 5-year longitudinal study. Educational Psychology , 21(1), 103—114.
13.
Dwyer, K. (2002). Tools for building safe, effective schools. In M. R. Shinn, G. Stoner, & H. M. Walker (Eds.), Interventions for academic and behavior problems: Preventive and remedial approaches (pp. 167—211). Silver Spring, MD: National Association of School Psychologists.
14.
Dwyer, K., & Osher, D. (2000). Safeguarding our children: An action guide. Washington, DC: U.S. Departments of Education and Justice, American Institutes for Research. Retrieved April 14, 2002, from http://www.ed.gov/offices/OSERS/OSEP/ActionGuide
15.
Eckenrode, J., Laird, M., & Doris, J. (1993). School performance and disciplinary problems among abused and neglected children. Developmental Psychology, 29(1), 53—62.
16.
Farrington, D.P. (1989). Early predictors of adolescent aggression and adult violence. Violence and Victims, 4, 79—100.
17.
Flay, B.R., Allred, C.G., & Ordway, N. (2001). Effects of the Positive Action Program on achievement and discipline: Two matched-control comparisons. Prevention Science, 2, 71—89.
18.
Gersten, R., Keating, T., & Irvin, L.K. (1995). The burden of proof: Validity as improvement of instructional practice. Exceptional Children, 61, 510— 519.
19.
Gottfredson, D.C. (2001). Schools and delinquency. New York: Cambridge University Press.
20.
Gottfredson, G.D., & Gottfredson, D.C. (1999). Development and application of theoretical measures for evaluating drug and delinquency prevention programs: Technical manual for research editions of What About You (WAY). Ellicott City, MD: Gottfredson Associates.
21.
Gottfredson, G.D., Gottfredson, D.C., Czeh, E.R., Cantor, D., Crosse, S., & Hantman, I. (2000). A national study of delinquency prevention in schools. Ellicott City, MD: Gottfredson Associates.
22.
Gresham, F.M., & Elliott, S.N. (1990). Social skills rating system. Circle Pines, MN: American Guidance Service.
Guzmán, N. (1996). Leadership for successful inclusive schools: A study of principal behaviours. Journal of Educational Administration , 35, 439—450.
25.
Hagan-Burke, S., Sugai, G., Ingram, K., Thier, K., Garrett, U., Derickson, E., et al. (2004). Descriptive synthesis of school-wide discipline research. Manuscript submitted for publication.
26.
Hawkins, J.D., Herrenkohl, T., Farrington, D.P., Brewer, D., Catalano, R.F., & Harachi, T.W. (1998). A review of predictors of youth violence. In R. Loeber & D. P. Farrington (Eds.), Serious and violent juvenile offenders: Risk factors and successful interventions (pp. 106—146). Thousand Oaks, CA: Sage.
27.
Hirsch, E.D. (1967). Validity in interpretation. New Haven, CT: Yale University Press.
28.
Horner, R.H., Sugai, G., Lewis-Palmer, T., & Todd, A.W. (2001). Teaching school-wide behavioral expectations. Report on Emotional and Behavioral Disorders in Youth, 1, 77—79, 93—96.
29.
Huisinga, D., Esbensen, F.-A., & Weiher, A.W. (1991). Are there multiple paths to delinquency?Journal of Criminal Law and Criminology, 82, 83— 118.
30.
Huisinga, D., & Jakob-Chien, C. (1998). The contemporaneous co-occurrence of serious and violent juvenile offending and other problem behaviors. In R. Loeber & D. P. Farrington (Eds.), Serious and violent juvenile offenders: Risk factors and successful interventions (pp. 47—67). Thousand Oaks, CA: Sage.
31.
Jessor, R., & Jessor, S.L. (1977). Problem behavior and psycho-social development: A longitudinal study of youth. New York: Cambridge University Press.
32.
Kane, M.T. (1992). An argument-based approach to validity. Psychological Bulletin,112, 527—535.
33.
Knoff, H.M. (2000). Organizational development and strategic planning for the millennium: A blueprint toward effective school discipline, safety, and crisis prevention. Psychology in the Schools, 37, 17—32.
34.
Knoff, H.M., & Batsche, G.M. (1995). Project ACHIEVE: Analyzing a school process for at-risk and underachieving students. School Psychology Review , 24, 579—603.
35.
Kortering, L., Braziel, P., & Tompkins, J. (2002). The challenge of school completion among youths with behavioral disorders: Another side of the story. Behavioral Disorders, 27, 142—154.
36.
Lewis, T.J., & Sugai, G. (1999). Effective behavior support: A systems approach to proactive school-wide management. Focus onExceptional Children, 31(6), 1—24.
37.
Mather, C. (1975). Corderius Americanus: A discourse on the good education of children [Microfilm]. New Haven, CT: Research Publications. (Original work published 1828)
38.
May, S., Ard, W., III., Todd, A.W., Horner, R.H., Glasgow, A., Sugai, G., et al. (2002). School-wide information system. Eugene: University of Oregon, Educational and Community Supports.
39.
Mayer, G.R. (1995). Preventing antisocial behavior in the schools . Journal of Applied Behavior Analysis, 28, 467—478.
40.
Messick, S. (1988). The once and future issues of validity: Assessing the meaning and consequences of measurement. In H. Wainer & H. Braun (Eds.), Test validity (pp. 33—45). Hillsdale, NJ: Erlbaum.
41.
Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessment. Educational Researcher , 23(2), 13—23.
42.
Metzler, C.W., Biglan, A., Rusby, J.C., & Sprague, J.R. (2001). Evaluation of a comprehensive behavior management program to improve school-wide positive behavior support. Education and Treatment of Children, 24, 448— 479.
43.
Morgan-D'Atrio, C., Northup, J., LaFleur, L., & Spera, S. (1996). Toward prescriptive alternatives to suspensions: A preliminary evaluation. Behavioral Disorders, 21, 190—200.
44.
Morrison, G.M., Anthony, S., Storino, M., & Dillon, C. (2001). An examination of the disciplinary histories and the individual and educational characteristics of students who participate in an in-school suspension program. Education and Treatment of Children, 24, 276—293.
45.
Morrison, G.M., & Skiba, R. (2001). Predicting violence from school misbehavior: Promises and perils. Psychology in the Schools, 38, 173—184.
46.
Murdock, T.B., Anderman, L.H., & Hodge, S.A. (2000). Middle-grade predictors of students' motivation and behavior in high school. Journal of Adolescent Research , 15, 327—351.
47.
Najaka, S.S., Gottfredson, D.C., & Wilson, D.B. (2002). A meta-analytic inquiry into the relationship between selected risk factors and problem behavior. Prevention Science, 2, 257—271.
48.
Nakasato, J. (2000). Data-based decision making in Hawaii's behavior support effort. Journal of Positive Behavior Interventions , 2, 247—251.
49.
National School Safety Center. (1998). Checklist of characteristics of youth who have caused school-associated violent deaths. Westlake Village, CA: Author .
50.
Nelson, J.R., Benner, G.J., Reid, R.C., Epstein, M.H., & Currin, D. (2002). The convergent validity of office discipline referrals with the CBCL-TRF. Journal of Emotional and Behavioral Disorders , 10, 181—188.
51.
Nelson, J.R., Colvin, G., & Smith, D.J. (1996). The effects of setting clear standards on students' social behavior in common areas of the school. The Journal of At-Risk Issues, 3(1), 10—18.
52.
Nelson, J.R., Gonzales, J.E., Epstein, M.H., & Benner, G.J. (2003). Administrative discipline contacts: A review of the literature. Behavioral Disorders, 28, 249—281.
53.
Nelson, J.R., & Roberts, M. (2000). Ongoing reciprocal teacher—student interactions involving disruptive behavior in general education classrooms. Journal of Emotional andBehavior Disorders, 8, 27—37.
54.
Paine, S., & Paine, C.K. (2002). Promoting safety and success in school by developing students' strengths. In M. R. Shinn, G. Stoner, & H. M. Walker (Eds.), Interventions for academic and behavior problems: Preventive and remedial approaches (pp. 89—108). Silver Spring, MD: National Association of School Psychologists .
55.
Patterson, G.R., & Bank, L. (1986). Bootstrapping your way in the nomological thicket . Behavioral Assessment, 8, 49—73.
56.
Quay, H.C., & Peterson, D.R. (1988). Revised behavior problem checklist. Miami, FL: University of Miami.
57.
Ryan-Arredondo, K., Renouf, K., Egyed, C., Doxey, M., Dobbins, M., Sanchez, S., et al. (2001). Threats of violence in schools: The Dallas Independent School District's response. Psychology in the Schools, 38, 185—196.
58.
Sadler, C. (2000). Effective behavior support implementation at the district level: Tigard-Tualatin School District. Journal of Positive Behavior Interventions , 2, 241—243.
59.
Scott, T.M. (2001). A schoolwide example of positive behavioral support . Journal of Positive Behavior Interventions, 3, 88—94.
60.
Skiba, R., Peterson, R., & Williams, T. (1997). Office referrals and suspension: Disciplinary intervention in middle schools. Education and Treatment of Children, 20, 295—315.
61.
Skinner, B.F. (1953). Science and human behavior. New York: Macmillan.
62.
Smith, H., & Thomas, S.P. (2000). Violent and nonviolent girls: Contrasting perceptions of anger experiences, school, and relationships. Issues in Mental Health Nursing, 21, 547—575.
63.
Snell, J.L., Mackenzie, E.P., & Frey, K.S. (2002). Bullying prevention in elementary schools: The importance of adult leadership, peer group support, and student socio-emotional skills. In M. R. Shinn, G. Stoner, & H. M. Walker (Eds.), Interventions for academic and behavior problems: Preventive and remedial approaches (pp. 351—372). Silver Spring, MD: National Association of School Psychologists .
64.
Sprague, J.R., Colvin, G., & Irvin, L.K. (1995). The Oregon school safety survey. Eugene: University of Oregon.
65.
Sprague, J.R., Sugai, G., Horner, R.H., & Walker, H. (1999). Using office discipline referral data to evaluate school-wide discipline and violence prevention interventions. Oregon School Study Council, 42(2), 1—18.
66.
Sprague, J.R., Walker, H.M., Sowards, S., Van Bloem, C., Eberhardt, P., & Marshall, B. (2002). Sources of vulnerability to school violence: Systems-level assessment and strategies to improve safety and climate. In G. Stoner, M. Shinn, & H. M. Walker (Eds.), Interventions for achievement and behavior problems (pp. 295—314). Silver Spring, MD: National Association of School Psychologists .
67.
Sugai, G., & Horner, R.H. (2002). The evolution of discipline practices: School-wide positive behavior supports. Child & FamilyBehavior Therapy24(1/2), 23—50.
68.
Sugai, G., Horner, R.H., Dunlap, G., Hieneman, M., Nelson, C.M., Scott, T., et al. (2000). Applying positive behavior support and functional behavioral assessment in schools. Journal of Positive Behavior Interventions, 2, 131—143.
69.
Sugai, G., Horner, R.H., & Gresham, F. (2002). Behaviorally effective school environments. In M. R. Shinn, G. Stoner, & H. M. Walker (Eds.), Interventions for academic and behavior problems: Preventive and remedial approaches (pp. 315—350). Silver Spring, MD: National Association for School Psychologists.
70.
Sugai, G., Sprague, J.R., Horner, R.H., & Walker, H.M. (2000). Preventing school violence: The use of office discipline referrals to assess and monitor school-wide discipline interventions . Journal of Emotional and Behavioral Disorders, 8, 94—101.
71.
Tobin, T., & Irvin, L.K. (1996). The Olweus Bully/Victim Questionnaire: Evidence and consequences regarding use in the United States. Reclaiming Children and Youth: Journal of Emotional and Behavioral Problems, 5(1), 29—33.
72.
Tobin, T.J., Lewis-Palmer, T., & Sugai, G. (2002). School-wide and individualized effective behavior support: An explanation and an example. Behavior Analysis Today, 3(1), 51—75. Retrieved May 1, 2002, from http://www.behavior-analyst-online.org/BAT/BAT31.pdf
73.
Tobin, T.J., & Sugai, G.M. (1999a). Discipline problems, placements, and outcomes for students with serious emotional disturbance. Behavioral Disorders, 24, 109—121.
74.
Tobin, T.J., & Sugai, G.M. (1999b). Using sixth-grade school records to predict violence, chronic discipline problems, and high school outcomes. Journal of Emotional and Behavioral Disorders, 7, 40—53.
75.
Tobin, T., Sugai, G., & Colvin, G. (1996). Patterns in middle school discipline records. Journal of Emotional and Behavioral Disorders, 4, 82—94.
76.
Tobin, T., Sugai, G., & Colvin, G. (2000, May). Using discipline referrals to make decisions . NASSP Bulletin, 84(616), 106—117.
77.
Todd, A.W., Haugen, L., Anderson, K., & Spriggs, M. (2002). Teaching recess: Low-cost efforts producing effective results. Journal of Positive Behavior Interventions, 4, 46—52.
78.
U.S. Department of Education, National Center for Educational Statistics, National Forum on Educational Statistics. (2002 ). Safety in numbers: Collecting and using incident data to make a difference in schools (NCES 2002-312, prepared by I. Hantman, G. Bairu, A. Barwick, B. Smith, B. Mack, S. Meston, et al. of the Crime, Violence, and Discipline Task Force). Washington, DC: Authors. Retrieved February 5, 2003, from http://nces.ed.gov/pubs2002/2002312.pdf
79.
Walker, H.M., Block-Pedego, A., Todis, B., & Severson, H. (1991). School archival records search (SARS): User's guide and technical manual. Longmont, CO: Sopris West.
80.
Walker, H.M., Colvin, G., & Ramsey, E. (1995). Antisocial behavior in school: Strategies and best practices. Pacific Grove, CA: Brooks/Cole .
81.
Walker, H.M., & Shinn, M.R. (2002). Structuring school-based interventions to achieve integrated primary, secondary, and tertiary prevention goals for safe and effective schools. In M. R. Shinn, G. Stoner, & H. M. Walker (Eds.), Interventions for academic and behavior problems: Preventive and remedial approaches (pp. 1—25). Silver Spring, MD: National Association of School Psychologists.
82.
Walker, H.M., Shinn, M.R., O'Neill, R.E., & Ramsey, E. (1987). A longitudinal assessment of the development of antisocial behavior in boys: Rationale, methodology, and first-year results . Remedial and Special Education, 8(4), 7—16.
83.
Walker, H.M., Stieber, S., & O'Neill, R.E. (1990). Middle school behavioral profiles of antisocial and at-risk control boys: Descriptive and predictive outcomes. Exceptionality, 1, 61—77.
84.
Walker, H.M., Stieber, S., Ramsey, E., & O'Neill, R.E. (1990). School behavioral profiles of arrested versus nonarrested adolescents. Exceptionality , 1, 249—265.
85.
Walker, H.M., Stieber, S., Ramsey, E., & O'Neill, R.E. (1991). Longitudinal prediction of the school achievement, adjustment, and delinquency of antisocial versus at-risk boys. Remedial and Special Education, 12(4), 43—51.
86.
Walker, H.M., Stieber, S., Ramsey, E., & O'Neill, R.E. (1993). Fifth grade school adjustment and later arrest rate: A longitudinal study of middle school antisocial boys . Journal of Child and Family Studies, 2, 295—315.
87.
Wright, J.A., & Dusek, J.B. (1998). Compiling base rates for disruptive behaviors from student discipline referral data. School Psychology Review , 27, 147—183.
88.
Zins, J.E., & Ponti, C.R. (1990). Best practices in school-based consultation. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology (Vol. 2, pp. 673—694). Washington, DC: National Association of School Psychologists.
