Using Office Discipline Referral Data for Decision Making About Student Behavior in Elementary and Middle Schools

Abstract

In this evaluation we used Messick's construct validity as a conceptual framework for an empirical study assessing the validity of use, utility, and impact of office discipline referral (ODR) measures for data-based decision making about student behavior in schools. The Messick approach provided a rubric for testing the fit of our theory of use of ODR measures with empirical data on reported and actual use. It also facilitated our demonstration of Messick's principle that validation is both a developmental and an ongoing collaborative process among developers of educational and psychological measures, researchers interested in theories underlying such measures, and educators who use these measures in professional practice. We used a single-group, nonexperimental evaluation design to survey users of ODR measures from the standardized School Wide Information System in 22 elementary and 10 middle schools; respondents included school staff involved exclusively with data entry and staff actively involved in data-based decision making. Results were highly consistent across 2 independent data sources—electronic database records of actual access of summaries of ODR measures and self-report survey responses regarding frequencies and types of uses of ODR measures for decision making. Results indicated that ODR measures are regularly used for a variety of types of data-based decision making and are regarded as both efficient and effective for those purposes. We discuss implications of our SWIS ODR validity evaluation results within the context of the Messick framework.

Get full access to this article

View all access options for this article.

References

Blonigen, B. , Harbaugh, W. , Singell, L. , Horner, R.H. , Irvin, L. , & Smolkowski, K. (2005). Application of economic analysis to school-wide positive behavior support (SW-PBS) programs. Manuscript in preparation.

Gersten, R. , Keating, T. , & Irvin, L.K. (1995). The burden of proof: Validity as improvement of instructional practice. Exceptional Children, 61, 510—520.

Irvin, L.K. , Tobin, T.J. , Sprague, J.R. , Sugai, G. , & Vincent, C.G. (2004). Validity of office discipline referral measures as indices of school-wide behavioral status and effects of school-wide behavioral interventions. Journal of Positive Behavior Interventions , 6, 131—147.

Kane, M.T. (1992). An argument-based approach to validity. Psychological Bulletin, 112, 527—535.

LeTendre, G. (2000). The “problem” of minority education in an international perspective. International Journal of Educational Research, 33, 577—653.

May, S. , Ard, W., III , Todd, A.W. , Horner, R.H. , Glasgow, A. , Sugai, G. , et al. (2003). Schoolwide information system. Eugene: Educational and Community Supports, University of Oregon.

Messick, S. (1988). The once and future issues of validity: Assessing the meaning and consequences of measurement. In H. Wainer & H. Braun , (Eds.), Test validity (pp. 33—45). Hillsdale, NJ: Erlbaum.

Nakasato, J. (2000). Data-based decision making in Hawaii's behavior support effort. Journal of Positive Behavior Interventions , 2, 247—251.

Posavac, E.J. , & Carey, R.G. (1997). Program evaluation: Methods and case studies. Upper Saddle River, NJ: Prentice Hall.

10.

Putnam, R.F. , Luiselli, J.K. , & Handler, M.W. (2001). Improving discipline practices in public schools: Description of a whole-school and district-wide model of behavior analysis consultation. The Behavior Analyst Today, 2, 20—27.

11.

Sugai, G. , & Horner, R.H. (2002). The evolution of discipline practices: School-wide positive behavior supports. Child & Family Behavior Therapy , 24(1/2), 23—50.

12.

Sugai, G. , Sprague, J.R. , Horner, R.H. , & Walker, H.M. (2000). Preventing school violence: The use of office discipline referrals to assess and monitor school-wide discipline interventions . Journal of Emotional and Behavioral Disorders, 8, 94—101.

13.

Taylor-Greene, S. , Brown, D. , Nelson, L. , Longton, J. , Gassman, T. , Cohen, J. , et al. (1997). School-wide behavioral support: Starting the year off right. Journal of Behavioral Education, 7, 99—112.

14.

Taylor-Greene, S.J. , & Kartub, D.T. (2000). Durable implementation of school-wide behavior support: The high five program. Journal of Positive Behavior Interventions, 2, 233—235.

15.

Tobin, T. , Sugai, G. , & Colvin, G. (2000). A guide to the creation and use of discipline referral graphs. NASSP Bulletin, 84(616), 106—117.