Challenges in the measurement of racial disparities in school discipline are a significant barrier to identifying policy and programmatic reforms that are effective at closing gaps. This article reviews key measurement issues and presents a set of empirical analyses as an illustrative case study. Specifically, we reframe the interpretation of discipline data in light of initiatives designed to reduce racial discipline disparities. We also characterize common metrics and recognize several additional ones for use in discipline disproportionality outcome evaluations. Leveraging a statewide policy reform as an example, we report findings from a quasi-experimental evaluation, which demonstrated that the various metrics can point to differing conclusions. We conclude with proposed guiding principles for the selection and use of discipline disproportionality metrics in evaluations.
American Psychological Association Zero Tolerance Task Force. (2008). Are zero tolerance policies effective in the schools? An evidentiary review and recommendations. American Psychologist, 63(9), 852–862. https://doi.org/10.1037/0003-066X.63.9.852
3.
AndersonA. R.ChristensonS. L.SinclairM. F.LehrC. A. (2004). Check & Connect: The importance of relationships for promoting engagement with school. Journal of School Psychology, 42(2), 95–113.
4.
AnyonY.LechugaC.OrtegaD.DowningB.GreerE.SimmonsJ. (2018). An exploration of the relationships between student racial background and the school sub-contexts of office discipline referrals: A critical race theory analysis. Race, Ethnicity and Education, 21(3), 390–406. https://doi.org/10.1080/13613324.2017.1328594
5.
BalA.KozleskiE. B.SchraderE. M.RodriguezE. M.PeltonS. (2014). Systemic transformation from the ground-up: Using learning lab to design culturally responsive schoolwide positive behavioral supports. Remedial and Special Education, 35(6), 327–339.
BottianiJ. H.BradshawC. P.GregoryA. (2018). Nudging the gap: Introduction to the special issue “Closing in on discipline disproportionality.”School Psychology Review, 47(2), 109–117. https://doi.org/10.17105/spr-2018-0023.v47-2
8.
BottianiJ. H.BradshawC. P.MendelsonT. (2017). A multilevel examination of racial disparities in high school discipline: Black and White adolescents’ perceived equity, school belonging, and adjustment problems. Journal of Educational Psychology, 109(4), 532–545. https://doi.org/10.1037/edu0000155
9.
BradshawC. P.DebnamK. J.Lindstrom JohnsonS.PasT. E.HershfeldtP.AlexanderA.BarrettS.LeafP. (2014). Maryland’s evolving system of social, emotional, and behavioral interventions in public schools: The Maryland Safe and Supportive Schools Project. Adolescent Psychiatry, 4(3), 194–206. https://doi.org/10.2174/221067660403140912163120
10.
BradshawC. P.MitchellM. M.O’BrennanL. M.LeafP. J. (2010). Multilevel exploration of factors contributing to the overrepresentation of black students in office disciplinary referrals. Journal of Educational Psychology, 102(2), 508. https://doi.org/10.1037/a0018450
11.
BradshawC. P.PasE. T.BottianiJ. H.DebnamK. J.ReinkeW. M.HermanK. C.RosenbergM. S. (2018). Promoting cultural responsivity and student engagement through double check coaching of classroom teachers: An efficacy study. School Psychology Review, 47(2), 118–134. https://doi.org/10.17105/spr-2017-0119.v47-2
12.
BradshawC. P.PasE. T.DebnamK. J.Lindstrom JohnsonS. (2015). A focus on implementation of positive behavioral interventions and supports (PBIS) in high schools: Associations with bullying and other indicators of school disorder. School Psychology Review, 44(4), 480–498.
13.
BradshawC. P.WaasdorpT. E.DebnamK. J.JohnsonS. L. (2014). Measuring school climate in high schools: A focus on safety, engagement, and the environment. Journal of School Health, 84(9), 593–604. https://doi.org/10.1111/josh.12186
14.
BradshawC. P.WaasdorpT. E.LeafP. J. (2012). Effects of school-wide positive behavioral interventions and supports on child behavior problems. Pediatrics, 130(5), e1136–e1145. https://doi.org/10.1542/peds.2012-0243
15.
CamachoK. A.KrezmienM. P. (2019). Individual and school level factors contributing to disproportionate suspension rates: A multilevel analysis of one state. Journal of Emotional and Behavioral Disorders, 27(4), 209–220. doi:10.1177/1063426618769065
16.
CamachoK. A.KrezmienM. P. (2020). A statewide analysis of school discipline policies and suspension practices. Preventing School Failure: Alternative Education for Children and Youth, 64(1), 55–66. https://doi.org/10.1080/1045988X.2019.1678010
ChinM. J.QuinnD. M.DhaliwalT. K.LovisonV. S. (2020). Bias in the air: A nationwide exploration of teachers’ implicit racial attitudes, aggregate bias, and student outcomes. Educational Researcher, 1–13. https://doi.org/10.3102/0013189X20937240
19.
CholewaB.HullM. F.BabcockC. R.SmithA. D. (2018). Predictors and academic outcomes associated with in-school suspension. School Psychology Quarterly, 33(2), 191.
20.
CrenshawK.OcenP.NandaJ. (2015). Black girls matter: Pushed out, overpoliced, and underprotected. Center for Intersectionality and Social Policy Studies, Columbia University.
21.
CurranF. C. (2020). A matter of measurement: How different ways of measuring racial gaps in school discipline can yield drastically different conclusions about racial disparities in discipline. Educational Researcher, 49(5), 382–387. doi:10.3102/0013189X20923348
FeaginJ. O. E. R. (2020). White racial frame centuries of racial framing and counter framing (3rd ed.). Routledge.
26.
GageN. A.Grasley-BoyN.Peshak GeorgeH.ChildsK.KincaidD. (2019). A quasi-experimental design analysis of the effects of school-wide positive behavior interventions and supports on discipline in Florida. Journal of Positive Behavior Interventions, 21(1), 50–61.
27.
GarayM. M.RemediosJ. D. (2021). A review of White-centering practices in multiracial research in social psychology. Social and Personality Psychology Compass, 15(10), e12642.
GirvanE. J.GionC.McIntoshK.SmolkowskiK. (2017). The relative contribution of subjective office referrals to racial disproportionality in school discipline. School Psychology Quarterly, 32(3), 392.
30.
GirvanE. J.McIntoshK.Santiago-RosarioM. R. (2021). Associations between community-level racial biases, office discipline referrals, and out-of-school suspensions. School Psychology Review, 50(2–3), 288–302.
31.
GirvanE. J.McIntoshK.SmolkowskiK. (2019). Tail, tusk, and trunk: What different metrics reveal about racial disproportionality in school discipline. Educational Psychologist, 54(1), 40–59. doi:10.1080/00461520.2018.1537125
32.
GLSEN (2016). Educational exclusion: Drop out, push out, and school-to-prison pipeline among LGBTQ youth. Author.
GregoryA.OsherD.BearG. G.JagersR. J.SpragueJ. R. (2021). Good intentions are not enough: Centering equity in school discipline reform. School Psychology Review, 1–15. https://doi.org/10.1080/2372966X.2020.1861911
35.
GriffinC. B.StittR. L.HendersonD. X. (2020). Investigating school racial climate and private racial regard as risk and protector factors for Black high school students’ school engagement. Journal of Black Psychology, 46(6–7), 514–549. https://doi.org/10.1177/0095798420946895
36.
GulloG. L.BeachumF. D. (2020). Does implicit bias matter at the administrative level? A study of principal implicit bias and the racial discipline severity gap. Teachers College Record, 122(3), 1–28.
HuangF. L. (2020). Prior problem behaviors do not account for the racial suspension gap. Educational Researcher, 49(7), 493–502.
39.
IrvinL. K.TobinT. J.SpragueJ. R.SugaiG.VincentC. G. (2004). Validity of office discipline referral measures as indices of school-wide behavioral status and effects of school-wide behavioral interventions. Journal of Positive Behavior Interventions, 6(3), 131–147.
40.
JagersR. J.Rivas-DrakeD.WilliamsB. (2019). Transformative social and emotional learning (SEL): Toward SEL in service of educational equity and excellence. Educational Psychologist, 54(3), 162–184.
41.
KangS.HarveyE. A. (2020). Racial differences between Black parents’ and White teachers’ perceptions of attention-deficit/hyperactivity disorder behavior. Journal of Abnormal Child Psychology, 48(5), 661–672. doi.org/10.1007/s10802-019-00600-y
Maryland State Department of Education (MSDE). 2017. Reducing and eliminating disproportionate impact: Technical assistance guide for local education agencies and schools to address disproportionality in school discipline. https://eric.ed.gov/?id=ED589789
46.
McCaffreyD. F.RidgewayG.MorralA. R. (2004). Propensity score estimation with boosted regressions for evaluating causal effects in observational studies. Psychological Methods, 9(4), 403–425. https://doi.org/10.1037/1082-989X.9.4.403
47.
McIntoshK.BarnesA.EliasonB. M.MorrisK. (2014). Using discipline data within SWPBIS to identify and address disproportionality: A guide for school teams. Eugene: Center on Positive Behavioral Interventions and Supports. University of Oregon.
48.
McIntoshK.CampbellA. L.CarterD. R.ZumboB. D. (2009). Concurrent validity of office discipline referrals and cut points used in schoolwide positive behavior support. Behavioral Disorders, 34(2), 100–113.
49.
McIntoshK.GirvanE. J.Fairbanks FalconS.McDanielS. C.SmolkowskiK.BastableE.Santiago-RosarioM. R.IzzardS. A.NeseS. C.RhondaN. T.BaldyT. S. (2021). Equity-focused PBIS approach reduces racial inequities in school discipline: A randomized controlled trial. School Psychology, 36(6), 433.
NguyenB. M. D.NogueraP.AdkinsN.TeranishiR. T. (2019). Ethnic discipline gap: Unseen dimensions of racial disproportionality in school discipline. American Educational Research Journal, 56(5), 1973–1972. doi:10.3102/0002831219833919
52.
NishiokaV.ShigeokaS.LolichE. (2017). School discipline data indicators: A guide for districts and schools (REL 2017–240). U.S. Department of Education, Institute of Education Sciences. http://ies.ed.gov/ncee/edlabs
53.
PasE. T.BradshawC. P.MitchellM. M. (2011). Examining the validity of office discipline referrals as an indicator of student behavior problems. Psychology in the Schools, 48(6), 541–555.
54.
PasE. T.RyooJ. H.MusciR. J.BradshawC. P. (2019). A state-wide quasi-experimental effectiveness study of the scale-up of school-wide positive behavioral interventions and supports. Journal of School Psychology, 73, 41–55. https://doi.org/10.1016/j.jsp.2019.03.001
55.
PegueroA. A.ShekarkharZ. (2011). Latino/a student misbehavior and school punishment. Hispanic Journal of Behavioral Sciences, 33(1), 54–70.
56.
R Core Team. (2017). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org
57.
RiddleT.SinclairS. (2019). Racial disparities in school-based disciplinary actions are associated with county-level rates of racial bias. Proceedings of the National Academy of Sciences, 116(17), 8255–8260.
58.
RidgewayG.McCaffreyD. F.MorralA.GriffinB. A.BurgetteL. F. (2016). twang: Toolkit for weighting and analysis of nonequivalent groups. https://CRAN.R-project.org/package=twang
59.
RosenbaumP. R.RubinD. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41–55. https://doi.org/10.1093/biomet/70.1.41
60.
ScanlanJ. P. (2016). The mis-measure of health disparities. Journal of Public Health Management and Practice, 22(4), 415–419.
61.
SkibaR. J.ChungC. G.TrachokM.BakerT. L.SheyaA.HughesR. L. (2014). Parsing discipline disproportionality: Contributions of infractions, student, and school characteristics to out-of-school suspension and expulsion. American Educational Research Journal, 51(4), 640–670. doi: 10.3102/0002831214541670
62.
SkibaR. J.LosenD. J. (2016). From reaction to prevention: Turning the page on school discipline. American Educator, 39(4), 4–12.
63.
SkibaR. J.MichaelR. S.NardoA. C.PetersonR. L. (2002). The color of discipline: sources of racial and gender disproportionality in school punishment. Urban Review, 34(4), 317–342.
64.
SkibaR. J.SimmonsA. B.RitterS.GibbA. C.RauschM. K.CuadradoJ.ChungC.-G. (2008). Achieving equity in special education: History, status, and current challenges. Exceptional Children, 74(3), 264–288.
65.
SmolkowskiK.GirvanE. J.McIntoshK.NeseR. N.HornerR. H. (2016). Vulnerable decision points for disproportionate office discipline referrals: Comparisons of discipline for African American and White elementary school students. Behavioral Disorders, 41(4), 178–195.
66.
StataCorp. (2015). Stata statistical software: Release 14. Author.
67.
SugaiG.HornerR. R. (2006). A promising approach for expanding and sustaining school-wide positive behavior support. School Psychology Review, 35(2), 245–259.
68.
SugaiG.HornerR. H.DunlapG.HienemanM.LewisT. J.NelsonC. M.ScottT.LiaupsinC.SailorW.TurnbullA. P.TurnbullH. R.IIIWickhamD.WilcoxB.RuefM. (2000). Applying positive behavior support and functional behavioral assessment in schools. Journal of Positive Behavior Interventions, 2(3), 131–143.
69.
U.S. Department of Education, Office of Civil Rights. (2019). Civil rights data collection (CRDC), 2009-2016 [Data set]. http://ocrdata.ed.gov/
70.
U.S. Department of Education, Office of Civil Rights. (2021). Civil rights data collection (CRDC), 2017 to 2018 [Data set]. from http://ocrdata.ed.gov/
VincentC. G.RandallC.CartledgeG.TobinT. J.Swain-BradwayJ. (2011). Towards a conceptual integration of cultural responsiveness and school-wide positive behavior support. Journal of Positive Behavior Interventions, 13(4), 219–229. https://doi.org/10.1177/1098300711399765
73.
VincentC. G.SpragueJ. R.CHiXapkaidM.TobinT. J.GauJ. M. (2015). Effectiveness in schoolwide positive behavior interventions and supports, in reducing racially inequitable discipline exclusion. In LosenD. (Ed.), Closing the school discipline gap: Equitable remedies for excessive exclusion (pp. 207–221). Teachers College Press.
74.
VincentC. G.Swain-BradwayJ.TobinT. J.MayS. (2011). Disciplinary referrals for culturally and linguistically diverse students with and without disabilities: Patterns resulting from school-wide positive behavior support. Exceptionality, 19(3), 175–190.
75.
VincentC. G.TobinT. J. (2011). The relationship between implementation of school-wide positive behavior support (SWPBS) and disciplinary exclusion of students from various ethnic backgrounds with and without disabilities. Journal of Emotional and Behavioral Disorders, 19(4), 217–232.
76.
WinshipC.MorganS. L. (1999). The estimation of causal effects from observational data. Annual Review of Sociology, 25, 659–706.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
