The Effect of School-Wide Positive Behavior Interventions and Supports on Disciplinary Exclusions: A Conceptual Replication

Abstract

Disciplinary exclusions, particularly out-of-school suspension and expulsions, are a pressing concern for schools, as research demonstrates that they are associated with myriad deleterious outcomes such as increased risk for poor academic achievement, school dropout, and contact with juvenile justice. Research suggests that School-Wide Positive Behavior Interventions and Supports (SWPBIS), a prevention and intervention framework for addressing school-based problem behavior, can have a significant and meaningful impact on reducing the likelihood of student suspensions and expulsions. In this study, we conceptually replicated a series of previous studies conducted in other states and examined the effect of universal SWPBIS on disciplinary exclusions in California. Using propensity score matching, we examine differences in suspension and expulsion rates for 98 schools implementing universal SWPBIS with fidelity and 98 comparison schools not implementing SWPBIS. Results suggest that schools implementing SWPBIS with fidelity have significantly fewer suspensions. No effects were found for expulsions. Implications and recommendations for future research are discussed.

Keywords

schoolwide positive behavior supports disciplinary exclusions suspensions quasi-experimental design propensity score matching

Get full access to this article

View all access options for this article.

References

Algozzine

Barrett

Eber

George

Horner

Lewis

. . . Sugai

(2014). School-wide PBIS Tiered Fidelity Inventory. OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports. https://www.pbisapps.org/Resources/SWIS%20Publications/SWPBIS%20Tiered%20Fidelity%20Inventory%20(TFI).pdf

American Academy of Pediatrics. (2013). Policy statement: Out-of-school suspension and expulsion. Pediatrics, 131, e1000–e1007. https://doi.org/10.1542/peds.2012-3932

Anderson

K. P.

Ritter

G. W.

Zamarro

(2019). Understanding a vicious cycle: The relationship between student discipline and student academic outcomes. Educational Researcher, 48(5), 251–262. https://doi.org/10.3102/0013189X19848720

Azur

M. J.

Stuart

E. A.

Frangakis

Leaf

P. J.

(2011). Multiple imputation by chained equations: What is it and how does it work? International Journal of Methods in Psychiatric Research, 20, 40–49. https://doi.org/10.1002/mpr.329

Borenstein

Hedges

L. V.

Higgins

J. P. T.

Rothstein

H. R.

(2009). Introduction to meta analysis. John Wiley.

Bradshaw

C. P.

Koth

C. W.

Bevans

K. B.

Ialongo

Leaf

P. J.

(2008). The impact of school-wide Positive Behavioral Interventions and Supports (PBIS) on the organizational health of elementary schools. School Psychology Quarterly, 23, 462–473. https://doi.org/10.1037/a0012883

Bradshaw

C. P.

Waasdorp

T. E.

Leaf

P. J.

(2012). Effects of school-wide positive behavioral interventions and supports on child behavior problems. Pediatrics, 130, e1136–e1145. https://doi.org/10.1001/archpediatrics.2011.755

Chin

J. K.

Dowdy

Jimerson

S. R.

Rime

W. J.

(2012). Alternatives to suspensions: Rationale and recommendations. Journal of School Violence, 11, 156–173. https://doi.org/doi.org/10.1080/15388220.2012.652912

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum.

10.

Coyne

M. D.

Cook

B. G.

Therrien

W. J.

(2016). Recommendations for replication research in special education: A framework of systematic, conceptual replications. Remedial and Special Education, 37, 244–253. https://doi.org/10.1177/0741932516648463

11.

Crone

D. A.

Hawken

L. S.

Horner

R. H.

(2010). Responding to problem behavior in schools: The behavior education program (2nd ed.). Guilford Press.

12.

Dong

Peng

C. Y. J.

(2013). Principled missing data methods for researchers. SpringerPlus, 2, 1–17.

13.

Fabelo

Thompson

M. D.

Plotkin

Carmichael

D. P.

Marchbanks

III Booth

(2011). Breaking schools’ rules: A statewide study of how school discipline relates to students’ success and juvenile justice involvement. Council of State Governments Justice Center. https://csgjusticecenter.org/wp-content/uploads/2012/08/Breaking_Schools_Rules_Report_Final.pdf

14.

Fortson

Verbitsky-Savitz

Kopa

Gleason

(2012). Using an experimental evaluation of charter schools to test whether nonexperimental comparison group methods can replicate experimental impact estimates (NCEE 2012-4019). National Center for Education Evaluation and Regional Assistance.

15.

Gage

N. A.

Grasley-Boy

George

H. P.

Childs

Kincaid

(2019). A quasi-experimental design analysis of the effects of school-wide positive behavior interventions and supports on discipline in Florida. Journal of Positive Behavior Interventions, 21, 50–61. https://doi.org/10.1177/1098300718768208

16.

Gage

N. A.

Lee

Grasley-Boy

George

H. P.

(2018). The impact of school-wide positive behavior interventions and supports on school suspensions: A statewide quasi-experimental analysis. Journal of Positive Behavior Interventions, 20, 217–226. https://doi.org/10.1177/1098300718768204

17.

Gage

N. A.

Leite

Childs

Kincaid

(2017). Average treatment effect of school-wide positive behavioral interventions and supports on school-level academic achievement in Florida. Journal of Positive Behavior Interventions, 19, 158–167. https://doi.org/10.1177/1098300717693556

18.

Gage

N. A.

Stevens

R. N.

(2018). Rigor, replication, and reproducibility: Increasing the relevance of behavioral disorders research. Education and Treatment of Children, 41, 567–588.

19.

Gage

N. A.

Whitford

D. K.

Katsiyannis

(2018). A review of schoolwide positive behavior interventions and supports as a framework for reducing disciplinary exclusions. The Journal of Special Education, 52, 142–151. https://doi.org/10.1177/0022466918767847

20.

Grasley-Boy

Gage

N. A.

Lombardo

(2019). Effects of SWPBIS on disciplinary exclusions for students with and without disabilities. Exceptional Children, 86, 25–39. https://doi.org/10.1177/0014402919854196

21.

Hansen

B. B.

Fredrickson

Buckner

Errickson

Solenberger

(2018). Optmatch: Functions for optimal matching (R package version 0.9-8). https://cran.r-project.org/web/packages/optmatch/

22.

Imai

King

Stuart

Whitworth

(2017). MatchIt: Nonparametric preprocessing for parametric causal inference. Journal of Statistical Software, 42(8), 1–28. http://www.jstatsoft.org/v42/i08/

23.

Kang-Brown

Trone

Fratello

Daftary-Kapur

(2013). A generation later: What we’ve learned about zero tolerance in schools. Center on Youth Justice, Vera Institute of Justice. https://storage.googleapis.com/vera-web-assets/downloads/Publications/a-generation-later-what-weve-learned-about-zero-tolerance-in-schools/legacy_downloads/zero-tolerance-in-schools-policy-brief.pdf

24.

Kincaid

Childs

George

H. P.

(2005, 2010). School-Wide Benchmarks of Quality (BoQ) [Unpublished instrument]. Department of Child and Family Studies, University of South Florida. http://flpbs.fmhi.usf.edu/ProceduresTools.cfm

25.

Leite

(2017). Propensity score methods using R. SAGE.

26.

Losen

D. J.

Martin

(2018). The unequal impact of suspension on the opportunity to learn in California: What the 2016-17 rates tell us about progress. Center for Civil Rights Remedies at the UCLA Center for Civil Rights. https://www.civilrightsproject.ucla.edu/research/k-12-education/school-discipline/the-unequal-impact-of-suspension-on-the-opportunity-to-learn-in-ca/Suspension_Impact_California_2018_R6.pdf

27.

Losen

D. J.

Martinez

T. E.

(2013). Out of school and off track: The overuse of suspensions in American middle and high schools. The Center for Civil Rights Remedies at the Civil Rights Project of the University of California at Los Angeles.

28.

McIntosh

Massar

M. M.

Algozzine

R. F.

George

H. P.

Horner

R. H.

Lewis

T. J.

Swain-Bradway

(2017). Technical adequacy of the SWPBIS Tiered Fidelity Inventory. Journal of Positive Behavior Interventions, 19, 3–13. https://doi.org/10.1177/1098300716637193

29.

Mercer

S. H.

McIntosh

Hoselton

(2017). Comparability of fidelity measures for assessing tier 1 school-wide positive behavioral interventions and supports. Journal of Positive Behavior Interventions, 19, 195–204. https://doi.org/10.1177/1098300717693384

30.

Mitchell

B. S.

Hatton

Lewis

T. J.

(2018). An examination of the evidence-base of school-wide positive behavior interventions and supports through two quality appraisal processes. Journal of Positive Behavior Interventions, 20, 239–250. https://doi.org/1098300718768217

31.

Noltemeyer

A. L.

Ward

R. M.

McLoughlin

(2015). Relationship between school suspension and student outcomes: A meta-analysis. School Psychology Review, 44, 224–240. https://doi.org/10.17105/spr-14-0008.1

32.

OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports. (2017). Positive behavioral interventions & supports [Website]. https://www.pbis.org/

33.

Pas

E. T.

Ryoo

J. H.

Musci

R. J.

Bradshaw

C. P.

(2019). A state-wide quasi-experimental effectiveness study of the scale-up of school-wide positive behavioral interventions and supports. Journal of School Psychology, 73, 41–55. https://doi.org/10.1016/j.jsp.2019.03.001

34.

Raghunathan

T. E.

Lepkowski

J. M.

Van Hoewyk

Solenberger

(2001). A multivariate technique for multiply imputing missing values using a sequence of regression models. Survey Methodology, 27, 85–96.

35.

Raudenbush

S. W.

Bryk

A. S.

(2002). Hierarchical linear models: Applications and data analysis methods (Vol. 1). SAGE.

36.

R Core Team. (2014). R: A language and environment for statistical computing. R Foundation for Statistical Computing. http://www.R-project.org/

37.

Rosenbaum

P. R.

Rubin

D. B.

(1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41–55.

38.

Rumberger

R. W.

Losen

D. J.

(2016). The high cost of harsh discipline and its disparate impact. The Civil Rights Project/Proyecto Derechos Civiles.

39.

Ryoo

J. H.

Hong

Bart

W. M.

Shin

Bradshaw

C. P.

(2018). Investigating the effect of school-wide positive behavioral interventions and supports on student learning and behavioral problems in elementary and middle schools. Psychology in the Schools, 55, 629–643. https://doi.org/10.1002/pits.22134

40.

Shadish

W. R.

Cook

T. G.

Campbell

D. T.

(2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin Harcourt.

41.

Stuart

E. A.

Azur

Frangakis

Leaf

(2009). Multiple imputation with large data sets: A case study of the children’s mental health initiative. American Journal of Epidemiology, 169, 1133–1139. https://doi.org/10.1093/aje/kwp026

42.

Sugai

Lewis-Palmer

Todd

Horner

R. H.

(2005). School-wide Evaluation Tool, v. 2.1. Educational and Community Supports, University of Oregon.

43.

Sullivan

A. L.

Van Norman

E. R.

Klingbeil

D. A.

(2014). Exclusionary discipline of students with disabilities: Student and school characteristics predicting suspension. Remedial and Special Education, 35, 199–210. https://doi.org/10.1177/0741932513519825

44.

U.S. Department of Education. (2018). California compilation of school discipline laws and regulations. https://safesupportivelearning.ed.gov/sites/default/files/discipline-compendium/California%20School%20Discipline%20Laws%20and%20Regulations.pdf

45.

U.S. Department of Education, National Center for Educational Statistics. (2019). Status and trends in the education of racial and ethnic groups: Indicator 15: Retention, suspensions, and expulsion. https://nces.ed.gov/programs/raceindicators/indicator_RDA.asp

46.

van Buuren

Groothuis-Oudshoorn

. (2011). mice: Multivariate imputation by chained equations in R. Journal of Statistical Software, 45, 1–67. http://www.jstatsoft.org/v45/i03/

47.

Walker

H. M.

Seeley

J. R.

Small

Severson

H. H.

Graham

B. A.

Feil

E. G.

. . . Forness

S. R.

(2009). A randomized controlled trial of the first step to success early intervention: Demonstration of program efficacy outcomes in a diverse, urban school district. Journal of Emotional and Behavioral Disorders, 17, 197–212. https://doi.org/10.1177/1063426609341645

48.

What Works Clearinghouse. (2015). Procedures and standards handbook, Version 3.0. https://ies.ed.gov/ncee/wwc/Docs/referenceresources/wwc_procedures_v3_0_standards_handbook.pdf