In this 5-year prospective study, we identified practice, school, and district variables predicting sustained implementation of positive behavioral interventions and supports (PBIS), a widely implemented framework for improving social, emotional, behavioral, and academic outcomes. Uniquely, the study captured implementation before and after the onset of the COVID-19 pandemic. Personnel from 595 schools across 22 U.S. states completed a validated measure of implementation strategies during the 2018–2019 school year. Analyses included a multilevel logistic regression of school and district fidelity of implementation data, as well as school characteristics and district PBIS data. Results showed that most schools had sustained implementation after 5 years, and adequate fidelity of implementation and better team use of data for decision-making in Year 1 were the strongest predictors of sustained implementation in the 2022–2023 school year. School grade levels served and school locale were also predictive of sustained implementation. We discuss implications for how teams can improve sustained implementation of evidence-based practices through massive school disruptions.
AlgozzineR. F.BarrettS.EberL.GeorgeH.HornerR. H.LewisT. J.PutnamR. F.Swain-BradwayJ.McIntoshK.SugaiG. (2014). SWPBIS tiered fidelity inventory. OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports. http://www.pbis.org
2.
AndreouT. E.McIntoshK.RossS. W.KahnJ. D. (2015). Critical incidents in the sustainability of school-wide positive behavioral interventions and supports. Journal of Special Education, 49, 157–167. https://doi.org/10.1177/0022466914554298
Bacher-HicksA.ChiO. L.OrellanaA. (2023). Two years later: How COVID-19 has shaped the teacher workforce. Educational Researcher, 52(4), 219–229. https://doi.org/10.3102/0013189x231153659
5.
BarnardJ.RubinD. B. (1999). Small-sample degrees of freedom with multiple imputation. Biometrika, 86, 948–955.
6.
BatesD.MächlerM.BolkerB.WalkerS. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01
7.
BradshawC. P.PasE. T. (2011). A state-wide scale-up of Positive Behavioral Interventions and Supports (PBIS): A description of the development of systems of support and analysis of adoption and implementation. School Psychology Review, 40, 530–548.
8.
BradshawC. P.PasE. T.DebnamK. J.JohnsonS. L. (2021). A randomized controlled trial of MTSS-B in high schools: Improving classroom management to prevent EBDs. Remedial and Special Education, 42(1), 44–59. https://doi.org/10.1177/0741932520966727
9.
Center on Positive Behavioral Interventions and Supports. (2020). Positive Behavioral Interventions and Supports (PBIS) District Systems Fidelity Inventory (DSFI). University of Oregon. https://www.pbis.org/resource/dsfi
10.
ChoiN. G.Nathan MartiC.ChoiB. Y. (2024). Serious suicidal thoughts, suicide plans, and mental health service use among youth during the COVID-19 pandemic. Children and Youth Services Review, 161, 107665. https://doi.org/10.1016/j.childyouth.2024.107665
11.
CoffeyJ. H.HornerR. H. (2012). The sustainability of schoolwide positive behavior interventions and supports. Exceptional Children, 78(4), 407–422. https://doi.org/10.1177/001440291207800402
12.
CohenR.KincaidD.ChildsK. E. (2007). Measuring school-wide positive behavior support implementation: Development and validation of the benchmarks of quality. Journal of Positive Behavior Interventions, 9, 203–213.
13.
CollieR. J. (2023). Job demands and resources, teachers’ subjective vitality, and turnover intentions: An examination during COVID-19. Educational Psychology, 43(5), 452–471. https://doi.org/10.1080/01443410.2022.2036323
14.
CondliffeB.ZhuP.DoolittleF.van DokM.PowerH.DenisonD.KurkiA. (2022). Study of training in multi-tiered systems of support for behavior: Impacts on elementary school students’ outcomes (NCEE 2022-008). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. http://ies.ed.gov/ncee
15.
ElderK. I.ProchnowJ. E. (2016). PB4L school-wide: What will support the sustainability of the initiative?New Zealand Journal of Educational Studies, 51(1), 83–97. https://doi.org/10.1007/s40841-016-0036-1
16.
FoxR. A.LeifE. S.MooreD. W.FurlongerB.AndersonA.SharmaU. (2022). A systematic review of the facilitators and barriers to the sustained implementation of school-wide positive behavioral interventions and supports. Education and Treatment of Children, 45(1), 105–126.
17.
GageN. A.WhitfordD. K.KatsiyannisA. (2018). A review of schoolwide positive behavior interventions and supports as a framework for reducing disciplinary exclusions. The Journal of Special Education, 52, 142–151.
18.
Grasley-BoyN. M.GageN. A.LombardoM.AndersonL. (2022). The additive effects of implementing advanced tiers of SWPBIS with fidelity on disciplinary exclusions. Journal of Positive Behavior Interventions, 24(3), 183–195.
19.
GrundS.RobitzschA.LuedtkeO. (2021). mitml: Tools for multiple imputation in multilevel modeling [Computer software] (0.4-3 ed.). https://CRAN.R-project.org/package=mitml
HornerR. H.SugaiG.AndersonC. (2010). Examining the evidence base for school-wide positive behavior support. Focus on Exceptional Children, 42(8), 1–14. https://doi.org/10.17161/fec.v42i8.6906
22.
HornerR. H.ToddA. W.Lewis-PalmerT.IrvinL. K.SugaiG.BolandJ. B. (2004). The School-wide Evaluation Tool (SET): A research instrument for assessing school-wide positive behavior support. Journal of Positive Behavior Interventions, 6, 3–12.
KittelmanA.MercerS. H.McIntoshK.MorrisK. R.HattonH. L. (2022). Validation of a measure of district systems implementation of positive behavioral interventions and supports. Remedial and Special Education. https://doi.org/10.1177/07419325221114472
26.
LathamG. (1988). The birth and death cycles of educational innovations. Principal, 68, 41–43.
27.
LeeA.GageN. A. (2020). Updating and expanding systematic reviews and meta-analyses on the effects of school-wide positive behavior interventions and supports. Psychology in the Schools, 57(5), 783–804. https://doi.org/10.1002/pits.22336
28.
LittleR. J. A. (1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83(404), 1198–1202. https://doi.org/10.1080/01621459.1988.10478722
29.
MassarM.McIntoshK.MercerS. H.HoseltonR. (2017). Factor analysis of a fidelity of implementation measure for social behavior systems. Remedial and Special Education, 40(1), 16–24. https://doi.org/10.1177/0741932517736515
30.
MathewsS.McIntoshK.FrankJ. L.MayS. L. (2014). Critical features predicting sustained implementation of school-wide positive behavior support. Journal of Positive Behavior Interventions, 16, 168–178.
31.
McIntoshK.DoolittleJ.VincentC. G.HornerR. H.ErvinR. A. (2009). School-wide universal behavior sustainability index: School teams. University of British Columbia.
32.
McIntoshK.GirvanE. J.Fairbanks FalconS.McDanielS. C.SmolkowskiK.BastableE.Santiago-RosarioM. R.IzzardS.AustinS. C.NeseR. N. T.BaldyT. S. (2021). An equity-focused PBIS approach reduces racial inequities in school discipline: A randomized controlled trial. School Psychology, 36(6), 433–444. https://doi.org/10.1037/spq0000466
33.
McIntoshK.MacKayL. D.HumeA. E.DoolittleJ.VincentC. G.HornerR. H.ErvinR. A. (2011). Development and initial validation of a measure to assess factors related to sustainability of school-wide positive behavior support. Journal of Positive Behavior Interventions, 13(4), 208–218. https://doi.org/10.1177/1098300710385348
34.
McIntoshK.MassarM. M.AlgozzineR. F.GeorgeH. P.HornerR. H. J. T. L.Swain-BradwayJ. (2017). Technical adequacy of the SWPBIS tiered fidelity inventory. Journal of Positive Behavior Interventions, 19(1), 3–13. https://doi.org/10.1177/1098300716637193
35.
McIntoshK.MercerS. H.HumeA. E.FrankJ. L.TurriM. G.MathewsS. (2013). Factors related to sustained implementation of schoolwide positive behavior support. Exceptional Children, 79(3), 293–311.
36.
McIntoshK.MercerS. H.NeseR. N. T.Strickland-CohenM. K.HoseltonR. (2016). Predictors of sustained implementation of school-wide positive behavioral interventions and supports. Journal of Positive Behavior Interventions, 18, 209–218.
37.
McIntoshK.MercerS. H.NeseR. N. T.Strickland-CohenM. K.KittelmanA.HoseltonR.HornerR. H. (2018). Factors predicting sustained implementation of a universal behavior support framework. Educational Researcher, 47(5), 307–316. https://doi.org/10.3102/0013189X18776975
38.
McIntoshK.PredyL. K.UpretiG.HumeA. E.TurriM. G.MathewsS. (2014). Perceptions of contextual features related to implementation and sustainability of school-wide positive behavior support. Journal of Positive Behavior Interventions, 16(1), 31–43. https://doi.org/10.1177/1098300712470723
39.
MercerS. H.McIntoshK.HoseltonR. (2017). Comparability of fidelity measures for assessing tier 1 school-wide positive behavioral interventions and supports. Journal of Positive Behavior Interventions, 19, 195–204.
40.
MercerS. H.McIntoshK.Strickland-CohenM. K.HornerR. H. (2014). Measurement invariance of an instrument assessing sustainability of school-based universal behavior practices. School Psychology Quarterly, 29(2), 125–137. https://doi.org/10.1037/spq0000054
41.
Metis Associates. (2024). Nevada’s school climate transformation project: Building multi-tiered systems of support 2022-23 evaluation report. Nevada PBIS TA Center.
MolloyL. E.MooreJ. E.TrailJ.Van EppsJ. J.HopferS. (2013). Understanding real-world implementation quality and “active ingredients” of PBIS. Prevention Science, 14, 593–605.
44.
MurphyA.PinkertonL. M.BrucknerE.RisserH. J. (2021). The impact of the novel coronavirus disease 2019 on therapy service delivery for children with disabilities. The Journal of Pediatrics, 231, 168–177. https://doi.org/10.1016/j.jpeds.2020.12.060
NeseR. N. T.McIntoshK.NeseJ. F. T.BloomJ.JohnsonN. W.PhillipsD.RichterM.HoseltonR. (2016). Predicting abandonment of school-wide behavior interventions. Behavioral Disorders, 42, 261–270.
48.
Ozamiz-EtxebarriaN.Idoiaga MondragonN.Bueno-NotivolJ.Pérez-MorenoM.SantabárbaraJ. (2021). Prevalence of anxiety, depression, and stress among teachers during the COVID-19 pandemic: A rapid systematic review with meta-analysis. Brain Sciences, 11(9), 1172. https://www.mdpi.com/2076-3425/11/9/1172
49.
PressleyT.HaC.LearnE. (2021). Teacher stress and anxiety during COVID-19: An empirical study. School Psychology, 36(5), 367–376. https://doi.org/10.1037/spq0000468
R Core Team. (2023). R: The R project for statistical computing.
52.
SafranS. P. (2006). Using the effective behavior supports survey to guide development of schoolwide positive behavior support. Journal of Positive Behavior Interventions, 8, 3–9.
53.
Santiago-RosarioM. R.McIntoshK.IzzardS.Cohen LissmanD.CalhounT. E. (2023). Is positive behavioral interventions and supports (PBIS) an evidence-based practice? University of Oregon. www.pbis.org
54.
SimonsenB.GoodmanS.RobbieK.PowerM.RodriguezC.BurnsD. (2021). Effective instruction as a protective factor. Center on PBIS, University of Oregon. www.pbis.org
55.
SteinerE. D.DoanS.WooA.GittensA. D.LawrenceR. A.BerdieL.WolfeR. L.GreerL.SchwartzH. L. (2022). Restoring teacher and principal well-being is an essential step for rebuilding schools: Findings from the state of the American teacher and state of the American principal surveys. RAND Corporation. https://doi.org/10.7249/RRA1108-4
56.
SugaiG.HornerR. H. (2020). Sustaining and scaling positive behavioral interventions and supports: Implementation drivers, outcomes, and considerations. Exceptional Children, 86, 120–136.
57.
SugaiG.HornerR. H.ToddA. W. (2000). PBIS self-assessment survey 2.0. Educational and Community Supports. http://www.pbisapps.org
WardC. S.HarmsA. L.St MartinK.CusumanoD.RussellC.HornerR. H. (2022). Development and technical adequacy of the district capacity assessment. Journal of Positive Behavior Interventions, 24(2), 87–98.
60.
WolfK.SchmitzJ. (2024). Scoping review: Longitudinal effects of the COVID-19 pandemic on child and adolescent mental health. European Child & Adolescent Psychiatry, 33(5), 1257–1312. https://doi.org/10.1007/s00787-023-02206-8
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
