Sustainability of effective practices in schools is a critical area for research in any domain. The purpose of this article is to describe and evaluate the validity and reliability of a recently developed research instrument designed to evaluate schools’ capacity to sustain school-wide positive behavior support (SWPBS) efforts at the universal tier. The School-Wide Universal Behavior Sustainability Index–School Teams (SUBSIST) was created to assess factors (of the context, implementer practices, and outcomes) that enhance or prevent sustainability of SWPBS. Content of the web-based survey was identified through literature review, and initial validation analyses included ratings of content validity by an expert panel (n = 21) and assessment of internal consistency, test–retest reliability, interrater reliability, and concurrent validity (with SWPBS fidelity of implementation data) through a pilot study (n = 25). Results indicated strong psychometric properties for assessing sustainability. The authors discuss the results in terms of future research in enhancing SWPBS sustainability.
AlgozzineK.AlgozzineB. (2007). Classroom instructional ecology and school-wide positive behavior support. Journal of Applied School Psychology, 24, 29–47.
2.
AlreckP. L.SettleR. B. (1995). The survey research handbook: Guidelines and strategies for conducting a survey. New York, NY: McGraw-Hill.
3.
BambaraL.NonnemacherS.KernL. (2009). Sustaining school-based individualized Positive Behavior Support: Perceived barriers and enablers. Journal of Positive Behavior Interventions, 11, 161–178.
4.
BermanP.McLaughlinM. W. (1976). Implementation of educational innovations. Educational Forum, 40, 345–370.
5.
BiglanA. (2004). Contextualism and the development of effective prevention practices. Prevention Science, 5, 15–21.
6.
BradshawC. P.MitchellM. M.LeafP. J. (2010). Examining the effects of school-wide positive behavioral interventions and supports on student outcomes: Results from a randomized controlled effectiveness trial in elementary schools. Journal of Positive Behavior Interventions, 12, 133–148.
7.
Center on Positive Behavioral Interventions and Supports. (2010). Schools that are implementing SWPBIS. Retrieved from www.pbis.org
8.
CurtisM. J.CastilloJ. M.CohenR. M. (2008). Best practices in system-level change. In ThomasA.GrimesJ. P. (Eds.), Best practices in school psychology (Vol. 5, pp. 887–901). Bethesda, MD: National Association of School Psychologists.
9.
DavisL. (1992). Instrument review: Getting the most from your panel of experts. Applied Nursing Research, 5, 194–197.
10.
DillmanD. A. (1983). Mail and other self-administered questionnaires. In RossiP.WrightJ. D.AndersonA. B. (Eds.), The handbook of survey research (pp. 359–377). New York, NY: Academic Press.
11.
DoolittleJ. H. (2006). Sustainability of positive behavior supports in schools (Unpublished doctoral dissertation). University of Oregon, Eugene.
12.
ElliottS. N.WittJ. C.KratochwillT. R.StoiberK. C. (2002). Selecting and evaluating classroom interventions. In ShinnM. R.WalkerH. M.StonerG. (Eds.), Interventions for academic and behavior problems II: Preventive and remedial approaches (pp. 243–294). Bethesda, MD: National Association of School Psychologists.
13.
FixsenD. L.NaoomS. F.BlaseK. A.FriedmanR. M.WallaceF. (2005). Implementation research: Synthesis of the literature (FMHI Publication No. 231). Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, the National Implementation Research Network.
14.
FullanM. (2005). Leadership and sustainability. Thousand Oaks, CA: Corwin Press.
15.
GableR. K.WolfJ. W. (1993). Instrument development in the affective domain: Measuring attitudes and values in corporate and school settings. Boston, MA: Kluwer Academic.
16.
GrantJ. S.DavisL. L. (1997). Selection and use of content experts for instrument development. Research in Nursing & Health, 20, 269–274.
17.
HanS. S.WeissB. (2005). Sustainability of teacher implementation of school-based mental health programs. Journal of Abnormal Child Psychology, 33, 665–679.
18.
HienemanM.DunlapG. (2001). Factors affecting the outcomes of community-based behavioral support: II. Factor category importance. Journal of Positive Behavior Interventions, 3, 67–74.
19.
HornerR. H. (2009, March). Expanding the science, values and vision of Positive Behavior Support. Paper presented at the Sixth International Association for Positive Behavior Support Conference, Jacksonville, FL.
20.
HornerR. H.SugaiG.SmolkowskiK.EberL.NakasatoJ.ToddA. W.EsperanzaJ. (2009). A randomized, wait-list controlled effectiveness trial assessing school-wide positive behavior support in elementary schools. Journal of Positive Behavior Interventions, 11, 133–144.
21.
HornerR. H.ToddA. W.Lewis-PalmerT.IrvinL. K.SugaiG.BolandJ. B. (2004). The School-wide Evaluation Tool (SET): A research instrument for assessing school-wide positive behavior support. Journal of Positive Behavior Interventions, 6, 3–12.
22.
KincaidD.ChildsK.BlaseK. A.WallaceF. (2007). Identifying barriers and facilitators in implementing schoolwide positive behavior support. Journal of Positive Behavior Interventions, 9, 174–184.
23.
KincaidD.ChildsK.GeorgeH. (2005). School-wide benchmarks of quality (Unpublished instrument). University of South Florida, Tampa.
24.
LassenS. R.SteeleM. M.SailorW. (2006). The relationship of school-wide positive behavior support to academic achievement in an urban middle school. Psychology in the Schools, 43, 701–712.
25.
LathamG. (1988). The birth and death cycles of educational innovations. Principal, 68, 41–43.
26.
LorhmannS.FormanM.MartinS.PalmieriS. (2008). Social context and personal belief barriers that impede the adoption of PBS at the universal level implementation. Journal of Positive Behavior Interventions, 10, 256–259.
27.
LynnM. (1986). Determination and quantification of content validity. Nursing Research, 35, 382–385.
28.
McIntoshK. (2010). SUBSIST sustainability checklist. Vancouver, Canada: University of British Columbia.
29.
McIntoshK.ChardD. J.BolandJ. B.HornerR. H. (2006). Demonstration of combined efforts in school-wide academic and behavioral systems and incidence of reading and behavior challenges in early elementary grades. Journal of Positive Behavior Interventions, 8, 146–154.
30.
McIntoshK.DoolittleJ. D.VincentC. G.HornerR. H.ErvinR. A. (2009). School-wide universal behavior sustainability index: School teams. Vancouver, Canada: University of British Columbia.
31.
McIntoshK.HornerR. H.SugaiG. (2009). Sustainability of systems-level evidence-based practices in schools: Current knowledge and future directions. In SailorW.DunlapG.SugaiG.HornerR. H. (Eds.), Handbook of positive behavior support (pp. 327–352). New York, NY: Springer.
32.
RansfordC. R.GreenbergM. T.DomitrovichC. E.SmallM.JacobsonL. (2009). The role of teachers’ psychological experiences and perceptions of curriculum supports on the implementation of a social and emotional learning curriculum. School Psychology Review, 38, 510–532.
33.
RubioD. M.Berg-WagnerM.TebbS. S.LeeE. S.RauchS. (2003). Objectifying content validity: Conducting a content validity study in social work research. Social Work Research, 27, 94–104.
34.
SarasonS. B. (1971). The culture of the school and the problem of change. Boston, MA: Allyn & Bacon.
35.
ScottT. M.BarrettS. B. (2004). Using staff and student time engaged in disciplinary procedures to evaluate the impact of school-wide PBS. Journal of Positive Behavior Interventions, 6, 21–27.
36.
SheatsleyP. B. (1983). Questionnaire construction and item writing. In RossiP.WrightJ. D.AndersonA. B. (Eds.), The handbook of survey research (pp. 195–230). New York, NY: Academic Press.
37.
SugaiG.HornerR. H. (2009). Defining and describing schoolwide positive behavior support. In SailorW.DunlapG.SugaiG.HornerR. H. (Eds.), Handbook of positive behavior support (pp. 307–326). New York, NY: Springer.
38.
SugaiG.HornerR. H.SailorW.DunlapG.EberL.LewisT. J.. . . NelsonC. M. (2005). School-wide positive behavior support: Implementers’ blueprint and self-assessment. Eugene: University of Oregon.
39.
SugaiG.Lewis-PalmerT. L.ToddA. W.HornerR. H. (2001). School-wide Evaluation Tool (SET). Eugene, OR: Educational and Community Supports. Retrieved from http://www.pbis.org
40.
VincentC. G.SpauldingS. A.TobinT. J. (2010). A reexamination of the psychometric properties of the School-wide Evaluation Tool (SET). Journal of Positive Behavior Interventions, 12, 161–179.
41.
WalzC. F.StricklandO.LenzE. (1991). Measurement in nursing research (Vol. 2). Philadelphia, PA: F.A. Davis.
