In recent years, there has been increased attention regarding systems-level implementation to support the sustained use of evidence-based interventions and supports in authentic early childhood settings. With this comes a need to accurately measure implementation fidelity of the critical features within a framework as well as individual practices. Program-Wide Support for Pyramid Model Implementation (PWS-PMI) provides an approach for early childhood programs to develop such a framework that can underpin evidence-based practices in their classrooms. This article describes an evaluation of the technical properties of the Supporting Program-wide Implementation Fidelity Instrument (SPIFI), a fidelity tool that was developed to be used by typical evaluators to measure PWS-PMI in these settings. Findings suggest that the instrument reliably demonstrated construct validity when used by typical evaluators to assess PWS-PMI and provides initial validation of the SPIFI as an objective measure for use in evaluative research and technical assistance.
BlairK.-S. C.FoxL.LentiniR. (2010). Use of positive behavior support to address the challenging behavior of young children within a community early childhood program. Topics in Early Childhood Special Education, 30(2), 68–79. https://doi.org/10.1177/0271121410372676
2.
BurchinalM.VandergriftN.PiantaR.MashburnA. (2010). Threshold analysis of association between child care quality and child outcomes for low-income children in pre-kindergarten programs. Early Childhood Research Quarterly, 25(2), 166–176. https://doi.org/10.1016/j.ecresq.2009.10.004
3.
DomitrovichC. E.MooreJ. E.GreenbergM. T. (2012). Maximizing the effectiveness of social-emotional interventions for young children through high-quality implementation of evidence-based interventions. In KellyB.PerkinsD. F. (Eds.), Handbook of implementation science for psychology in education (pp. 207–229). Cambridge University Press. https://doi.org/10.1017/CBO9781139013949.017
4.
DurlakJ. A. (2016). Programme implementation in social and emotional learning: Basic issues and research findings. Cambridge Journal of Education, 46, 333–345. https://doi.org/10.1080/0305764X.2016.1142504
5.
FixsenD. L.BlaseK. A.NaoomS. F.WallaceF. (2009). Core implementation components. Research on Social Work Practice, 19(5), 531–540. https://doi.org/10.1177/1049731509335549
6.
FixsenD. L.NaoomS. F.BlaseK. A.FriedmanR. M.WallaceF. (2005). Implementation research: A synthesis of the literature (FMHI Publication #231). The National Implementation Research Network, University of South Florida.
7.
FoxL.DunlapG.HemmeterM. L.JosephG. E.StrainP. S. (2003). The teaching pyramid: A model for supporting social competence and preventing challenging behavior in young children. Young Children, 58(4), 48–52. https://doi.org/10.1002/cbl.20134
8.
FoxL.HemmeterM. L. (2009). A programwide model for supporting social emotional development and addressing challenging behavior in early childhood settings. In Handbook of positive behavior support (pp. 177–202). Springer.
9.
FoxL.HemmeterM. L.JackS.BinderD. (2017). Early childhood program-wide PBS benchmarks of quality. National Center for Pyramid Model Innovations.
10.
FoxL.HemmeterM. L.SnyderP. S. (2014). Teaching Pyramid Observation Tool for Preschool Classrooms (TPOT™), research edition. Paul H. Brookes.
HallgrenK. A. (2012). Computing inter-rater reliability for observational data: An overview and tutorial. Tutorials in Quantitative Methods for Psychology, 8(1), 23–34. https://doi.org/10.20982/tqmp.08.1.p023
13.
HaytonJ. C.AllenD. G.ScarpelloV. (2004). Factor retention decisions in exploratory factor analysis: A tutorial on parallel analysis. Organizational Research Methods, 7(2), 191–205. https://doi.org/10.1177/1094428104263675
14.
HemmeterM. L.BartonE.FoxL.VatlandC.HenryG.PhamL.HorthK.TaylorA.BinderD. P.von der EmbseM.VeguillaM. (2022). Program-wide implementation of the Pyramid Model: Supporting fidelity at the program and classroom levels. Early Childhood Research Quarterly, 59, 56–73. https://doi.org/10.1016/j.ecresq.2021.10.003
15.
HemmeterM. L.FoxL.SnyderP. (2013). A tiered model for promoting social-emotional competence and addressing challenging behavior. In BuysseV.Peisner-FeinbergE. (Eds.), Handbook of response to intervention in early childhood (pp. 85–101). Brookes.
16.
HemmeterM. L.FoxL.SnyderP.AlginaJ.HardyJ. K.BishopC.VeguillaM. (2021). Corollary child outcomes from the Pyramid Model professional development intervention efficacy trial. Early Childhood Research Quarterly, 54, 204–218.
17.
HemmeterM. L.SnyderP. A.FoxL.AlginaJ. (2016). Evaluating the implementation of the Pyramid Model for Promoting Social-Emotional Competence in early childhood classrooms. Topics in Early Childhood Special Education, 36(3), 133–146. https://doi.org/10.1177/0271121416653386
18.
HornJ. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30(2), 179–185. https://doi.org/10.1007/BF02289447
19.
LandisJ.KochG. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174. https://doi.org/10.2307/2529310
20.
MeekS. E.GilliamW. S. (2016). Expulsion and suspension as matters of social justice and health equity [Discussion Paper]. National Academy of Medicine.
21.
MetzA.BartleyL.BallH.WilsonD.NaoomS.RedmondP. (2015). Active implementation frameworks for successful service delivery: Catawba county child wellbeing project. Research on Social Work Practice, 25(4), 415–422. https://doi.org/10.1177/1049731514543667
22.
PiantaR. C.LaParoK. M.HamreB. K. (2008). Classroom Assessment Scoring System™ (CLASS™) Manual: Pre-K. Paul H. Brookes Publishing.
23.
QuesenberryA. C.HemmeterM. L.OstroskyM. M. (2011). Addressing challenging behaviors in Head Start: A closer look at program policies and procedures. Topics in Early Childhood Special Education, 30(4), 209–220. https://doi.org/10.1177/0271121410371985
24.
RaudenbushS. W.BrykA. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). SAGE.
25.
ShavelsonR.WebbN. (1991). Generalizability theory: A primer. SAGE.
26.
SnijdersT.BoskerR. (2012). Multilevel Analysis. An introduction to basic and advanced multilevel modeling (2nd ed.). SAGE.
27.
SnyderP. A.HemmeterM. L.FoxL. (2022). Essentials of practice-based coaching. Supporting effective practices in early childhood. Paul H. Brookes.
28.
SnyderP. A.HemmeterM. L.FoxL.BishopC. C.MillerM. D. (2013). Developing and gathering psychometric evidence for a fidelity instrument: The Teaching Pyramid Observation Tool–Pilot version. Journal of Early Intervention, 35(2), 150–172. https://doi.org/10.1177/1053815113516794
29.
StataCorp. (2017). Stata Statistical Software: Release 15.
30.
SteedE. A.PomerleauT.HornerR. H. (2012). Preschool-wide evaluation tool (PreSET). Paul H. Brookes.
31.
SteedE. A.WebbM. Y. L. (2013). The Psychometric Properties of the Preschool-Wide Evaluation Tool (PreSET). Journal of Positive Behavior Interventions, 15(4), 231–241. https://doi.org/10.1177/1098300712459357
32.
SugaiG.HornerR. (2002). The evolution of discipline practices: School-wide positive behavior supports. Child & Family Behavior Therapy, 24(1–2), 23–50. https://doi.org/10.1300/J019v24n01_03
WasikB. A.MatteraS. K.LloydC. M.BollerK. (2013). Intervention dosage in early childhood care and education: It’s complicated (OPRE Research Brief OPRE 2013-15). Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
35.
WillisG. B. (2004). Cognitive interviewing: A tool for improving questionnaire design. SAGE.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
