Implementation of evidence-based, Tier 1 social-emotional learning (SEL) programming that supports healthy relationships skills is recognized as a key mental health promotion and violence prevention strategy for youth. However, work specifically exploring how to support the high-quality implementation of such programming with Canadian teachers and schools is just beginning to emerge. Drawing on implementation frameworks that emphasize the importance of setting, provider, and implementation process characteristics for understanding program implementation outcomes, this prospective, longitudinal study explores implementation of the Fourth R, a SEL-based healthy relationships program, in a Western Canadian province using a sample of middle school teachers. The aim of this mixed-methods study was to illuminate relationships among teachers’ attitudes toward evidence-based programming, perceptions of organizational climate, and training experience with program implementation outcomes (dosage, quality, fidelity). Findings of this exploratory study identified that these characteristics influenced implementation dosage, quality, and fidelity in differential ways. Qualitative data drawn from teacher interviews supported quantitative findings and highlighted the importance of organizational support for high-quality implementation. We discuss areas for further study, given that there are significant gaps in knowledge about teachers’ attitudes toward evidence-based programming, school climate, and other systemic factors in the Canadian context, as well as relevance of study findings to the field of school psychology.
AaronsG. A. (2006). Transformational and transactional leadership: Association with attitudes toward evidence-based practice. Psychiatric Services, 57, 1162-1169.
2.
AaronsG. A.EhrhartM. G.FarahnakL. R.SklarM. (2013). Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annual Review of Public Health, 35, 255-274.
3.
AaronsG. A.GlissonC.HoagwoodK.KelleherK.LandsverkJ.CafriG., & The Research Network on Youth Mental Health. (2010). Psychometric properties and United States national norms of the Evidence-Based Practice Attitude Scale (EBPAS). Psychological Assessment, 22, 356-365.
4.
AaronsG. A.SawitzkyA. C. (2006). Organizational culture and climate and mental health provider attitudes toward evidence-based practice. Psychological Services, 3, 61-72.
BarwickM. A.BaracR.AkrongL. M.JohnsonS.ChabanP. (2014). Bringing evidence to the classroom: Exploring educator notions of evidence and preferences for practice change. International Education Research, 2(4), 1-15.
8.
BeetsM. W.FlayB. R.VuchinichS.AcockA. C.LiK.AllredC. (2008). School climate and teachers’ beliefs and attitudes associated with implementation of the Positive Action program: A diffusion of innovations model. Prevention Science, 9, 264-275.
9.
BradshawC. P.DebnamK.KothC. W.LeafP. (2009). Preliminary validation of the implementation phases inventory for assessing fidelity of schoolwide positive behavior supports. Journal of Positive Behavior Interventions, 11, 145-160.
10.
BradshawC. P.KothC. W.ThorntonL. A.LeafP. J. (2009). Altering school climate through school-wide positive behavioral interventions and support: Findings from a group-randomized effectiveness trial. Prevention Science, 10, 100-115.
11.
BrownsonR. C.BakerE. A.LeetT. L.GillespieK. N. (2003). Evidence-based public health. New York, NY: Oxford University Press.
12.
Collaborative for Academic, Social, and Emotional Learning. (2015). CASEL guide: Effective social and emotional learning programs—Middle and high school edition. Retrieved from http://secondaryguide.casel.org/description-page.html#4R
13.
CrooksC. V.ChiodoD.ZwarychS.HughesR.WolfeD. A. (2013). Predicting implementation success of an evidence-based program to promote healthy relationships among students two to eight years after teacher training. Canadian Journal of Community Mental Health, 32, 125-138.
14.
CrooksC. V.ScottK. L.BrollR.ZwarychS.HughesR.WolfeD. A. (2015). Does an evidence-based healthy relationships program for 9th graders show similar effects for 7th and 8th graders? Results from 57 schools randomized to intervention. Health Education Research, 30, 513-519.
15.
CrooksC. V.ScottK.EllisW.WolfeD. A. (2011). Impact of a universal school-based violence prevention program on violent delinquency: Distinctive benefits for youth with maltreatment histories. Child Abuse & Neglect, 35, 393-400.
DamschroderL. J.AronD. C.KeithR. E.KirschS. R.AlexanderJ. A.LoweryJ. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, Article 50.
18.
DomitrovichC. E.BradshawC. P.PoduskaJ. M.HoagwoodK.BuckleyJ. A.OlinS.. . . IalongoN. S. (2008). Maximizing the implementation quality of evidence-based preventive interventions in schools: A conceptual framework. Advances in School Mental Health Promotion, 1, 6-28.
19.
DomitrovichC. E.PasE. T.BradshawC. P.BeckerK. D.KeperlingJ. P.. . . IalongoN. (2015). Individual and school organizational factors that influence implementation of the PAX good behavior game intervention. Prevention Science, 16, 1064-1074.
20.
DozoisE.WellsL.CrooksC. (2016). Developing a comprehensive, evidence-based strategy for violence prevention: Lessons learned from Alberta, Canada. In TaylorM. F.KhanU.PooleyJ. A. (Eds.), Crime and violence prevention: Moving beyond hot-stove policing and perpetrator rehabilitation (pp. 215-230). New York: Nova Publishers.
21.
DurlakJ. A. (2015). Studying program implementation is not easy but it is essential. Prevention Science, 16, 1123-1127.
22.
DurlakJ. A.DuPreE. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327-350.
23.
Exner-CortensD.WellsL.LeeL.SpiricV. (2019). Building a culture of intimate partner violence prevention in Alberta, Canada through the promotion of healthy youth relationships. Prevention Science. Advance online publication. doi:10.1007/s11121-019-01011-7
24.
FixsenD. L.BlaseK. A.NaoomS. F.WallaceF. (2009). Core implementation components. Research on Social Work Practice, 19, 531-540.
25.
FlayB. R.BiglanA.BoruchR. F.CastroF. G.GottfredsonD.KellamS.. . . JiP. (2005). Standards of evidence: Criteria for efficacy, effectiveness and dissemination. Prevention Science, 6, 151-175.
26.
FormanS. G.ShapiroE. S.CoddingR. S.GonzalesJ. E.ReddyL. A.RosenfieldL. A.. . . StoiberK. C. (2013). Implementation science and school psychology. School Psychology Quarterly, 28, 77-100.
27.
GottfredsonD. C.CookT. D.GardnerF. E. M.Gorman-SmithD.HoweG. W.SandlerI. N.ZafftK. M. (2015). Standards of evidence for efficacy, effectiveness, and scale-up research in prevention science: Next generation. Prevention Science, 16, 893-926.
28.
GottfredsonD. C.GottfredsonG. D. (2002). Quality of school-based prevention programs: Results from a national survey. Journal of Research in Crime & Delinquency, 39, 3-35.
29.
GregoryA.HenryD. B.SchoenyM. E., & The Metropolitan Area Child Study Research Group. (2007). School climate and the implementation of a preventive intervention. American Journal of Community Psychology, 40, 250-260.
30.
HaignereC. S.CulhaneJ. F.BalsleyC. M.LegosP. (1996). Teachers’ receptiveness and comfort teaching sexuality education and using non-traditional teaching strategies. Journal of School Health, 66, 140-144.
31.
HanS. S.WeissB. (2005). Sustainability of teacher implementation of school-based mental health programs. Journal of Abnormal Child Psychology, 33, 665-679.
32.
JohnsonS. R.PasE. T.LohD.DebnamK. J.BradshawC. P. (2017). High school teachers’ openness to adopting new practices: The role of personal resources and organizational climate. School Mental Health, 9, 16-27.
33.
KelmJ. L.McIntoshK.CooleyS. (2014). Effects of implementing school-wide positive behavioural interventions and supports on problem behaviour and academic achievement in a Canadian elementary school. Canadian Journal of School Psychology, 29, 195-212.
34.
LyonA. R.CookC. R.BrownE. C.LockeJ.DavisC.EhrhartM.AaronsG. A. (2018). Assessing organizational implementation context in the education sector: Confirmatory factor analysis of measures of implementation leadership, climate, and citizenship. Implementation Science, 13, Article 5.
35.
MalloyM.AcockA.DuBoisD. L.VuchinichS.SilverthornN.. . . FlayB. (2015). Teachers’ perceptions of school organizational climate as predictors of dosage and quality of implementation of a social-emotional and character development program. Prevention Science, 16, 1086-1095.
36.
MathiesenS. G.HohmanM. (2013). Revalidation of an evidence-based practice scale for social work. Journal of Social Work Education, 49, 451-460.
37.
McCallR. B. (2009). Evidence-based programming in the context of practice and policy. Social Policy Report, 28(3), 3-19.
38.
McIntoshK.MonizC.CraftC. B.GolbyR.Steinwand-DeschambeaultT. (2014). Implementing school-wide positive behavioural interventions and supports to better meet the needs of Indigenous students. Canadian Journal of School Psychology, 29, 236-257.
39.
MelnykB. M.Fineout-OverholtE.FeinsteinN. F.SadlerL. S.Green-HernandezC. (2008). Nurse practitioner educators’ perceived knowledge, beliefs, and teaching strategies regarding evidence-based practice: Implications for accelerating the integration of evidence-based practice into graduate programs. Journal of Professional Nursing, 24, 7-13.
40.
MihalicS. F.FaganA. A.ArgamasoS. (2008). Implementing the LifeSkills Training drug prevention program: Factors related to implementation fidelity. Implementation Science, 3, 1-16.
41.
NationM.CrustonC.WandersmanA.KumpferK. L.SeyboltD.Morrissey-KaneE.DavinoK. (2003). What works in prevention: Principles of effective prevention programs. American Psychologist, 58, 449-456.
42.
NovinsD. K.GreenA. E.LeghaR. K.AaronsG. A. (2013). Dissemination and implementation of evidence-based practices for child and adolescent mental health: A systematic review. Journal of the American Academy of Child & Adolescent Psychiatry, 52, 1009-1025.
43.
OwensJ. S.LyonA. R.BrandtN. E.WarnerC. M.NadeemE.SpielC.WagnerM. (2014). Implementation science in school mental health: Key constructs in developing a research agenda. School Mental Health, 6, 99-111.
44.
ReinkeW. M.StormontM.HermanK. C.NewcomerL. (2014). Using coaching to support teacher implementation of classroom-based interventions. Journal of Behavioral Education, 23, 150-167.
45.
RiceK.HwangJ.Abrefa-GyanT.PowellK. (2010). Evidence-based practice questionnaire: A confirmatory factor analysis in a social work sample. Advances in Social Work, 11, 158-173.
46.
RingwaltC. L.EnnettS.JohnsonR.RohrbachL. A.Simons-RudolphA.. . . ThorneJ. (2003). Factors associated with fidelity to substance use prevention curriculum guides in the nation’s middle schools. Health Education & Behavior, 30, 375-391.
47.
SandelowskiM. (2000). Whatever happened to qualitative description?Research in Nursing & Health, 23, 334-340.
48.
ShortK. (2016). Intentional, explicit, systematic: Implementation and scale-up of effective practices for supporting student mental well-being in Ontario schools. International Journal of Mental Health Promotion, 18, 33-48.
49.
SpothR.GuyllM.RedmondC.GreenbergM.FeinbergM. (2011). Six-year sustainability of evidence-based intervention implementation quality by community-university partnerships: The PROSPER study. American Journal of Community Psychology, 48, 412-425.
Tanner-SmithE. E.DurlakJ. A.MarxR. A. (2018). Empirically based mean effect size distributions for universal prevention programs targeting school-aged youth: A review of meta-analyses. Prevention Science, 19, 1091-1101. doi:10.1007/s11121-018-0942-1
52.
UbbinkD. T.GuyattG. H.VermeulenH. (2013). Framework of policy recommendations for implementation of evidence-based practice: A systematic scoping review. BMJ Open, 3, e001881.
WanlessS. B.DomitrovichC. E. (2015). Readiness to implement school-based social-emotional learning interventions: Using research on factors related to implementation to maximize quality. Prevention Science, 16, 1037-1043.
55.
WolfeD. A.CrooksC. V.ChiodoD.HughesR.EllisW. (2012). Observations of adolescent peer resistance skills following a classroom-based healthy relationship program: A post-intervention comparison. Prevention Science, 13, 196-205.
56.
WolfeD. A.CrooksC.JaffeP.ChiodoD.HughesR.EllisW.. . . DonnerA. (2009). A school-based program to prevent adolescent dating violence: A cluster randomized trial. Archives of Pediatrics and Adolescent Medicine, 163, 692-699.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
