Key Considerations and Recommended Strategies for Conducting a School-Based Longitudinal RE-AIM Evaluation: Insights From a 28-School Cluster Randomized Trial
Restricted accessResearch articleFirst published online January, 2023
Key Considerations and Recommended Strategies for Conducting a School-Based Longitudinal RE-AIM Evaluation: Insights From a 28-School Cluster Randomized Trial
RE-AIM is an implementation science framework that provides a structure to promote data collection and analysis in the domains of Reach, Efficacy/Effectiveness, Adoption, Implementation, and Maintenance. The RE-AIM approach promotes more data collection and reporting than typical evaluations so potential adopters can determine the potential program fit for their setting. RE-AIM has been used to evaluate a variety of programs; however, there are few resources that provide strategies for conducting a RE-AIM evaluation in the school setting. The purpose of this article is to inform future studies by elucidating experiences conducting a large, complex, multisite, longitudinal RE-AIM evaluation in schools in partnership with the state’s Departments of Education and Health, and to share strategies for overcoming obstacles. With the long-term goal of facilitating the translation of school-based intervention research into practice, we provide key considerations and recommend strategies for school-based RE-AIM evaluation success.
AustinG.BellT.CaperchioneC.MummeryW. K. (2011). Translating research to practice: Using the RE-AIM framework to examine an evidence-based physical activity intervention in primary school settings. Health Promotion Practice, 12(6), 932–941. https://doi.org/10.1177/1524839910366101
2.
BauerM. S.DamschroderL.HagedornH.SmithJ.KilbourneA. M. (2015). An introduction to implementation science for the non-specialist. BMC Psychology, 3(1), Article 32. https://doi.org/10.1186/s40359-015-0089-9
3.
BhattacharyyaO.ReevesS.ZwarensteinM. (2009). What is implementation research? Rationale, concepts, and practices. Research on Social Work Practice, 19(5), 491–502. https://doi.org/10.1177/1049731509335528
4.
CassarS.SalmonJ.TimperioA.NaylorP.-J.van NassauF.Contardo AyalaA. M.KoortsH. (2019). Adoption, implementation and sustainability of school-based physical activity and sedentary behaviour interventions in real-world settings: A systematic review. International Journal of Behavioral Nutrition and Physical Activity, 16(1), Article 120. https://doi.org/10.1186/s12966-019-0876-4
5.
ChenH. T. (2014). Practical program evaluation. Sage.
6.
CheneyD. A.YongM. (2014). RE-AIM checklist for integrating and sustaining tier 2 social-behavioral interventions. Intervention in School and Clinic, 50(1), 39–44. https://doi.org/10.1177/1053451214532343
7.
ChristianD. L.ToddC.RanceJ.StrattonG.MackintoshK. ARapportF.BrophyS. (2020). Involving the headteacher in the development of school-based health interventions: A mixed-methods outcome and process evaluation using the RE-AIM framework. PLOS ONE, 15(4), Article e0230745. https://doi.org/10.1371/journal.pone.0230745
8.
CookC. R.LyonA. R.LockeJ.WaltzT.PowellB. J. (2019). Adapting a compilation of implementation strategies to advance school-based implementation research and practice. Prevention Science, 20(6), 914–935. https://doi.org/10.1007/s11121-019-01017-1
9.
DaleyE. M.MarhefkaS. L.WangW.NobleC. A.MahonyH.ArzolaS., et al (2019). Longitudinal evaluation of the Teen Outreach Programme: Impacts of a health promotion programme on risky sexual behaviours. Health Education Journal. 78(8), 916–30. https://doi.org/10.1177/0017896919857777.
10.
DzewaltowskiD. A.GlasgowR. E.KlesgesL. M.EstabrooksP. A.BrockE. (2004). RE-AIM: Evidence-based standards and a web resource to improve translation of research into practice. Annals of Behavioral Medicine, 28(2), 75–80. https://doi.org/10.1371/journal.pone.0230745
11.
FormanS. G.OlinS. S.HoagwoodK. E.CroweM.SakaN. (2009). Evidence-based interventions in schools: Developers’ views of implementation barriers and facilitators. School Mental Health, 1(1), 26–36. https://doi.org/10.1007/s12310-008-9002-5
12.
Foster-FishmanP. G.BerkowitzS. L.LounsburyD. W.JacobsonS.AllenN. A. (2001). Building collaborative capacity in community coalitions: A review and integrative framework. American Journal of Community Psychology, 29(2), 241–261. https://doi.org/10.1023/A:1010378613583
13.
GaglioB.ShoupJ. A.GlasgowR. E. (2013). The RE-AIM framework: A systematic review of use over time. American Journal of Public Health, 103(6), e38–e46. https://doi.org/10.2105/ajph.2013.301299
14.
GlasgowR. E. (2003). Translating research to practice: Lessons learned, areas for improvement, and future directions. Diabetes Care, 26(8), 2451–2456. https://doi.org/10.2337/diacare.26.8.2451
15.
GlasgowR. E.EstabrooksP. E. (2018). Peer reviewed: Pragmatic applications of RE-AIM for health care initiatives in community and clinical settings. Preventing Chronic Disease, 15, Article 170271. https://doi.org/10.5888/pcd15.170271
16.
GlasgowR. E.HardenS. M.GaglioB.RabinB. A.SmithM. L.PorterG. C.OryM. G.EstabrooksP. A. (2019). RE-AIM planning and evaluation framework: Adapting to new science and practice with a twenty-year review. Frontiers in Public Health, 7, Article 64. https://doi.org/10.3389/fpubh.2019.00064
17.
GlasgowR. E.McKayH. G.PietteJ. D.ReynoldsK. D. (2001). The RE-AIM framework for evaluating interventions: What can it tell us about approaches to chronic illness management?Patient Education and Counseling, 44(2), 119–127.
18.
GlasgowR. E.VogtT. M.BolesS. M. (1999). Evaluating the public health impact of health promotion interventions: The RE-AIM framework. American Journal of Public Health, 89(9), 1322–1327. https://doi.org/10.2105/AJPH.89.9.1322
19.
HardenS. M.SmithM. L.OryM. G.Smith-RayR. L.EstabrooksP. A.GlasgowR. E. (2018). RE-AIM in clinical, community, and corporate settings: Perspectives, strategies, and recommendations to enhance public health impact. Frontiers in Public Health, 6, Article 71. https://doi.org/10.3389/fpubh.2018.00071
20.
High/Scope Educational Research Foundation. (2005). Youth program quality assessment. High/Scope Press
21.
JiP.DuBoisD. L.FlayB. R.BrechlingV. (2008). “Congratulations, you have been randomized into the control group!(?)”: Issues to consider when recruiting schools for matched-pair randomized control trials of prevention programs. Journal of School Health, 78(3), 131–139. https://doi.org/10.1111/j.1746-1561.2007.00275.x
22.
KesslerR. S.PurcellE. P.GlasgowR. E.KlesgesL. M.BenkeserR. M.PeekC. (2013). What does it mean to “employ” the RE-AIM model?Evaluation & the Health Professions, 36(1), 44–66.
23.
KwanB. M.McGinnesH. L.OryM. G.EstabrooksP. A.WaxmonskyJ. A.GlasgowR. E. (2019). RE-AIM in the real world: Use of the RE-AIM framework for program planning and evaluation in clinical and community settings. Frontiers in Public Health, 7, Article 345. https://doi.org/10.3389/fpubh.2019.00345
24.
LounsberyM. A. F.McKenzieT. L.TrostS.SmithN. J. (2011). Facilitators and barriers to adopting evidence-based physical education in elementary schools. Journal of Physical Activity and Health, 8(Suppl. 1), S17–S25. https://doi.org/10.1123/jpah.8.s1.s17
25.
LyonA. R.CookC. R.LockeJ.DavisC.PowellB. J.WaltzT. J. (2019). Importance and feasibility of an adapted set of implementation strategies in schools. Journal of School Psychology, 76, 66–77. https://doi.org/10.1016/j.jsp.2019.07.014
NaylorP.-J.NettlefoldL.RaceD.HoyC.AsheM. C.HigginsJ. W.McKayH. A. (2015). Implementation of school based physical activity interventions: A systematic review. Preventive Medicine, 72, 95–115. https://doi.org/10.1016/j.ypmed.2014.12.034
28.
NilsenP. (2015). Making sense of implementation theories, models and frameworks. Implementation Science, 10(1), Article 53. https://doi.org/10.1186/s13012-015-0242-0
29.
PayneA. A.EckertR. (2010). The relative importance of provider, program, school, and community predictors of the implementation quality of school-based prevention programs. Prevention Science, 11(2), 126–141. https://doi.org/10.1007/s11121-009-0157-6
30.
PowellB. J.WaltzT. J.ChinmanM. J.DamschroderL. J.SmithJ. L.MatthieuM. M.ProctorE. K.KirchnerJ. E. (2015). A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science, 10(1), Article 21. https://doi.org/10.1186/s13012-015-0209-1
RichmondS. A.DonaldsonA.MacphersonA.BridelW.FinchC. F.HagelB.EmeryC. A. (2020). Facilitators and barriers to the implementation of iSPRINT: A sport injury prevention program in junior high schools. Clinical Journal of Sport Medicine, 30(3), 231–238. https://doi.org/10.1097/JSM.0000000000000579
33.
RosasS. R. (2017). Systems thinking and complexity: Considerations for health promoting schools. Health Promotion International, 32(2), 301–311. https://doi.org/10.1093/heapro/dav109
34.
ScheirerM. A. (2013). Linking sustainability research to intervention types. American Journal of Public Health, 103(4), e73–e80. https://doi.org/10.2105/ajph.2012.300976
35.
ScheirerM. A.HartlingG.HagermanD. (2008). Defining sustainability outcomes of health programs: Illustrations from an on-line survey. Evaluation and Program Planning, 31(4), 335–346. https://doi.org/10.1016/j.evalprogplan.2008.08.004
36.
SummitA. K.FriedmanE.SteinT. B.GoldM. (2019). Integration of onsite long-acting reversible contraception services into school-based health centers. Journal of School Health, 89(3), 226–231. https://doi.org/10.1111/josh.12732
37.
WaltzT. J.PowellB. J.FernándezM. E.AbadieB.DamschroderL. J. (2019). Choosing implementation strategies to address contextual barriers: Diversity in recommendations and future directions. Implementation Science, 14(1), Article 42. https://doi.org/10.1186/s13012-019-0892-4
38.
Walsh-BuhiE. R.MarhekfaS.L.WangW.DebateR.PerrinK.SingletonA., et al (2016). The Impact of the Teen Outreach Program on Sexual Intentions and Behaviors. Journal of Adolescent Health. 59(3), 283–90. http://dx.doi.org/10.1016/j.jadohealth.2016.05.007.
