Abstract
Background
School-based universal prevention programs, like the Michigan Model for Health™ (MMH), hold promise for enhancing youth behavioral health but often face implementation challenges due to insufficiently addressing priority student issues. Previous research identified trauma-sensitive content as a student need in the MMH. Enhanced Replicating Effective Programs (REP), a multicomponent implementation strategy, is well suited to support program providers in addressing priority health issues among youth.
Method
This pilot cluster-randomized controlled trial compared Enhanced REP (tailored curriculum, training, and implementation facilitation with trauma-sensitive content) to standard REP (standard curriculum, initial training, as-needed technical assistance) across eight high schools serving low-income students. Through semistructured interviews at three time points, we assessed teacher perceptions of feasibility, acceptability, and appropriateness related to REP core and enhanced components.
Results
Teachers generally found Enhanced REP to deliver MMH satisfactory and suitable. However, the school environment, notably administrative support, influenced feasibility compared to standard REP. Enhanced REP teachers reported benefits in meeting student needs that were not seen in the standard REP group. The standard REP data helped to understand the comparative value of the enhanced strategy during a time of notable upheaval and mental health challenges due to the COVID-19 pandemic.
Conclusions
While some schools may succeed with less intensive strategies (REP), many may require more intensive approaches for effective implementation. Enhanced REP shows promise in tailoring curriculum delivery and providing additional support to meet student needs, but its success may hinge on organizational support, especially from leadership. Future research should investigate the addition of organizational-level strategies, such as leadership training, to optimize implementation and explore the comparative effectiveness of Enhanced versus standard REP.
Plain Language Summary
Background
Behavioral health challenges among adolescents are concerning, with a 109% rise in overdose deaths from 2019 to 2021 (Tanz et al., 2022), and in 2022, among high school students, 20% seriously considered attempting suicide (CDC, 2023), highlighting the need for universal prevention approaches during this vulnerable developmental stage (Hasin et al., 2007). Universal prevention centers on mitigating risk and building protective factors that influence multiple, interrelated adverse health outcomes, including substance misuse and mental health (Catalano et al., 2012; Hawkins et al., 2015; Monahan et al., 2014). The Michigan Model for HealthTM (MMH) and other similar skills-based curricula (Griffin & Botvin, 2010; Sussman et al., 2014) focus on reducing risk and enhancing protective factors (e.g., self-management, decision-making, refusal skills, and communication). Comprehensive universal prevention evidence-based interventions (EBIs) have demonstrated effectiveness and cost-effectiveness and reach diverse and underserved youth populations (Greenberg, 2010; Hale et al., 2014; Hill et al., 2020). Prevention EBIs can reduce drug use morbidity and mortality and related adverse consequences (Compton et al., 2019; Hale et al., 2014).
Prevention EBIs, however, are rarely delivered as intended (i.e., with optimal fidelity) and often fail to achieve desired public health outcomes; this is especially detrimental in economically marginalized communities where substance use and mental health disorders are prevalent and access to resources is limited (Baumann, Shelton et al., 2023; Brownson et al., 2021; Kariisa et al., 2022; Moir, 2018; U.S. Department of Education, 2011). Multiple factors influence the implementation of prevention interventions, including innovation, provider, and setting characteristics. For example, ensuring the EBI meets youth health needs via tailoring is vital to effective implementation. Providers also need sufficient training and implementation assistance to deliver the intervention effectively (Mihalic et al., 2008; Smith et al., 2022). Finally, understanding the organizational and contextual landscape for providers and EBI implementation is essential. Determining the salient factors affecting implementation success can aid in identifying and developing impactful strategies for diverse settings (Powell et al., 2019). This includes pragmatic considerations such as aligning implementation strategies with existing infrastructure and practices to leverage existing resources and minimize disruption.
The MMH Intervention
MMH is an evidence-based, universal, comprehensive health curriculum deployed across Michigan and 40 states across the United States and Canada (State of Michigan, 2023). MMH is aligned with state and national health education standards and has demonstrated efficacy in reducing substance use and improving mental health outcomes in high school students (CDC, 2019; O’neil et al., 2011; Shope et al., 1996). The core components of MMH are listed in Table 1. Researchers found that 91% of Michigan health teachers use MMH; estimates vary, but most high school teachers (71–89%) do not meet state-designated fidelity standards (dose delivered: 80% or more of the curriculum), based on teacher self-report (Eisman et al., 2022; Rockhill, 2017). MMH curriculum standard implementation is consistent with Replicating Effective Programs (REP), a low-intensity strategy that includes three components: (1) a curriculum manual, (2) curriculum training, and (3) as-needed technical assistance initiated by health teachers (Kilbourne et al., 2018; Neumann & Sogolow, 2000). REP, however, is often insufficient to implement complex behavioral interventions successfully (Kilbourne et al., 2013). A recent study found, for example, that REP was insufficient for 88% of school professionals implementing a mental health EBI (Smith et al., 2022). Consequently, researchers designed Enhanced REP, a more intensive implementation strategy tailored to the population and contextual needs that includes (1) tailored curriculum materials, (2) tailored training, and (3) hands-on support via implementation facilitation through trained implementation support practitioners (Kilbourne et al., 2014). See Table 2.
Core Components of the Michigan Model for HealthTM (Eisman et al., 2024a)
Enhanced REP and Standard REP Components
Implementation strategy per SISTER taxonomy.
Justification for standard implementation is that it is consistent with standard practice; specification of strategies modified from original description by Kilbourne et al. (2007, 2014) to align with recommendations from Proctor et al.(2013).
REP = Replicating Effective Programs; MMH = Michigan Model for Health; i-PARIHS = integrated Promoting Action on Research Implementation in Health Services; RESAs = Regional Education Service Agencies.
Target implementation outcome for a larger-scale trial: MMH fidelity.
Implementation Strategy: Enhanced REP
We describe the Enhanced REP implementation strategy development elsewhere Eisman et al. (2024b), but review it briefly here and include a summary in Table 2. Academic and state agency collaborators formed an advisory board to tailor the MMH curriculum to fit various contexts better, focusing on core foundational health skills, social-emotional learning, and drug-use prevention units. This adaptation process aims to improve the fit of the intervention with population health needs—crucial in the face of barriers such as the lack of trauma-sensitive content for youth in under-resourced areas (Baumann, Cabassa, & Stirman, 2023; Taxman & Friedmann, 2009). Our partners identified trauma-sensitive focus as a need due to notable trauma exposure in low-resource communities and its influence on substance use initiation, mental health, and related problems (Bethell et al., 2014; Eisman et al., 2022; Ellis & Dietz, 2017; Sacks & Murphey, 2018).
Modifications to MMH include trauma-informed alternatives for assignments, language, and classroom activities. Trauma-sensitive training for teachers and implementation facilitation strategies complement these content changes. Implementation facilitation (IF) involves school health coordinators who provide personalized, data-driven assistance and support to health teachers based on the integrated Promoting Action on Research Implementation in Health Services framework (Harvey & Kitson, 2016; Kilbourne et al., 2014). IF with teachers can include ongoing consultation, coaching, and support from health coordinators, with a focus on providers (Harvey & Kitson, 2016; Smith et al., 2022).
The purpose of this study was to evaluate the feasibility, acceptability, and appropriateness of Enhanced REP for implementing MMH in high schools versus standard REP. Specifically, we aimed to understand teachers’ perceptions of the enhanced strategy and whether they found it more feasible, acceptable, and appropriate than the standard version. By comparing teachers’ experiences in both the Enhanced REP and standard REP groups, we explored more intensive implementation strategies for school-based prevention EBIs versus current less intensive standard implementation (REP). Our central research question is to what extent do teachers perceive Enhanced REP as feasible, acceptable, and appropriate compared to standard REP for implementing the MMH in high schools, and what factors contribute to these perceived differences?
Methods
Study Setting and Participants
Michigan's education system takes a local control approach to addressing the wide variety of school districts with varying challenges and needs. Individual school districts and Regional Education Service Agencies (RESAs) have a great deal of autonomy to allow for localized decision-making to respond to the needs of the area. Still, it can also exacerbate inequities in EBI implementation, funding, and resources (Smrekar & Crowson, 2015). Michigan's network of school health coordinators is a network of implementation support practitioners who work with local schools and districts and navigate their unique needs and operations (MiSHCA, 2024).
In collaboration with two school health coordinator partners, we recruited schools in three Michigan counties during the 2021–2022 school year. Health coordinators contacted potentially eligible health education teachers in their regions to assess interest, and the research team shared study information and recruitment materials. We focused on health teachers to leverage health coordinators’ consistent and direct collaboration with educators. This strategy aligns with standard practices and considers that school leadership varies widely in its involvement with health coordinators. High school health teachers in participating counties were eligible if they (1) had 20% or more of students eligible for free and reduced-price lunch, (2) used less than 80% of the MMH curriculum, the state-identified fidelity standard, and/or (3) faced two or more barriers to implementation, such as meeting needs of trauma-exposed youth. We provide participating school demographics in Table 3.
School-Level Demographics (State of Michigan, N.D.)
Procedures
The study team randomized schools to either standard implementation (consistent with standard REP) or Enhanced REP (see Figure 1). The initial group-level comparisons supported comparable groups by the school size and free and reduced lunch status. We invited all health teachers in a building to participate, although the majority of high schools have one health teacher, the school was included if at least one agreed. We consulted with the administration at each school to review procedures and teacher participation. The health coordinators and study team met with relevant participating school staff to share study information and procedures. Teachers provided informed consent before the start of the study.
Pilot Study Design Overview
Study Design
The design was a pilot cluster randomized trial comparing Enhanced REP to deliver MMH versus standard REP. We use this design to assess the enhanced strategies’ feasibility, acceptability, and appropriateness to support MMH delivery versus standard REP (Pearson et al., 2020). This pilot study design also tests study procedures, informs if progression to a large-scale trial is warranted (Hallingberg et al., 2018), and helps test recruitment and randomization procedures (Kistin & Silverstein, 2015).
Our study included qualitative data collection from teachers in the Enhanced REP and standard REP groups across multiple time points to enhance the robustness of the findings due to the small sample size for a pilot study. This longitudinal approach can capture the dynamic nature of the context and the evolution of participants’ experiences throughout the study (Nevedal et al., 2019).
Data and Measures
We conducted semistructured interviews pre- (T1), interim- (T2), and postimplementation (T3) for each term that a teacher implemented MMH during the 2021–2022 school year (see Figure 1). Enhanced REP components and the Consolidated Framework for Implementation Research (CFIR) informed the structured interview guide. Each interview lasted 30–45 min and we used videoconferencing software. We recorded interviews, transcribed them, and deidentified the transcripts.
Data Analytic Plan
We conducted an inductive-deductive thematic analysis following procedures for the reflexive thematic analysis approach to accommodate both data-driven (inductive) coding and the incorporation of preexisting (deductive) knowledge or theory (Braun & Clarke, 2022). We implemented a six-phase thematic analysis method, consistent with Braun and Clarke (2022), which involves familiarizing with the data, generating initial codes, searching for themes, reviewing themes, defining and naming themes, and producing the report. Three research team members trained in qualitative methods read the transcripts and provided initial impressions (Phase 1). We generated initial codes guided by CFIR constructs, which the coders independently modified (Phase 2). We systematically collated preliminary codes into potential themes using a constant comparative method (Phase 3; Glaser & Strauss, 1967). To ensure data extracts illustrated the themes and identified the subthemes, we reviewed the themes and subthemes against the original transcripts after the review sessions to ensure the analysis provided a well-organized and thorough view of the data (phases 4 and 5). We generated a report of the themes using QSR Nvivo (Phase 6). Consistent with the reflexive analysis, we continually questioned and examined our biases, assumptions, and their potential influence on the research process. We held regular analysis meetings to examine analytic decisions and possible alternative interpretations to question our evolving analysis (Braun & Clarke, 2019).
Member Checking to Support Methodological Integrity
Methodological integrity refers to enhancing the rigor of qualitative research to improve how well the findings accurately capture participants’ experiences (Creswell, 2015; Levitt et al., 2021). We assessed whether the study information was accurate through member checking; we presented a summary of the findings to key implementation partners, including members of the Michigan School Health Coordinators Association and the Michigan Department of Education and Health and Human Services staff (Lietz et al., 2006).
Results
We enrolled nine schools in the pilot study, four in the Enhanced REP condition, and five in the standard implementation condition. One Enhanced REP school discontinued after the preimplementation interview (T1) with no subsequent data collection; thus, we report on the eight schools (nine teachers) that completed more than one study activity (e.g., participating in Enhanced REP activities, completing lesson tracking). Six schools (two Enhanced REP and four standard REP), one health teacher each, completed the T1 and T3 interviews, with half also completing the T2 interviews. The other two schools (one standard REP and two Enhanced REP health teachers) completed at least one study activity beyond the T1 interview and thus are included in the analysis, resulting in 18 total interview transcripts.
In the T1 interviews, we asked teachers to describe their experience teaching MMH and to briefly describe their training experiences. The majority of teachers (n = 6) implemented the MMH curriculum in part or in conjunction with another intervention (e.g., a textbook curriculum required by the school). Four teachers (three control and one Enhanced REP) reported teaching MMH for at least five years. All teachers reported they had been trained to teach MMH, with all but three (two REP, one Enhanced REP) teachers reporting their training and/or a refresher course had happened within a “couple of years.”
Qualitative Results
We identified key themes related to each implementation outcome of interest: feasibility, acceptability, and appropriateness. See Table 3 for a summary of themes.
Feasibility
We found notable variability in how teachers discussed Enhanced REP component feasibility versus standard implementation, particularly concerning IF.
Enhanced REP feasibility and school support for universal prevention. The implementation climate reflects staff perceptions of the school's support for universal prevention and health promotion, particularly MMH (Lyon et al., 2018). At the T2 and T3 interviews, Enhanced REP teachers indicated that engaging with the Enhanced strategy varied based on the school-level support and prioritization, a characteristic of the intervention-specific (versus molar) implementation climate. One Enhanced REP teacher reported that student well-being and health education were a school priority and stated that Enhanced REP was more feasible than standard REP as it provided necessary tailoring, training, and hands-on assistance to better serve youth from an economically disadvantaged community where most students face substantial adversity (e.g., limited community resources, high rates of adverse childhood experiences). “I do have the students who are vulnerable … (and) have to modify a couple of things …” But her modifications (reported during T3) centered on student engagement versus substantive curriculum changes because “the update(s) … include everything I need.” However, another Enhanced REP teacher reported that she appreciated the tailored materials and assistance. Still, it was less feasible than standard MMH implementation because of school preferences and priorities: “I love the updated materials,” but that “I feel like I'm on an island by myself … my coworkers (say), ‘Let's do what we've always done’ and … they didn't … want to (use the updated curriculum),” stated one teacher during the T2 interview. They remarked that fully employing the tailored MMH components was challenging as they “still had to teach parts of (the old curriculum)” because of school-level requirements for a common final exam.
The health education implementation climate also impacted the feasibility of tailored training and IF versus standard REP. While some Enhanced REP teachers maintained protected planning time and could engage with the health coordinator (consistent with teacher-focused IF) during, for example, their planning hour, others reported challenges due to competing school-level demands. Some teachers frequently had to cancel or reschedule facilitation meetings with health coordinators due to other initiatives taking priority (e.g., covering classes and sticking to a strict professional development schedule). One teacher stated (T3), “I was so busy with teaching and everything that … a lot of times when (the health coordinator) was willing to help me, I just didn't have the time to do it.” In the same interview, the teacher reported that their school's priorities focused on reactively responding to urgent staffing needs versus health education, which made meeting with the health coordinator challenging; for example, “sub(bing during) your planning time.” The school prioritized addressing staffing issues versus the teacher receiving hands-on assistance (i.e., IF).
Control group teachers expressed a lack of training or access to training offered by their administration: “(health professional development is) in our free time and … unfortunately it usually means we’re taking days to go to (another school district) to get that training that our district doesn’t offer” (T3).
Enhanced REP feasibility and leadership support. Teachers discussed the notable influence of strategic implementation leadership on the feasibility of Enhanced REP in their schools. This type of leadership centers on specific behaviors that may support (or inhibit) EBI implementation (Lyon et al., 2018). When teachers reported “tangible” leadership support, including proactive problem-solving and steps to assist with the implementation process, deploying Enhanced REP was more feasible than standard implementation as it aided teachers in better meeting students’ needs. One Enhanced REP teacher reported in the postinterview, “I told (administration) that I was going to be a part of this partnership, and there was no hesitation about it … they just gave me (support) to do whatever I needed to do.”
In contrast, less strategic implementation leadership support notably reduced the Enhanced REP feasibility. For example, we found that administrators who provided permission without tangible assistance created the expectation of time, typically outside working hours, to participate. One Enhanced REP teacher remarked at T3, “(Administration was) very supportive of me doing it, but they never checked in (…) What did I actually get from them? Support-wise, nothing.” In this case, the teacher reported low Enhanced REP feasibility because of school-level leadership expectations, such as planning time to cover other classes and limited flexibility to permit time for training and IF meetings.
Similarly, one control teacher shared that, while granted permission to participate in the project, she felt a lack of engaged support from administrators preoccupied with other pressing issues, leaving her feeling isolated in dealing with current health challenges: “they really have no idea what I am doing.” This difference suggests that while administrative “green light” is a necessary first step, active and engaged leadership is crucial for prevention EBI implementation broadly and perhaps even more important when engaging in more intensive strategies like Enhanced REP.
Acceptability
Tailored curriculum and student needs. Overall, Enhanced REP teachers reported benefits in meeting student needs not seen in the standard REP group. Specifically, Enhanced REP teachers highlighted the value of the tailored curriculum with trauma-sensitive content in addressing the unique needs of their students. One teacher in the Enhanced REP group with a high proportion (93%) of economically disadvantaged students noted during her T3 interview that she was grateful she did not have to start from scratch tailoring the curriculum for her students’ needs. Two Enhanced REP teachers working in the same school found that with the additional tailoring and support they noticed a difference during curriculum delivery stating that the students were “talking less” and more engaged in the group work (T2). Another teacher, in their T3 interview, reported high acceptability of the materials and activities, stating, “I've already talked to our curriculum director about possibly using (the trauma-sensitive MMH curriculum) as our main teaching source in the next couple of years.”
The control teachers, in contrast, highlighted at T3 issues that reduced curriculum acceptability for standard MMH implementation as they spent time and effort focused on more substantive adaptations to incorporate “up-to-date statistics … to supplement what's not there” to meet students’ needs, including those around behavioral health issues. Another reported spending “a lot of time updating … tailoring to the class.”
Acceptability of implementation facilitation. Overall, Enhanced REP teachers reported high acceptability of IF with teachers by the school health coordinators at T3. One teacher stated: “It's been very valuable having (the facilitator) as a resource.” Another reported that the health coordinator was “awesome … always willing to meet, to help … Always check in to see how things were going … very, very helpful.” While control teachers had access to health coordinator support via technical assistance, the engagement was infrequent as they reported occasional email contact “as needed” across the study time points.
Building-level influences on tailored curriculum acceptability. We found that inner context factors had a notable impact on acceptability. One teacher who reported school leadership support for MMH at T3 indicated high acceptability of Enhanced REP as it notably reduced her lesson prep time (from “two to three hours” to “20–30 min”). She reported she was able to focus on tailoring specifically for her student population (e.g., adapting the reading level). Another teacher stated at T3 that, given the limited resources of her school, the tailored curriculum was “a huge opportunity … to get new lessons and new things to kind of revive my own spirit and then also of the students.” However, this same teacher also reported low acceptability as school leadership (and fellow health teachers) did not support Enhanced REP. “I didn't have other peers to collaborate with versus (the) curriculum that my school has used forever, and not that it's great, but we collaborate because we're all teaching it together, so (using something different) was really rough” (T3); identifying a need to include fellow teachers and administrators to ensure successful deployment.
In contrast, standard REP teachers also expressed an unmet need for building-level support, primarily focused on advocating for health education within the district and establishing program continuity within the school. “I feel the hardest part of my job is reminding my district of what (health teachers) do” (T3). These findings underscore the importance of the inner context, including both tangible leadership support and peer collaboration, for enhancing the acceptability of implementation strategies. The challenges observed are notable given that standard REP implementation is a low-level strategy. More intensive strategies, such as Enhanced REP, may encounter similar or even greater difficulties in achieving broad acceptance and successful deployment without adequate support systems in place.
Appropriateness
Appropriateness of the tailored curriculum and training. In the Enhanced REP condition, one teacher mentioned that their students were “very engaged” in the updated curriculum and in discussions of important social/emotional, substance misuse and other issues. Another teacher mentioned at T3 with curriculum updates, “I loved the (updated) materials, I actually would love for our school to completely go that way.” Teachers mentioned that they still needed additional tailoring for their student population. One teacher stated at T3, “Common sense is not so common, and diversity is real, so sometimes I just have to be cognizant of that and just … Break it down to a novice level.”
Teachers in the standard REP group discussed the need for tailored behavioral health training: “We could always use more training on these things like … emotional well-being, meeting students where they’re at. That would be great” (T3).
Appropriateness of implementation facilitation. Enhanced REP teachers consistently reported that IF from trained health coordinators fit their needs and practices. Coordinators varied their support based on teacher needs, which is consistent with the theoretical foundations of IF (Harvey & Kitson, 2016), coordinators varied their support based on teacher needs. One teacher reported at T3 that the health coordinator gave them “tips and tricks” and valuable resources for students and families. Another stated (T3) that their only regret was not being able to meet with the health coordinator more regularly. One coordinator reported that this project provided an opportunity to guide teachers on additional tailoring consistent with the MMH core functions (i.e., fidelity-consistent modifications; see Table 1).
Teachers in the standard REP group mentioned the need for more support at T1 “(we need) some … more guidance about exactly what to do (to meet student's needs).” At T2 and T3 REP teachers repeated the need for more scaffolding, training, and access to MMH implementation support.
Methodological Integrity Via Member Checking
We shared our findings with key interested parties, including health coordinators via the Michigan School Health Coordinators’ Association and personnel from the Michigan Department of Health and Human Services to ensure robustness and methodological integrity of the findings. Their validation indicates that the study's outcomes align closely with the real-world experiences and perspectives of individuals actively involved in the school health domain. This methodological approach was rigorous but resonant with the realities of the participants’ experiences, fortifying overall quality and validity.
Discussion
Our results indicate that Enhanced REP, a more intensive approach than standard implementation (REP), is a promising universal prevention intervention implementation strategy. Our findings suggest that tailored training and curriculum materials with IF may provide substantive support, information and flexibility to meet student needs. We also found that IF, consistent with current practices, was centered on teachers and may benefit from a more intentional focus on the inner setting in some cases. Our findings suggest that building-level factors influenced implementation in both standard and Enhanced REP groups (Chan et al., 2021) and were not always addressed sufficiently. Our results reinforce the importance of a supportive inner context for implementation strategy deployment. While factors such as sufficient administrator involvement and building-level collaboration are important for the successful implementation of any school-based EBI (Locke et al., 2019; Lyon et al., 2022), the more intensive nature of Enhanced REP versus REP may make it vulnerable to challenges in achieving broad acceptance and successful deployment if adequate support systems are lacking.
Feasibility
Our findings reveal that the feasibility of implementing universal prevention interventions, like MMH using standard REP or the more intensive Enhanced REP, is shaped by features of the context, with all teachers highlighting building-level factors. While teacher-centered IF can enhance individual teacher capabilities and address some barriers (Eiraldi et al., 2015; Kolko et al., 2022; McAlister et al., 2008), our results, consistent with other research (Aarons et al., 2011; Damschroder et al., 2022; Moullin et al., 2020), highlight the critical importance of building-level factors, particularly strategic implementation leadership and climate. The more intensive nature of Enhanced REP may heighten the need for robust leadership support and a collaborative school climate to address implementation challenges effectively. Specific leadership actions, such as actively communicating the value of the intervention, fostering collaboration among staff, providing resources, and removing barriers, can contribute to a more supportive inner context (Locke et al., 2024). Furthermore, providing leadership-focused IF may be necessary to build the skills and strategies to effectively champion and support deploying more intensive strategies like Enhanced REP when indicated (Langley et al., 2010; Locke et al., 2019).
Acceptability
For the packaging component of Enhanced REP, teachers reported the curriculum was generally acceptable and engaging for students without needing substantial time to modify it. Teachers in the Enhanced REP group reported that the adapted curriculum provided adequate delivery to address student needs. They could tailor further with health coordinator support to engage their students. Enhanced REP was also acceptable in its straightforward approach and practicality, specifically because health coordinators provided relevant resources for teachers to use with students and their families. In some cases, standard REP teachers reported low acceptability; they took notable time to make substantive adaptations, often without technical assistance from health coordinators to supplement what was missing to meet their students’ needs.
We also found instances when contextual factors, such as organizational or school setting features, influenced the acceptability of Enhanced REP. Given the interdependence of teachers and their school setting, cases when teacher and leadership priorities were not aligned reduced acceptability. Teachers who expressed favorable perceptions of Enhanced REP components acknowledged that it became less appealing when organizational factors did not adequately support its deployment. Alignment with the setting is an essential component of acceptability: satisfaction with the intervention and supports (e.g., IF) and how and when they are delivered (Meichsner et al., 2019). Leadership IF may be needed in addition to teacher IF to create sufficient setting alignment and ensure implementation strategy acceptability.
Appropriateness
The additional tailoring and support foundational to the Enhanced REP strategy overall were appropriate to address the identified need for additional trauma-sensitive content and its delivery (Eisman et al., 2022). Teachers in the Enhanced REP group noted that they could spend extra time customizing for specific population issues, such as the literacy level, but that the enhanced strategy provided them with the bandwidth to complete this customization as the other components were well-suited to student needs. Teachers in the standard REP group noted the need for additional support to meet student needs.
Our results collectively suggest that Enhanced REP with teacher IF may often be necessary for successful implementation but not always sufficient. The results point to nuanced and complex implementation challenges related to school health interventions. Despite evidence that schools can improve academic outcomes by effectively delivering health education and promotion interventions (Michael et al., 2015), organizational factors, such as limited leadership support and lack of favorable implementation climate for health education, remain notable barriers (Herlitz et al., 2020; Hunt et al., 2015). While schools have made advances in integrating health education and related health initiatives in schools (Temkin et al., 2020), this research and other recent work (Herlitz et al., 2020; Shoesmith et al., 2021) underscore that multilevel implementation strategies, including adaptive implementation strategies, may be vital to accelerate the effective integration of complex health interventions in schools. Consistent with the i-PARHIS framework (Harvey & Kitson, 2016; Kidwell & Almirall, 2023), recent studies suggest that further refinement in IF, for example, adding leadership facilitation (i.e., LEAD) in addition to teacher-level when indicated, may enhance the effectiveness and efficiency of EBI implementation (Kolko et al., 2022).
Limitations
Given the nature of the study as a pilot, the results should be interpreted with caution. School-level attrition, particularly during the COVID-19 pandemic, may limit the generalizability of our findings. However, it's important to note that our primary focus, consistent with a pilot study, is not on statistically representative school participation. One reason was unforeseen circumstances (e.g., community, teacher, or school-specific circumstances), leading to an unbalanced control and intervention group. While this imbalance could limit direct statistical comparisons, the qualitative data from both groups provides valuable insights into the perceived advantages of Enhanced REP, particularly regarding its ability to address student needs related to behavioral health issues. The inclusion of a standard REP condition, even with limitations, allows for a more nuanced understanding of the factors contributing to the perceived feasibility, acceptability, and appropriateness of the more intensive Enhanced REP.
While some of the circumstances were beyond control of the study team and influenced by an exceptionally tumultuous time in education (e.g., COVID-19), this pilot informed recruitment next steps (e.g., recruiting multiple teachers in a school if more than one health teacher is there) for a larger trial, streamlining student surveys to include approaches such as planned missing data to reduce student burden, and allowing for additional planning with teachers when participating in strategies such as implementation facilitation to create additional synergy with teacher workflow. This would likely be less impactful with a larger sample size; for example, one of the schools lost due to extenuating circumstances was the largest in the study (multiple teachers and classrooms) and thus significantly impacted the overall teacher and student enrollment. These steps will allow us to gather robust data while acknowledging research challenges in a dynamic educational landscape. Despite the temporal challenges in part due to an unprecedented global pandemic, this study provided valuable insights into comparing the feasibility, acceptability, and appropriateness of Enhanced REP versus REP for prevention in schools in preparation for a larger trial.
We experienced within-group variation of implementation styles (e.g., one vs. two semesters, one-on-one vs. small group facilitation, planning period use, administrative support, varying levels of familiarity and experience with MMH). While notable heterogeneity is typical for schools, accounting for this heterogeneity will be critical in successfully executing a larger trial. Finally, building-level characteristics (e.g., percent free/reduced lunch eligible) and contextual factors (e.g., leadership support of MMH) may be important moderating factors that influence strategy deployment, and this is an important area for future research. Finally, while previous research suggests both the tailored intervention and training components can independently enhance implementation (Baker et al., 2015; Botvin et al., 2018), our focus is on evaluating the feasibility of the Enhanced REP package versus REP. While the intervention and training tailoring are vital components of the package, we found notable variation and impacts on feasibility with IF. As such, an essential and practical next step in this research is to examine variations in facilitation, consistent with other research (Kolko et al., 2022; Smith et al., 2022). As MMH is a comprehensive health curriculum aimed at universal prevention in a classroom setting, we focus our efforts on health teachers. Further research should expand this to include perspectives of other key school personnel, including curriculum directors and school administrators.
Conclusions
In summary, this study offers promising insights into enhancing the implementation of comprehensive prevention programs in schools through Enhanced REP. Our findings suggest that incorporating trauma-sensitive content within a tailored curriculum, identified by educators as an evolving student need and priority, combined with implementation facilitation, may be key to its perceived benefits over standard REP. However, successful implementation often requires multilevel strategies, including those targeting both teachers and school leadership in addition to the curriculum and training. While teacher-focused IF proved valuable, our findings also indicate potential benefits from refining IF to include leadership as needed, aligning with recent calls to refine facilitation at multiple levels (Kolko et al., 2022). Such refinements could involve training and coaching to improve strategic implementation leadership and climate (Aarons et al., 2014; Ehrhart et al., 2014; Lyon et al., 2022). Given the potential cost and time investment of Enhanced REP, and specifically IF, future research should explore adaptive implementation strategies tailored to each school's unique needs and context. The lessons learned from this pilot study regarding Enhanced REP versus standard REP provide preliminary insights and identify important next steps for optimizing the implementation of other school-based prevention interventions and improving public health outcomes.
Footnotes
Acknowledgments
The authors used Jenni.ai and Grammarly to improve the clarity, grammar, and overall readability of the manuscript. This technology was chosen due to its advanced language-processing capabilities and efficiency in addressing grammatical and stylistic inconsistencies. All AI-generated edits were carefully reviewed by the authors, who decided whether to incorporate AI-generated suggestions into the manuscript. No substantive changes to the scientific content, data interpretation, or conclusions of the manuscript were made by the AI.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethical Considerations
This study was approved by the Wayne State University Institutional Review Board (IRB-20-10-2821). The study was conducted in accordance with the ethical standards laid down in the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Institute on Drug Abuse at the National Institutes of Health (K01 DA044279).
