Abstract
Background
Opioid overdoses are a pervasive public health crisis. Overdose education and naloxone distribution (OEND), and medications for opioid use disorder (MOUD) are evidence-based practices (EBPs) that can reduce opioid-related deaths. The HEALing Communities Study (HCS) worked with coalitions of community partner organizations during the Communities That HEAL (CTH) intervention to increase OEND and MOUD access. This study aims to explore how EBP strategies, or the mechanisms by which EBPs were delivered, were sustained after the study ended.
Method
Interviews with representatives from organizations who participated in the implementation of an OEND and/or MOUD EBP strategy were conducted 6–8 months after the HCS intervention period to understand how the EBP strategies had been adapted, maintained, or discontinued. A total of 135 interviews were transcribed, coded, and analyzed to identify the barriers and facilitators to sustainment.
Results
We identified five themes related to the sustainment of EBP strategies after the end of the HCS intervention period. Our findings highlight the importance of (a) organizational capacity building, (b) collaboration and partnership between implementing organizations, (c) integration of EBP strategies into routine care during the study, (d) adaptation of services after the intervention to better respond to evolving community needs, and (e) community support for the implementation of EBP strategies.
Conclusion
Future studies should explore how explicit attention to capacity building, collaboration within and between community agencies, and opportunities for adapting EBPs to better fit the needs and contexts of communities may facilitate sustainment of effective EBP strategies in other public health contexts.
ClinicalTrials.gov identifier
NCT0411939
Plain Language Summary
Why was the study done?
Opioid overdoses are a major public health issue in the United States. One way to address the overdose crisis is to provide overdose reversal medicine, naloxone. Another way is by supporting patients in accessing and continuing use of medications for opioid use disorder. The HEALing Communities Study provided funding and support to communities in four states to help them develop strategies that could increase access to and use of both types of services. This article seeks to understand how these services continued to be delivered in communities in the 6–8 months after the study.
What did the researchers do?
After the end of the study, researchers interviewed staff members who worked on these strategies. They explored these interviews to find commonly discussed factors contributing to organizations’ ability to keep delivering services after the study.
What did the researchers find?
Researchers found that organizations’ ability to continue delivering services relied largely on money to fund activities. If organizations found funding to support their work, other factors within their organizations and communities, such as staff capacity, partnerships between organizations, incorporation of study strategies into everyday processes, modification of study strategies to respond to client needs, and community support for their efforts, continued to positively impact the ways they delivered services after the study.
What do the findings mean?
These findings mean that research studying solutions to public and community health issues needs to focus more on building organizational capacity to acquire funding, promoting collaboration among community organizations, and appropriately integrating or modifying strategies to respond to changing needs and contexts within communities in order to ultimately prepare for the sustainment of effective EBP strategies.
Keywords
Introduction
In 2022 alone, 81,806 individuals in the US died from opioid overdose (Spencer et al., 2023). While provisional data from 2023 suggest a reduction in opioid-related fatalities among those with opioid use disorder (OUD) (CDC, 2024), efforts are still needed to mitigate the harms of the opioid epidemic. Overdose education and naloxone distribution (OEND) and medications for opioid use disorder (MOUD) are evidence-based practices (EBPs) that reduce opioid-related deaths (Tabatabai et al., 2023; Wakeman et al., 2020). Studies demonstrate the effectiveness of scaling up OEND and MOUD to reduce opioid overdose deaths, but these EBPs are underutilized (Beletsky & Tiako, 2024; Irvine et al., 2022; Krawczyk et al., 2022). Given the scope of the opioid epidemic and the effectiveness of OEND and MOUD, there is an urgent need to sustainably promote the delivery of such services.
The determinants of successful EBP implementation have been comprehensively explored in the Implementation Science (IS) literature; however, less is known about how and why EBPs reach sustainment following periods when implementation strategies were deployed (Aarons et al., 2010; Chambers et al., 2013; Glasgow et al., 2019; Margolies et al., 2023). This gap is particularly notable for community-engaged research that often relies on enduring partnerships (Holt & Chambers, 2017; Lindquist et al., 2025; Luger et al., 2020). Of studies that do explore sustainment in community-based settings, few are guided by implementation frameworks or consider the adaptations that may be necessary to promote sustainment (George et al., 2018; Hailemariam et al., 2019).
Sustainability is defined as the ability of a clinical intervention or program to continue delivering services after a specified period of time (Moore et al., 2017). Sustainment is the outcome that indicates whether such services continue to be delivered over time (Moullin et al., 2020). Understanding the processes and decisions that predict EBP sustainment is imperative because even after successful implementation, public health and clinical impact can be diminished by organizational decisions to scale back, significantly alter, or abandon delivery of an EBP (Aarons et al., 2016; Hailemariam et al., 2019). Particularly for EBPs that are implemented as part of community-engaged research studies, the ending of the intervention period is frequently accompanied by discontinuation of funding to continue and assess effectiveness (Bearman et al. 2020). Further, client and community needs evolve over time, necessitating potential adaptations to shifting circumstances when considering sustainment (Chambers et al., 2013). It is thus necessary to understand how transitions within the internal context (i.e., organization) and external context (i.e., community, state) of implemented EBPs shape sustained service delivery after initial implementation.
To address the gaps in the literature surrounding determinants of EBP sustainment, this article aims to identify and explore factors that promoted or inhibited EBP sustainment after the HEALing Communities Study (HCS) concluded for the intervention communities. HCS was a large community-engaged implementation research study that worked with community coalitions in 66 communities across four states to support the selection, design, implementation, and sustainment of OEND and MOUD EBP strategies. EBP strategies were defined as the specific approach by which OEND and MOUD services were delivered in communities; they allow for individualization to specific community contexts with the understanding that overdose reduction efforts are interconnected across a continuum of care and should not exist as isolated steps (Winhusen et al., 2020; Walsh et al., 2020). OEND EBP strategies focused on promoting active OEND, passive OEND, and naloxone administration. MOUD EBP strategies included expanding MOUD treatment availability, linking individuals to services, and promoting treatment engagement and retention. A more detailed outline of EBP strategy menu options is provided in Winhusen et al., 2020.
Once selected, EBP strategies were implemented by organizations in the healthcare, behavioral health, and criminal legal system sectors within each community. Through qualitative analysis of interviews with organizational staff involved in EBP strategy implementation and sustainability planning, we examined the factors contributing to EBP strategy sustainment in the 6–8 months after HCS. We considered the internal and external factors influencing short- and long-term sustainment. Short-term sustainment was defined as the initial ongoing delivery of services during the 6 months after the intervention period. Long-term sustainment was defined as the integration of new practices into organizational infrastructure and routine after the 6-month short-term sustainment period. From our analyses, we offer recommendations for researchers and community partners to enhance the sustainment of EBP strategies. Our findings have important implications related to the HCS goal of reducing opioid overdose deaths.
Method
Intervention Overview
HCS was a National Insitutes of Health funded implementation study conducted in four states (Kentucky, Massachusetts, New York, and Ohio), and was a multi-site, wait-listed, community-level cluster-randomized trial (ClinicalTrials.gov identifier: NCT04111939) that implemented an intervention, called Communities That HEAL (CTH), in 66 communities across the four states (Chandler et al., 2020; Walsh et al., 2020; Winhusen et al., 2020). To test the effectiveness of the CTH intervention, communities were randomized to participate in either Wave 1, which occurred from January 2020 to June 2022, or Wave 2, which occurred from July 2022 to December 2023. The goal of HCS was to reduce fatal opioid overdoses through the implementation of EBP strategies to increase OEND, MOUD, and safer prescribing practices in communities with high numbers of opioid overdose deaths. This article focuses on the sustainment of the OEND and MOUD EBP strategies after Wave 1. Safer prescribing EBP strategies represented a small proportion of EBP strategies selected by the coalitions for implementation, as many communities had already made significant improvements in opioid prescribing practices.
Using coalitions as a main implementation strategy, communities selected EBP strategies from menus of OEND and MOUD options and tailored them to their unique contexts (Sprague Martinez et al., 2020). Coalitions consisted of representatives from organizations that implemented EBP strategies (referred to as “implementing organizations”) and a range of other community members, including persons with lived or living experience of substance use; some implementing organization representatives were not coalition members.
Implementing organizations included those across the behavioral health, health care, and criminal legal system sectors such as hospitals, community-based organizations, and jails. In some states, coalitions selected and developed requests for funding for each EBP strategy to then be implemented by selected organizations. These requests were then submitted to the research site for approval and dispersal of funding. In other states, coalitions selected EBP strategies, but the research site worked directly with implementing organizations to provide resources (e.g., providing naloxone, paying for staffing). Funding levels varied among different EBP strategies. Many implementing organizations implemented one or more EBP strategies during the study, and some EBP strategies were implemented by multiple organizations working together.
The CTH intervention promoted the design and implementation of EBP strategies optimized for uptake and sustainment beyond the intervention period (Sprague Martinez et al., 2020). To do so, HCS supported coalitions to (a) build capacity by hiring and training community members, (b) engage with Learning Collaboratives to encourage collaboration between organizations and communities in the transmission of information, problem identification, and solution development, and (c) apply data-driven decision making during and well beyond the intervention period. Coalition members also participated in conversations with study staff to identify funding sources and concrete plans for sustaining effective and beneficial EBP strategies after the intervention period (Walsh et al., 2020). Participation from and collaboration across a wide range of organizations allowed for a unique and rich opportunity to examine determinants of sustainment in a broader way than studies that examine single organizations.
Data Collection
HCS included a mixed-methods assessment of the CTH intervention grounded in the RE-AIM/PRISM (Reach, Effectiveness, Adoption Implementation, Maintence/ Practical Robust Implementation and Sustainability Model) evaluation framework (Glasgow et al., 2019; Knudsen et al., 2020). One component of this assessment involved qualitative interviews with representatives from implementing organizations who participated in the implementation of an OEND and/or MOUD strategy during Wave 1 of HCS. Advarra, the HCS single Institutional Review Board (sIRB) approved the study protocol (Protocol Number: Pro00038088). Eligibility to participate in this round of interviews was determined based on involvement in the implementation of a EBP strategy, rather than individual interviewee characteristics. Interviews were conducted by members of the HCS research team familiar with the study processes and highly trained in qualitative data collection. All interview participants provided verbal informed consent prior to the beginning of the interview. Participants were offered a $50 gift card for their time. The interview guide development was guided by the PRISM/RE-AIM evaluation framework. The research team participated in a series of conversations to develop and modify the guide to most effectively inquire about how internal and external facilitators and barriers influenced EBP strategy sustainment. The interview guide is included in the supplemental materials section. Questions explored which EBP strategies organizations implemented, factors affecting implementation, which EBP strategies were sustained or adapted, and factors affecting sustainment 6–8 months after the end of the intervention period. These interviews focused on Wave 1 communities because Wave 2 communities were still participating in the active intervention and had not entered the sustainment period. To recruit participants, the research team utilized internal site databases to purposively select a sample of representatives from organizations located in both rural and urban communities and across sectors. Eligible organizations were those that implemented at least one EBP strategy during the CTH intervention period, but the sustainment of that EBP strategy was not required. Eligible participants were invited to participate via email. There were a total of 135 interviews conducted across the four states between January 2023 and March 2023. As some implementing organizations chose to conduct small group interviews with multiple staff members, there were a total of 179 interview participants. Per IRB protocol, interviews were conducted and recorded on Zoom, transcribed by a professional transcription service and reviewed by a member of the study team.
Data Analysis
All coders had experience in qualitative coding and participated in an HCS-specific coding training prior to the start of the consensus coding process. Data were coded using a two-stage coding process. First, all interview data were deductively coded at a high level using a codebook based on the PRISM/RE-AIM framework. For each EBP, this codebook contained codes corresponding to each of the constructs of the RE-AIM framework. (i.e., Reach-OEND, Maintenance-OEND, etc.). To examine sustainment, data for this analysis were extracted from the Maintenance-OEND and Maintenance-MOUD codes to create a subset of data that was inductively coded to identify themes related to the determinants of sustained service delivery and adaptations made to the EBP strategies after the intervention period.
To develop the inductive codebook, a team of four authors (one representative from each site) conducted an open coding process. Each member of this team reviewed excerpts from the subset of data independently and generated potential codes. The team then reconvened to discuss similarities and differences in their generated codes and developed an initial codebook. They shared this codebook with the full authorship group, discussing necessary modifications regarding the definitions and inclusion criteria for the codes to finalize the inductive codebook. Both codebooks are included in the supplemental materials section of this paper.
The same four authors then completed two rounds of consensus coding to ensure consistent application of the codes to the data. At the end of each round of consensus coding the Nvivo files were merged. Any variations in coding were identified via coding stripes, discussed, and resolved through conversation among the group (O’Connor et al., 2020). Once this discussion-based consensus was established across the four sites, the representative from each site who participated in consensus coding then led a similar consensus coding process with their own site's coding team to establish state-specific consensus. Once consensus was reached at the state level, all transcripts from each state, including those used for the consensus coding processes, were divided among the respective coding teams to complete the coding. There were a total of eight coders across all four sites, and NVivo 12 qualitative coding software was used.
Throughout the coding process, the coders identified the number of EBP strategies discussed in each transcript and provided qualitative justification for whether each strategy was sustained. This information was compiled in a shared document among the coding team to inform the development of Table 1.
Sustainment Status of Strategies Interviewed.
After coding was complete, the coding team performed thematic analysis (Vaismoradi et al., 2013). They developed an initial outline of themes and subthemes, which was presented to the full authorship group for discussion. The themes and subthemes were refined through an iterative process of conversations among the coding team and the larger authorship group. Once the themes were agreed upon, the coding team constructed the descriptions and identified the representative quotes presented in this paper.
Results
EBP Strategy Sustainment
Across implementing organizations that participated in the interviews, 279 EBP strategies were discussed. Of these, 139 were focused on OEND and 140 related to MOUD. Sustainment among discussed EBP strategies was high: of 139 OEND EBP strategies discussed, 136 (98%) were sustained 6–8 months after the end of the HCS intervention; of 140 MOUD EBP strategies discussed, 112 (80%) were sustained 6–8 months after the end of the HCS intervention. More information about EBP strategies can be found in Table 1. The interview sample was predominantly White, female, and highly educated. Full demographic information can be found in Table 2.
Characteristics of Interview Participants.
**Race = “other” includes individuals who identified as Asian/Native Hawaiian/Other Pacific Islander, Native American/Alaska Native, Mixed Race-Ethnicity, and Other Unspecified Race.
% rounded to the nearest tenth.
Qualitative Findings
Qualitative analysis identified five themes related to the sustainment of EBP strategies approximately 6–8 months after the HCS intervention period. First, organizational capacity building during the intervention facilitated funding acquisition to sustain EBP strategies after the study. Second, collaboration and partnerships between implementing organizations in the community enhanced sustainment. Third, integration of EBP strategies into routine care during the study eased the transition to post-intervention service delivery. Fourth, adapting services after the intervention to better respond to evolving community needs was integral to the sustainment of some EBP strategies. Finally, support for implementing EBP strategies within communities combatted stigma's negative effects and enhanced service delivery after the intervention. These themes and illustrative quotes are presented below.
Organizational capacity building during the intervention facilitated funding acquisition to sustain EBP strategies after the study.
Most participants expressed interest in continuing the EBP strategies implemented during HCS because of their evident benefit to the community. However, because EBP strategy implementation was supported by HCS funding, post-intervention sustainment depended upon the acquisition of external funding. Participants reported that the inability to find funding to continue to pay program staff, rent, and purchase supplies was the principal reason for discontinuation. Among sustained EBP strategies, participants discussed how different organizational capacity building strategies facilitated funding acquisition after the study. These strategies included identifying and hiring staff who were familiar with grant application processes, integrating sustainability considerations into EBP strategy implementation and strengthening partnerships with other implementing organizations in the coalition.
Identifying staff familiar with funding landscapes and grant applications facilitated funding acquisition and promoted sustainment.
Participants discussed how identifying and applying for funding to support the work of the organization was difficult as it required individuals to understand the intricacies of their EBP strategies and the complexities of funding landscapes. To combat this challenge, organizations built capacity during the study by identifying knowledgeable and experienced staff who had the time and skillsets to write grant applications. One participant emphasized the significant time requirement as the funding landscape for substance use-related grants has become increasingly competitive. I spend a lot of time looking for and writing grants or finding a grant … but it's highly competitive, and it's getting more so. (OH, OEND: Behavioral Health, #2774)
Post-intervention funding challenges were also mitigated by applying for and securing concurrent funding during the HCS intervention period. Participants described their organizational staff's foresight to build capacity in financially preparing for post-intervention EBP strategy sustainment. We received a five-year grant from [Funder] that started in 2020. So, we're still working with that grant, and that grant is very much aligned with what HCS was doing. (NY, OEND: Health Care #0084)
In the funding acquisition process, it was also reportedly important for staff to be aware of the ways that their organization would need to adapt to comply with guidelines and restrictions of new funding sources that differed from HCS. Thus, staff needed to assess their organization's capacity to respond effectively to such changes. For example, one participant described their organization's ability to comply with new data collection requirements. With [New Funder], I have to submit an Excel spreadsheet with all the demographics on it, which takes a little more time than what I got to do with you all…. It's not a problem to do it, it's just different. (KY, MOUD: Health Care, #1211)
However, at times, new requirements posed challenges that were more difficult to overcome as some funding sources imposed strict guidelines on the programmatic use of money. One participant described such restrictions on purchasing outreach materials including naloxone, printed handouts with educational and contact information, and other supplies, which contrasted with how they could spend money during HCS. They got X amount of dollars for outreach materials with very little oversight of what they were ordering. So now they're housed under a [Funder] grant, and it's really strict as to what they can buy. (MA, OEND: Behavioral Health, #1655)
HCS integrated sustainability considerations during initial EBP strategy implementation, which built organizational capacity for funding acquisition and continued service delivery.
Participants discussed how the infrastructural support from the study itself was beneficial in considering, planning for, and achieving sustainment. When the staff of implementing organizations were not already familiar with funding acquisition processes, HCS staff provided support in this area to help implementing organizations prepare for the end of the intervention. Participants appreciated conversations with HCS staff members to develop plans for post-intervention EBP strategy sustainment. One participant described that a major factor contributing to funding acquisition efforts was, “just [HCS Staff Member] laying it out for us to know what to do, who to reach out to, those things, we're able to keep it going” (KY, MOUD: Behavioral Health, #2768).
Participants also discussed how participation in the study provided implementing organizations’ staff with experiences, skills, and data that aided them in building capacity for success in the grant application process throughout the intervention period. Specifically, many appreciated the utility of data collection and EBP strategy monitoring skills that demonstrated organizational readiness for sustained service delivery. For example, one participant shared “Now I feel really comfortable writing those [grants], because I can point to what we've accomplished” (MA, OEND: Behavioral Health, #5151).
HCS staff also contributed to capacity building during the intervention period by working with implementing organizations to consider sustainability in the design and implementation of EBP strategies. Participants described the benefits of developing infrastructure that could easily be sustained after the intervention with little financial input. For example, one participant shared how the impact of a remote program to prescribe MOUD was beneficial to their organization and that, “the financial resource to ever get it (telehealth) started was probably the biggest help. Then it became self-sustaining in the sense” (KY, MOUD: Health Care, #2810).
Hiring and retention of dedicated and skilled staff were integral to sustained service delivery.
Sustained service delivery depended upon the extent to which implementing organization staff were dedicated to and had the capacity to continue the work started during the study. One participant described the “moral commitment to the community” that both motivated and necessitated sustained service delivery (NY, OEND: Health Care, #0202).
Additionally, implementing organizations recognized that positive working environments were also crucial to retaining staff and the sustainment of EBP strategies. One participant described that facilitating a supportive and collaborative work culture allowed them to “not necessarily just see patients, but to support the rest of the staff seeing patients, and make sure that the workflow of the clinic carried on” (KY, MOUD: Health Care, #2810).
Participants also discussed barriers to staff capacity and hiring that resulted from the COVID-19 pandemic which limited in-person activities, outreach, and posed challenges to filling positions needed to continue to deliver services. However, with specific respect to MOUD provision, participants also discussed how staff adapted to organizational needs and increased their comfortability with telehealth and remote service provision, ultimately promoting increased access to services in their communities. One participant described their staff's commitment to the EBP strategy despite the challenges the pandemic presented: “We’ll continue to be a menu of resources for the SUD population … whether it's telehealth or in-person here in the community. That's what we do” (NY, MOUD: Criminal Legal, #5236).
Collaboration and partnership between implementing organizations in the community enhanced sustainment.
Participants discussed that when multiple implementing organizations in one community came together to create inter-organizational plans for continued service delivery, funding was more easily acquired. While some partnerships and relationships between organizations preceded the start of HCS, many organizations agreed that participating in the coalition helped strengthen collaboration and support within and among organizations. One participant explained the important role of inter-organizational collaboration in capacity building for the grant application process. Working in partnership is also what I would advise because for number one it increases your likelihood of funding when you can show multiple entities are working together, and number two it's just better for your community to do that. (OH, MOUD: Criminal Legal, #0259)
Another participant described how an EBP strategy housed under a police department prevented them from seeking funding to continue a naloxone distribution program after the intervention. However, collaboration with a local non-profit in the community allowed for funding acquisition and the continuation of the program for both organizations. We partnered with a nonprofit, because we're a police department, so we can't do really fundraising. So, we get money, so it goes into a local 501(c) who sort of takes care of our funding stuff. (MA, OEND: Criminal Legal, 1561)
Partnerships between implementing organizations within the community also facilitated the exchange of non-financial resources, such as bus passes and naloxone kits, that aided in the sustainment of certain EBP strategies. For example, one participant expressed how concerns surrounding the future of their program were alleviated by a partnership with their community's health department that provided the necessary resources to continue. Before we weren't sure how we would be able to finance all of the free naloxone that we would have to be sending out for people prior to our partnership with the health department. And so that helped that piece of the puzzle. (OH, OEND: Health Care, #2003)
Ultimately, such partnerships that formed and strengthened during the intervention period allowed implementing organizations to continue to comprehensively respond to the needs of their communities and clients after the study. The spirit of collaboration, I would say is probably the number one plus that came out of HCS … if that can be replicated in communities around the country and have all these disjointed agencies all of a sudden start working together, the patient's going to benefit from that. (MA, MOUD: Health Care, #1580)
Integration of EBP strategies into routine care during the study eased the transition to post-intervention service delivery.
Participants discussed how routinization, or the integration of EBP strategies into the organization's day-to-day practices, during the intervention period allowed them to more easily continue service delivery after the study. During the intervention period staff became familiar with EBP strategy processes. Many participants discussed how these processes became “second nature” during the study, enhancing the organization's ability to navigate the transition to post-intervention service delivery. Really nothing's changed for us. We still operate as if we're currently practicing within this study. (KY, OEND: Health Care, #4356)
In addition, routinization facilitated a smoother transition for clients as they had also become familiar with the processes of organizations, EBP strategies, and staff members from whom they received services during the study. When asked how their organization was able to sustain their EBP strategy post-intervention, one participant described how clients had developed familiarity with organization processes which created certain expectations for services in the community, enhancing continued uptake of services. I think that the knowledge that the communities had already with the staff that was already in place, there was a built-in trust factor there and I think that was really very beneficial for us to being able to move forward with these interventions. (NY, MOUD: Health Care, #5251)
Adapting services after the intervention to better respond to evolving community needs was integral to the sustainment of some EBP strategies.
Implementing organizations were attentive to the ever-evolving contexts and needs of community members who access the services they provide. Responses to these needs resulted in adaptation of service delivery accordingly, ultimately promoting sustainment. While there were study restrictions to adhere to the CTH manual during the intervention period, organizations had more flexibility to adapt their service delivery processes more efficiently post-intervention. I would say that the peer distribution model changed … It had to be a flexible program… I surveyed participants, changed it based on their needs. (MA, OEND: Behavioral Health, #2292)
When organizational budgets evolved after the study, participants supported and argued for the continued inclusion of HCS EBP strategies that had proved meaningful and necessary for community members. For example, one participant discussed advocating for a transportation strategy to be included in their post-intervention budget because of the great need in the community and the significant impact it demonstrated during the intervention period. Now that we're operating as a sustainable clinic, we have specific budget parameters …, and we advocated for transportation being a part of it. That has just been a no-problem kind of thing, which is amazing. (MA, MOUD: Health Care, #5186)
Support for implementing EBP strategies within communities combatted stigma's negative effects and enhanced service delivery after the intervention.
Participants discussed how increased awareness of the opioid epidemic in the larger community contributed to demonstrated support for the work of EBP strategies. This support facilitated the acceptability of and advocacy for sustained service delivery. We have, like, I said, a growing … support coming from family members of the people that we’re serving … and the collateral impacts of what we're doing. (OH, MOUD: Criminal Legal, #0259)
Participants also identified support from local governing officials and legislation around the use of EBPs to promote awareness and combat substance use related stigma. As I mentioned before, [Politician Name] has really been a champion for us in all aspects as far as finding us money and supporting us on different events and using his platforms of social media to help get the word out. (NY, OEND: Behavioral Health, #0551)
However, it is also important to note that persistent stigma within communities interfered, at times, with the locations and organizations with which individuals delivering services could engage. Some of the stigma is going away there's still stigma associated with … you know drug usage. Some of the places that we want to get into, like some of these hotels and things like tha t… you know they're still struggling. (OH, OEND: Criminal Legal, #1982)
Discussion
Through an analysis of diverse organizational perspectives, we examined early sustainment of OEND and MOUD EBP strategies after HCS. We studied how implementing organizations built capacity to acquire funding for sustainment, explored how participants perceived organizational and community-level factors to shape the functioning and evolution of EBP strategies that occurred during the first 6–8 months after study completion. The findings from this study contribute to filling a literature gap surrounding the determinants of EBP sustainment after community-engaged research studies have ended.
First, we found that organizational capacity building for funding acquisition was crucial to EBP strategy sustainment post-intervention. While capacity building is reflected in the literature as a facilitator of EBP sustainment, our findings underscored the importance of integrating capacity building into interventions for the specific purpose of funding acquisition (Hacker et al., 2012; Hunter et al., 2017, 2020; Ramanadhan et al., 2020). This mechanism of capacity building is important as our study and others point to funding as the fundamental sustainment factor (Aarons et al., 2010; Hailemariam et al., 2019). Future community-engaged studies can integrate capacity-building activities into interventions by developing grant-writing and funding acquisition workshops, empowering organizations to secure post-intervention sustainment funding (King et al., 2015).
Institutionalization and the development of self-sustaining infrastructure can mitigate the costs associated with sustainment and ease anxieties about long-term funding. This preparatory step is especially important considering participants' comments about how grant funds typically included restrictions that may require adaptations to EBP strategies inconsistent with desired practices, had unclear timelines for funds distribution and termination dates, and ultimately left organizations in a constant state of uncertainty (Buchanan et al., 2005; Chambers et al., 2013; Glasgow et al., 2003; Shelton et al., 2018). Future studies can promote this process by working with communities to design and set up infrastructure that can be sustained after funding ends.
Our findings also emphasized the crucial role of dedicated staff in the continued functioning of EBP strategies after an intervention ends. Selection and retention of organizational staff were integral to the delivery of services during and after the intervention period, so sustainment efforts must assign sufficient attention to staffing challenges and issues (Aarons et al., 2010; Hacker et al., 2012). Staff dedication and value alignment can be fostered within a supportive culture and may offset costly turnover, which was identified as a sustainment challenge in our analysis (Aarons et al., 2010; Fixsen et al., 2005; Hunter et al., 2017, 2020; Lartey et al., 2013; Rodriguez et al., 2018). Hiring practices should be tailored to identify individuals who express dedication to serving people with OUD (Fixsen et al., 2005). Staff relationships with clients are also seen to be particularly important in the context of sustaining OEND and MOUD as the delivery of these services requires the establishment of trust (Ackland et al., 2023; Wenger et al., 2022).
Participants discussed the need for EBP strategy adaptations to more effectively meet the changing needs of organizations, communities, and clients as the study came to an end. Literature reflects our findings that adaptation is a common and necessary sustainment process and can facilitate effective responses to evolving needs (Carvalho et al., 2013; Chambers et al., 2013; Hunter et al., 2018; Krist et al., 2014; Stanton et al., 2005). Specifically, in the context of OEND and MOUD maintenance, research highlights the ongoing need for programs to adapt to the needs of clients and communities as they respond to the ever-changing opioid epidemic (Ackland et al., 2023; Bangham et al., 2023). Moreover, future research should explore how more static community contexts, such as rurality, may also influence sustainment.
The development of community networks during implementation can mitigate cost burdens and promote concerted sustainment efforts post-intervention. As was the case with HCS, community coalitions are an effective avenue for facilitating connections to disseminate resources (Butterfoss, 2007). Inter-organizational networks are key to implementation in many contexts, providing service provision support and opportunities to share resources and information (Butterfoss, 2007; Granner & Sharpe, 2004). Future research should explore how intricacies and power dynamics within these partnerships impact sustained service delivery.
Further, our findings suggest that the inter-organizational relationship development and resource sharing were particularly important for sustaining programs that involve the distribution of harm reduction supplies. This finding, taken together with the high cost of staffing organizations with medical personnel to prescribe MOUD, may also help explain the relatively higher sustainment rates of OEND EBP strategies compared to MOUD EBP strategies (Garcia et al., 2023).
Finally, community buy-in and awareness of the opioid crisis contributed to the sustainment of EBP strategies after the HCS intervention period. Research points to the importance of advocacy and culture shifts surrounding awareness about the opioid epidemic that can influence the sustainment of community-based EBP strategies that provide substance use-related services (Ackland et al., 2023). Overall, our findings emphasize the importance of assessing and utilizing community contexts during EBP strategy design and implementation to facilitate sustainment.
Limitations
This paper has limitations. First, our analysis presents the perspectives of representatives from implementing organizations who chose to engage with the study approximately 6–8 months after the intervention. It is likely that those organizations that sustained EBP strategies were more willing to participate in an interview than those that did not. Organizational representatives who participated in these interviews were not able to comment on all EBP strategies and may not reflect the experiences of these organizations with strategies that were discontinued during or after the intervention period. The study only explores early sustainment occurring 6–8 months after the intervention. The timing of data collection may have also contributed to the higher presence of sustained EBP strategies. Findings from extended sustainment periods may provide new insights. This bias may contribute to our greater emphasis on approaches to sustainment of EBPs rather than de-implementation and discontinuation. Future investigation could also explore the role of contextual factors such as geography, organization sector, or community attributes as factors in the sustainment of EBPs. Finally, it is also important to note that qualitative research is not generalizable. Despite these limitations, our findings offer important contributions to the understanding of EBP sustainment after a community-engaged intervention has been implemented.
Conclusion
Through semi-structured interviews with implementing organizations from multiple sectors in four states, participants described how sustainment was shaped by participation in the study itself, and by other internal organizational and external community contexts. Funding was a major factor influencing sustainment. For organizations that secured post-study funding, factors at the organizational and community levels shaped EBP strategy evolution and sustainment. Our findings highlight the importance of prioritizing capacity building at every phase of intervention design and implementation to support sustainment. Given that EBPs can only reach their full impact through sustainment, more research is needed to understand facilitators and barriers to sustainment and the role of adaptation, particularly over longer periods of time. Finally, future research should explore how this study's findings can be translated and applied to efforts to improve long-term EBP sustainability in other public health contexts.
Supplemental Material
sj-docx-1-irp-10.1177_26334895261450989 - Supplemental material for “We Have a Moral Commitment to the Community to Sustain It”: Exploring Determinants of Strategy Sustainment After the HEALing Communities Study
Supplemental material, sj-docx-1-irp-10.1177_26334895261450989 for “We Have a Moral Commitment to the Community to Sustain It”: Exploring Determinants of Strategy Sustainment After the HEALing Communities Study by Elizabeth Jadovich, Jill Davis, Shaquita Andrews-Higgins, Sandra Back-Haddix, Michael Goetz, Erin Kim, Christine Mayotte, Lauren A. Voss, Daniel Walker, Donna Beers, Nishita Dsouza, Ann Scheck Mcalearney, Dean Rivera, Hannah Knudsen and Mari-Lynn Drainoni in Implementation Research and Practice
Supplemental Material
sj-docx-2-irp-10.1177_26334895261450989 - Supplemental material for “We Have a Moral Commitment to the Community to Sustain It”: Exploring Determinants of Strategy Sustainment After the HEALing Communities Study
Supplemental material, sj-docx-2-irp-10.1177_26334895261450989 for “We Have a Moral Commitment to the Community to Sustain It”: Exploring Determinants of Strategy Sustainment After the HEALing Communities Study by Elizabeth Jadovich, Jill Davis, Shaquita Andrews-Higgins, Sandra Back-Haddix, Michael Goetz, Erin Kim, Christine Mayotte, Lauren A. Voss, Daniel Walker, Donna Beers, Nishita Dsouza, Ann Scheck Mcalearney, Dean Rivera, Hannah Knudsen and Mari-Lynn Drainoni in Implementation Research and Practice
Supplemental Material
sj-pdf-3-irp-10.1177_26334895261450989 - Supplemental material for “We Have a Moral Commitment to the Community to Sustain It”: Exploring Determinants of Strategy Sustainment After the HEALing Communities Study
Supplemental material, sj-pdf-3-irp-10.1177_26334895261450989 for “We Have a Moral Commitment to the Community to Sustain It”: Exploring Determinants of Strategy Sustainment After the HEALing Communities Study by Elizabeth Jadovich, Jill Davis, Shaquita Andrews-Higgins, Sandra Back-Haddix, Michael Goetz, Erin Kim, Christine Mayotte, Lauren A. Voss, Daniel Walker, Donna Beers, Nishita Dsouza, Ann Scheck Mcalearney, Dean Rivera, Hannah Knudsen and Mari-Lynn Drainoni in Implementation Research and Practice
Supplemental Material
sj-pdf-4-irp-10.1177_26334895261450989 - Supplemental material for “We Have a Moral Commitment to the Community to Sustain It”: Exploring Determinants of Strategy Sustainment After the HEALing Communities Study
Supplemental material, sj-pdf-4-irp-10.1177_26334895261450989 for “We Have a Moral Commitment to the Community to Sustain It”: Exploring Determinants of Strategy Sustainment After the HEALing Communities Study by Elizabeth Jadovich, Jill Davis, Shaquita Andrews-Higgins, Sandra Back-Haddix, Michael Goetz, Erin Kim, Christine Mayotte, Lauren A. Voss, Daniel Walker, Donna Beers, Nishita Dsouza, Ann Scheck Mcalearney, Dean Rivera, Hannah Knudsen and Mari-Lynn Drainoni in Implementation Research and Practice
Footnotes
Acknowledgments
The authors express sincere appreciation to the communities that participated in the HEALing Communities Study and the organizational representatives who participated in the interviews analyzed in this manuscript. We also acknowledge and thank the members of the HCS research team who supported the qualitative data collection processes.
ORCID iDs
Ethical Approval and Informed Consent Statement
Advarra, the HCS single Institutional Review Board (sIRB) approved the study protocol (Pro00038088). Per this protocol, prior to the start of the interview, all interviewees provided verbal informed consent to participate and be recorded.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the National Institutes of Health through the NIH HEAL Initiative under award numbers UM1DA049394, UM1DA049406, UM1DA049412, UM1DA049415, and UM1DA049417. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or its NIH HEAL Initiative.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability Statement
The qualitative interview transcripts are not available due to confidentiality and institutional data-sharing policies.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
