Sage Journals: Discover world-class research

Abstract

Background

Sustaining healthcare interventions once they have been implemented is a pivotal public health endeavor. Achieving sustainability requires context-sensitive adaptations to evidence-based practices (EBPs) or the implementation strategies used to ensure their adoption. For replicability of adaptations beyond the specific setting in question, the underlying logic needs to be clearly described, and adaptations themselves need to be plainly documented. The goal of this project was to describe the process by which implementation facilitation was adapted to improve the uptake of clinical care practices that are consistent with the collaborative chronic care model (CCM).

Method

Quantitative and qualitative data from a prior implementation trial found that CCM-consistent care practices were not fully sustained within outpatient general mental health teams that had received 1 year of implementation facilitation to support uptake. We undertook a multistep consensus process to identify adaptations to implementation facilitation based on these results, with the goal of enhancing the sustainability of CCM-based care in a subsequent trial. The logic for these adaptations, and the resulting adaptations themselves, were documented using two adaptation-oriented implementation frameworks (the iterative decision-making for evaluation of adaptations [IDEA] and the framework for reporting adaptations and modifications to evidence-based implementation strategies [FRAME-IS], respectively).

Results

Three adaptations emerged from this process and were documented using the FRAME-IS: (a) increasing the scope of implementation facilitation within the medical center, (b) having the internal facilitator take a greater role in the implementation process, and (c) shortening the implementation timeframe from 12 to 8 months, while increasing the intensity of facilitation support during that time.

Conclusions

EBP sustainability may require careful adaptation of EBPs or the implementation strategies used to get them into routine practice. Recently developed frameworks such as the IDEA and FRAME-IS may be used to guide decision-making and document resulting adaptations themselves. An ongoing funded study is investigating the utility of the resulting adaptations for improving healthcare.

Plain Language Summary

Evidence-based treatments may not be sustained after they have been implemented in healthcare settings. To address this, treatments and implementation strategies may need to be adapted to fit the local context or the patient population. Maximizing the usefulness of such adaptations requires documenting the decision-making process. Understanding how an implementation strategy has been adapted for a given study or setting is crucial to ensuring that adaptations don’t compromise fidelity to the implementation strategy while enabling its replicability in similar settings. This article uses two adaptation frameworks to describe the process by which implementation facilitation, a common implementation strategy, was adapted to help establish and sustain effective mental health clinical teams in VA medical centers. It is our hope that our description of this process may help healthcare researchers, administrators, and policymakers to describe and document adaptations to implementation strategies in their own settings.

Keywords

sustainability adaptation collaborative care team-based care FRAME-IS‌

Introduction

Healthcare interventions will have minimal public health impact if they are not sustained after initial implementation (Wiltsey Stirman et al., 2012). While substantial literature has investigated the adoption of evidence-based interventions in healthcare settings, research on sustainability is less developed (Shelton et al., 2018). Achieving sustainability may require the adaptation of clinical interventions or implementation strategies to fit the local context in which they are being used (Baumann et al., 2017; Chambers & Norton, 2016), and this adaptation may need to occur during the preimplementation phase to maximize effectiveness (Pluye et al., 2004). For replicability and usability of such adaptations beyond the specific context in question, the underlying logic for adaptation needs to be clearly described, and the adaptations themselves need to be plainly documented. To this end, a rapidly growing body of literature elucidates ways to design, enact, and document adaptations to clinical interventions (e.g., (Escoffery et al., 2018; Miller, Wiltsey-Stirman, et al., 2020; Wiltsey Stirman et al., 2019) and the implementation strategies that are used to get them into routine practice (Miller et al., 2021).

A subset of this literature has focused primarily on ways to plan adaptations to increase the reach, feasibility, and overall effectiveness of implementation efforts. One tool, called the iterative decision-making for evaluation of adaptations (the IDEA; Miller, Wiltsey-Stirman, et al., 2020), is a practical guide for adapting evidence-based practices (EBPs). The IDEA incorporates feedback loops, allowing it to iteratively assess the possible need for adaptations to an EBP. While the IDEA has been used by other research groups in the context of adaptations to EBPs (Sripada et al., 2022), to our knowledge, it has not yet been applied in the context of adaptations to implementation strategies. This is an important gap, as adaptations to implementation strategies are common—but have not been studied as comprehensively as adaptations to EBPs. Understanding how an implementation strategy has been adapted for a given study or setting is crucial to ensuring that adaptations don’t compromise fidelity to the implementation strategy while enabling its replicability in similar settings.

Similarly, while frameworks dating back over a decade have provided guidance on the documentation of adaptations to EBPs (Stirman et al., 2013; Wiltsey Stirman et al., 2019), research on documenting adaptations to implementation strategies is relatively nascent. For example, the framework for reporting adaptations and modifications to evidence-based implementation strategies (FRAME-IS; Miller et al., 2021) was designed to document the ways that an implementation strategy (or strategies) has been adapted to better fit with the local context in which a specific EBP is being implemented. The FRAME-IS includes seven core and optional modules to better meet the needs of researchers or project teams with varying resources and time available for this type of documentation. Other research teams have begun using the FRAME-IS to document adaptations to implementation strategies in their own studies (Lane et al., 2022), but more work is needed to determine the best ways to apply the FRAME-IS in ongoing implementation trials.

Thus, in this article, we describe the process by which an implementation strategy (implementation facilitation; Kirchner et al., 2014) was adapted to enhance the sustainability of the collaborative chronic care model (CCM; (Coleman et al., 2009; Wagner et al., 1996), an evidence-based approach to team-based mental health care. The need for these adaptations is rooted in qualitative and quantitative data from a prior implementation study (Bauer et al., 2021; Miller et al., 2023). Below, we describe this process in more detail using the IDEA and FRAME-IS, which to our knowledge have never been applied in concert to plan and describe adaptations to an implementation strategy. Then, we discuss the implications of this process for future efforts to design, document, and evaluate adaptations in implementation science.

Method

Project Background

Clinical Context: U.S. Department of Veterans Affairs General Mental Health Outpatient Clinics

The U.S. Department of Veterans Affairs (VA) represents the largest nationally integrated healthcare system in the United States, with over 150 VA medical centers and over 1,000 affiliated community-based outpatient clinics. Over nine million U.S. military veterans are enrolled in VA care, six million of whom receive healthcare services in a given year (National Center for Veterans Analysis and Statistics, 2021). About a third of these veterans have at least one mental health diagnosis (Government Accountability Office, 2021) accounting for a large share of VA healthcare costs (Watkins et al., 2011). Most of these veterans have their mental health care provided within VA General Mental Health (GMH) outpatient clinics, which are meant to serve as the mental health “home” for these veterans, akin to patient-centered medical homes for primary care (Jackson et al., 2013; Kearney et al., 2014).

To ensure that care within GMH clinics is well-coordinated, in 2013, the VA's Office of Mental Health and Suicide Prevention (OMHSP) released national guidance encouraging GMH clinics to adopt a team-based structure by establishing Behavioral Health Interdisciplinary Program (BHIP) teams. The BHIP staffing model recommends that BHIP teams include a multidisciplinary mix of at least nine staff, including licensed clinicians, nonlicensed providers, and administrative support personnel, serving a panel of about 1,000 Veteran patients (Bauer et al., 2016).

EBP: The Collaborative CCM

To help frontline BHIP teams deliver care in an evidence-based manner, OMHSP adopted the collaborative CCM (Coleman et al., 2009; Wagner et al., 1996) as the guiding clinical model for BHIP teams. The CCM consists of six elements that are meant to be flexibly applied in a given clinical setting (Figure 1) to enhance continuity and quality of care. Randomized trials have suggested that mental health care built around these principles results in improved patient outcomes, at no additional cost when compared to traditional care delivery (Miller et al., 2013; Woltmann et al., 2012).

Figure 1.

CCM Goal and Elements.

Implementation Strategy:Implementation Facilitation

To help BHIP teams structure their care processes in accordance with the CCM, in 2015, VA's Quality Enhancement Research Initiative (QUERI) funded an implementation trial known as the BHIP-CCM Enhancement Project (Bauer et al., 2016, 2019). This was a nine-site, stepped wedge, hybrid type II trial that used implementation facilitation (Kirchner et al., 2014; Smith et al., 2020) as its implementation strategy. Briefly, implementation facilitation is “a process of interactive problem-solving and support that occurs in the context of a recognized need for improvement and a supportive interpersonal relationship” (Ritchie et al., 2020, p. 3). It typically includes an external facilitator with expertise in the EBP being implemented, working closely with an internal facilitator or site champion who has knowledge of the clinical site that is attempting to implement the EBP. It is an inherently flexible strategy, which can involve the delivery of several discrete implementation tasks (e.g., obtaining leadership buy-in; clarifying roles and responsibilities; setting goals and priorities; and identifying and solving problems). Results from the BHIP-CCM Enhancement Project suggested that implementation facilitation was successful in improving collaboration within the involved BHIP teams and was associated with reduced mental health hospitalizations among patients treated by those teams (Bauer et al., 2019), at a substantial cost saving to VA (Miller, Griffith, et al., 2020).

Mixed-Methods Analysis of Sustainability

We undertook two follow-up analyses to determine the extent to which the gains described above were sustained over time. First, we repeated our analyses of mental health hospitalizations from the BHIP-CCM Enhancement Project, 1 year postfacilitation (Bauer et al., 2021). Second, we conducted qualitative interviews with staff from BHIP teams that had participated in the BHIP-CCM Enhancement Project, as described in more detail in previously published work (Miller et al., 2023). Briefly, we interviewed 30 total BHIP administrators and clinicians across the nine participating BHIP-CCM enhancement project sites, 3–4 years after the completion of implementation facilitation. Interviews were conducted, and qualitative analyses were undertaken, by a multidisciplinary team with extensive qualitative research experience. The interview guide questions and qualitative codebook were based on a variety of sources, with a primary emphasis on the sustainability of collaborative BHIP team-based mental health care. The analytic approach was built around directed content analysis (Hsieh & Shannon, 2005), and analyses were conducted in NVivo 12.

Consensus Process for Designing Implementation Strategy Adaptations

In 2020, QUERI funded a follow-up implementation trial, the BHIP-CCM Enhancement Project 2.0, to spread CCM-based care to additional BHIP teams. One goal of this trial was to enhance the sustainability of CCM-consistent care processes. We therefore planned to incorporate feedback from multiple sources to inform this adaptation process (see Table 1 for details). Each of the decisions made and documented regarding the adaptations was by consensus among the project team members based on multiple sources of input. As described in more detail in the Introduction section, we planned to use the IDEA and FRAME-IS to document this process and the resulting adaptations, in the hopes that such documentation would (a), aid in the interpretation of trial results and (b), contextualize trial findings for other implementation scientists and health system leaders contemplating adaptations to implementation strategies in their own settings.

Table 1

Sources of Input for Adaptations

Input source	Format for obtaining input	Timing of input
Project team	Weekly project team meetings	Adaptations discussed periodically during months leading up to facilitation launch
Office of Mental Health and Suicide Prevention (OMHSP)	Weekly meetings between Principal Investigator (PI) and OMHSP representative	Adaptations discussed periodically during months leading up to facilitation launch
Behavioral Health QUERI Stakeholder Council ^a	Quarterly meetings between project team representative and Stakeholder Council	Sustainability findings and planned adaptations presented to Stakeholder Council during two calls, during which the group provided input
Behavioral Health QUERI Technical Expert Panel ^b	Quarterly meetings between project team representative and Technical Expert Panel	Sustainability findings and planned adaptations presented to Stakeholder Council during two calls, during which the group provided input

Note. QUERI = Quality Enhancement Research Initiative.

Stakeholder Council consists of veterans, family members, and advocates. We are currently undergoing a process to rename the Stakeholder Council, consistent with CDC guidance regarding the use of the term “Stakeholder” (CDC, 2023).

Technical Expert Panel consists of current and former health system leaders from within and outside VA.

Results

We share below the results of systematically planning and documenting our adaptations to the implementation facilitation strategy to promote sustainable uptake of the CCM by BHIP teams in VA medical center GMH outpatient clinics. The results are organized in two parts: First, we describe our use of IDEA to assess the need for and plan implementation strategy adaptation. Second, we outline our use of FRAME-IS to document the resulting adaptations to implementation facilitation.

Using IDEA to Assess the Need for and Plan Implementation Strategy Adaptation

The IDEA is presented as a series of steps, which can be pursued in a nonlinear/recursive fashion. Below, we summarize these steps in developing an adapted approach to implementation facilitation for the BHIP-CCM Enhancement Project 2.0. Each step closely sought and incorporated input from the multiple sources noted in Table 1.

IDEA Step A: Do Existing Data Suggest That an Adaptation Is Needed?

As described above, there were two sources of follow-up data investigating the sustainability of CCM-based BHIP care from the BHIP-CCM Enhancement Project. Quantitative data indicated that there was no evidence that reduced mental health hospitalizations had been sustained 1-year postfacilitation (Bauer et al., 2021). These findings applied both to the original patients using BHIP services during that trial, as well as new patients treated by the same BHIP teams during the postfacilitation year.

These quantitative findings, while important, gave no indication of why the clinical impact of BHIP-CCM care may have faded over time. Thus, we conducted qualitative interviews with 30 clinicians who worked in the BHIP teams that participated in the BHIP-CCM Enhancement Project (Miller et al., 2023). Key findings from those qualitative interviews relevant to enhancing sustainability are summarized in Table 2, along with information relevant to sustainability derived from the study team. While these findings did not suggest that broad changes to the EBP (i.e., the CCM) were needed, they did suggest that the implementation strategy (i.e., implementation facilitation) would need to be adapted to enhance the sustainability of CCM-based BHIP care, as shown in the third column of Table 2.

Table 2

Findings Relevant to Enhancing Sustainability

Finding	Relevant example	Implications for implementation facilitation
Turnover of frontline BHIP clinicians or BHIP managers contributed to lack of sustainment of CCM-based BHIP care	One BHIP team had three BHIP clinicians (fully half the team) retire or otherwise leave their position within a 3-month span	Rather than working with a single BHIP team, work with a local BHIP implementation team including mental health leaders responsible for several BHIP teams
Many BHIP-related process improvements appeared to attain sustainability in the context of higher-level mental health service line input and resources	Measurement-based care champions, evidence-based psychotherapy coordinators, and clinical applications coordinators (external to the BHIP teams themselves) supported CCM-based BHIP care processes with facility-level support
While external facilitator expertise was appreciated, role clarity between the internal and external facilitators was difficult to establish	In some cases, implementation facilitation was seen by mental health leadership and frontline clinicians as being driven primarily by “the external consultants”	Frame the implementation process as being primarily driven by the internal facilitator throughout the implementation process
Implementation facilitation effort was reduced during months 6–12 of implementation at many sites	Time-motion tracking data (Kim et al., 2022) suggested reduced time spent on implementation facilitation after 6 months, with commensurate reduced pace of CCM-based process improvements	Instead of having 6 months of active facilitation and 6 months of step-down, opt for 8 months of active implementation facilitation at each site

Note. BHIP = Behavioral Health Interdisciplinary Program; CCM = Collaborative Chronic Care Model.

IDEA Step B: Are Core Elements or Functions Known?

While implementation facilitation is an inherently flexible implementation strategy, recent work points toward certain core activities that should be undertaken by implementation facilitators to promote successful uptake (Table 3; Smith et al., 2020). For our purposes, we considered these activities to represent the core functions (Perez Jolles et al., 2019) of implementation facilitation.

Table 3

Core Activities of Implementation Facilitation

Activity	Core for preimplementation phase	Core for implementation phase	Core for sustainment phase
Action/implementation planning	X
Engaging interested parties; obtaining buy-in	X
Identification/selection of local change agents	X
Data collection to assess context and baseline performance	X
Describing/clarifying roles and responsibilities	X
Goal/priority setting	X
Problem identification	X
Administrative tasks	X	X
Problem solving		X
Fostering organizational change—structural		X
Managing group/team processes		X
Adapting program to local context without compromising fidelity		X
Providing support		X	X
Providing updates and feedback		X	X
Conducting ongoing monitoring of program implementation		X	X
Pulling back and letting sites take the lead			X

Note. Adapted from Smith et al. (2020).

IDEA Step C: Can Barriers/Concerns Be Addressed While Preserving Core Elements or Functions?

There is a general consensus that core functions of EBPs and implementation strategies alike should be retained, with adaptations limited to the “adaptable periphery” (Damschroder et al., 2009) to minimize the chances that a crucial component is left out when adaptations are made. In this case, the sustainability challenges identified in Step A did not appear to necessitate adapting the core activities of implementation facilitation. That is, it was deemed unlikely by the study that team adaptations to implementation facilitation would necessarily involve eliminating any of the core activities listed in Table 3.

IDEA Step D: Does Timeframe Allow Pilot?

OMHSP is committed to supporting broad-based efforts to spread CCM-based BHIP care, meaning that their timeframe would not support a small-scale pilot study. Instead, adaptations to implementation facilitation, described in more detail in the next section, will be evaluated in the BHIP-CCM Enhancement Project 2.0. The remaining steps from the IDEA will be populated when that trial has been completed. These include steps and feedback loops focused on whether yet-to-be-collected results from the adapted approach suggest the need for further adaptation.

Using FRAME-IS to Document Resulting Adaptations to Implementation Facilitation

As described briefly above, the FRAME-IS consists of seven modules designed to catalog modifications to an implementation strategy. Table 2 summarizes the modifications that will be made to implementation facilitation based on the sustainability data summarized above. Table 4 also documents these modifications according to the FRAME-IS. In short, these adaptations are intended to enhance the sustainability of CCM-based care (Core Module 4, Part 1). This manifested primarily in three ways. First, the original BHIP-CCM Enhancement Project involved the external facilitator and internal facilitator working with one BHIP team. Postfacilitation qualitative interviews, however, indicated the potential pitfalls of this approach. In particular, interview results suggested that clinician turnover within the BHIP team could quickly undo months of CCM implementation work, as a few staff departures within the BHIP team could easily result in the team reverting to care as usual or being disbanded entirely. Furthermore, qualitative findings suggested that higher-level mental health support structures were essential for maintaining institutional knowledge supporting CCM-based BHIP care. Thus, for the BHIP-CCM Enhancement Project 2.0, for each site, the internal and external facilitators will instead be working with a local BHIP implementation team, consisting of facility-level mental health leaders, BHIP supervisors, and representatives from several frontline BHIP clinical teams. The specific constitution of these BHIP implementation teams will be left up to mental health leadership at the participating sites. Ideally, working at the level of the BHIP implementation team will provide some measure of insulation against staff turnover, as BHIP-related implementation work will be able to continue even if one or more key staff depart or are reassigned to different job duties. This will represent a fundamental shift in how implementation facilitation is delivered.

Table 4

Completion of the FRAME-IS for Adaptations to Implementation Facilitation

FRAME-IS module	Adaptation no. 1	Adaptation no. 2	Adaptation no. 3
Module 1
– The EBP being implemented is:	The collaborative chronic care model (CCM)
– The implementation strategy being modified is:	Implementation facilitation
– The modification being made is:	Internal and external facilitators work with a BHIP implementation team representing several BHIP teams (instead of one BHIP team)	Internal facilitator taking a relatively larger role in driving the implementation effort (when compared to the external facilitator)	Timeframe for implementation shortened from 1 year to 8 months
– The reason for the modification is:	To enhance the sustainability of the EBP (in this case, CCM-based BHIP care)
Module 2
– What is modified?	Context (population)	Context (personnel)	Content
Module 3
– What is the nature of the modification?	N/A	N/A	Shortening/condensing (while maintaining fidelity to core activities)
Module 4
– Part 1: what is the goal?	To enhance the sustainability of the EBP (in this case, CCM-based BHIP care)
– Part 2: What is the level of rationale?	Organizational level
Module 5
– When is the modification initiated	Preimplementation phase
Module 6
– Who participates in the decision to modify?	See Table 1
– Who makes the ultimate decision?	Project team
Module 7
– How widespread is the modification?	Implementation/facilitation team

Note. FRAME-IS = framework for reporting adaptations and modifications to evidence-based implementation strategies; EBP = evidence-based practice.

Second, post-facilitation qualitative interviews indicated that sites with relatively more active internal facilitators were more likely to have robust sustainability of CCM-based BHIP teams within their GMH clinic. In addition, interviews conducted with the external facilitators found that it was often difficult to establish role clarity between internal and external facilitators. External facilitators frequently felt that they, as opposed to the internal facilitators, were expected to drive change processes (Connolly et al., 2020). Thus, for the BHIP-CCM Enhancement Project 2.0, our approach to implementation facilitation will involve internal facilitators playing a larger role in leading the implementation process (including leading the BHIP implementation team), relative to the external facilitator. This will require adjusting plans for the preimplementation coordination process, and the incorporation of frequent discussions between the internal and external facilitators regarding role clarity throughout the implementation period at each site.

Third, active facilitation, as demonstrated by the time–motion study conducted during the original BHIP-CCM Enhancement Project, was delivered as 6 months of structured activities followed by ad hoc provision of as-needed support to the sites for an additional 6 months (i.e., a total of 12 months of facilitation; Kim et al., 2022). Based on sustainability data and multiple sources of input, the project team reached the conclusions that (a) the unstructured support (provided in months 6–12) was of limited utility and (b) making structured facilitation available to the sites for a longer duration of time would be helpful to enhance sustainability. Thus, the BHIP-CCM Enhancement Project 2.0 will have 8 months of structured facilitation. To maintain facilitation resources at a reasonable level even with this increased duration of active facilitation, the overall active facilitation period will be limited to these 8 months instead of 12 months total (Kim et al., 2022). Ideally, this provision of longer active facilitation—but in the context of a shorter overall implementation time period—will efficiently support the sustainability of CCM-based BHIP teams at the sites enrolled in the BHIP-CCM Enhancement Project 2.0.

Discussion

Effective healthcare interventions will have minimal public health impact if they are not sustained after initial implementation. In many cases, achieving sustainability requires that aspects of the clinical intervention and/or implementation strategy be adapted to fit the local context. Thus, in this article, we described the process by which we adapted implementation facilitation, a common and effective implementation strategy (Kirchner et al., 2014), to enhance the sustainability of CCM-consistent care processes in frontline mental health teams known as BHIP teams. Initial quantitative and qualitative data indicated that our previous attempts to implement CCM-based care in such teams did not result in robust sustainability (Bauer et al., 2021; Miller et al., 2023), and pointed toward possible adaptations. The need for these adaptations was cataloged in detail using the IDEA (Miller, Wiltsey-Stirman, et al., 2020), and we engaged in a robust consensus process to decide which adaptations to implementation facilitation would be most likely to maximize sustainability in a follow-up trial. We documented these adaptations using the FRAME-IS (Miller et al., 2021), which to our knowledge is the only implementation framework specifically designed to describe adaptations to implementation strategies. These results have implications for future efforts to implement CCM-based care specifically, as well as implications for designing and documenting adaptations to implementation strategies more broadly.

Implications for Using Implementation Facilitation to Promote Sustained Uptake of CCM-Based Care

As described in Table 4, study results suggest several possible sustainability-focused adaptations to implementation facilitation. First, consistent with previous literature (Woltmann et al., 2008), our results indicate the importance of insulating implementation efforts from the negative impact of staff turnover. To accomplish this, in our follow-up trial, we plan to work with a BHIP implementation team overseeing the work of several BHIP teams, rather than working with just one BHIP team. Ideally, in addition to ensuring that implementation gains are sustained even if key frontline clinicians leave the BHIP teams for other positions, this will have the additional benefits of more firmly engaging mental health leadership in the implementation process (Farahnak et al., 2020); creating opportunities for enhanced standardization across BHIP teams; and promoting communication among BHIP team members across the medical center. Second, role clarity between the internal and external facilitators has been recognized as pivotal to implementation success (Smith et al., 2020). Consistent with this, in our follow-up trial, we intend to shift the locus of control for the implementation process more firmly toward the internal facilitator, to minimize the chances that process improvements dwindle after the external facilitator ceases working at the site. This will require modifying documents to ensure that the internal facilitator is featured more prominently, along with up-front and ongoing discussions between the facilitators regarding this balance in roles. Third, while implementing the CCM is a complex process, our results indicate that a 1-year implementation period including an extended step-down phase may not represent the most effective use of limited resources; thus, in the follow-up trial, we instead plan to use 8 months of intensive facilitation per site.

Implications for Planning and Documenting Adaptations to Implementation Strategies

Research on adapting implementation strategies lags behind research on adapting EBPs (Miller et al., 2021), limiting the ability of implementation scientists and healthcare administrators to determine how best to adapt implementation strategies for their settings. Ideally, standardized reporting of adaptations will help to close this gap, by ensuring that lessons learned in one particular setting or study can be replicated in other settings. In the current report, we extended the application of the IDEA (Miller, Wiltsey-Stirman, et al., 2020), originally designed for EBPs rather than implementation strategies, to lay out the logic for adapting implementation facilitation. The incorporation of both quantitative and qualitative data was pivotal to this process (Palinkas et al., 2019) with quantitative data providing the “what” (i.e., the extent to which the clinical gains from CCM-based care were sustained), and qualitative data providing the “why” and “how” (i.e., what may have interfered with sustainability). Multiple parties contributed to the decision-making process around adaptations (as outlined in Table 1), consistent with an inclusive and representative approach to healthcare implementation (Boaz et al., 2018) and increasing our confidence in the value of the adaptations themselves.

Limitations

This work should be considered in the context of several limitations. First, while we undertook a rigorous process to identify and document adaptations to implementation facilitation, we do not yet have data on the clinical or implementation outcomes that will result from the application of this adapted approach. The BHIP-CCM Enhancement Project 2.0, which will use these adaptations in an eight-site implementation trial, is currently underway. Even if that trial does not demonstrate the sustainability of CCM-based care, it will provide valuable quantitative and qualitative data to inform further refinements of implementation facilitation. Second, we acknowledge that the literature contains various methodologies and frameworks for conceptualizing adaptations (e.g., the Model for Adaptation Design and Impact; (Kirk et al., 2020) and sustainability (e.g., the dynamic sustainability framework (Chambers et al., 2013) that we did not formally incorporate into our approach. Our process for identifying and documenting adaptations, however, incorporated many of the elements that are highlighted in these frameworks (e.g., accounting for contextual factors; attention to core elements or functions; and carefully delineating the rationale for adaptations).

Conclusions

Sustaining EBPs is challenging and may require careful adaptation of the EBPs themselves or the implementation strategies used to integrate them into routine practice. Recently developed frameworks such as the IDEA and FRAME-IS may be used to plan and document adaptations to implementation strategies, and may provide a clearer picture of the complex decision-making that may inform the adaptation process. Follow-up research will provide concrete data regarding the utility of the resulting adaptations for improving healthcare. More broadly, we hope that continued use of these frameworks will help the field refine the ways that adaptations are conceptualized, documented, and evaluated.

Footnotes

Declaration of Conflicting Interests

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Dr. Bo Kim is an Associate Editor of Implementation Research and Practice. As such, she was not involved in the peer review process for this article. All other authors declare no conflicting interests.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the U.S. Department of Veterans Affairs, Behavioral Health Quality Enhancement Research Initiative (BH QUERI; grant number: QUE 20-026).

ORCID iD

Madisen Brown

References

Bauer

M. S.

Miller

Kim

Lew

Weaver

Coldwell

Henderson

Holmes

Seibert

M. N.

Stolzmann

Elwy

A. R.

Kirchner

(2016). Partnering with health system operations leadership to develop a controlled implementation trial. Implementation Science, 11, 22. https://doi.org/10.1186/s13012-016-0385-7

Bauer

M. S.

Miller

C. J.

Kim

Lew

Stolzmann

Sullivan

Riendeau

Pitcock

Williamson

Connolly

Elwy

A. R.

Weaver

(2019). Effectiveness of implementing a collaborative chronic care model for clinician teams on patient outcomes and health status in mental health: A randomized clinical trial. JAMA Network Open, 2(3), Article e190230. https://doi.org/10.1001/jamanetworkopen.2019.0230

Bauer

M. S.

Stolzmann

Miller

C. J.

Kim

Connolly

S. L.

Lew

(2021). Implementing the collaborative chronic care model in mental health clinics: Achieving and sustaining clinical effects. Psychiatric Services, 72(5), 586–589. https://doi.org/10.1176/appi.ps.202000117

Baumann

Cabassa

L. J.

Stirman

S. W.

(2017). Adaptation in dissemination and implementation science. In Dissemination and implementation research in health: Translating science to practice (2nd ed., Vol. 2, pp. 286–300). Oxford University Press. https://doi.org/10.1093/oso/9780190683214.003.0017

Boaz

Hanney

Borst

O'Shea

Kok

(2018). How to engage stakeholders in research: Design principles to support improvement. Health Research Policy and Systems, 16(1), 60. https://doi.org/10.1186/s12961-018-0337-6

CDC. (2023). Centers for Disease Control and Prevention, CDC: Preferred terms for select population groups and communities . https://www.cdc.gov/healthcommunication/Preferred_Terms.html

Chambers

D. A.

Glasgow

R. E.

Stange

K. C.

(2013). The dynamic sustainability framework: Addressing the paradox of sustainment amid ongoing change. Implementation Science, 8(1), 1–11. https://doi.org/10.1186/1748-5908-8-117

Chambers

D. A.

Norton

W. E.

(2016). The adaptome: Advancing the science of intervention adaptation. American Journal of Preventive Medicine, 51(4 Suppl 2), S124–S131. https://doi.org/10.1016/j.amepre.2016.05.011

Coleman

Austin

B. T.

Brach

Wagner

E. H.

(2009). Evidence on the chronic care model in the new millennium. Health Affairs, 28(1), 75–85. https://doi.org/10.1377/hlthaff.28.1.75

10.

Connolly

S. L.

Sullivan

J. L.

Ritchie

M. J.

Kim

Miller

C. J.

Bauer

M. S.

(2020). External facilitators’ perceptions of internal facilitation skills during implementation of collaborative care for mental health teams: A qualitative analysis informed by the i-PARIHS framework. BMC Health Services Research, 20(1), 165. https://doi.org/10.1186/s12913-020-5011-3

11.

Damschroder

L. J.

Aron

D. C.

Keith

R. E.

Kirsh

S. R.

Alexander

J. A.

Lowery

J. C.

(2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, 50. https://doi.org/10.1186/1748-5908-4-50

12.

Escoffery

Lebow-Skelley

Haardoerfer

Boing

Udelson

Wood

Hartman

Fernandez

M. E.

Mullen

P. D.

(2018). A systematic review of adaptations of evidence-based public health interventions globally. Implementation Science, 13(1), 125. https://doi.org/10.1186/s13012-018-0815-9

13.

Farahnak

L. R.

Ehrhart

M. G.

Torres

E. M.

Aarons

G. A.

(2020). The influence of transformational leadership and leader attitudes on subordinate attitudes and implementation success. Journal of Leadership & Organizational Studies, 27(1), 98–111. https://doi.org/ 10.1177/1548051818824529

14.

Government Accountability Office. (2021). Veterans’ growing demand for mental health services. The Government Accountability Office. https://www.gao.gov/assets/gao-21-545sp.pdf

15.

Hsieh

H. F.

Shannon

S. E.

(2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288. https://doi.org/10.1177/1548051818824529

16.

Jackson

G. L.

Powers

B. J.

Chatterjee

Bettger

J. P.

Kemper

A. R.

Hasselblad

Dolor

R. J.

Irvine

R. J.

Heidenfelder

B. L.

Kendrick

A. S.

Gray

Williams

J. W.

(2013). The patient centered medical home. A systematic review. Annals of Internal Medicine, 158(3), 169–178. https://doi.org/10.7326/0003-4819-158-3-201302050-00579

17.

Kearney

L. K.

Post

E. P.

Pomerantz

A. S.

Zeiss

A. M.

(2014). Applying the interprofessional patient aligned care team in the Department of Veterans Affairs: Transforming primary care. American Psychologist, 69(4), 399–408. https://doi.org/10.1037/a0035909

18.

Kim

Miller

C. J.

Ritchie

M. J.

Smith

J. L.

Kirchner

J. E.

Stolzmann

Connolly

S. L.

Drummond

K. L.

Bauer

M. S.

(2022). Time–motion analysis of external facilitation for implementing the collaborative chronic care model in general mental health clinics: Use of an interval-based data collection approach. Implementation Research and Practice, 3, Article 26334895221086275. https://doi.org/10.1177/26334895221086275

19.

Kirchner

J. E.

Ritchie

M. J.

Pitcock

J. A.

Parker

L. E.

Curran

G. M.

Fortney

J. C.

(2014). Outcomes of a partnered facilitation strategy to implement primary care-mental health. Journal of General Internal Medicine, 29(Suppl 4), 904–912. https://doi.org/10.1007/s11606-014-3027-2

20.

Kirk

M. A.

Moore

J. E.

Wiltsey Stirman

Birken

S. A.

(2020). Towards a comprehensive model for understanding adaptations’ impact: The model for adaptation design and impact (MADI). Implementation Science, 15, 1–15. https://doi.org/10.1186/s13012-019-0962-7

21.

Lane

Wolfenden

Hall

Sutherland

Naylor

P. J.

Oldmeadow

Leigh

Shoesmith

Bauman

McCarthy

Nathan

(2022). Optimising a multi-strategy implementation intervention to improve the delivery of a school physical activity policy at scale: Findings from a randomised noninferiority trial. International Journal of Behavioral Nutrition and Physical Activity, 19(1), 106. https://doi.org/10.1186/s12966-022-01345-6

22.

Miller

C. J.

Barnett

M. L.

Baumann

A. A.

Gutner

C. A.

Wiltsey-Stirman

(2021). The FRAME-IS: A framework for documenting modifications to implementation strategies in healthcare. Implementation Science, 16(1), 36. https://doi.org/10.1186/s13012-021-01105-3

23.

Miller

C. J.

Griffith

K. N.

Stolzmann

Kim

Connolly

S. L.

Bauer

M. S.

(2020). An economic analysis of the implementation of team-based collaborative care in outpatient general mental health clinics. Medical Care, 58(10), 874–880. https://doi.org/10.1097/MLR.0000000000001372

24.

Miller

C. J.

Grogan-Kaylor

Perron

B. E.

Kilbourne

A. M.

Woltmann

Bauer

M. S.

(2013). Collaborative chronic care models for mental health conditions: Cumulative meta-analysis and metaregression to guide future research and implementation. Medical Care, 51(10), 922–930. https://doi.org/10.1097/MLR.0b013e3182a3e4c4

25.

Miller

C. J.

Kim

Connolly

S. L.

Spitzer

E. G.

Brown

Bailey

H. M.

Weaver

Sullivan

J. L.

(2023). Sustainability of the collaborative chronic care model in outpatient mental health teams three years post-implementation: A qualitative analysis. Administration and Policy in Mental Health and Mental Health Services Research, 50(1), 151–159. https://doi.org/10.1007/s10488-022-01231-0

26.

Miller

C. J.

Wiltsey-Stirman

Baumann

A. A.

(2020). Iterative decision-making for evaluation of adaptations (IDEA): A decision tree for balancing adaptation, fidelity, and intervention impact. Journal of Community Psychology, 48(4), 1163–1177. https://doi.org/10.1002/jcop.22279

27.

National Center for Veterans Analysis and Statistics. (2021). National Center for Veterans Analysis and Statistics . https://www.va.gov/vetdata/docs/Quickfacts/Stats_at_a_glance_12_31_21.PDF

28.

Palinkas

L. A.

Mendon

S. J.

Hamilton

A. B.

(2019). Innovations in mixed methods evaluations. Annual Review of Public Health, 40, 423–442. https://doi.org/10.1146/annurev-publhealth-040218-044215

29.

Perez Jolles

Lengnick-Hall

Mittman

B. S.

(2019). Core functions and forms of complex health interventions: A patient-centered medical home illustration. Journal of General Internal Medicine, 34(6), 1032–1038. https://doi.org/10.1007/s11606-018-4818-7

30.

Pluye

Potvin

Denis

J.-L.

(2004). Making public health programs last: Conceptualizing sustainability. Evaluation and Program Planning, 27(2), 121–133. https://doi.org/10.1016/j.evalprogplan.2004.01.001

31.

Ritchie

M. J.

Dollar

K. M.

Miller

C. J.

Smith

J. L.

Oliver

K. A.

Kim

Woodward

Ochoa-Olmos

Day

Lindsay

J. A.

Kirchner

J. E.

(2020). Using implementation facilitation to improve healthcare (version 3). Veterans Health Administration, Behavioral Health Quality Enhancement Research Initiative (QUERI). https://www.queri.research.va.gov/tools/Facilitation-Manual.pdf

32.

Shelton

R. C.

Cooper

B. R.

Stirman

S. W.

(2018). The sustainability of evidence-based interventions and practices in public health and health care. Annual Review of Public Health, 39, 55–76. https://doi.org/10.1146/annurev-publhealth-040617-014731

33.

Smith

J. L. E. A.

Ritchie

M. J.

Kim

Miller

C. J.

Chinman

M. J.

Kelly

P. A.

Landes

S. J.

Kirchner

J. E.

(2020). Getting to fidelity: Scoping review and expert panel process to identify core activities of implementation facilitation strategies. PrePrint. https://doi.org/10.31235/osf.io/6xfvj

34.

Sripada

R. K.

Rodriguez

J. L.

Wright

T. P.

Hyland

J. A.

Walters

H. M.

Ganoczy

Haft

S. M.

Smith

E. R.

Porter

K. E.

Driesenga

S. A.

DeJong

T. M.

Rauch

S. A. M.

(2022). Feasibility and acceptability of group-facilitated prolonged exposure therapy for PTSD in VA residential rehabilitation treatment programs. Behavior Therapy, 53(4), 714–724. https://doi.org/10.1016/j.beth.2022.02.004

35.

Stirman

S. W.

Miller

C. J.

Toder

Calloway

(2013). Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implementation Science, 8, 65. https://doi.org/10.1186/1748-5908-8-65

36.

Wagner

E. H.

Austin

B. T.

Von Korff

(1996). Organizing care for patients with chronic illness. The Milbank Quarterly, 74(4), 511–544. https://doi.org/10.2307/3350391

37.

Watkins

K. E.

Pincus

H. A.

Smith

Paddock

S. M.

Mannle

T. E.

Woodroffe

Jr Solomon

Sorbero

M. E.

Farmer

C. M.

Hepner

K. A.

Adamson

D. M.

Forrest

Call

(2011). The cost and quality of VA mental health services. RAND Corporation. https://doi.org/10.7249/RB9594

38.

Wiltsey Stirman

Baumann

A. A.

Miller

C. J.

(2019). The FRAME: An expanded framework for reporting adaptations and modifications to evidence-based interventions. Implementation Science, 14(1), 58. https://doi.org/10.1186/s13012-019-0898-y

39.

Wiltsey Stirman

Kimberly

Cook

Calloway

Castro

Charns

(2012). The sustainability of new programs and innovations: A review of the empirical literature and recommendations for future research. Implementation Science, 7, 17. https://doi.org/10.1186/1748-5908-7-17

40.

Woltmann

Grogan-Kaylor

Perron

Georges

Kilbourne

A. M.

Bauer

M. S.

(2012). Comparative effectiveness of collaborative chronic care models for mental health conditions across primary, specialty, and behavioral health care settings: Systematic review and meta-analysis. American Journal of Psychiatry, 169(8), 790–804. https://doi.org/10.1176/appi.ajp.2012.11111616

41.

Woltmann

E. M.

Whitley

McHugo

G. J.

Brunette

Torrey

W. C.

Coots

Lynde

Drake

R. E.

(2008). The role of staff turnover in the implementation of evidence-based practices in mental health care. Psychiatric Services, 59(7), 732–737. https://doi.org/10.1176/ps.2008.59.7.732

Adaptation for sustainability in an implementation trial of team-based collaborative care

Abstract

Background

Method

Results

Conclusions

Plain Language Summary

Keywords

Introduction

Method

Project Background

Clinical Context: U.S. Department of Veterans Affairs General Mental Health Outpatient Clinics

EBP: The Collaborative CCM

Implementation Strategy:Implementation Facilitation

Mixed-Methods Analysis of Sustainability

Consensus Process for Designing Implementation Strategy Adaptations

Results

Using IDEA to Assess the Need for and Plan Implementation Strategy Adaptation

IDEA Step A: Do Existing Data Suggest That an Adaptation Is Needed?

IDEA Step B: Are Core Elements or Functions Known?

IDEA Step C: Can Barriers/Concerns Be Addressed While Preserving Core Elements or Functions?

IDEA Step D: Does Timeframe Allow Pilot?

Using FRAME-IS to Document Resulting Adaptations to Implementation Facilitation

Discussion

Implications for Using Implementation Facilitation to Promote Sustained Uptake of CCM-Based Care

Implications for Planning and Documenting Adaptations to Implementation Strategies

Limitations

Conclusions

Footnotes

Declaration of Conflicting Interests

Funding

ORCID iD

References