Barriers to implementing motivational interviewing in addiction treatment: A nominal group technique process evaluation

Abstract

Background:

The successful implementation of evidence-based practices (EBPs) in real-world settings requires an adaptive approach and ongoing process evaluation and tailoring. Although conducting a needs assessment during the preparation phase of implementation is beneficial, it is challenging to predict all barriers to EBP implementation that may arise over the course of implementation and sustainment. This article describes a process evaluation that identified emergent and persistent barriers that impacted the implementation of an EBP across multiple behavioral health organizations and clinics.

Methods:

This study was conducted during the first cohort of a cluster randomized controlled trial testing the effectiveness of the Leadership and Organizational Change for Implementation (LOCI) strategy to implement motivational interviewing (MI) in substance use disorder treatment agencies and clinics. We used a modified nominal group technique (NGT) in which clinic leaders identified barriers faced during the implementation process. Barriers were categorized, then ranked and rated according to leaders’ perceptions of each barrier’s influence on implementation. The barriers were then contextualized through individual qualitative interviews.

Results:

Fifteen barriers were identified, grouped into staff-level barriers, management-level barriers, and implementation program barriers. Time and resistance to MI were rated as the most influential staff-level barriers. Among management-level barriers, time was also rated highest, followed by turnover and external contractual constraints. The most influential implementation barrier was client apprehension of recording for fidelity assessment and feedback. Individual interviews supported these findings and provided suggested adaptations for future implementation efforts.

Conclusion:

EBP implementation is an ongoing process whereby implementation strategies must be proactively and strategically tailored to address emergent barriers. This research described a process evaluation that was used to identify 15 emergent and/or persistent barriers related to staff, management, and the implementation program. Using implementation strategies that can be tailored and/or adapted to such emergent barriers is critical to implementation effectiveness.

Plain Language Summary

Unforeseen barriers often arise during the course of implementation. Conducting evaluations during implementation allows for tailoring the implementation strategy. As part of a larger study using the Leadership and Organizational Change for Implementation (LOCI) strategy to implement motivational interviewing (MI), we collected data from the first cohort of LOCI clinic leaders to identify barriers to MI implementation that persisted despite advanced planning and to understand unanticipated barriers that arose during implementation. Leaders identified 15 barriers faced during the implementation process that fell into three categories: staff-level barriers, management-level barriers, and implementation program barriers. The leaders ranked time as the most influential barrier at both the staff and management levels. Staff apprehension, resistance to MI implementation, and staff turnover were also of significant concern to leaders. Future implementation efforts may benefit from conducting a similar process evaluation during the implementation phase.

Implementation of evidence-based practices (EBPs) ideally involves a combination of methodical planning, and implementation strategy tailoring and adaptation. Planning efforts often consider the phases of implementation, previously identified implementation determinants, and those specific to the context in which implementation is to occur (Fernandez et al., 2019). Many factors have been shown to influence the implementation of EBPs, including barriers such as organizational culture and climate, support, and expertise (Aarons et al., 2014; Bach-Mortensen et al., 2018). Multiple frameworks, differing in structure and goals, have emerged to assist in the planning and implementation of EBPs across disciplines (Moullin et al., 2015). This study focused on the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework, which can be used to guide stakeholders through the process of identifying and selecting an EBP suitable to the needs or constraints of service systems, agencies, providers, and patients; preparing for its uptake; and for identifying or developing strategies for adopting, implementing, and sustaining the EBP (Aarons et al., 2011; Moullin et al., 2019). In addition, EPIS assists in identifying key implementation determinants and mechanisms (i.e., barriers or facilitators), including factors within the inner organizational-level context, outer system-level context, and those bridging the inner and outer contexts, which will ultimately support effective implementation (Lengnick-Hall et al., 2020; Moullin et al., 2019). As an agency and its staff move through the EPIS phases, it is important to conduct process evaluations and elicit feedback from relevant stakeholders to determine and address barriers to successful EBP adoption. Implementation strategies may vary in the degree to which they can be adapted and tailored to address barriers during the implementation process. To the extent possible, it is beneficial for this adaptation process to incorporate the perspectives and values of all relevant stakeholders involved in the process (von Thiele Schwarz et al., 2019).

Despite careful planning, unanticipated circumstances are likely to arise during the implementation process, making adaptation and tailoring desirable. Implementation strategies may need to be adapted and/or tailored in response to context, process, and clinician and/or patient constraints or opportunities (Baumann et al., 2017; Cabassa & Baumann, 2013; Chambers & Norton, 2016; Powell et al., 2017). In this article, we describe a process evaluation, which used a modified nominal group technique (NGT) and interviews, to identify emergent and persistent barriers that impacted the implementation of an EBP during an effectiveness trial of the Leadership and Organizational Change for Implementation (LOCI) strategy (Aarons et al., 2017).

The LOCI study

The present study is part of a larger cluster randomized trial of the LOCI strategy in implementing motivational interviewing (MI) in substance use disorder treatment clinics (Aarons et al., 2017). LOCI focuses on improving first-level leaders’ full-range and implementation leadership behaviors and aligning organizational-level strategies to develop and support clinic-level implementation climate and provider engagement to assist in the use of the EBP with fidelity (Aarons et al., 2017). First-level leaders are those who supervise clinical service staff who provide direct services such as therapists and counselors (Priestland & Hanig, 2005). To accomplish its goals, LOCI operates at multiple organizational levels and employs multiple components, including: (1) providing leadership and implementation training to first-level clinic leaders coupled with coaching to support ongoing progress, (2) using data to provide feedback on progress toward goals across all levels of an agency (i.e., executives, middle management, first-level leaders, and direct service staff), and (3) multi-level planning and strategies to align the strategic focus on EBP implementation to communicate that the EBP is expected, supported, and rewarded (Klein et al., 2001; Schein, 2010). Through these components, LOCI engages key implementation determinants to positively influence implementation process mechanisms and outcomes.

In consultation with the funding agency, MI was selected as the EBP to be implemented for adults engaging in services to address substance use disorders. MI utilizes an empathic client-centered approach that emphasizes the client’s role in evoking change talk and resolving their ambivalence regarding change (Miller & Rollnick, 2012). Although MI training is widely available and delivered, leaders and agencies often do not provide additional support beyond initial training sessions to effectively implement MI as a counseling technique (Madson & Campbell, 2006). To develop provider-level MI skills, a 2-day training was offered to all participating direct service staff and first-level leaders. As part of the implementation strategy, first-level leaders were included as participants in the MI training so that they may demonstrate to their direct service staff their own knowledge of, and commitment to, implementing MI. Direct service staff were asked to audio-record sessions with clients, upload their recordings to a HIPAA-compliant web portal, and have one recording randomly selected to be coded for fidelity to MI by an external consultant. Recording devices and technical assistance were provided to service staff in the MI training and ongoing throughout the project. Feedback reports were provided to service staffs’ first-level leader, who in turn delivered the feedback to their staff members.

Purpose of the present study

The present study utilized data from the first cohort of first-level clinic leaders who completed LOCI to identify barriers to EBP implementation and determine if there were any unanticipated barriers for which the LOCI strategy could be tailored.

Methods

The present analysis focuses on data combined from two procedures completed during the first cohort of the LOCI study: (1) a modified nominal group technique process and (2) post-implementation semi-structured interviews. The study was approved by the University of California, San Diego Institutional Review Board. All participants were given the opportunity to consent or decline participation in any of the research study components. Participants were able to cease participation in any part of the research study at any time.

Participants

Participant demographics differed slightly across the three study activities: nominal group technique brainstorming session (n = 9), ranking and rating survey (n = 14), and qualitative interviews (n = 15; detailed in Table 1). The brainstorming session took place first, during a LOCI leadership booster training session; therefore, all participants were in the LOCI condition. Four months later, participants in both the LOCI and control conditions participated in the ranking and rating survey, as well as the individual interviews. As shown in Table 1, overall, the sample was majority female, Caucasian, and non-Hispanic.

Table 1.

Sociodemographic characteristics of participants.

Baseline characteristic	Nominal group technique		Ranking and rating		Qualitative interview
	n = 9	%	n = 14	%	n = 15 a	%
Gender
Female	5	56	11	79	10	71
Male	4	44	3	21	4	29
Race
Caucasian	3	33	8	57	7	50
Black	3	33	3	21	3	21
Other	3	33	3	21	4	29
Hispanic	3	33	6	43	5	36
Highest educational level
High school	0	0	1	7	0	0
Some college	5	56	5	36	6	46
College	2	22	5	36	4	31
Some graduate work	0	0	1	7	1	8
Master’s	2	22	2	14	2	15
	M	SD	M	SD	M	SD
Age	52.09	11.71	46.19	11.89	48.69	12.34
Time with agency	5.11	6.45	4.14	5.64	11.78	9.63

a

No demographics available for one participant in the qualitative interview group.

Modified nominal group technique process

The modified nominal group technique process included an in-person brainstorming session and a follow-up survey. For a timeline displaying when this process occurred relative to other components of the study, see Figure 1. Nominal group technique is designed to minimize the effect of intragroup dynamics and encourage openness and contributions from all participants. This is particularly useful when some attendees are more vocal than others or if certain attendees think better in silence, by ensuring each member contributes ideas (Harvey & Holmes, 2012).

Figure 1.

Modified nominal group technique timeline.

Brainstorming

The brainstorming session was conducted at the final LOCI strategy training session, at the 8-month study time point. Nine first-level leaders assigned to the LOCI strategy condition participated in the guided process.

To begin the session, a member of the research team JM posed the following question to the participants: “Identify barriers/issues that you have faced during the MI implementation process” and clarified the statement as needed until all participants understood it. Each participant then silently considered the question for 5 minutes and wrote down as many ideas related to the subject that they could think of on post-it notes. Next, each participant stated aloud one idea that he or she had written, and this continued from participant to participant, going around the group one idea at a time, until all ideas were collected. All participants were given a chance to voice their idea without discussion, questions, or interruptions.

After the session, a member of the research team thematically organized participant responses into distinct barriers and then categorized these barriers into three categories that were representative of the content. These barriers and categories were reviewed and affirmed by three other members of the research team.

Rating and ranking

As part of a web-based survey conducted 4 months after the NGT activity, six leaders from the LOCI condition and eight leaders from the webinar control condition answered two questions as a follow-up to the nominal group technique brainstorming session. First, participants were asked to rate how influential, on a sliding scale from 0—not at all influential to 10—extremely influential, each of the identified barriers were on implementing MI with fidelity for their team. Second, participants were asked to rank the barriers, separated into each of the three categories, in order of their degree of influence on implementing MI with fidelity for their team, ranking #1 as the most influential factor. Finally, the research team combined the rating and ranking information to provide an overall ranking of the barriers. To do so, the participants’ ratings were converted into rankings by ordering the median scores for the barriers with the highest median first. For example, for management level factors in the LOCI condition, managers’ time had the highest median of 8.5, so was ranked first, followed by staffing (median = 7), ranked second, and support from other levels (median = 5.5), ranked third. Both rating columns were transformed into rankings in this manner and then the research team calculated the overall ranking of barriers, which summarizes the perceived influence of each barrier, by averaging the resulting four sets of rankings.

Qualitative interviews

Four months after the NGT activity, following their year-long participation in either study condition, all first-level leaders were invited to participate in a 1-hr semi-structured qualitative interview. Conducted by a medical anthropologist, the overall purpose of these interviews was to learn about leaders’ experiences with MI implementation strategies that were executed over the course of the year, their opinions about how to improve specific components of the implementation process, and the overall study. The interviews also included specific questions about barriers, facilitators, and strategies to implementing MI within their clinic, for example: “What challenges are counselors encountering/experiencing in maintaining fidelity to MI?” and “How has the agency supported the implementation and use of MI over the last 12 months?”

All interviews were audio recorded, transcribed, and coded using a primarily deductive approach to contextualize the variables that were discovered within the modified nominal group technique process. Additional memos were used to discuss emergent ideas mentioned in the interviews that were not identified in the nominal group technique. Definitions for each barrier were developed using field notes from the brainstorming session and used as codes by which to examine the qualitative data. As a starting point, data were extracted from four interview transcripts and independently coded by two researchers MH and KC. Codes were discussed and refined, and two additional transcripts were used to confirm 100% agreement around which passages applied to each code. The remaining interviews were coded once. Quotes from each barrier were summarized to conceptualize the way in which leaders perceived the barrier across agency affiliation and condition assignment, and emergent ideas were discussed.

Results

Identified barriers

Fifteen barriers were identified in the nominal group technique brainstorming session and grouped into three categories: (1) staff-level barriers, (2) management-level barriers, and (3) implementation program barriers. Table 2 displays the results from the ranking and rating activities. Overall rankings were calculated by converting the ratings to rankings (i.e., highest median rating ranked first) and then averaging all four rankings. These barriers were contextualized using qualitative interviews.

Table 2.

Rating and ranking of barriers identified by nominal group technique brainstorming activity, ranked and rated according to leaders’ perception of influence on the implementation process.

Category of identified barriers	Rating median		Ranking median		Overall rankings
	LOCI (n = 6)	Control (n = 8)	LOCI (n = 6)	Control (n = 8)
Staff level barriers
Staff lack of time	8.5	8.0	1	1	1
Staff resistance	6.5	5.0	2	2	2
Staff lack of confidence	6.5	6.0	5	4	3
Staff competency in using MI	7.0	5.5	4	6	4
Staff apprehension	5.0	5.0	3	3	5
Staff lack of buy-in to MI	6.0	6.0	6	5	6
Management (supervisor) level factors
Leaders’ time	8.5	6.5	1	1	1
Staffing (staff turnover and short staffing)	7.0	6.0	2	2	2
External contractual constraints	5.0	5.5	5	3	3
Support from other levels	5.5	4.5	4	4	4
Financial resources	4.0	4.0	3	5	5
Leadership turnover	1.0	5.0	6	6	6
Implementation program factors
Clients’ apprehension and resistance to recording	5.5	8.0	2	1	1
Staff’s apprehension and resistance to recording	5.5	5.5	1	2	2
Technology issues	3.0	3.5	3	3	3

Ratings indicate how influential leaders in each condition perceived each factor to be, on a scale of 0 (not at all influential) to 10 (extremely influential). Rankings indicate how influential each factor was perceived to be in relation to the other factors in that category.

Staff-level barriers

Staff-level barriers are situational factors and characteristics of the counselor population in the substance use disorder treatment centers that directly impacted the successful implementation of MI. Six staff-level barriers identified from the NGT and described from the perspective of leaders interviewed are detailed below:

1. Lack of staff time. MI implementation, particularly practicing MI skills, was viewed as overly time-consuming. Staff had large caseloads, meetings, and other job requirements, and these other time constraints took precedence over implementing MI in their sessions. Clinics that were not fully staffed required current employees to take on multiple day-to-day jobs just to maintain viability, and therefore, MI implementation could not be prioritized (e.g., “We were so horribly short-staffed I don’t know how I kept my doors open”). When new counselors were finally hired, these clinics remained in a state of “rebound” as all staff were behind on their regular work tasks. As a result, staff were both overworked and burned out, causing MI to be forgotten about or “put on the backburner.” One leader summarized this concern: “This would be something that they would probably have to take home with them to read up on and practice [outside of work hours] and come back in, but can I ask them to do that without paying them?”

2. Resistance. Certain staff were unwilling to use MI and/or engage in the MI feedback process because it was new and different than their regular routines, a sentiment amplified if a staff member already held general closed-mindedness and reticence toward any culture change. This was most common among the more seasoned counselors who had already developed a pattern of clinical practice over time (e.g., “[One of my staff members] was definitely one of the old school style of counselors, not very touchy feely for sure”) and among counselors who were less invested in their work (e.g., “I had conversations with my other counselor but it was just taken as ‘Yeah, okay. Whatever. I did my job.’”). As a whole, staff resistance to change dissipated over the course of the study.

3. Lack of confidence. Some staff lacked confidence in their ability to deliver MI with fidelity and therefore avoided using MI or participating in the MI feedback process all together. At times, this phenomenon occurred among staff who displayed sufficient competency in MI but did not realize their skill level (e.g., “A lot of times counselors feel like they really have to think about doing MI when they were actually already doing MI without even thinking about it”). Leaders stated that counselor confidence in MI improved with encouragement and practice and was higher among staff by the end of the study (e.g., “Not being good at something because it’s new and ‘I don’t know how to do it’ kind of deters them from partaking, in really having fidelity to the recording, so it kind of made them seem like they were not cooperative at first. But now that I feel like because they’ve gotten some really good feedback . . . their confidence has increased as they have learned.”).

4. Competency. Some staff lacked a general understanding of MI. For example, one leader stated that his staff became preoccupied with adhering to the MI model (e.g., how and when to use exact MI techniques), rather than the overall purpose and spirit of MI, which resulted in less effective therapy sessions. In addition, counselors varied in level of experience and training in MI implementation; some counselors had been trained in MI before the study whereas others had little to no exposure to the EBP. As a result, staff varied widely in their need for support and supervision, which complicated first-level clinic leaders’ plans for overall MI implementation.

5. Staff apprehension. Related to lack of confidence, certain staff were concerned about taking risks and trying something new, and subsequently avoided the MI feedback process. Rather than ask for help, staff made excuses about why they were not able to fully engage with MI (e.g., “Some of these people were so worried about how they might look and then they would make excuses like, ‘Oh, the clients don’t want to sign the waiver’”). Apprehension was frequently expressed at the beginning of the study, though one leader commented that this concern dissipated over time as counselors became more comfortable with the EBP.

6. Lack of staff buy-in to MI. A few staff discredited the value of and evidence behind MI and therefore did not accept agency efforts to implement MI (e.g., “If the counselors don’t see it as being significant, they’re not going to go in that direction [of putting time and energy into the MI feedback process]”). Despite first-level clinic leaders’ various efforts to increase the appeal of MI to their staff, some staff lacked interest in adopting MI because they did not understand how it could help their clients. Support from upper management and the provision of incentives sometimes helped mitigate these struggles: “Sometimes, you’ve really got to take the upper management and kind of come down and say, ‘Hey, you’ve got to get behind this.’” Unlike other staff-level barriers that dissipated with time, leaders stated that the level of staff buy-in fluctuated throughout the course of the study.

Management-level barriers

Management-level barriers are factors that disrupted implementation efforts through impeding support systems on which staff rely. Six management-level barriers were identified:

1. Leaders’ time. Participants felt that they, as first-level clinic leaders, were unable to dedicate the time, focus, and energy they felt necessary to adequately follow up on the initial MI training. Although they expressed a desire to provide education and supervision to their staff, leaders were diverted by their regular day-to-day workload, as well as abrupt clinic disruptions such as staff turnover and being short staffed, that required them to take on additional work (e.g., “It has been extremely difficult for me to fit the time commitments that were necessary to do an excellent job with this . . . Unfortunately, other kind of things trumped my attention”).

2. Staffing. Staffing concerns occurred at all levels in the agency, from counselors to executives. Agencies were short-staffed, either permanently or temporarily, due to a cycle of staff turnover. Participants were affected by turnover at levels above them (i.e., executive leadership) and below them (i.e., staff). Executive leadership turnover particularly affected MI implementation when executives recruited onto the LOCI study and who championed strategic development around MI, later left the agency with little transition to their replacement. Staff turnover required leaders to consider how newly hired staff differed from those who had left; leaders reported some new staff were less confident, less trained, and held different attitudes toward EBPs.

3. External contractual constraints. Participants’ agencies were funded by external contracts that demanded specific documentation and clinical service provision to maintain funding. As new external contracts were adopted by the agency, new mandates required extensive preparation, which interfered with the delivery of EBP services. When newer contracts were signed, there were competing demands agency-wide that staff, leaders, and executives had to focus on. This was in addition to their own deadlines that were high priority. In extreme cases, entire clinics closed when they were unable to keep up with their external contracts and get renewed funding. Within this study, a new waiver was introduced in California that diverted much of the attention of both leaders and staff away from MI implementation to new waiver-mandated requirements: “July 1st everything goes to drug MediCal, so we need all the time we have in case conference and other meetings to work on what needs to change by July 1st, so I think this kind of took a backseat to that.”

4. Support from other levels. Although leaders saw the value in adapting a new EBP model, other members of the leadership team, such as executives and clinical supervisors, offered limited support and/or resisted these strategic efforts (e.g., “We have to come together as an agency and [implementation] has to go all the way up, you know? Because I can set goals for my people that are under me, but I don’t have anybody above me who has ever like one time asked me about, ‘Hey, how is the MI going?’”). Leaders felt that executives did not provide sufficient guidance, support, or attention to reinforcing the importance of EBP implementation (e.g., “The hardest thing about this whole process is you know each program [within the agency] has kind of had to come up with its own way of doing this, and the support that we’ve received from corporate from the agency itself has been pretty low”).

5. Financial resources. As part of the LOCI intervention, the LOCI trainers introduced the benefits of establishing rewards and incentives to maintain motivation for new EBP initiatives. Although first-level clinic leaders were enthusiastic about this suggestion, their agencies lacked the financial resources to designate funds for this purpose. Therefore, some leaders were not able to provide rewards or incentives, while others paid for rewards out of their own pocket (e.g., “I could never justify gift card incentives for staff . . . I’d get in big trouble for that. So do you take it out of your own pocket? Um okay, seeing what non-profit people make . . .”). Leaders also discussed the need for financial resources to support provision of additional MI trainings, including refresher trainings for previously trained staff or full trainings for new staff members.

6. Leadership turnover (at the LOCI leader level). In addition to mentioning turnover at other levels in the organization, participants also highlighted the effects of LOCI leader turnover. Throughout the study, LOCI leaders left their clinic, were promoted to other positions, or stepped down from leadership positions. Replacement LOCI leaders came from positions both internal and external to agencies. Within the NGT, those replacement leaders commented on how difficult it was to enter in the LOCI leader role partway through the study.

Implementation program barriers

As part of the LOCI study, fidelity was measured through the coding of recorded MI sessions with clients. This process was part of the implementation program only and was removed after the one-year research study procedures were completed. There was significant resistance to this fidelity monitoring process due to technological challenges, as well as apprehension and resistance among staff who were already overburdened and/or insecure in their abilities to effectively implement MI. Three barriers were identified:

1. Clients’ apprehension and resistance to being recorded. Leaders and staff reported that many clients refused to have their sessions recorded. These clients, especially those in forensic or probation systems, may have been uncomfortable with any potential breaches in confidentiality or privacy. However, this barrier may also have been confounded with other barriers, as some leaders noted that clients’ reactions were based on staff’s introduction to recordings (e.g., “I think a lot of [clients’ resistance] was how the counselor presented the recording itself”) and staff’s perceptions (e.g., “[My staff] didn’t think their clients were going to want to do it”). In addition, one leader commented that clients’ concern with being recorded may change the nature of sessions: “[Some patients] take on a different role, they’re not themselves really . . . They want to be politically right . . . The patient won’t put down the barrier because the recorder.”

2. Staff’s apprehension and resistance to recording. Related to aforementioned staff-level issues, staff expressed discomfort around recording their sessions that would then be scored and reviewed in the feedback process. Counselors expressed concern regarding how they would be perceived after others listened to their recordings, how they would be graded or judged, and ultimately if they would face punitive action around their performance. This resistance was so prevalent that despite input from and problem-solving with higher-level executives, first-level clinic leaders struggled to motivate their staff to record sessions for upload.

3. Technology issues. Staff had difficulty using the technological equipment provided by the study team (e.g., audio recorders) and at their own agencies (e.g., computers, internet issues) that were necessary to record and upload MI sessions. As a result, staff required more training and/or additional help from their information technology support staff or LOCI staff to navigate these issues, which resulted in more time and coordination than anticipated. One leader remarked, “If we would get past [the technology issues], then we would have focused more on the [MI/clinical] issues.”

Ranking and rating the identified barriers

The rankings and ratings provided by the leaders did not align in all cases, but the overall rankings display the trend (Table 2). Across conditions, leaders rated and ranked time as the most influential factor at both the management and staff levels. Within the management-level barriers, the category of staffing issues (turnover and being short staffed) was the second greatest concern, followed by the burden of external contractual constraints, support from other levels, financial constraints, and finally leadership turnover. At the staff level, resistance to MI implementation and staff confidence in their own implementation skills were ranked second and third most influential. Staff competency in using MI and staff apprehension were also rated as significant barriers to implementing MI with fidelity, followed by staff buy-in. Both client and staff apprehension and resistance to recording were rated as influential barriers to the fidelity measuring process. Technological issues were not as influential, as displayed by their low ranking.

Discussion

This study sought to identify barriers, as identified by first-level leaders, to EBP implementation that arose over the course of a longitudinal effort focused on implementing MI in outpatient substance use treatment agencies. Through identifying these barriers, this study aimed to anticipate barriers in future implementation efforts by tailoring LOCI or other EBP implementation strategies to individual organizations, especially pertaining to preparing and motivating staff, promoting implementation climate within an agency, and monitoring and measuring EBP fidelity. There were 15 barriers to EBP implementation identified by first-level leaders during and directly following their participation in the LOCI implementation strategy.

At the staff level, leaders rated and ranked staff’s lack of time as the primary barrier to MI implementation. Time pressure is a universal issue in substance use disorder treatment and implementation of EBPs (Willenbring et al., 2004). Leaders advocated for providing protected time for staff’s participation in ongoing refresher trainings to mitigate other staff-level barriers, such as resistance, apprehension, lack of confidence, and perceived lack of competency. Ongoing training and support would help to ensure that both seasoned and newly hired staff members have the minimum knowledge needed to effectively and confidently implement an EBP within clinical sessions. This would potentially increase the staff’s understanding of the EBP and promote increasing their buy-in to its value. Group supervision could also serve as an opportunity to increase peer support.

Also within the inner context and mirroring staff-level barriers, leaders ranked and rated their own time as the most influential management-level barrier to successfully implementing MI. Qualitative data showed that leaders’ time was inherently interrelated with the other barriers identified, as leaders necessarily focused on problem-solving staffing issues, strategies to maintain the requirements of financial contracts, and enforcing the priorities of their executive leadership. To this end, two adaptations to the LOCI strategy may function to target leader-level barriers.

First, care must be taken to situate implementation of a given EBP in relation to other initiatives being undertaken at each agency. Doing so may require a more intentional effort to tailor LOCI activities and messages for each level of the agency throughout the entirety of the implementation time frame. Clinicians, first-level leaders, and executive leadership differed in the amount of value they each placed on EBP implementation and leaders cited a lack of support from upper management for MI initiatives. Although financial resources are generally sparse for substance use disorder treatment agencies (Beaston-Blaakman et al., 2007; French & McGeary, 1997), creative strategies can help to ease this burden (Jaramillo et al., 2019), and LOCI operates to develop new strategies through monthly ongoing contact and problem-solving with agency executives. Given that the LOCI strategy was delivered as part of a research study, it is likely that researcher-level and agency-level priorities differ. Therefore, this gap must be continually revisited to ensure that both partners maintain mutual understanding of the agency’s context and to maintain agency’s buy-in into implementation efforts.

Second, also related to financial resources, incorporating system-level priorities into implementation strategies may help align policies with agencies and clients (McCarty & Chandler, 2009). Throughout the duration of the study, changes in external policies and contractual constraints drew executive leaders’ attention away from strategic initiatives such as MI implementation, as they prioritized the system-level mandates that ensure each agency’s financial viability. Therefore, including system-level stakeholders may better align policymakers’ priorities, policies, and contractual initiatives with on-the-ground efforts to improve clinical practice (Lengnick-Hall et al., 2020).

Leaders identified specific implementation program processes associated with the research study, such as recording MI sessions, as significant barriers to MI implementation. This could be attributed to a lack of organizational readiness. In order for an intervention to be effective, certain preconditions must be in place, such as a workforce that is willing and able to dedicate time and resources to training and sustainment efforts. This may also suggest a significant advantage if an agency independently decides and initiates the implementation of a specific EBP, rather than being offered a designated EBP by an external academic entity, as is commonly practiced in research. Some of the barriers that arose in this study during the nominal group technique activity may have been influenced by this process. However, MI is the most commonly used EBP in addiction settings and there was concern from the funding agency that there is room for improvement in the quality of MI implementation and use.

This study provides insight into both persistent and emergent barriers perceived by first-level leaders to hinder successful EBP implementation. However, these findings must be considered in light of the study’s limitations. First, our sample was relatively small and differed slightly across the three portions of the modified nominal group technique, as not all leaders participated in each component. Only individuals in the LOCI condition were able to participate in the brainstorming session; however, all first-level leaders had the opportunity to rate and rank the resulting barriers and participate in qualitative interviews. Second, although the rankings and ratings were weighted equally in our calculations, they did not align in all cases, such that some items were rated as important but ranked below other factors. Third, the EBP in this study was selected by the funding agency, rather than by the agencies directly involved in the implementation process. Finally, our population comprised first-level leaders who reported on their perceptions of barriers to implementation within their treatment organization. In the future, it may be helpful to elicit the perspectives of providers and executives in a similar manner, to better understand how the perception of barriers may differ by job role and organizational level. Future research should also consider whether these barriers change over time, throughout active implementation and sustainment. Despite these limitations, we believe the study provides substantive information about the perspective of individuals within the treatment organizations and that both the modified nominal group technique methodology and the barriers identified may help inform future implementation efforts.

Conclusion

Understanding potential and emergent barriers to implementation is critical to ensure that the necessary preconditions are in place prior to EBP implementation and to adapt implementation strategies during the implementation process. The barriers to EBP implementation identified using the NGT spanned three different levels: staff-level barriers, management-level barriers, and implementation program barriers. After evaluating the perspectives of leaders nearing the completion of the LOCI strategy, considerations for future adaptations to LOCI include broadening the evaluation of organizations’ implementation climate to target inner context barriers, collaborating with system-level entities to mitigate outer context barriers, and careful attention to organizational readiness for fidelity monitoring processes. In addition to anticipating barriers for future EBP implementation efforts, the evaluation efforts described here could also be utilized during the implementation phase to allow for context-specific barriers to be identified prior to the sustainment phase.