Sage Journals: Discover world-class research

Abstract

Background

Efforts to maximize the impact of healthcare improvement interventions are hampered when intervention components are not well defined or described, precluding the ability to understand how and why interventions are expected to work.

Method

We partnered with two organizations delivering province-wide quality improvement interventions to establish how they envisaged their interventions lead to change (their underlying causal assumptions) and to identify active ingredients (behavior change techniques [BCTs]). The interventions assessed were an audit and feedback report and an academic detailing program. Both focused on supporting safer opioid prescribing in primary care in Ontario, Canada. Data collection involved semi-structured interviews with intervention developers (n = 8) and a content analysis of intervention documents. Analyses unpacked and articulated how the interventions were intended to achieve change and how this was operationalized.

Results:

Developers anticipated that the feedback report would provide physicians with a clear understanding of their own prescribing patterns in comparison to others. In the feedback report, we found an emphasis on BCTs consistent with that assumption (feedback on behavior; social comparison). The detailing was designed to provide tailored support to enable physicians to overcome barriers to change and to gradually enact specific practice changes for patients based on improved communication. In the detailing materials, we found an emphasis on instructions on how to perform the behavior, for a range of behaviors (e.g., tapering opioids, treating opioid use disorder). The materials were supplemented by detailer-enacted BCTs (e.g., social support [practical]; goal setting [behavior]; review behavioral goal[s]).

Conclusions

The interventions included a small range of BCTs addressing various clinical behaviors. This work provides a methodological example of how to apply a behavioral lens to surface the active ingredients, target clinical behaviors, and causal assumptions of existing large-scale improvement interventions that could be applied in other contexts to optimize effectiveness and facilitate scale and spread.

Plain Language Summary

What is already known about the topic?: The causal assumptions and key components of implementation interventions are often not well described, which limits the influence of implementation science on implementation practice. What does this paper add?: This work provides an approach for surfacing the causal assumptions from intervention developers (through interviews with eight participants) and active ingredients from intervention materials, focusing on two real-world interventions already delivered at scale and designed to promote safer opioid prescribing. The analysis provides a comprehensive intervention description and reveals the extent to which final interventions align with developers’ intentions. What are the implications for practice, research, or policy?: The findings provide a foundation for future work which will describe the effectiveness of these interventions (alone and in combination) and explore whether they achieve change in the intended ways, thereby providing an example of a more fulsome intervention evaluation. More broadly, our methods can be used by implementation practitioners to review and reflect on their intervention development process and support comprehensive intervention descriptions.

Keywords

intervention description causal assumptions behavior change techniques process evaluation audit and feedback academic detailing/educational outreach opioid prescribing

Introduction

There is considerable variation in the effectiveness of healthcare quality improvement (QI) interventions (Flodgren et al., 2019; Forsetlund et al., 2021; Ivers et al., 2012; O’Brien et al., 2007; Pantoja et al., 2019). Understanding the source and nature of variation is challenging when details about component parts are not comprehensively described (Powell et al., 2019; Proctor et al., 2013). This lack of detail limits the application of implementation science to implementation practice (i.e., deployment of real-world QI interventions) (Westerlund et al., 2019) and undermines efforts to optimize intervention effectiveness.

Specifying how an intervention is anticipated to lead to desired changes (i.e., the underlying causal assumptions) and fully describing intervention components can help to make implicit assumptions explicit and identify any weaknesses or inconsistencies in the causal logic (Davidoff et al., 2015; Moore et al., 2015). Such insights are central to understanding whether, where, and how to adjust interventions to improve effectiveness. Behavioral science approaches have proven valuable in achieving these aims in several key ways: (1) by systematizing the identification of intervention active ingredients and clinical behaviors targeted for change (Mc Sharry et al., 2016); (2) by providing a theoretical basis to support elucidation of barriers to and enablers of target behaviors (Patey et al., 2012); and (3) by demonstrating ways in which the mechanisms of change can be better understood (Curran et al., 2013; Steinmo et al., 2015).

In Ontario, Canada, QI organizations have implemented large-scale interventions to support safer opioid prescribing in primary care, in response to rising opioid-related morbidity and mortality (Gomes et al., 2014; Spooner et al., 2016). Amongst these is an audit and feedback report (AF) led by Ontario Health, the provincial agency designed to modernize Ontario's healthcare system to ensure that residents receive the best possible patient-centered care. Another is an academic detailing (AD) service led by the Centre for Effective Practice, an independent not-for-profit organization supporting QI in primary care. These initiatives are understood to have the greatest reach in the Province, and with existing large investments by the Province, are likely to only continue and expand. Both AF and AD are effective interventions, but it is unclear how to maximize their effectiveness (Ivers et al., 2012; O’Brien et al., 2007). In partnership with Ontario Health and the Centre for Effective Practice, our team are conducting (i) a quasi-experimental quantitative evaluation of the impact of these interventions on opioid prescribing and (ii) a mixed method process evaluation to identify active ingredients and causal assumptions, and assess fidelity, perceived impact, and underlying mechanism(s) of action. Here, we unpack the design and development of these two large-scale real-world implementation interventions. Our specific objectives were to:

Understand how implementation practitioners envisaged that their interventions would lead to desired changes in practice (i.e., their underlying causal assumptions).

Delineate how these expectations were operationalized by identifying and mapping the embedded behavior change techniques (i.e., the active ingredients) to targeted clinical behaviors.

Method

Our methods were informed by the UK Medical Research Council process evaluation guidance (Moore et al., 2015). This project was approved by the Ottawa Health Science Network Research Ethics Board at the Ottawa Hospital Research Institute (20180885-01H). The study was also formally reviewed by institutional authorities at Women's College Hospital and was deemed not to require Research Ethics Board approval. This study is reported in accordance with the COnsolidated Criteria for REporting Qualitative Research (COREQ) Checklist (Tong et al., 2007) (Supplement 1). Further Supplemental Materials include methods and results supplements.

Design

We conducted semi-structured one-to-one qualitative telephone interviews with those involved in designing the interventions and examined all associated intervention documents.

Interventions

AF: This report provides primary care physicians with information about their practice via email every 6 months. Physicians’ performance is compared to their peers and their own past performance. Indicators are reported in aggregate, i.e., the percentage of the physicians’ patients receiving specific items of care per month. Peer comparators included the percentage of patients receiving these items of care locally and across the Province. “Change ideas” are included, providing links to educational resources and practice-based tools to support QI. At the end of 2017, four opioid prescribing indicators pertaining to non-palliative care patients were added to the report. These comprised monthly percentages of patients dispensed (i) an opioid prescription; (ii) a new opioid prescription; (iii) an opioid and benzodiazepine; and (iv) at least one high-dose opioid of >90 mg morphine equivalents per day. Over 3,000 physicians received this report in 2019.

AD: This service provides physicians with up to three educational outreach visits to their own practice by a trained pharmacist, focused on safer opioid prescribing and appropriate management of chronic noncancer pain (CNCP). Each visit centers on a specific subtopic: caring for patients on opioid therapy (opioid tapering/rotation); caring for patients with CNCP (non-pharmacological therapies/pharmacological alternatives to opioids/trialing opioids); and caring for patients with opioid use disorder (screening, naloxone treatment, additional supports). For each visit, detailers have a range of evidence-based documents they can use to support their discussions, including step-by-step guides for specific actions, screening tools, patient handouts, and lists of local resources. The AD is not a case consult service (i.e., not intended to provide physicians with recommendations about individual patients in their care) but provides educational support for broader categories of patients. Five hundred physicians received visits in 2019.

Data Collection and Analysis

Interviews

The intervention developers facilitated the recruitment of key individuals involved in development by providing a list of their names and email addresses. Author NMc (PhD trained, experienced in interviewing, and who identifies as a woman) invited these individuals to take part and conducted the interviews. Verbal consent was obtained. The interviewer had met some of the interviewees previously. An interview guide was developed for each intervention covering content development, targeted clinical behaviors, and how the developers envisaged that the active ingredients in the interventions would support these behaviors. The guide was updated following a pilot interview. Interviews were audio-recorded and then transcribed verbatim by an external third party. No field notes were made.

NVivo 11 (QSR International) software was used to facilitate data analysis. Transcripts were analyzed using the codebook approach to thematic analysis (Braun & Clarke, 2020a, 2020b). This involves coding (assigning descriptive labels to meaningful units of text) (Braun & Clarke, 2006; Gale et al., 2013) and theme generation (considering how different codes may combine to form patterns of shared meaning) (Braun & Clarke, 2020b). An initial coding framework was developed, and two researchers (NMc and CL) independently applied it to the first two transcripts, refined it, and then applied it to the remaining transcripts. Coding discrepancies were resolved by discussion. Theme development involved an iterative process whereby one researcher (NMc) generated themes that were discussed regularly with the second researcher (CL) and then the wider team. Saturation was less relevant since we spoke to all who led intervention development. Transcripts were not returned to participants. Preliminary results were discussed with leaders from each organization separately to ensure the results resonated with their experience.

Document Review

Intervention documents from Ontario Health comprised a sample AF report containing mock data. Materials from the Centre for Effective Practice included any document used during an AD visit. Documents were coded by two independent coders (NMc and CL) to identify active ingredients using the Behaviour Change Techniques Taxonomy version 1 (BCTTv1) (Michie et al., 2013) and informed by current methodological literature on BCT coding (Black et al., 2019). NVivo 11 was again used to facilitate analysis. Coding involved identifying each separate instance of a BCT and specifying the corresponding behavior addressed by the BCT. Coding was achieved using definitions and examples from the published taxonomy, supplemented with a coding manual to track decision rules and help achieve coding consistency. Coding discrepancies were resolved either by discussion or the involvement of a third researcher (JP). Frequencies of BCTs and behaviors were calculated and input into https://rawgraphs.io/ to create alluvial diagrams, which support visualization of frequencies and relationships between entities.

Results

Eight interviews (23–57 min) were conducted between March and May 2019. Interviewees had differing experiences: either clinical, leadership, health informatics, project management, and/or QI. AF developers (n = 5) had a range of roles from report development to distribution and provided clinical expertise, data analytic expertise, QI expertise, or senior leadership oversight. AD developer (n = 3) roles included topic selection, developing key messages and materials, developing detailer training, providing detailer support, and strategic oversight. None were detailers themselves. One AF report and 21 documents used in academic detailing were coded. The results are presented by intervention.

AF Intervention

Causal Assumptions

AF developers envisioned that the report would enable practice change amongst physicians in five ways (Table 1). In addition, the report was explicitly designed to prevent unintended behavior change.

Table 1

Audit and Feedback—Supporting Quotes/Examples for Causal Assumptions and Active Ingredients

Causal assumptions—key themes	Quotes
Encouraging understanding of data and reflection on practice	We also included, right on those indicator pages beside the graph, some kind of reflective questions … we had hoped that by posing the right questions we would start that process of reflecting and getting curious about the data. (OH-01)
Shaping knowledge of appropriate practices and available supports	I think we also know that there's a little bit, somewhat I understand, of a knowledge gap in how do I prescribe opioids for certain situations? So, I think the other part for me is to point them to these existing resources to help I guess fill in that gap that physicians may have. (OH-02)
Supporting prioritization for specific patients	The first step is always to better understand whether or not those numbers are right and how you can figure out which patients they are … that first QI one is usually around making some sort of list of patients or understanding who it is and then fanning out from there. (OH-03)
Facilitating subsequent enactment of direct clinical behaviors	They can actually look into the EMR, understand who those people are, and start thinking about do they start tapering. (OH-02)
Supporting connections to other QI tools	Identifying where in your process or where in your clinical workflow is there an opportunity to make a change … identifying or taking advantage of either a local or provincial-based support. (OH-01)
Preventing unintended behavior change	The most important thing when physicians got this report is that they did not all of a sudden take action like cancelling that patient, i.e., getting rid of that patient as a member of their panel. They didn’t all of a sudden stop giving them the medications, that in fact this report was not intended to do that… So, that was almost more important or as important as what we wanted them to do with the report. (OH-04)

Active ingredients—BCT labels and descriptions^a	BCT examples and behaviors addressed
Feedback on behavior: Monitor and provide informative or evaluative feedback on the performance of the behavior (e.g., form, frequency, duration, intensity)	Number of my patients with a high-dose opioid >90 mg MEQ daily within the last 6 months By Me: 21. (OH report page 9) Behavior: Prescribing high-dose opioids
Social comparison: Draw attention to others’ performance to allow comparison with the person's own performance	My group and LHIN percentages are 4.2% and 4.0%, respectively. (OH report page 7) Behavior: Initiating new opioid prescriptions
Credible source: Present verbal or visual communication from a credible source in favor of or against the behavior	The expert perspective is that opioids and benzodiazepines should very rarely be prescribed together (1). (OH report page 8) Behavior: Prescribing opioids and benzodiazepines
Information about health consequences: Provide information (e.g., written, verbal, visual) about the health consequences of performing the behavior	Moderate quality evidence suggests a dose-dependent increase in risk as the prescribed dose of opioids increases. Some patients may gain important benefit at a dose of more than 90 mg MEQ daily (1). (OH report page 9) Behavior: Prescribing high-dose opioids
Instruction on how to perform the behavior: Advise or agree on how to perform the behavior	Run a search in the electronic medical record (EMR) to verify my data and to identify a list of patients being prescribed opioids by me… Where can I access EMR queries to generate a list of my patients being prescribed an opioid? AFHTO: Opioid Query. (OH report page 10) Behavior: Identifying patients prescribed opioids
Problem-solving: Analyze, or prompt the person to analyze, factors influencing the behavior and generate or select strategies that include overcoming barriers and/or increasing facilitators	Identify patients who have a clear clinical indication for tapering, and if appropriate, initiate tapering according to the recommended guidelines. Do not abruptly discontinue the use of opioids but rather seek help through one or more of the supports listed below when dealing with a challenging case (Resources: CEP Opioid Tapering Template; RXFiles Opioid tapering template). (OH report page 11) Behavior: Tapering opioids
Social support (practical): Advise on, arrange, or provide practical help (e.g., from friends, relatives, colleagues, “buddies” or staff) for performance of the behavior	Who can help me make these lists? OntarioMD Peer Leaders, EMR Practice Enhancement Consultants, and/or Practice Advisors For Family Health Teams: Quality Improvement Decision Support Specialists My office administrative staff. (OH report page 10) Behavior: Identifying patients prescribed opioids

Note. AFHTO = Association of Family Health Teams of Ontario; BCT = behavior change technique; CEP = Centre for Effective Practice; EMR = electronic medical record; LHIN = Local Health Integration Network; MEQ = morphine equivalent; OH = Ontario Health; QI = quality improvement.

Labels and definitions taken from BCT Taxonomy v1 report (Michie et al., 2013).

Encouraging Understanding of Data and Reflection on Practice. Interviewees primarily emphasized that their key intention was to support physicians to understand their practice data and to encourage reflection on this data as a foundation for behavior change. Providing physicians with their own prescribing data was considered essential to help them understand their own practice patterns over time. The report was purposefully designed to encourage reflection. For example, reflective questions were included (e.g., “How can I reflect on my opioid prescribing patterns in my practice? [page 10]").

Shaping Knowledge of Appropriate Practices and Available Supports. Participants anticipated that physicians may have knowledge gaps regarding the appropriate use of opioids and the ways in which their practice could potentially align better with evidence-based guidelines. Therefore, AF developers intended to shape physicians’ knowledge of current guidelines and the available resources, supports, and continuing professional development (CPD) opportunities for helping to optimize practice.

Supporting Prioritization for Specific Patients. The AF report provided aggregate rather than patient-level data, but participants stated that the report should support physicians to prioritize their actions by encouraging the identification of patients who might benefit most from reassessment. To facilitate prioritization, the number of patients prescribed opioids by the physicians themselves was reported separately from those prescribed for their patients by other professionals. The “change ideas” page was also designed to focus on prescribing by physicians themselves, including guidance on how to identify patients using their EMR system. This was described as a key strategy to support actionability in terms of case finding as a prerequisite for changing clinical behaviors (e.g., introducing tapering), and was thought to increase trust in the data. However, the potential for tension caused by highlighting the prescribing of others was acknowledged. The report was one of the only sources of information on this for physicians, and the amount of prescribing by others can be relatively high. This may frustrate physicians due to the potential safety issues for their patients. In Ontario, it can be difficult or impossible to identify which specific patients receive prescriptions from others, which explains the approach of prioritizing action related to a physician's own prescribing.

Facilitating Subsequent Enactment of Direct Clinical Behaviors. Some participants discussed their intent to change clinical behavior after the physician had reflected, reviewed the supports available, and prioritized their actions. The most discussed behavior was tapering opioids. Others mentioned were de-prescribing benzodiazepines for patients currently prescribed both opioids and benzodiazepines, as well as broader behaviors, such as altering pain management, improving opioid prescribing, and reducing opioid initiation.

Supporting Connections to Other QI Tools. Developers intended that physicians respond to the “change ideas” by mapping out the steps in their care process for patients with CNCP; identifying where there are opportunities for improvement; pursuing the practice supports outlined; and following up with patients to enact changes where appropriate. However, participants also acknowledged that although the report supports access to further QI opportunities, physicians routinely find it difficult to find time to reflect on their practice and pursue such activities, which would limit the changes that could be achieved with the materials provided by this intervention.

Preventing Unintended Clinical Behavior Change. Developers emphasized that the report should not encourage physicians to stop seeing patients on opioids or stop providing opioids. Participants stressed the challenges in delivering AF that encourages reduced prescribing where clinically appropriate but also discourages abrupt withdrawal which can harm patients. Since there is no appropriate target to aim for, they did not want to use comparator data as a goal, although Ontario averages were still provided. Specific features were embedded in the report to try and reduce the likelihood of unintended consequences and explicit target setting. For example, on the indicator summary page, the information on opioid prescribing was physically separated from other report indicators, and did not include the percentile rankings or color coding used to indicate priorities for practice improvement. Efforts were also made to use nonjudgemental language to discourage stopping of all opioids and target setting, to be supportive of opioid prescribing where clinically appropriate, and to encourage reflection and accessing supports before making changes.

Active Ingredients

Table 1 defines each BCT identified in the AF report and provides examples. Figure 1 displays the frequency of coding of each BCT and the frequency of each corresponding behavior addressed. We coded seven BCTs (problem-solving; credible source; information about health consequences; feedback on behavior; social comparison; instruction on how to perform the behavior; and social support [practical]). These BCTs addressed six clinical behaviors: tapering opioids; prescribing opioids and benzodiazepines; prescribing high-dose opioids; prescribing opioids; initiating new opioid prescriptions; and identifying patients prescribed opioids.

Figure 1

BCTs and Corresponding Behaviors Coded in the Audit and Feedback Report

Social comparison and feedback on behavior were the most often-coded BCTs (34 and 20 instances, respectively). Prescribing of high-dose opioids and prescribing of opioids and benzodiazepines were the most often-addressed behaviors (17 instances each); these were addressed mainly by social comparison and feedback on behavior but were also reinforced with information about health consequences and a credible source. Problem-solving, social support (practical), and instruction on how to perform the behavior were used once and addressed the behaviors “identifying patients prescribed opioids” and “tapering opioids.”

Some report content did not contain any BCTs but focused more broadly on shaping knowledge and was coded separately by creating and assigning “non-BCT” codes. There were eight instances of this coding, focused on shaping the knowledge of CPD programs; guidelines; provincial/local resources, programs, or supports; opioid treatment agreements; and provincial information for patients.

Intent Versus Operationalization. Six of the seven BCTs coded could also be identified from the interviews. Participants did not discuss problem-solving, which was coded once. No new BCTs were identified in the interviews which were not already identified in the report.

AD Intervention

Causal Assumptions

AD developers intended that the detailing would enable practice change amongst physicians in five ways (Table 2). In addition, detailers were explicitly supported to reframe patient-specific questions to focus on broader examples.

Table 2

Academic Detailing—Supporting Quotes/Examples for Causal Assumptions and Active Ingredients

Causal assumptions—key themes	Quotes
Providing a set structure whilst encouraging tailoring of content	A normal academic detailing visit should always start with touching base on how things have gone since the last time you were there … they don’t always have to cover every single key message … it's really about focusing in on the key messages that are most relevant to that provider. Whether that's because they said that's the scenario that they need or whether through your informal needs assessment you kind of pick out that maybe that's something that they would find helpful. Or they’d mentioned, this type of patient is a patient that I’m really struggling with right now, so you’re focusing on the information that's relevant to that. (CEP-03)
Moving beyond education and motivation to enable immediate clinical action	When a physician is walking through how they might implement one of these best practices, you have all the tools in your back pocket to help them do that, whether that be a knowledge gap, a skill gap, a comfort gap, a conversation gap, actual services in your area gap, any of those things, the detailer knows how to focus in on that specific need. (CEP-03)
Encouraging gradual change based on improved communication	One of the most useful things I think that we’ve heard as we were preparing it, as we were detailing, was the value of just having some of those little scripts, ideas as to how some of their colleagues worded this particular message they wanted to get across. So, I think those talking points for this topic especially were very important and I had a lot of comments from the detailers of certain clinicians that really valued those … because that's kind of where the rubber hits the road as to whether they’re going to have to accept again moving a patient towards tapering or not. (CEP-02)
Shaping knowledge of appropriate practices and available supports	And it can also help for the provider to have something that they can look back to later on when they say, “oh, yeah, which drug was that again that I might consider using if the person is really going through sweats and everything?”. (CEP-02)
Aligning with other QI tools	Definitely there is alignment. One of the reasons why we set up the Partners Table, to ensure that's actually happening, and we’re not providing conflicting messages to the physicians… We are really trying to make sure that we align as much as possible. (CEP-01)
Reframing case consult requests to focus on broader examples	You asked what it should be, what it should not be is case consult. We’ve been quite clear about that, that this is not a clinical case consultation service. We know and expect that the physician will say, well, I have a patient, XYZ, but the detailer should be able to say, to provide information that the provider can then apply across similar patients. Now having said that, though, because we have two models, one is pharmacist detailers who are out in the community … and then FHT pharmacist detailers … we expect, and anecdotally know that there will be a little bit of case consultation because that's just the nature of the way in which their relationship works. (CEP-01)

Active ingredients—BCT labels and descriptions^a	BCT examples and behaviors addressed
Instruction on how to perform the behavior: Advise or agree on how to perform the behavior	Example of buprenorphine-naloxone tapering protocol. (CEP Opioid Use Disorder Tool) Behavior: Providing OAT for OUD
Information about health consequences: Provide information (e.g., written, verbal, visual) about the health consequences of performing the behavior	Remember overdose risk is significant even at low doses. (CEP Opioid Manager Tool) Behavior: Prescribing opioids
Adding objects to the environment: Add objects to the environment in order to facilitate the performance of the behavior	Clinical Opiate Withdrawal Scale (COWS) (whole document) Behavior: Providing OAT for OUD
Problem-solving: Analyse, or prompt the person to analyse, factors influencing the behavior and generate or select strategies that include overcoming barriers and/or increasing facilitators	Identifies aspects of situations which can pose challenges to conducting conversations about tapering, and outlines solutions for mitigating these challenges (CEP talking points for visit 1) Behavior: Tapering opioids
Credible source: Present verbal or visual communication from a credible source in favor of or against the behavior	WATCHFUL DOSE: Guidelines recommend reassessing the benefit/risk of doses ≥50 MME/day and to “avoid or justify increasing dosage” at doses ≥90 MME/day.2,21. (CEP management of chronic noncancer pain tool) Behavior: Prescribing opioids
Social support (unspecified): Advise on, arrange, or provide social support (e.g., from friends, relatives, colleagues, buddies, or staff) or non-contingent praise or reward for performance of the behavior. It includes encouragement and counseling, but only when it is directed at the behavior	Encourages providers to seek support from colleagues in specific situations (CEP talking points for visit 1) Behavior: Tapering opioids
Behavior substitution: Prompt substitution of the unwanted behavior with a wanted or neutral behavior	When considering therapy for patients with chronic noncancer pain, optimize non-opioid pharmacotherapy and nonpharmacological therapy, rather than initiating a trial of opioids. (CEP opioid manager tool) Behavior: Providing opioids
Salience of consequences: Use methods specifically designed to emphasize the consequences of performing the behavior with the aim of making them more memorable (goes beyond informing about consequences)	Overdose risk box visually stands out and includes table highlighting proportions of fatal overdoses associated with opioids (CEP opioid manager tool) Behavior: Prescribing opioids
Social support (practical)^b: Advise on, arrange, or provide practical help (e.g., from friends, relatives, colleagues, “buddies” or staff) for the performance of the behavior	If it's something they’ve thought about, but they don’t really know how to do, it's getting them aware that there's the RAAM clinics and how to refer to those… So, it might be helping to connect them with the RAAM. (CEP-03)
Pros and cons^b: Advise the person to identify and compare reasons for wanting (pros) and not wanting to (cons) change the behavior (includes “decisional balance”)	To discuss the pros, cons, advantages, disadvantages of something along with the provider, and to think through whether there's a potential way to reinforce behaviour or change something to incorporate a little better approach. (CEP-02)
Behavioral practice/rehearsal^b: Prompt practice or rehearsal of the performance of the behavior one or more times in a context or at a time when the performance may not be necessary, in order to increase habit and skill	I did have a couple of detailers who did end up role playing a couple of things … it certainly means that the provider has bought into trying this out to see if they can make it work, and not just having it as a theory, but actually practicing their skill. (CEP-02)
Goal setting (behavior)^b: Set or agree on a goal defined in terms of the behavior to be achieved	Remembering from the last visit, what were the key things that provider said they were going to work on or implement … checking back on how the key behaviour change things that they committed to at their last visit are going … and trying to have the provider commit to the realistic behaviour change that they’ll be able to do after, so you can capture that on a revisit plan at the next visit. (CEP-03)
Commitment^b: Ask the person to affirm or reaffirm statements indicating a commitment to change the behavior
Review behavior goal(s)^b: Review behavior goal(s) jointly with the person and consider modifying goal(s) or behavior change strategy in light of achievement. This may lead to re-setting the same goal, a small change in that goal or setting a new goal instead of (or in addition to) the first, or no change

Note. BCT = behavior change technique; CEP = Centre for Effective Practice; MME = morphine milligram equivalents; OAT = opioid agonist therapy; OUD = opioid use disorder; QI = quality improvement; RAAM = Rapid Access to Addictions Medicine.

Labels and definitions taken from BCT Taxonomy v1 report (Michie et al., 2013).

Additional BCT identified from interviews: specific behaviors addressed not defined.

Providing a Set Structure While Encouraging Tailoring of Content. Participants noted that detailers should follow a set structure. The first visit was to focus on introductions and rapport building, followed by a content introduction and a needs assessment. The conversation should align with pre-set key messages and be interactive and responsive to physicians’ concerns, whilst providing a mix of education and support and incorporating the materials provided. The detailer closes the session by summarizing the discussion and supporting the physician to reflect on how they will change. Subsequent visits should follow the same structure, building on previous conversations and asking about discussed changes.

Whilst this structure guides the visit, developers emphasized there is no “typical visit” because the content should be tailored. Detailers were trained to be flexible and to tailor their support to the specific needs of each physician rather than trying to cover all material. Each conversation should focus on areas where physicians may be facing challenges, as identified by the needs assessment. The materials provided should support the conversation, and although the main tool relevant for each visit is typically given, any relevant supplemental material can be provided. This tailoring was expected to ensure that value is provided to each physician to help them optimize their practice.

Moving Beyond Education and Motivation to Enable Immediate Clinical Action. Developers recognized that whilst physicians may be motivated to change, implementing change is difficult and often requires access to additional support. Detailing was explicitly designed to serve as a support for moving beyond education and motivation to enable immediate action in physicians’ day-to-day practice. Strategies to encourage behavior change were explicitly embedded, and detailers were trained to support physicians to overcome a range of different barriers to change. Participants noted that the detailer should center the session around a potential action that could significantly improve patient care.

Encouraging Gradual Change Based on Improved Communication. All participants discussed how detailing should encourage gradual change over time, helping the physician to build on a foundation of improved communication with patients. A lot of detailer time was to be spent on supporting physicians to improve how they approach discussions around various issues related to opioids and CNCP. To facilitate this, the “talking points” (which outlined specific phrasing that physicians could use) formed a key part of the detailing materials. Based on conversations with physicians, a new communicative tool was developed to support the discussion of a multimodal approach to treat and manage CNCP, to help patients understand the range of supports needed other than pharmacological treatments.

Shaping Knowledge of Appropriate Practices and Available Supports. Participants noted that the detailing was also designed to target known knowledge gaps in this area amongst physicians. Detailers should summarize the evidence base where needed to support a specific recommendation. The detailing sessions should also serve to increase knowledge about the different non-pharmacological and alternative pharmacological treatment options available, how they work to help people with CNCP, and how to adjust opioid doses safely, with the detailers leaving tools and resources behind to help solidify knowledge (e.g., information about which drugs to use for specific symptoms or situations). Participants felt that physicians are often unaware of resources in their local area, such as cognitive behavioral therapy services or Rapid Access Addiction Medicine Clinics and their referral processes, so detailing also focused on increasing this knowledge. Finally, the sessions could also provide information to increase knowledge about relevant CPD courses and other educational/QI opportunities.

Aligning With Other QI Tools. Participants discussed how detailing was designed to align with other QI supports and resources, to help ensure that physicians did not receive mixed messages. It was noted that detailers should highlight those other supports where relevant and could potentially serve as a “navigator” to help guide physicians to the resources most relevant for their specific issue and to emphasize the value of those resources.

Reframing Case Consult Requests to Focus on Broader Examples. All participants emphasized that the academic detailing was not designed to be a case consult service. Participants had anticipated that physicians may expect this type of support, and so the detailer training covered how to reframe the conversation should this occur. Such reframing should involve discussion of a more general category of patients, whereby the detailer discusses potential treatment or management options for patients in similar situations.

Active Ingredients

Table 2 defines each BCT identified and provides examples. Figure 2 displays the frequency of coding of each BCT and the frequency of each corresponding behavior addressed. We identified numerous behaviors: to provide a high-level overview, we grouped these behaviors to create higher-order behaviors. For example, the higher-order behavior “tapering opioids” is a grouping of more specific behaviors such as assessing readiness for tapering, discussing tapering, enacting the tapering, documenting tapering, and managing withdrawal symptoms.

Figure 2

BCTs and Corresponding Behaviors Coded in the Academic Detailing Documents

We coded eight BCTs (problem-solving; adding objects to the environment; social support (unspecified); instruction on how to perform the behavior; information about health consequences; salience of consequences; behavior substitution; and credible source). These BCTs addressed seven higher-order behaviors (providing non-opioid treatments; providing opioids; switching opioids; tapering opioids; identifying problems with opioid use; providing opioid agonist treatment for opioid use disorder; and providing ongoing support, assessment, and review). Instruction on how to perform the behavior was the most often-coded BCT (112 instances). Tapering opioids was the most often-addressed behavior (51 instances). Instruction on how to perform the behavior and adding objects to the environment mapped to all behaviors. As with the AF, some AD documents did not contain any BCTs but focused more broadly on shaping knowledge about different aspects of CNCP care and the resources available.

Intent Versus Operationalization. Seven of the eight BCTs coded could also be identified in the interviews. An additional six BCTs were identified from the interviews. These were delivered by the detailers and therefore not identifiable from the materials alone. Participants noted that the detailers were equipped to help connect physicians to local services (social support [practical]); support physicians to understand the pros, cons, advantages, and disadvantages of adopting a specific practice (pros and cons); role-play patient discussions using the scripts provided in the materials (behavioral practice or rehearsal); and help physicians set practice change goals, commit to those goals, and review progress toward meeting those goals (goal setting [behavior]; commitment; review behavioral goal[s]).

Discussion

We surfaced causal assumptions and identified active ingredients of two interventions implemented at scale to improve opioid prescribing. AF developers intended for the report to facilitate change by encouraging understanding and reflection on data; shaping knowledge; and supporting prioritization, subsequent enactment of clinical practice changes, and connections to other QI tools, all whilst preventing unintended changes. Key active ingredients were feedback on prescribing and comparison with others. The AD was designed to enable change by encouraging tailoring of content to meet individual needs; moving beyond education and motivation to enable action; encouraging gradual change based on improved patient communication; shaping knowledge; and aligning with other QI tools, with the detailers also expected to reframe case consult requests to focus on broader examples. Detailers were equipped to provide social support, encourage rehearsal, and support goal setting and review. Active ingredients in the AD materials included over 100 instances of providing instructions on how to enact behaviors such as providing opioids safely, tapering opioids, and providing non-opioid treatments. By considering the causal assumptions, intervention content, and the wider literature on features of effective interventions and pathways of change, we can draw out some conclusions regarding the follow-through from intent to operationalization in intervention design and some implications about the extent of change that these interventions may achieve.

AF Intent, Operationalization, and Implications for Change

Clinical Performance Feedback Intervention Theory (CP-FIT) posits that understanding data is a foundational component of the processes through which AF leads to change (Brown et al., 2019), aligning well with the intended focus of the AF report and the two most frequently used BCTs (feedback on behavior and social comparison). Whilst AF developers should carefully consider the number of comparators (Brehaut et al., 2016; Gude et al., 2019), including multiple comparators, as in this report, can help feedback recipients assess whether there is a true discrepancy between their practice and that of others and mitigate questions about comparison fairness, ultimately encouraging acceptance of the data (Gude et al., 2019). AF developers reflected evidence in the desire to support physicians to carry out actions in response to the feedback, starting with how to identify specific patients (Brown et al., 2019). This was borne out in the identification of BCTs instruction on how to perform the behavior and social support (practical), albeit these were identified only once, with much more frequent use of BCTs targeting understanding of prescribing in comparison with colleagues.

Coherence between intent and operationalization was less clear for other aspects, which may affect the desired impact of the AF. After receipt of feedback, clinicians often need more comprehensive support to change behaviors (Desveaux et al., 2021). Whilst developers discussed their intent to encourage opioid tapering, only one instance of an associated BCT (problem-solving) was identified. Given the myriad challenges reported by family physicians in initiating tapering (Desveaux et al., 2019; Kennedy et al., 2018), more support is likely needed. Whilst the report did include links to other QI tools and supports to assist physicians with identifying individual patients, an additional step is needed for physicians to access those tools. The extent to which physicians take these steps and initiate changes for specific patients, as well as any challenges experienced, will be explored in further components of the process evaluation. Should barriers remain, the AF literature can be drawn upon to inform future iterations of the AF. For example, to increase actionability and reduce the need to seek additional support, there may be opportunities to embed solutions to common practice problems directly within the report, such as examples of effective actions that can be taken with patients (Desveaux et al., 2021; Willis et al., 2022). Finally, the embedded features aiming to reduce the likelihood of target setting, motivated by the important aim of preventing unintended changes, may temper the impact of the AF on behavior change since setting a goal for improvement can enhance AF effectiveness (Ivers et al., 2012). Preventing unintended consequences is important: physicians have concerns about harms from inappropriate opioid discontinuation (Langford et al., 2021), and QI interventions can lead to decreased follow-up of patients who discontinue opioids (Husain et al., 2019). Future work should focus on understanding where to direct emphasis, how, and in what proportion relative to other content.

AD Intent, Operationalization, and Implications for Change

A fundamental principle of AD is that it be tailored (Soumerai & Avorn, 1990; Yeh et al., 2016). AD should be action-oriented, including specific recommendations about change with a strong emphasis on detailer training on giving solutions to help recipients enact those recommendations (Kennedy et al., 2021; Yeh et al., 2016). Alignment with these principles is evident in our findings. The AD service included a range of BCTs across many different clinical behaviors that, if delivered with high fidelity by the detailers, may increase the likelihood of change. Developers emphasized that their detailers were trained to help physicians set goals pertaining to specific actions they could take to improve care and that the first step in a follow-up visit should involve reviewing progress toward those goals.

Instruction on how to perform behaviors was identified considerably more often than other BCTs. Whilst greater knowledge is associated with improved opioid prescribing (Gray et al., 2021), there are many other barriers to change, including the heavy emotional toll caused by the anticipation of conflict with patients, the belief that following guidelines alone risks destabilizing patients which can lead to the use of illicit drugs, as well as a perceived threat to the therapeutic relationship caused by initiating conversations about tapering (Desveaux et al., 2019). The large number of possible behaviors to address in the AD sessions may make it challenging to detect a quantifiable change in target outcomes at a service level since the focus of each AD visit varies across physicians. Carefully considering the number and complexity of clinical behaviors to be targeted is both challenging and important, as it can be difficult to interpret the results of AD targeting numerous complex behaviors simultaneously (O’Brien et al., 2007). Additionally, the AD was designed to support gradual change over time which may reduce the likelihood of identifying behavior changes made because of the AD. Finally, developers emphasized that the detailing was not designed to be a case consult service; detailers were encouraged to steer physicians toward more general examples. A key recommendation for AD is that challenging cases are discussed and feasible solutions provided (Yeh et al., 2016). The extent to which the general examples and provided solutions align with the cases brought forward by the physicians may impact the extent of behavior change. Future phases of this work will reveal the extent to which physicians were able to apply the recommendations to make changes in care provision for specific patients. Should challenges remain, the AD literature can be drawn upon to inform future iterations (for example, reassessing the ease with which recommendations can be actioned in daily practice) (Luetsch et al., 2023).

Opening the “Black Box” of Implementation Intervention Design

There is an established need to examine the details within the “black box” of improvement interventions (Caton et al., 2021). Previous contributions have focused on establishing more fulsome descriptions of interventions and their pathways of change (Rodrigues et al., 2022; Vachon et al., 2013). Our work serves as a unique example of how a behavioral lens, focused on surfacing developers’ implicit theories of change and identifying embedded BCTs, can help open the “black box” of implementation intervention design by revealing the extent to which intervention content aligned with developers’ intentions. Such work forms the basis for subsequent rigorous evaluation focusing on the extent to which change is achieved in the ways intended. On the whole, we found that the causal assumptions were well reflected in the BCTs and target behaviors identified. Areas where developers’ intent and active ingredients and target behaviors were less aligned emphasize the challenges in maintaining intervention fidelity throughout the development process.

This work provides a methodological example of applying a behavioral lens to surface the active ingredients, target clinical behaviors, and causal assumptions of QI interventions. For others to apply these methods, we recommend some immersion in the literature on behavior change-informed implementation science approaches (Curran et al., 2013; French et al., 2012; Lorencatto et al., 2016; Mc Sharry et al., 2016; Patey et al., 2023; Presseau et al., 2019, 2022; Rodrigues et al., 2022; Steinmo et al., 2015) and completion of training in BCTTv1 use. Whilst QI practitioners may not have the time or resources to follow our methods exactly, there may be ways of adapting this approach. For example, group discussions with intervention developers could help identify causal assumptions, and identification of BCTs and associated target behaviors could be completed as a group coding exercise. Resources available to support these activities include the BCTTv1 taxonomy (Michie et al., 2013), our interview guides and coding manual (Supplemental Materials), the AACTT Framework to guide behavior specification (Presseau et al., 2019), and literature on intervention logic models (Davidoff et al., 2015). These activities can also be conducted as initiatives are being developed, with the logic model being revisited throughout the development process to maintain a clear record of causal assumptions and coherence with intervention components.

This work was made possible through a trusting partnership with intervention developers who allowed access to their materials and described their process and rationale to us in detail. In return, the researchers discussed the evolving findings with the developers at numerous stages to support them in reflecting on their own processes and in considering ideas for intervention optimization (which will be elaborated after reflecting on all project components, including engagement with Lived Experience Advisors [Nicolas-Angl et al., 2023]). We hope this serves as an example of the value of investing in such partnered work in that it can contribute to a more fulsome intervention description as the basis for supporting understanding of how and why interventions work (or don’t), which can be achieved using process evaluation methods (Moore et al., 2015) to investigate mechanisms of change (e.g., surveying recipients to establish whether an intervention designed to operate via knowledge actually increases knowledge).

Strengths and Limitations

Strengths include partnership working with organizations to help understand how real-world QI interventions are developed and designed and application of existing frameworks to build on cumulative knowledge concerning intervention development and evaluation. Authors of a recent systematic review of interventions to influence opioid prescribing for CNCP highlighted that no included studies specified and evaluated mechanisms of change (Asamoah-Boaheng et al., 2021). This paper describes the first step in our program of work aiming to achieve this, by firstly identifying causal assumptions, BCTs included, and behaviors addressed.

The interviews were conducted after the interventions were launched, so we may have missed some reflections that occurred during intervention development. There were also some challenges with using the BCTTv1 to retrospectively code intervention content. We attempted to adhere strictly to coding recommendations and only code clear instances of use in accordance with BCT descriptions. Coding disagreements required us to develop some additional coding guidance. Whilst most previous work has focused on identifying the BCTs included in an intervention, we attempted to identify each instance of the use of each BCT, which often proved challenging. Linking each instance of use with a specific behavior helped alleviate these challenges; however, we acknowledge that methods for distinguishing between instances of the use of a BCT and for conceptualizing dose in behavior change interventions are still in their infancy.

We did not consider dimensions of equity in our approach for surfacing causal assumptions and active ingredients. Known disparities in the provision of care for individuals from equity-denied groups experiencing CNCP should be important targets for QI interventions. For example, non-White patients are less likely to be prescribed opioids than White patients (De Sola et al., 2020): consistent undertreatment of Black patients’ pain (Meghani et al., 2012) likely underlies this. Future work focusing on a more fulsome description of QI interventions could explore whether and how equity factors were considered during intervention design: for example, by embedding questions on this for intervention developers into the interview guide. This would support either subsequent assessment of the impact of the embedded equity considerations or further discussion of how equity considerations may be incorporated in future iterations of the intervention. Equity factors can also be explicitly considered at the intervention design stage. Any active ingredients included to target equity factors should be clearly specified, along with a delineation of how this content is expected to achieve impact.

Conclusion

The content of and causal assumptions underlying implementation interventions are not often well-specified, which hampers the interpretation of intervention effects in individual studies and evidence syntheses. This work provides an example of methods for applying a behavioral lens to surface the active ingredients, target clinical behaviors, and causal assumptions of existing large-scale QI initiatives. Subsequent reports from this program of work will describe the effectiveness of these interventions alone and in combination, intervention fidelity, the implementation process, and mechanisms of impact. The results will support efforts to optimize the interventions, facilitated by existing partnerships.

Supplemental Material

sj-docx-1-irp-10.1177_26334895231206569 - Supplemental material for Surfacing the causal assumptions and active ingredients of healthcare quality improvement interventions: An application to primary care opioid prescribing

Supplemental material, sj-docx-1-irp-10.1177_26334895231206569 for Surfacing the causal assumptions and active ingredients of healthcare quality improvement interventions: An application to primary care opioid prescribing by Nicola McCleary, Celia Laur, Justin Presseau, Gail Dobell, Jonathan M.C. Lam, Sharon Gushue, Katie Hagel, Lindsay Bevan, Lena Salach, Laura Desveaux and Noah M. Ivers in Implementation Research and Practice

Supplemental Material

sj-pdf-2-irp-10.1177_26334895231206569 - Supplemental material for Surfacing the causal assumptions and active ingredients of healthcare quality improvement interventions: An application to primary care opioid prescribing

Supplemental material, sj-pdf-2-irp-10.1177_26334895231206569 for Surfacing the causal assumptions and active ingredients of healthcare quality improvement interventions: An application to primary care opioid prescribing by Nicola McCleary, Celia Laur, Justin Presseau, Gail Dobell, Jonathan M.C. Lam, Sharon Gushue, Katie Hagel, Lindsay Bevan, Lena Salach, Laura Desveaux and Noah M. Ivers in Implementation Research and Practice

Supplemental Material

sj-pdf-3-irp-10.1177_26334895231206569 - Supplemental material for Surfacing the causal assumptions and active ingredients of healthcare quality improvement interventions: An application to primary care opioid prescribing

Supplemental material, sj-pdf-3-irp-10.1177_26334895231206569 for Surfacing the causal assumptions and active ingredients of healthcare quality improvement interventions: An application to primary care opioid prescribing by Nicola McCleary, Celia Laur, Justin Presseau, Gail Dobell, Jonathan M.C. Lam, Sharon Gushue, Katie Hagel, Lindsay Bevan, Lena Salach, Laura Desveaux and Noah M. Ivers in Implementation Research and Practice

Supplemental Material

sj-pdf-4-irp-10.1177_26334895231206569 - Supplemental material for Surfacing the causal assumptions and active ingredients of healthcare quality improvement interventions: An application to primary care opioid prescribing

Supplemental material, sj-pdf-4-irp-10.1177_26334895231206569 for Surfacing the causal assumptions and active ingredients of healthcare quality improvement interventions: An application to primary care opioid prescribing by Nicola McCleary, Celia Laur, Justin Presseau, Gail Dobell, Jonathan M.C. Lam, Sharon Gushue, Katie Hagel, Lindsay Bevan, Lena Salach, Laura Desveaux and Noah M. Ivers in Implementation Research and Practice

Supplemental Material

sj-docx-5-irp-10.1177_26334895231206569 - Supplemental material for Surfacing the causal assumptions and active ingredients of healthcare quality improvement interventions: An application to primary care opioid prescribing

Supplemental material, sj-docx-5-irp-10.1177_26334895231206569 for Surfacing the causal assumptions and active ingredients of healthcare quality improvement interventions: An application to primary care opioid prescribing by Nicola McCleary, Celia Laur, Justin Presseau, Gail Dobell, Jonathan M.C. Lam, Sharon Gushue, Katie Hagel, Lindsay Bevan, Lena Salach, Laura Desveaux and Noah M. Ivers in Implementation Research and Practice

Supplemental Material

sj-docx-6-irp-10.1177_26334895231206569 - Supplemental material for Surfacing the causal assumptions and active ingredients of healthcare quality improvement interventions: An application to primary care opioid prescribing

Supplemental material, sj-docx-6-irp-10.1177_26334895231206569 for Surfacing the causal assumptions and active ingredients of healthcare quality improvement interventions: An application to primary care opioid prescribing by Nicola McCleary, Celia Laur, Justin Presseau, Gail Dobell, Jonathan M.C. Lam, Sharon Gushue, Katie Hagel, Lindsay Bevan, Lena Salach, Laura Desveaux and Noah M. Ivers in Implementation Research and Practice

Supplemental Material

sj-docx-7-irp-10.1177_26334895231206569 - Supplemental material for Surfacing the causal assumptions and active ingredients of healthcare quality improvement interventions: An application to primary care opioid prescribing

Supplemental material, sj-docx-7-irp-10.1177_26334895231206569 for Surfacing the causal assumptions and active ingredients of healthcare quality improvement interventions: An application to primary care opioid prescribing by Nicola McCleary, Celia Laur, Justin Presseau, Gail Dobell, Jonathan M.C. Lam, Sharon Gushue, Katie Hagel, Lindsay Bevan, Lena Salach, Laura Desveaux and Noah M. Ivers in Implementation Research and Practice

Supplemental Material

sj-docx-8-irp-10.1177_26334895231206569 - Supplemental material for Surfacing the causal assumptions and active ingredients of healthcare quality improvement interventions: An application to primary care opioid prescribing

Supplemental material, sj-docx-8-irp-10.1177_26334895231206569 for Surfacing the causal assumptions and active ingredients of healthcare quality improvement interventions: An application to primary care opioid prescribing by Nicola McCleary, Celia Laur, Justin Presseau, Gail Dobell, Jonathan M.C. Lam, Sharon Gushue, Katie Hagel, Lindsay Bevan, Lena Salach, Laura Desveaux and Noah M. Ivers in Implementation Research and Practice

Supplemental Material

sj-docx-9-irp-10.1177_26334895231206569 - Supplemental material for Surfacing the causal assumptions and active ingredients of healthcare quality improvement interventions: An application to primary care opioid prescribing

Supplemental material, sj-docx-9-irp-10.1177_26334895231206569 for Surfacing the causal assumptions and active ingredients of healthcare quality improvement interventions: An application to primary care opioid prescribing by Nicola McCleary, Celia Laur, Justin Presseau, Gail Dobell, Jonathan M.C. Lam, Sharon Gushue, Katie Hagel, Lindsay Bevan, Lena Salach, Laura Desveaux and Noah M. Ivers in Implementation Research and Practice

Footnotes

Acknowledgements

We would like to thank all of the participants for taking part in this research. We would also like to thank Zack van Allen (Ottawa Hospital Research Institute) for his advice on data visualization.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by a Canadian Institutes of Health Research—Strategy for Patient-Oriented Research Innovative Clinical Trial Multi-Year Grant. The funding source had no role in the study design, collection, analysis, or interpretation of the data, writing the manuscript, or the decision to submit the paper for publication. NMI is supported by a Canada Research Chair (Tier 2) in the Implementation of Evidence Based Practice, and a Clinician Scholar Award from the University of Toronto Department of Family and Community Medicine.

ORCID iD

Nicola McCleary

Supplemental Material

Supplemental material for this article is available online.

References

Asamoah-Boaheng

Badejo

O. A.

Bell

L. V.

Buckley

Busse

J. W.

Campbell

T. S.

Corace

Cooper

Flusk

Garcia

D. A.

Hossain

M. A.

Iorio

Lavoie

K. L.

Poulin

P. A.

Skidmore

Rash

J. A.

(2021). Interventions to influence opioid prescribing practices for chronic noncancer pain: A systematic review and meta-analysis. American Journal of Preventive Medicine, 60(1), e15–e26. https://doi.org/10.1016/j.amepre.2020.07.012

Black

Williams

A. J.

Javornik

Scott

Johnston

Eisma

M. C.

Michie

Hartmann-Boyce

West

Viechtbauer

de Bruin

(2019). Enhancing behavior change technique coding methods: Identifying behavioral targets and delivery styles in smoking cessation trials. Annals of Behavioral Medicine, 53(6), 583–591. https://doi.org/10.1093/abm/kay068

Braun

Clarke

(2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Braun

Clarke

(2020a). Can I use TA? Should I use TA? Should I not use TA? Comparing reflexive thematic analysis and other pattern-based qualitative analytic approaches. Counselling and Psychotherapy Research, 21(1), 37–47. https://doi.org/10.1002/capr.12360

Braun

Clarke

(2020b). One size fits all? What counts as quality practice in (reflexive) thematic analysis? Qualitative Research in Psychology, 18(3), 328–352. https://doi.org/10.1080/14780887.2020.1769238

Brehaut

J. C.

Colquhoun

H. L.

Eva

K. W.

Carroll

Sales

Michie

Ivers

Grimshaw

J. M.

(2016). Practice feedback interventions: 15 suggestions for optimizing effectiveness. Annals of Internal Medicine, 164(6), 435–441. https://doi.org/10.7326/M15-2248

Brown

Gude

W. T.

Blakeman

van der Veer

S. N.

Ivers

Francis

J. J.

Lorencatto

Presseau

Peek

Daker-White

(2019). Clinical Performance Feedback Intervention Theory (CP-FIT): A new theory for designing, implementing, and evaluating feedback in health care based on a systematic review and meta-synthesis of qualitative research. Implementation Science, 14(1), 40. https://doi.org/10.1186/s13012-019-0883-5

Caton

Shen

Miele

G. M.

Darfler

Sandoval

J. R.

Urada

McGovern

M. P.

(2021). Opening the “black box”: Four common implementation strategies for expanding the use of medications for opioid use disorder in primary care. Implementation Research and Practice, 2, 26334895211005810. https://doi.org/10.1177/26334895211005809

Curran

J. A.

Brehaut

Patey

A. M.

Osmond

Stiell

Grimshaw

J. M.

(2013). Understanding the Canadian adult CT head rule trial: Use of the theoretical domains framework for process evaluation. Implementation Science, 8(1), 25. https://doi.org/10.1186/1748-5908-8-25

10.

Davidoff

Dixon-Woods

Leviton

Michie

(2015). Demystifying theory and its use in improvement. BMJ Quality & Safety, 24(3), 228–238. https://doi.org/10.1136/bmjqs-2014-003627

11.

De Sola

Dueñas

Salazar

Ortega-Jiménez

Failde

(2020). Prevalence of therapeutic use of opioids in chronic non-cancer pain patients and associated factors: A systematic review and meta-analysis. Frontiers in Pharmacology, 11, 564412. https://www.frontiersin.org/articles/10.3389/fphar.2020.564412. https://doi.org/10.3389/fphar.2020.564412

12.

Desveaux

Ivers

N. M.

Devotta

Ramji

Weyman

Kiran

(2021). Unpacking the intention to action gap: A qualitative study understanding how physicians engage with audit and feedback. Implementation Science, 16(1), 19. https://doi.org/10.1186/s13012-021-01088-1

13.

Desveaux

Saragosa

Kithulegoda

Ivers

N. M.

(2019). Understanding the behavioural determinants of opioid prescribing among family physicians: A qualitative study. BMC Family Practice, 20(1), 59. https://doi.org/10.1186/s12875-019-0947-2

14.

Flodgren

O’Brien

M. A.

Parmelli

Grimshaw

J. M.

(2019). Local opinion leaders: Effects on professional practice and healthcare outcomes. The Cochrane Database of Systematic Reviews, 2019(6), CD000125. https://doi.org/10.1002/14651858.CD000125.pub5

15.

Forsetlund

O’Brien

M. A.

Forsén

Mwai

Reinar

L. M.

Okwen

M. P.

Horsley

Rose

C. J.

(2021). Continuing education meetings and workshops: Effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews, 2021(9), CD003030. https://doi.org/10.1002/14651858.CD003030.pub3

16.

French

S. D.

Green

S. E.

O’Connor

D. A.

McKenzie

J. E.

Francis

J. J.

Michie

Buchbinder

Schattner

Spike

Grimshaw

J. M.

(2012). Developing theory-informed behaviour change interventions to implement evidence into practice: A systematic approach using the Theoretical Domains Framework. Implementation Science, 7, 38. https://doi.org/10.1186/1748-5908-7-38

17.

Gale

N. K.

Heath

Cameron

Rashid

Redwood

(2013). Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Medical Research Methodology, 13(1), 117. https://doi.org/10.1186/1471-2288-13-117

18.

Gomes

Mamdani

M. M.

Dhalla

I. A.

Cornish

Paterson

J. M.

Juurlink

D. N.

(2014). The burden of premature opioid-related mortality. Addiction, 109(9), 1482–1488. https://doi.org/10.1111/add.12598

19.

Gray

B. M.

Vandergrift

J. L.

Weng

Lipner

R. S.

Barnett

M. L.

(2021). Clinical knowledge and trends in physicians’ prescribing of opioids for new onset back pain, 2009-2017. JAMA Network Open, 4(7), e2115328. https://doi.org/10.1001/jamanetworkopen.2021.15328

20.

Gude

W. T.

Brown

van der Veer

S. N.

Colquhoun

H. L.

Ivers

N. M.

Brehaut

J. C.

Landis-Lewis

Armitage

C. J.

de Keizer

N. F.

Peek

(2019). Clinical performance comparators in audit and feedback: A review of theory and evidence. Implementation Science, 14(1), 39. https://doi.org/10.1186/s13012-019-0887-1

21.

Husain

J. M.

LaRochelle

Keosaian

Xuan

Lasser

K. E.

Liebschutz

J. M.

(2019). Reasons for opioid discontinuation and unintended consequences following opioid discontinuation within the TOPCARE trial. Pain Medicine, 20(7), 1330–1337. https://doi.org/10.1093/pm/pny124

22.

Ivers

N. M.

Jamtvedt

Flottorp

Young

J. M.

Odgaard-Jensen

French

S. D.

O’Brien

M. A.

Johansen

Grimshaw

Oxman

A. D.

(2012). Audit and feedback: Effects on professional practice and healthcare outcomes. The Cochrane Database of Systematic Reviews, 6, CD000259. https://doi.org/10.1002/14651858.CD000259.pub3

23.

Kennedy

A. G.

Regier

Fischer

M. A.

(2021). Educating community clinicians using principles of academic detailing in an evolving landscape. American Journal of Health-System Pharmacy, 78(1), 80–86. https://doi.org/10.1093/ajhp/zxaa351

24.

Kennedy

L. C.

Binswanger

I. A.

Mueller

S. R.

Levy

Matlock

D. D.

Calcaterra

S. L.

Koester

Frank

J. W.

(2018). “Those conversations in my experience don’t go well”: A qualitative study of primary care provider experiences tapering long-term opioid medications. Pain Medicine, 19(11), 2201–2211. https://doi.org/10.1093/pm/pnx276

25.

Langford

A. V.

Gnjidic

Lin

C.-W. C.

Bero

Penm

Blyth

F. M.

Schneider

C. R.

(2021). Challenges of opioid deprescribing and factors to be considered in the development of opioid deprescribing guidelines: A qualitative analysis. BMJ Quality & Safety, 30(2), 133–140. https://doi.org/10.1136/bmjqs-2020-010881

26.

Lorencatto

Gould

N. J.

McIntyre

S. A.

During

Bird

Walwyn

Cicero

Glidewell

Hartley

Stanworth

S. J.

Foy

Grimshaw

J. M.

Michie

Francis

J. J.

, & for the AFFINITIE programme. (2016). A multidimensional approach to assessing intervention fidelity in a process evaluation of audit and feedback interventions to reduce unnecessary blood transfusions: A study protocol. Implementation Science, 11(1), 163. https://doi.org/10.1186/s13012-016-0528-x

27.

Luetsch

Wong

Rowett

(2023). A realist synthesis of educational outreach visiting and integrated academic detailing to influence prescribing in ambulatory care: Why relationships and dialogue matter. BMJ Quality & Safety. https://doi.org/10.1136/bmjqs-2022-015498

28.

McSharry

Murphy

P. J.

Byrne

(2016). Implementing international sexual counselling guidelines in hospital cardiac rehabilitation: Development of the CHARMS intervention using the Behaviour Change Wheel. Implementation Science, 11(1), 134. https://doi.org/10.1186/s13012-016-0493-4

29.

Meghani

S. H.

Byun

Gallagher

R. M.

(2012). Time to take stock: A meta-analysis and systematic review of analgesic treatment disparities for pain in the United States. Pain Medicine (Malden, Mass.), 13(2), 150–174. https://doi.org/10.1111/j.1526-4637.2011.01310.x

30.

Michie

Richardson

Johnston

Abraham

Francis

Hardeman

Eccles

Cane

Wood

(2013). The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: Building an international consensus for the reporting of behavior change interventions. Annals of Behavioral Medicine, 46(1), 81–95. https://doi.org/10.1007/s12160-013-9486-6

31.

Moore

G. F.

Audrey

Barker

Bond

Bonell

Hardeman

Moore

O’Cathain

Tinati

Wight

Baird

(2015). Process evaluation of complex interventions: Medical Research Council guidance. BMJ, 350, h1258. https://doi.org/10.1136/bmj.h1258

32.

Nicolas-Angl

Laur

Strange

Sklar

Tadrous

Ivers

N. M.

(2023). Bringing your whole self: Exploring the varying roles of lived experience advisors in an implementation science project on management of opioid prescribing. Submitted for publication..

33.

O’Brien

M. A.

Rogers

Jamtvedt

Oxman

A. D.

Odgaard-Jensen

Kristoffersen

D. T.

Forsetlund

Bainbridge

Freemantle

Davis

D. A.

Haynes

R. B.

Harvey

E. L.

(2007). Educational outreach visits: Effects on professional practice and health care outcomes. The Cochrane Database of Systematic Reviews, 4, CD000409. https://doi.org/10.1002/14651858.CD000409.pub2

34.

Pantoja

Grimshaw

J. M.

Colomer

Castañon

Leniz Martelli

(2019). Manually-generated reminders delivered on paper: Effects on professional practice and patient outcomes. The Cochrane Database of Systematic Reviews, 2019(12), CD001174. https://doi.org/10.1002/14651858.CD001174.pub4

35.

Patey

A. M.

Fontaine

Francis

J. J.

McCleary

Presseau

Grimshaw

J. M.

(2023). Healthcare professional behaviour: Health impact, prevalence of evidence-based behaviours, correlates and interventions. Psychology & Health, 38(6), 766–794. https://doi.org/10.1080/08870446.2022.2100887

36.

Patey

A. M.

Islam

Francis

J. J.

Bryson

G. L.

Grimshaw

J. M.

(2012). Anesthesiologists’ and surgeons’ perceptions about routine pre-operative testing in low-risk patients: Application of the Theoretical Domains Framework (TDF) to identify factors that influence physicians’ decisions to order pre-operative tests. Implementation Science, 7(1), 52. https://doi.org/10.1186/1748-5908-7-52

37.

Powell

B. J.

Fernandez

M. E.

Williams

N. J.

Aarons

G. A.

Beidas

R. S.

Lewis

C. C.

McHugh

S. M.

Weiner

B. J.

(2019). Enhancing the impact of implementation strategies in healthcare: A research agenda. Frontiers in Public Health, 7, 3. https://doi.org/10.3389/fpubh.2019.00003

38.

Presseau

Byrne-Davis

L. M. T.

Hotham

Lorencatto

Potthoff

Atkinson

Bull

E. R.

Dima

A. L.

Dongen

A. v.

French

Hankonen

Hart

Hoor

G. A. t.

Hudson

Kwasnicka

Lieshout

S. v.

McSharry

Olander

E. K.

Powell

, … Byrne

(2022). Enhancing the translation of health behaviour change research into practice: A selective conceptual review of the synergy between implementation science and health psychology. Health Psychology Review, 16(1), 22–49. https://doi.org/10.1080/17437199.2020.1866638

39.

Presseau

McCleary

Lorencatto

Patey

A. M.

Grimshaw

J. M.

Francis

J. J.

(2019). Action, actor, context, target, time (AACTT): A framework for specifying behaviour. Implementation Science, 14(1), 102. https://doi.org/10.1186/s13012-019-0951-x

40.

Proctor

E. K.

Powell

B. J.

McMillen

J. C.

(2013). Implementation strategies: Recommendations for specifying and reporting. Implementation Science, 8(1), 139. https://doi.org/10.1186/1748-5908-8-139

41.

Rodrigues

A. M.

Wearn

Haste

Mallion

Evison

Howle

Haighton

(2022). Understanding the implementation strategy of a secondary care tobacco addiction treatment pathway (the CURE project) in England: A strategic behavioural analysis. BMJ Open, 12(6), e054739. https://doi.org/10.1136/bmjopen-2021-054739

42.

Soumerai

S. B.

Avorn

(1990). Principles of educational outreach (‘academic detailing’) to improve clinical decision making. JAMA: The Journal of the American Medical Association, 263(4), 549–556. https://doi.org/10.1001/jama.1990.03440040088034

43.

Spooner

Fernandes

Martins

Juurlink

Mamdani

Paterson

J. M.

Singh

Gomes

(2016). High-dose opioid prescribing and opioid-related hospitalization: A population-based study. PLoS ONE, 11(12), e0167479. https://doi.org/10.1371/journal.pone.0167479

44.

Steinmo

S. H.

Michie

Fuller

Stanley

Stapleton

Stone

S. P.

(2015). Bridging the gap between pragmatic intervention design and theory: Using behavioural science tools to modify an existing quality improvement programme to implement “Sepsis Six”. Implementation Science, 11(1), 14. https://doi.org/10.1186/s13012-016-0376-8

45.

Tong

Sainsbury

Craig

(2007). Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care, 19(6), 349–357. https://doi.org/10.1093/intqhc/mzm042

46.

Vachon

Désorcy

Camirand

Rodrigue

Quesnel

Guimond

Labelle

Fournier

Grimshaw

(2013). Engaging primary care practitioners in quality improvement: Making explicit the program theory of an interprofessional education intervention. BMC Health Services Research, 13(1), 106. https://doi.org/10.1186/1472-6963-13-106

47.

Westerlund

Nilsen

Sundberg

(2019). Implementation of implementation science knowledge: The research-practice gap paradox. Worldviews on Evidence-Based Nursing, 16(5), 332–334. https://doi.org/10.1111/wvn.12403

48.

Willis

T. A.

Wood

Brehaut

Colquhoun

Brown

Lorencatto

Foy

(2022). Opportunities to improve the impact of two national clinical audit programmes: A theory-guided analysis. Implementation Science Communications, 3(1), 32. https://doi.org/10.1186/s43058-022-00275-5

49.

Yeh

J. S.

Van Hoof

T. J.

Fischer

M. A.

(2016). Key features of academic detailing: Development of an expert consensus using the Delphi Method. American Health & Drug Benefits, 9(1), 42–50.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

0.30 MB

0.02 MB

0.58 MB

0.04 MB

3.09 MB

0.95 MB

0.33 MB