Theories,models,and frameworks for de-implementation of low-value care: A scoping review of the literature

Abstract

Background:

The aim of this scoping review was to identify theories, models, and frameworks for understanding the processes and determinants of de-implementing low-value care (LVC). We investigated theories, models, and frameworks developed specifically for de-implementation of LVC (conceptual studies) and those that were originally developed for implementation of evidence-based practices but were applied in studies to analyze de-implementation of LVC (empirical studies).

Methods:

We performed a scoping review to identify theories, models, and frameworks used to describe, guide, or explain de-implementation of LVC, encompassing four stages following the identification of the research question: (1) identifying relevant studies; (2) study selection; (3) charting the data; and (4) collating, summarizing, and reporting the results. The database searches yielded 9,642 citations. After removing duplicates, 6,653 remained for the abstract screening process. After screening the abstracts, 76 citations remained. Of these, 10 studies were included in the review.

Results:

We identified 10 studies describing theories, models, and frameworks that have been used to understand de-implementation of LVC. Five studies presented theories, models, or frameworks developed specifically for de-implementation of LVC (i.e., conceptual studies) and five studies applied an existing theory, model, or framework concerning implementation of evidence-based practices (i.e., empirical studies).

Conclusion:

Most of the theories, models, and frameworks that are used to analyze LVC suggest a multi-level understanding of de-implementation of LVC. The role of the patient is inconsistent in these theories, models, and frameworks; patients are accounted for in some but not in others. The findings point to the need for more research to identify the most important processes and determinants for successful de-implementation of LVC and to explore differences between de-implementation and implementation.

Plain language abstract

Achieving an evidence-based practice not only depends on implementation of evidence-based interventions (programs, methods, etc.) but also requires de-implementing interventions that are not evidence-based, that is, low-value care (LVC). Thus, de-implementation is the other side of the coin of an evidence-based practice. However, this is quite a new topic and knowledge is lacking concerning how de-implementation and implementation processes and determinants might differ. It is almost mandatory for implementation researchers to use theories, models, and frameworks (i.e., “theoretical approaches”) to describe, guide, or explain implementation processes and determinants. To what extent are such approaches also used with regard to de-implementation of LVC? And what are the characteristics of such approaches when analyzing de-implementation processes? We reviewed the literature to explore issues such as these. We identified only 10 studies describing theoretical approaches that have been used concerning de-implementation of LVC. Five studies presented approaches developed specifically for de-implementation of LVC and five studies applied an already-existing approach usually applied to analyze implementation processes. Most of the theoretical approaches we found suggest a multi-level understanding of de-implementation of LVC, that is, successfully de-implementing LVC may require strategies that target teams, departments, and organizations and merely focus on individual health care practitioners. The findings point to the need for more research to identify the most important processes and determinants for successful de-implementation of LVC, and to explore differences between de-implementation and implementation. In terms of practice and policy implications, the study underscores the relevance of addressing multiple levels when attempting to de-implement LVC.

Keywords

De-implementation health care health scoping review low-value care

Background

The birth of implementation science is usually linked to the emergence of evidence-based medicine and practice in the 1990s. The evidence-based movement has popularized the notion that research findings and empirically supported (“evidence-based”) practices (e.g., preventive, diagnostic or therapeutic interventions, services, programs, methods, techniques, and routines) should be more broadly and/or efficiently implemented in various settings to achieve improved health and welfare of populations. However, there is increasing recognition that an evidence-based practice may also require de-implementing (i.e., abandoning) practices that are not evidence-based, usually referred to as low-value care (LVC), that is, “care that is unlikely to benefit the patient given the harms, cost, available alternatives, or preferences of the patient” (Verkerk et al., 2018, p. 737). Research has documented considerable prevalence of LVC in the United States and other countries (Davidson et al., 2017). It has been estimated that 10% to 30% of all health care practices have little or no benefit to the patient (Morgan et al., 2017). Hence, as noted by Willson (2015, p. 1), LVC “constitutes a pervasive problem.”

There has been increased emphasis in recent years on health authority and policy initiatives to develop and spread strategies to facilitate de-implementation of LVC (Burton et al., 2019; Davidoff, 2015; McKay et al., 2018). Publishing “do-not-do” recommendations has become a common strategy to address LVC issues, for example, Choosing Wisely, Smarter Medicine, Slow Medicine, and Preventing Overdiagnosis initiatives, by encouraging health care professionals to de-implement specific LVC practices (Grimshaw et al., 2020; Morgan et al., 2017; Powell et al., 2013; Schlesinger & Grob, 2017; van Bodegom-Vos et al., 2017). These initiatives provide guidance on which practices should be abandoned, but they do not specify how they should be de-implemented or what factors need to be considered for achieving successful de-implementation. Thus, more knowledge is needed about the process and determinants of de-implementation of LVC to explain how and why such strategies may be effective.

In contrast to de-implementation of LVC, there is an extensive literature on implementation of evidence-based practices. Factors influencing implementation success have been described in numerous so-called determinant frameworks, for example, Consolidated Framework for Implementation Research (CFIR) (Damschroder et al., 2009) and the integrated Promoting Action on Research Implementation in Health Services (i-PARIHS) (Harvey & Kitson, 2016). Many of these frameworks point to the relevance of four types of determinants (i.e., determinant domains) for successful implementation: the characteristics of the practice itself (e.g., the empirical evidence); the patients; the individual health care professionals; and the social, organizational, and wider context of implementation (Nilsen, 2015). It has been speculated that partially different types of determinants influence de-implementation processes. Theory of Risk Aversion, that is, aversion to losses (e.g., abandoning a familiar practice) is stronger than attraction to gains (e.g., adopting a new practice) (Burton et al., 2019; Davidoff, 2015; McKay et al., 2018; van Bodegom-Vos et al., 2017; Willson, 2015), has been applied to explain why the reversal process of abandoning the use of well-established practices may be more difficult than adopting new practices. However, further research is needed to generate knowledge about the extent to which and how determinants involved in de-implementation of “non-evidence-based” practices differ from determinants of implementation of evidence-based practices.

De-implementation research hitherto has focused on studies within specific clinical and medical care areas; knowledge has not been widely disseminated beyond these areas. However, the need to synthesize knowledge on de-implementation of LVC has been raised by numerous scholars (Burton et al., 2019; Grimshaw et al., 2020; McKay et al., 2018; Niven et al., 2015; Norton et al., 2019; Voorn et al., 2018). There is a paucity of knowledge regarding to what extent theories, models, and frameworks are used to describe, guide, or explain the processes involved in the de-implementation of LVC. We also do not know whether de-implementation studies use the same theories, models, and frameworks as those applied in implementation studies or if specific theories, models, and frameworks have been developed to understand de-implementation of LVC.

Addressing key knowledge gaps, the aim of this scoping review was to identify theories, models, and frameworks for understanding the processes and determinants of de-implementing LVC. We investigated theories, models, and frameworks developed specifically for de-implementation of LVC (referred to as conceptual studies in this article) and those that were originally developed for implementation of evidence-based practices, but were applied in studies to analyze de-implementation of LVC (referred to as empirical studies in this article).

Methods

Study design

To address the study aims, we performed a scoping review to identify theories, models, and frameworks used to describe, guide, or explain de-implementation of LVC. A scoping review was chosen because it allows for synthesis of findings across a range of study types and designs, providing a broad overview of a topic (O’Brien et al., 2016).

The scoping review was based on a framework developed by Arksey and O’Malley (2005), encompassing four stages following the identification of the research question: (1) identifying relevant studies; (2) study selection; (3) charting the data; and (4) collating, summarizing, and reporting the results. The first three stages are described in the “Methods” section, and the fourth stage constitutes the “Results” section.

Identifying relevant studies

The first stage involved defining inclusion criteria (Arksey & O’Malley, 2005). To be included in the scoping review, studies were required to address theories, models, and frameworks used to describe, guide, or explain de-implementation of LVC in health care published from 2013 to June 2018 in English-speaking peer-reviewed journals. We used a broad definition of health care that covered primary care, different types of hospital care and mental health services. The rationale for the time frame was that an overview of the studies identified in the database searches showed that there was an increasing number of papers from 2013 onwards. It was also deemed important to limit the number of studies because the topic of the review yielded a large amount of irrelevant studies.

A search strategy was developed in collaboration with a team of informaticians at the Karolinska Institute university library. We first identified potential keywords through four means: (1) discussions in the research project team consisting of experts in the implementation research field, (2) discussions with the Swedish Agency for Health Technology Assessment and Assessment of Social Services (Statens Beredning för Medicinsk och Social Utvärdering, SBU), (3) examination of a review outlining the terminology used in de-implementation (Niven et al., 2015), and (4) examination of 17 example papers focusing on de-implementation (the list is available in the Supplementary Material). The search terms were tested and refined three times to ensure that they captured the 17 example papers and that they were sufficiently discriminant not to yield an overwhelming amount of references.

The search strategy included searches in four electronic databases: MEDLINE, Embase, CINAHL, and Web of Science. The search was conducted June 4, 2018. We also conducted manual searches of the journal Implementation Science and the reference lists from relevant articles. The informatician team removed duplicates before providing the full list of references produced by the search to the research project team. The databases were searched from January 1, 2013, to June 4, 2018. The search strategy used in Web of Science is detailed in Box 1.

Box 1.

Search strategy used in Web of Science.

Field labels
• TS/Topic = title, abstract, author keywords and Keywords Plus
• NEAR/x = within x words, regardless of order
• * = truncation of word for alternate endings
#1 TOPIC:, (((OR contradict* OR deadopt* OR “de-adopt*” OR disadopt* OR “dis-adopt*” OR decommission* OR “de-commission*” OR deimplement* OR “de-implement*” OR delist* OR “de-list*” OR disinvest* OR “dis-invest” OR deprescript* OR deprescrib* OR divest* OR inapprop* OR ineffective* OR “low-value” OR obsole* OR outmoded OR overuse OR reallocate* OR reassess* OR “re-assess*” OR refute* OR refuting OR “re-invest*” OR “medical revers*” OR supersed* OR unlearn*) NEAR/3 (care OR clinic* OR device* OR drug OR drugs OR evidence* OR health OR healthcare OR medical OR medication* OR prescrib* OR procedur* OR technolog* OR therap* OR treat*)))
#2, (((chang* or discontinu* or “dis-continu*” or decreas* or declin* or drop or reduc* or withdraw*) NEAR/1 (“use” or practice) NEAR/3 (care or clinic* or device* or drug or drugs or evidence* or health or healthcare or medical or medication* or prescrib* or procedur* or technolog* or therap* or treat*)))
#3 TS = (“choosing wisely” or “priority setting”) AND TS = (care or clinic* or device* or drug or drugs or evidence* or health or healthcare or medical or medication* or prescrib* or procedur* or technolog* or therap* or treat*)
#4 #3 OR #2 OR #1
#5 ((abandon* OR contradict* OR deadopt* OR “de-adopt*” OR disadopt* OR “dis-adopt*” OR decommission* OR “de-commission*” OR deimplement* OR “de-implement*” OR delist* OR “de-list*” OR disinvest* OR “dis-invest” OR discontinu* OR “dis-continu*” OR deprescipt* OR deprescrib* OR divest* OR inapprop* OR ineffective* OR “low-value” OR obsole* OR outmoded OR overuse OR reallocate* OR reassess* OR “re-assess*” OR refute* OR refuting OR “re-invest*” OR “medical revers*” OR supersed* OR unlearn* OR withdraw*) NEAR/3 (factor* OR barrier* OR engag* OR “evidence-based” OR facilitat* OR determinant* OR predict* OR model* OR framework* OR intervent* OR policy OR policies OR “practice pattern*” OR program* OR strateg* OR tool*))
#6 #5 AND #4
Refined by: LANGUAGES: (ENGLISH)

Study selection

The next stage involved the elimination of studies that did not address our research questions (Arksey & O’Malley, 2005). All studies were imported into Rayyan, a web and mobile application that organizes and facilitates an initial screening of titles and abstracts (Ouzzani et al., 2016). A multi-step test was conducted to examine the relevance of the eligibility criteria and ensure consensus among the reviewers concerning the included studies. As a first step, four reviewers tested the relevance of the criteria on a sample of 40 study abstracts. Inconsistencies were discussed, and some clarifications and modifications to the criteria were made. As a second step, the same reviewers conducted a second test on an additional 40 abstracts, which resulted in minor refinements of the criteria. Thereafter, five reviewers (three authors [P.N., H.A., S.I.] and two research assistants) applied the eligibility criteria to all the abstracts.

The abstract screening was done in duplicate so that each abstract was independently assessed by two reviewers. When all the abstracts had been screened, the reviewers discussed conflicting decisions. In cases where disagreement or uncertainty still existed, the whole reviewer group discussed until consensus was reached.

In the next step, the five reviewers assessed the full texts of the included citations for final inclusion. This process was also carried out in duplicate so that all full-text articles were assessed independently by two reviewers. Again, conflicting decisions were discussed between the two reviewers and in case of disagreement or uncertainty, the article was discussed by the full reviewer group until consensus was reached.

Charting the data

The next stage involved charting key items of information obtained from the studies that were reviewed (Arksey & O’Malley, 2005). We collected data on the following study characteristics: (1) authors; (2) publication year; (3) country of origin (with the first author’s affiliation used as a proxy for country of origin if information about origin was missing); (4) type of setting where the study was carried out; (5) LVC practice (e.g., intervention, treatment, routine, and program); (6) aim as specified in the study; (7) aim in relation to the aims of this study; and (8) data collection methods and/or knowledge sources used.

Extracted data were analyzed to distinguish between two types of studies: (1) conceptual studies that described theories, models, and frameworks developed specifically for de-implementation of LVC; and (2) empirical studies that featured theories, models, and frameworks originally developed in implementation science (i.e., for implementation of evidence-based practices), but which were applied to analyze de-implementation of LVC.

With regard to the conceptual studies, we extracted data concerning (1) type of theory, model, or framework described in the study (see below); (2) title given, if any, to the theory, model, or framework; (3) influences on the theory, model, or framework; and (4) components of the theory, model, or framework (i.e., activities and/or determinant domains). For the empirical studies, data were extracted on (1) which implementation theory, model, or framework was applied in the study; (2) components of the applied theory, model, or framework (i.e., activities and/or determinant domains); (3) perceived relevance of the theory, model, or framework, according to the authors; and (4) comments on the usefulness of the theory, model, or framework.

With regard to the type of theory, model, or framework described in the studies, this was classified according to the taxonomy by Nilsen (2015), which distinguishes between the following: (1) process models, (2) determinant frameworks, (3) classic theories, and (4) implementation theories. Process models are used to describe and/or guide change processes involved in translating research into practice. Determinant frameworks describe domains (i.e., general types or classes) of determinants that are hypothesized or have been found to influence implementation outcomes. Each determinant domain typically comprises a number of individual barriers (hinders, impediments) and/or enablers (facilitators), which are seen as independent variables that have an impact on implementation outcomes, that is, the dependent variable. Classic theories are theories borrowed from other fields such as psychology, sociology, and organizational theory. Implementation theories were developed or adapted by researchers for use in implementation science. A theory implies some predictive capacity and the ambition to explain causal mechanisms of implementation. Neither models nor frameworks specify the mechanisms of change; they are typically more like checklists of factors relevant to various aspects of implementation (Nilsen, 2015).

Results

Included studies

The database searches yielded 9,642 citations. After removing duplicates, 6,653 remained for the abstract screening process. After screening the abstracts, 76 citations remained. Of these, 10 studies were included in the review. The PRISMA flowchart (Figure 1) outlines the number included and excluded in each phase and provides information about the reasons for exclusion in the full-text screening.

Figure 1.

The PRISMA flowchart for this review.

Study characteristics

The 10 studies were published between 2013 and 2018 (Table 1). Four of the studies were conducted in the United States, two in Canada, two in the Netherlands, and one each in Ireland and Australia. Five studies applied an existing theory, model, or framework originally developed for implementation of evidence-based practices, but applied with regard to de-implementation of LVC (empirical studies) and five studies developed a new theory, model, or framework for understanding de-implementation of LVC (conceptual studies).

Table 1.

Information about the studies.

Authors and publication year	Origin of study	Type of setting	LVC practice	Aim as specified in the study	Aim in relation to the aims of this study	Data collection methods and knowledge sources
Curran et al. (2013)	Canada	Emergency departments	Use of computed tomography in adult patients with minor head injury	To explore whether the Theoretical Domains Framework (TDF) could be used to guide a retrospective process evaluation to better understand emergency physicians’ responses to the interventions employed in the Canadian Computed Tomography Head rule trial	To apply a determinant framework (TDF) to identify determinants of de-implementation of LVC	Interviews: Semi-structured interviews based on TDF were conducted with eight project leads and physicians
Powell et al. (2013)	United States	Primary care	LVC (in general) in primary care	To present a conceptual framework “to guide overuse research and intervention development”. More specifically, the purpose was “to provide a structure that can be used to identify barriers to appropriate use of services and to provide guidance on developing interventions to address barriers once they are identified”	To develop a theory for de-implementation of LVC	Experience/expertise: The study relied on the authors’ experience and experience regarding overuse and theories that they based the framework on (no specific sources are mentioned). Interventions to address barriers were identified by literature searches
Cullinan et al. (2014)	Ireland	Hospital care	Prescribing to older patients	To identify hospital doctors’ perceptions as to why potentially inappropriate prescribing occurs, to identify the barriers to addressing the issues identified and to determine which intervention types would be best suited to improving prescribing	To apply a determinant framework (TDF) to identify determinants of de-implementation of LVC	Interviews: Semi-structured interviews based on TDF were conducted with 22 hospital doctors
Voorn et al. (2014)	The Netherlands	Hospital care, total hip arthroplasty (THA) and total knee arthroplasty (TKA)	Blood-saving measures (BSMs) erythropoietin and intra- and postoperative blood salvage	To identify barriers that hinder physicians to stop the use of non-cost-effective BSMs in primary elective THA and TKA	To apply a determinant framework (TDF) to ensure that the questionnaire that was developed based on 20 interviews did not neglect any barriers to de-implementation of LVC	Interviews and questionnaire: Semi-structured interviews were conducted with 10 orthopedic surgeons and 10 anesthesiologists. Findings from the interviews were compared with TDF to ensure that no potentially relevant barriers were missed. A questionnaire was developed and a random sample of 400 orthopedic surgeons and 400 anesthesiologists were invited to respond
Niven et al. (2015)	Canada	Not specified	LVC (in general)	The study has several aims, but the relevant aim for this scoping review is that it will develop a “conceptual framework”	To develop a process model for de-implementation of LVC	Literature review: A scoping review was undertaken to describe the literature on de-adoption, map the literature to a proposed framework, identify gaps in the understanding of this concept, and identify opportunities for more detailed evidence syntheses and/or empirical research
Barnes et al. (2017)	United States	Nurse/pharmacist-run anticoagulation clinics	Laboratory testing for patients with stable warfarin control	To identify determinants of de-implementation of (i.e., reduced) laboratory testing for patients with stable warfarin control	To apply a determinant framework (TICD) to identify determinants of de-implementation of LVC	Interviews: Semi-structured interviews were conducted with seven nurses and one pharmacist. TICD was used as a framework for coding the interviews. The study also used a questionnaire survey but this was not used to analyze determinants of de-implementation
Harris, Green, & Elshaug (2017)	Australia	Not specified	LVC (in general)	The study has several aims, but the relevant aim for this scoping review is that it will propose a framework for disinvestment in the context of resource allocation in the local health care setting	To develop a combined process model and determinant framework for de-implementation of LVC	Literature review: The proposed framework was based on “messages from the literature” and drew specifically on three frameworks which identify steps in the disinvestment process; the Sustainability in Healthcare by Allocating Resources Effectively (SHARE) frameworks and models; and other frameworks for introduction of new technologies and clinical practices and evidence-based change
Morgan et al. (2017)	United States	Not specified	LVC (in general)	To develop a “framework for conceptualizing interventions to reduce medical overuse”	To develop a theory for de-implementation of LVC	Experience/expertise and literature review: Authors’ experience and experience as well as one referenced study, Reschovsky et al. (2015), were utilized. The framework was developed on the basis of this paper and expert opinion, discussion among the authors, and feedback from attendees at a meeting of clinicians, patients and policy makers interested in overuse
Parchman et al. (2017)	United States	Not specified	LVC (in general)	To propose an action-planning framework for use as a roadmap to guide engagement efforts for providers, patients, and all members of the health care team in efforts to reduce LVC	To develop a combined process model and determinant framework for de-implementation of LVC	Experience/expertise, literature review, and interviews: A multi-stakeholder advisory committee with eight-member stakeholder advisory committee that included patients, providers and health care leaders provided input for a literature review, informed selection of sites for the environmental scan interviews, and provided feedback and interpretation of findings to inform elements of the framework. A literature review was undertaken of peer-reviewed and gray literature. An environmental scan was carried out, encompassing semi-structured interviews with 23 leaders of initiatives to reduce LVC
Voorn et al. (2018)	The Netherlands	Hospital care, blood management in total hip arthroplasty (THA) and total knee arthroplasty (TKA)	Blood transfusions (the use of erythropoiesis-stimulating agents and blood salvage)	The study has several aims, but the relevant aim for this scoping review is that it will describe a systematic approach for de-implementation illustrated by an example regarding (cost-)effective patient blood management in total hip and knee arthroplasty	To apply a process model (Grol et al., 2005) to evaluate the process and determinants of de-implementation of LVC	Experience/expertise and literature review: The literature review covers issues concerning differences between the concepts of implementation and de-implementation and theories, models, and frameworks for potential use to understand de-implementation of LVC

Note. LVC = low-value care.

With the exception of one study (Powell et al., 2013), which focused on primary care, the conceptual studies did not specify a particular setting or LVC practice. Rather, the developed de-implementation theories, models, and frameworks were intended to apply to LVC in general. These studies utilized the authors’ experience and expertise, literature reviews, and interviews as knowledge sources.

The empirical studies were carried out in hospital care, emergency care, and nurse-/pharmacist-run clinics. The specific LVC practice that was addressed in these studies varied considerably. The studies collected empirical data by means of interviews and questionnaires.

Conceptual studies

Table 2 provides an overview of the five conceptual studies. The studies by Harris, Green, and Elshaug (2017) and Parchman et al. (2017) both featured models/frameworks that combine characteristics of process models and determinant frameworks, that is, encompassing both activities to guide de-implementation of LVC and determinants that might influence the outcomes of de-implementation of LVC. Harris, Green, and Elshaug (2017) listed 10 studies that informed their model/framework (referred to as “Framework for an Organisation-Wide Approach to Disinvestment in the Local Healthcare Setting”), of which seven were by the same main author. Parchman et al. (2017) did not mention any specific theory, model, or framework as an influence on their model/framework (referred to as “Framework for Taking Action on Overuse”).

Table 2.

Data on the conceptual studies featuring theories, models, and frameworks that were developed in the studies.

Authors (publication year)	Type of TMF(s) described in the study	Title of the TMF	Influences on the development of the TMF	Components of the TMF
Powell et al. (2013)	De-implementation theory	The theory is referred to as Theoretical Framework	Informed by Theory of Planned Behavior (classic theory) and “other theories of motivation and goal setting”	The theory consists of five determinant domains, referred to as “variables”, to “understand appropriateness of primary care provider service recommendations”:
				1. Provider beliefs
				2. Assessment intentions
				3. Assessment (of the appropriateness of the intervention)
				4. Appropriate recommendation
				5. Patient acceptance of recommendation
				A sequential chain of causal mechanisms is described. Provider beliefs affect assessment intentions, which in their turn affect the assessment of the appropriateness of the intervention. This assessment predicts an appropriate recommendation, which influences patient acceptance of recommendation. Provider beliefs are a function of outcome beliefs, normative beliefs, and control beliefs. Outcome beliefs concern beliefs about expected outcomes for patients, for the self and for the organization
Niven et al. (2015)	Process model for de-implementation of LVC	The model is referred to as Synthesis Model for the Process of De-adoption	Informed by Knowledge-to-Action (process model)	The model consists of seven activities in a “process of de-adoption”:
				1. Identify and prioritize low-value clinical practices
				2. Assess current use of low-value practice
				3. Adapt knowledge to local context
				4. Assess barriers and facilitators to de-adoption
				5. Select, tailor, and implement de-adoption intervention
				6. Evaluate de-adoption process and outcomes
				7. Sustain de-adoption
Harris, Green, & Elshaug (2017)	Combined process model and determinant framework for de-implementation of LVC	The model/framework is referred to as Framework for an Organization-Wide Approach to Disinvestment in the Local Healthcare Setting	The model/framework “builds on the work of others” and “draws heavily” on models and frameworks described in 10 specific studies (see table footnote)	The model/framework consists of three determinant domains, referred to as “interconnected and interdependent components” (also referred to as “elements”):
				1. Program for organization-wide decision-making (consisting of Principles for decision-making; Boundaries; Ethics; Governance; Structures; Processes; Stakeholder involvement; Resources; and Preconditions)
				2. Projects to implement decisions and evaluate outcomes consisting of: Identification of opportunities; Development of criteria; Prioritization and Decision-making; Development of a proposal; Implementation; Measurement of resource release; Monitoring, Evaluation and Reporting; Reinvestment (if required); Dissemination and Diffusion; and Maintenance
				3. Research to understand and improve the program and project activities overlaid across all components and sub-components of the model/framework to represent the potential for research in each aspect of the model/framework
Morgan et al. (2017)	De-implementation theory	The theory is referred to as Framework for Understanding and Reducing Overuse	Informed by one study by Reschovsky et al. (2015), “Factors contributing to variations in physicians’ use of evidence at the point of care: a conceptual model”	The theory consists of six determinant domains, referred to as “domains” and “drivers of overuse”:
				1. Culture of health care consumption
				2. Culture of professional medicine
				3. Practice environment
				4. Patient factors and experiences
				5. Clinician attitudes and beliefs
				6. Patient–clinician interaction
				The patient–clinician interaction yields the outcome Healthcare choice
				Causal mechanisms between the six domains are described. The culture of health care consumption influences patient factors and experiences; culture of professional medicine; and practice environment. The culture of professional medicine influences culture of health care consumption; clinician attitudes and beliefs; and practice environment. Practice environment influences clinician attitudes and beliefs; patient–clinician interaction; and patient factors and experiences. In addition to practice environment, the patient–clinician interaction is influenced by both patient factors and experiences and clinician attitudes and beliefs. Ultimately, the patient–clinician interaction influences health care choice, which is the “outcome”
Parchman et al. (2017)	Combined process model and determinant framework for de-implementation of LVC	The model is referred to as Framework for Taking Action on Overuse	No specific theories, models, or frameworks are mentioned as influences	The framework/model consists of four activities to “create conditions for change”:
				1. Prioritize the need to reduce LVC
				2. Build a culture of trust, innovation and improvement
				3. Establish a shared purpose and language (about overuse and LVC)
				4. Commit resources to measurement (on use of LVC)
				It also consists of four determinant domains, referred to as “Providers, care teams and patients change practice together”:
				1. Sense-making conversations among providers, patients, and care team members
				2. Conversations about potential for harm
				3. Provision of measures of how often a test or treatment is used
				4. Encouragement of coordinated action and ownership by providers and care teams
				Factors external to the health care system or setting represent a fifth determinant domain, although the framework/model “is not intended to provide guidance on addressing these factors since healthcare systems often cannot influence them”

Note. LVC: low-value care.

Harris, Green, and Elshaug (2017) listed the following 10 papers as influences on their combined process model/determinant framework: Harris, Allen, Brooke, et al. (2017), Harris, Allen, King, et al. (2017); Harris, Allen, Waller, and Brooke (2017); Harris, Allen, Waller, Green, et al. (2017); Harris et al. (2015); Harris, Ko, et al. (2017); Harris et al. (2016); Ibargoyen-Roteta et al. (2010); and Schmidt (2010).

Niven et al. (2015) developed a process model to guide the process of de-implementing LVC (referred to as “Synthesis Model for the Process of De-adoption”). The model was influenced by the Knowledge-to-Action model, which is based on a review of 31 theories, models, frameworks, and empirical studies. Knowledge-to-Action outlines stages of the research-to-practice process, from discovery and production of research-based knowledge to implementation and use of research in various settings (Graham et al., 2006).

Powell et al. (2013) and Morgan et al. (2017) both developed approaches that should be considered as theories (see explanation in the “Methods” section) because they specify causal mechanisms between determinant domains that show how they influence each other and the outcome. The theory by Powell et al. (2013) (referred to as “Theoretical Framework”) was informed by Theory of Planned Behavior and “other theories of motivation and goal setting” although they did not specify these “other” theories. Theory of Planned Behavior posits that intention toward a behavior, subjective norms, and perceived behavioral control together shape an individual’s behavioral intentions and behaviors (Ajzen, 1991). Meanwhile, Morgan et al. (2017) stated that their theory (referred to as “Framework for Understanding and Reducing Overuse”) was informed by one study by Reschovsky et al. (2015), which concerned factors contributing to variations in physicians’ use of evidence at the point of care.

Empirical studies

Table 3 provides information on the five empirical studies. Three of the empirical studies (Cullinan et al., 2014; Curran et al., 2013; Voorn et al., 2014) applied the Theoretical Domains Framework (TDF), an implementation determinant framework (Michie et al., 2005). The initial version of TDF integrated 33 psychological theories relevant to behavior change into 128 constructs that were sorted into 12 domains (Atkins et al., 2017). Nine of the 12 TDF domains were found to be relevant in one or more of the three studies that applied TDF, with the three domains “Memory, attention and decision processes,” “Environmental context and resources,” and “Social influences” identified as relevant in all three studies. Cullinan et al. (2014) and Voorn et al. (2014) did not comment on the usefulness of TDF for de-implementation purposes, but Curran et al. (2013, p. 7) believed it could provide a useful framework to guide “a retrospective process evaluation from a theoretical perspective.”

Table 3.

Data on the empirical studies that applied existing theories, models, and frameworks.

Authors (publication year)	Type of TMF(s) described in the study	Components of the TMF	Relevance of the components of the TMF	Comments on the TMF’s usefulness
Curran et al. (2013)	Applied the determinant framework TDF	TDF consists of 12 domains:	Six of the 12 TDF domains were found to be relevant:	The authors stated that the study demonstrated that TDF “could provide a useful framework to guide a retrospective process evaluation from a theoretical perspective” (p. 7)
		1. Knowledge	1. Beliefs about consequences
		2. Skills	2. Beliefs about capabilities
		3. Social/professional role and identity	3. Behavioral regulation
		4. Beliefs about capabilities	4. Memory, attention, and decision processes
		5. Beliefs about consequences	5. Environmental context and resources
		6. Motivation and goals	6. Social influences
		7. Memory, attention, and decision processes
		8. Environmental context and resources
		9. Social influences
		10. Emotion
		11. Behavioral regulation
		12. Nature of the behaviors
Cullinan et al. (2014)	Applied the determinant framework TDF	See Curran et al. (2013)	Five of the 12 TDF domains were found to be relevant:	The authors did not address the usefulness of TDF, but noted that the determinant domain social influences was the most “unexpected result”. They concluded that strategies for de-implementation of LVC “will have to be multifaceted” (p. 867)
			1. Environmental context and resources
			2. Knowledge
			3. Skills
			4. Social influences
			5. Memory/attention and decision processes
Voorn et al. (2014)	Applied the determinant framework TDF to ensure that no potentially relevant barriers would be neglected in a questionnaire that was developed based on interviews	See Curran et al. (2013)	Six of the 12 domains of TDF (encompassing 40 specific barriers) were found to be relevant:	The authors did not address the usefulness of TDF, but remarked that the identified barriers for de-implementation “are partly in line with literature concerning implementation of guidelines or evidence-based practice” (p. 2604)
			1. Memory, attention, and decision processes
			2. Social influences
			3. Motivation and goals
			4. Beliefs about consequences
			5. Knowledge
			6. Environmental context and resources
Barnes et al. (2017)	Applied the determinant framework TICD	TICD consists of seven domains:	Two of the seven TICD domains, encompassing four themes, were found to be relevant:	The authors did not address the usefulness of TICD
		1. Guideline factors	(1) Individual health professional factors; (1a) Staff overestimating their actual use of less frequent international normalized ratio testing; (1b) Importance of personalized nurse/pharmacist feedback
		2. Individual health professional factors
		3. Patient factors
		4. Professional interactions	(2) Incentives and resources: (2a) Barriers to appropriate patient engagement; (2b) Broad support for an electronic medical record flag to identify potentially eligible patients
		5. Incentives and resources
		6. Capacity for organizational change
		7. Social, political factors
Voorn et al. (2018)	Applied the process model by Grol et al. (2005)	Grol et al.’s model consists of five activities:	The authors concluded that their de-implementation strategy based on the process model was not effective in reducing LVC, concluding that “de-implementation may be more difficult than expected as other factors may play a role in effective de-implementation compared to implementation”	The authors encouraged others to use the process model to be able to generate more knowledge about how to reduce LVC
		1. Development of a concrete target for change
		2. Analysis of performance, target group and setting
		3. Selection of strategies and measures to change practice
		4. Development, testing and execution of the implementation plan
		5. Evaluation and adaption of the plan

Note. LVC = low-value care; TICD = Tailored Implementation for Chronic Disease.

The study by Barnes et al. (2017) applied Tailored Implementation for Chronic Disease (TICD), an implementation determinant framework based on 12 theories, models, and frameworks used in implementation science (Flottorp et al., 2013). Two of the seven TICD domains, “Individual health professional factors” and “Incentives and resources,” were found to be relevant in the study by Barnes et al. (2017). They did not address the usefulness of TICD.

Voorn et al. (2018) applied a process model by Grol et al. (2005), which describes five activities to be undertaken to guide implementation. The authors adapted the names of the five activities of the original process model: (1) “Detection of improvement goals”; (2) “Analysis of current care, target group and setting (problem analysis)”; (3) “Selection of the (de)implementation strategies”; (4) “Testing and execution of the (de)implementation plan”; and (5) “Evaluation and readjustment of the (de)implementation plan.” Voorn et al. (2018) did not comment on the usefulness of the model, but they encouraged others to use it to generate more knowledge about how to reduce LVC.

Discussion

This scoping review identified 10 theories, models, and frameworks that have been used to understand de-implementation of LVC. The review shows that few theories, models, and frameworks have been developed specifically for use in de-implementation of LVC, which is not surprising considering that de-implementation of LVC is a relatively new area of inquiry. Although results and lessons learned from implementation of evidence-based practices likely can inform de-implementation research, it should not be taken for granted that knowledge is easily transferable between the two areas. The outcomes of implementation and de-implementation differ according to a conceptual analysis by Prusaczyk et al. (2020). In line with this, Davidson et al. (2017, p. 464) have called for a distinct “de-implementation science” that would “recognize and identify problem areas of low-value and wasteful practice, carry out rigorous scientific examinations of the factors that initiate and maintain such practices, and then employ evidence-based interventions to extinguish these practices.”

We identified five studies that presented theories, models, or frameworks developed specifically for de-implementation of LVC (i.e., conceptual studies) and five studies that applied an existing theory, model, or framework concerning implementation of evidence-based practices (i.e., empirical studies). Of the conceptual studies, two studies (Morgan et al., 2017; Powell et al., 2013) described theories, one study (Niven et al., 2015) a process model, and two studies (Harris, Green, & Elshaug, 2017; Parchman et al., 2017) described “hybrid” approaches that combined elements from determinant frameworks and process models, according to the taxonomy by Nilsen (2015). In contrast, there were no theories among the empirical studies we included, of which four described determinant frameworks, TDF (in three studies: Cullinan et al., 2014; Curran et al., 2013; Voorn et al., 2014) and TICD (in one study: Barnes et al., 2017), while one study (Voorn et al., 2018) applied a process model. Thus, there appears to be some attempts to build theory specifically for de-implementation. Unlike determinant frameworks, theories describe mechanisms of change (Nilsen, 2015), thus implying that researchers have seen a need to develop theories specifically for de-implementation purposes.

The use of TDF and TICD point to the relevance of both individual and contextual determinants for de-implementation of LVC because both TDF and TICD account for determinants at different levels. Still, most of the 12 domains of TDF concern individual-level constructs (e.g., “Knowledge,” “Skills,” “Beliefs about capabilities,” and “Motivation and goals”). Two of the TDF domains have a contextual focus: “Environmental context and resources” and “Social influences.” TICD also accounts for individual-level factors (e.g., “Individual health professional factors”), but most of the seven domains concern contextual factors (e.g., “Social, political factors,” “Capacity for organizational change,” “Incentives and resources,” and “Professional interactions”).

A multi-level understanding of de-implementation is also evident in the de-implementation theory developed by Morgan et al. (2017). Contextual domains include “Culture and health care consumption,” “Culture of professional medicine,” and “Practice environment.” Similarly, Parchman et al.’s (2017) combined process model/determinant framework also highlights the importance of culture and collective-level aspects such as establishing a shared purpose and language, and committing resources to measurements. In contrast, the other de-implementation theory in the review, by Powell et al. (2013), focuses more on the individual level (e.g., “Provider beliefs” and “Assessment intentions”), which is to be expected because the theory was informed by the Theory of Planned Behavior, a social-cognitive theory about individuals’ behavior change.

The process model by Niven et al., (2015) is consistent with many process models regarding implementation of evidence-based practices: in that it provides more general guidance concerning important activities when de-implementing LVC, including activities such as assessing current use of LVC practices and assessing barriers and facilitators to de-adoption. Harris, Green, and Elshaug’s (2017) combined process model/determinant framework also offers more general recommendations for de-implementation, for example, activities such as developing structures, processes, and stakeholder involvement for de-implementation of LVC.

The scoping review cannot provide answers about the differences between the process of implementing evidence-based practices and de-implementing LVC. Still, it could be argued that patients are likely to have a more prominent role in de-implementation processes because they might be aware of and express preference for certain known (but LVC) options. It is also possible that patients request new care options, some of which may lack robust evidence, thereby contributing to establishing LVC. However, the results of this review do not support the notion that patients are driving the use of LVC or that they have a more important role in de-implementation processes. For example, neither the process model by Niven et al. (2015) nor the combined process models/determinant frameworks by Harris, Green, and Elshaug (2017) and Parchman et al. (2017) mention patients, focusing more on organizational conditions for de-implementation of LVC. However, the de-implementation theories presented by Powell et al. (2013) and Morgan et al. (2017) both feature the patient as a determinant domain, referring to “Patient acceptance of recommendations” and “Patient factors and experiences,” respectively. However, patients are not included in all implementation process models and determinant frameworks (Nilsen, 2015). For instance, TDF does not explicitly account for patients.

Voorn et al. (2014, p. 2604) noted in their empirical study that the identified determinants of de-implementing LVC practices are “partly in line with literature concerning implementation of guidelines or evidence-based practice.” This is consistent with a recent commentary on determinants of de-implementation by Norton and Chambers (2020), who proposed that the four types of determinants of implementation success—that is, the characteristics of the practice itself, the patients, the individual health care professionals, and the organizational context of implementation (Nilsen, 2015)—are also relevant with regard to de-implementation. Determinant frameworks are essentially socioecological frameworks for sorting various determinants that influence individuals’ behaviors. Applying the same basic principles for structuring determinants, it is obvious that there will be some overlap between implementation and de-implementation with regard to what determinant domains are described. However, the key issue is whether or the extent to which the relative importance of determinants differs between implementation and de-implementation.

Aside from operant learning theory, theories to understand and explain behavior change do not differentiate between increasing and decreasing frequency of a behavior, as noted by Patey et al. (2018). An LVC practice may consist of many behaviors, some of which may have to be increased and others decreased to de-implement the specific practice. For example, reducing the number of unnecessary computed tomography scans may require increased use of clinical decision rules and reduced decision-making based on one’s own clinical judgment (Curran et al., 2013). This makes it difficult to determine whether de-implementation of LVC is inherently more difficult or easier to achieve than implementing a new practice.

We used a scoping review approach because it is generally considered to be useful for answering broad questions and when the information on a topic has not been comprehensively reviewed and/or is diverse (Peters et al., 2015). Still, scoping reviews have shortcomings. For instance, scoping reviews require a substantial amount of time to complete due to the wide coverage of the search implicit in the approach (Sucharew & Macaluso, 2019). A potential limitation of our study is that the literature searches ended already in June 2018. The process from screening 6,570 records to identifying the 10 included studies was time- and resource-consuming, requiring a larger than usual research team. However, we have continued to conduct periodic surveillance of the literature until June 2020 and have thus far not been able to identify any studies that might meet our inclusion criteria. Still, it is possible that further studies of relevance have emerged after our deadline since they are published in many different areas, even with the focus on health care. However, further studies are unlikely to change the key results or conclusions of our review. Another potential limitation was the use of 2013 as a starting point for the literature searches. This was due to the scarcity of studies concerning theories, models, and frameworks prior to 2013. For example, there were only two studies using the keyword “de-implementation” in the Implementation Science journal before 2013 (Prusaczyk et al., 2020).

Conclusion

This scoping review demonstrates that few studies regarding de-implementation of LVC have applied or developed theories, models, and frameworks to understand the processes and determinants of de-implementing LVC. We found five studies that described theories, models, or frameworks developed specifically for de-implementation of LVC (i.e., conceptual studies) and five studies that applied existing an existing theory, model, or framework developed for implementation of evidence-based practices (i.e., empirical studies). Most of the theories, models, and frameworks identified suggest a multi-level understanding of de-implementation of LVC. The role of the patient is inconsistent in these theories, models, and frameworks; patients are accounted for in some but not in others. The findings point to the need to conduct more research to identify the most important activities and determinants for successful de-implementation of LVC and to explore differences between de-implementation and implementation.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Ajzen

(1991). The theory of planned behaviour. Organizational Behaviour and Human Decision Processes, 50, 179–211. https://doi.org/10.1016/0749-5978(91)90020-T

Arksey

O’Malley

(2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology, 8, 19–32. https://doi.org/10.1080/1364557032000119616

Atkins

Francis

Islam

O’Connor

Patey

Ivers

Foy

Duncan

E. M.

Colquhoun

Grimshaw

J. M.

Lawton

Michie

(2017). A guide to using the Theoretical Domains Framework of behaviour change to investigate implementation problems. Implementation Science, 12, 77. https://doi.org/10.1186/s13012-017-0605-9

Barnes

G. D.

Misirliyan

Kaatz

Jackson

E. A.

Haymart

Kline-Rogers

Kozlowski

Krol

Froehlich

J. B.

Sales

(2017). Barriers and facilitators to reducing frequent laboratory testing for patients who are stable on warfarin: A mixed methods study of de-implementation in five anticoagulation clinics. Implementation Science, 12, 87. https://doi.org/10.1186/s13012-017-0620-x

Burton

Williams

Bucknall

Edwards

Fisher

Hall

Harris

Jones

Makin

McBride

Meacock

Parkinson

Rycroft-Malone

Waring

(2019). Understanding how and why de-implementation works in health and care: Research protocol for a realist synthesis of evidence. Systematic Reviews, 8, 194. https://doi.org/10.1186/s13643-019-1111-8

Cullinan

Fleming

O’Mahoney

Ryan

O’Sullivan

Gallagher

Byrne

(2014). Doctors’ perspectives on the barriers to appropriate prescribing in older hospitalized patients: A qualitative study. British Journal of Clinical Pharmacology, 79, 860–869. https://doi.org/10.1111/bcp.12555

Curran

J. A.

Brehaut

Patey

A. M.

Osmond

Stiell

Grimshaw

J. M.

(2013). Understanding the Canadian adult CT head rule trial: Use of the Theoretical Domains Framework for process evaluation. Implementation Science, 8, 25. https://doi.org/10.1186/1748-5908-8-25

Damschroder

Aron

Keith

Kirsh

Alexander

Lowery

(2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, 50. https://doi.org/10.1186/1748-5908-4-50

Davidoff

(2015). On the undiffusion of established practices. JAMA Internal Medicine, 175, 809–811. https://doi.org/10.1001/jamainternmed.2015.0167

10.

Davidson

K. W.

Mensah

G. A.

(2017). Commentary: De-implementation science: A virtuous cycle of ceasing and desisting low-value care before implementing new high value care. Ethnicity & Disease, 27, 463–468. https://doi.org/10.18865/ed.27.4.463

11.

Flottorp

S. A.

Oxman

A. D.

Krause

Musila

N. R.

Wensing

Godycki-Cwirko

Baker

Eccles

M. P.

(2013). A checklist for identifying determinants of practice: A systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implementation Science, 8, 35. https://doi.org/10.1186/1748-5908-8-35

12.

Graham

I. D.

Logan

Harrison

M. B.

Straus

S. E.

Tetroe

Caswell

Robinson

(2006). Lost in knowledge translation: Time for a map? Journal of Continuing Education in the Health Professions, 26(1), 13–24. https://doi:10.1002/chp.47

13.

Grimshaw

J. M.

Patey

A. M.

Kirkham

K. R.

Hall

Dowling

S. K.

Rodondi

Ellen

Kool

van Dulmen

S. A.

Kerr

E. A.

Linklater

Levinson

Bhatia

R. S.

(2020). De-implementing wisely: Developing the evidence base to reduce low-value care. BMJ Quality & Safety, 29, 409–417. https://doi.org/10.1136/bmjqs-2019-010060

14.

Grol

Wensing

Eccles

(2005). Improving patient care: The implementation of change in clinical practice. Elsevier.

15.

Harris

Allen

Brooke

Dyer

Waller

King

Ramsey

Mortimer

(2017). Sustainability in health care by allocating resources effectively (SHARE) 6: Investigating methods to identify, prioritise, implement and evaluate disinvestment projects in a local healthcare setting. BMC Health Services Research, 17, Article 370. https://doi.org/10.1186/s12913-017-2269-1

16.

Harris

Allen

King

Ramsey

Kelly

Thiagarajan

(2017). Sustainability in health care by allocating resources effectively (SHARE) 2: Identifying opportunities for disinvestment in a local healthcare setting. BMC Health Services Research, 17, Article 328. https://doi.org/10.1186/s12913-017-2211-6

17.

Harris

Allen

Waller

Brooke

(2017). Sustainability in health care by allocating resources effectively (SHARE) 3: Examining how resource allocation decisions are made, implemented and evaluated in a local healthcare setting. BMC Health Services Research, 17, Article 340. https://doi.org/10.1186/s12913-017-2207-2

18.

Harris

Allen

Waller

Green

King

Ramsey

Kelly

Thiagarajan

(2017). Sustainability in health care by allocating resources effectively (SHARE) 5: Developing a model for evidence-driven resource allocation in the local healthcare setting. BMC Health Services Research, 17, Article 342. https://doi.org/10.1186/s12913-017-2208-1

19.

Harris

Garrubba

Allen

King

Kelly

Thiagarajan

Castleman

Ramsey

Farjou

(2015). Development, implementation and evaluation of an evidence-based program for introduction of new health technologies and clinical practices in a local healthcare setting. BMC Health Services Research, 15, Article 575. https://doi.org/10.1186/s12913-015-1178-4

20.

Harris

Green

Elshaug

A. G.

(2017). Sustainability in health care by Allocating Resources Effectively (SHARE) 10: Operationalising disinvestment in a conceptual framework for resource allocation. BMC Health Services Research, 17, Article 632. https://doi.org/10.1186/s12913-017-2506-7

21.

Harris

Waller

Sloss

Williams

(2017). Sustainability in health care by allocating resources effectively (SHARE) 4: Exploring opportunities and methods for consumer engagement in resource allocation in a local healthcare setting. BMC Health Services Research, 17, Article 329. https://doi.org/10.1186/s12913-017-2212-5

22.

Harris

Turner

Wilkinson

(2016). SEAchange: Guide to a pragmatic evidence-based approach to Sustainable, Effective and Appropriate change in health services. http://arrow.monash.edu.au/hdl/1959.1/1225377

23.

Harvey

Kitson

(2016). PARIHS revisited: From heuristic to integrated framework for the successful implementation of knowledge into practice. Implementation Science, 11, 33. https://doi.org/10.1186/s13012-016-0398-2

24.

Ibargoyen-Roteta

Gutierrez-Ibarluzea

Asua

(2010). Guiding the process of health technology disinvestment. Health Policy, 98, 218–226. https://doi.org/10.1016/j.healthpol.2010.06.018

25.

McKay

V. R.

Morshed

A. B.

Brownson

R. C.

Proctor

E. K.

Prusaczyk

(2018). Letting go: Conceptualizing intervention de-implementation in public health and social service settings. American Journal of Community Psychology, 62, 189–202. https://doi.org/10.1002/ajcp.12258

26.

Michie

Johnston

Abraham

Lawton

Parker

Walker

, & on behalf of the “Psychological Theory” Group. (2005). Making psychological theory useful for implementing evidence based practice: A consensus approach. BMJ Quality & Safety, 14, 26–33. https://doi.org/10.1136/qshc.2004.011155

27.

Morgan

D. J.

Leppin

A. L.

Smith

C. D.

Korenstein

(2017). A practical framework for understanding and reducing medical overuse: Conceptualizing overuse through the patient-clinician interaction. Journal of Hospital Medicine, 12, 346–351. https://doi.org/10.12788/jhm.2738

28.

Nilsen

(2015). Making sense of implementation theories, models and frameworks. Implementation Science, 10, 53. https://doi.org/10.1186/s13012-015-0242-0

29.

Niven

D. J.

Mrklas

K. J.

Holodinsky

J. K.

Straus

S. E.

Hemmelgarn

B. R.

Jeffs

L. P.

Stelfox

H. T.

(2015). Towards understanding the de-adoption of low-value clinical practices: A scoping review. BMC Medicine, 13, Article 55. https://doi.org/10.1186/s12916-015-0488-z

30.

Norton

W. E.

Chambers

D. A.

(2020). Unpacking the complexities of de-implementing inappropriate health interventions. Implementation Science, 15, 2. https://doi.org/10.1186/s13012-019-0960-9

31.

Norton

W. E.

Chambers

D. A.

Kramer

B. S.

(2019). Conceptualizing de-implementation in cancer care delivery. Journal of Clinical Oncology, 37, 93–96. https://doi.org/10.1200/JCO.18.00589

32.

O’Brien

K. K.

Colquhoun

Levac

Baxter

Tricco

A. C.

Straus

Wickerson

Nayar

Moher

O’Malley

(2016). Advancing scoping study methodology: A web-based survey and consultation of perceptions on terminology, definition and methodological steps. BMC Health Services Research, 16, Article 305. https://doi.org/10.1186/s12913-016-1579-z

33.

Ouzzani

Hammady

Fedorowicz

Elmagarmid

(2016). Rayyan—A web and mobile app for systematic reviews. Systematic Reviews, 5, 210. https://doi.org/10.1186/s13643-016-0384-4

34.

Parchman

M. L.

Henrikson

N. B.

Blasi

P. R.

Buist

D. S.

Penfold

Austin

Ganos

E. H.

(2017). Taking action on overuse: Creating the culture for change. Healthcare, 5, 199–203. https://doi.org/10.1016/j.hjdsi.2016.10.005

35.

Patey

A. M.

Hurt

C. S.

Grimshaw

J. M.

Francis

J. J.

(2018). Changing behaviour “more or less” —Do theories of behaviour inform strategies for implementation and de-implementation? A critical interpretive synthesis. Implementation Science, 13, 134. https://doi.org/10.1186/s13012-018-0826-6

36.

Peters

M. D.

Godfrey

C. M.

Khalil

McInerney

Parker

Soares

C. B.

(2015). Guidance for conducting systematic scoping reviews. International Journal of Evidence-Based Healthcare, 13, 141–146. https://doi.org/10.1097/XEB.0000000000000050

37.

Powell

A. A.

Bloomfield

H. E.

Burgess

D. J.

Wilt

T. J.

Partin

M. R.

(2013). A conceptual framework for understanding and reducing overuse by primary care providers. Medical Care Research and Review, 70, 451–472. https://doi.org/10.1177/1077558713496166

38.

Prusaczyk

Swindle

Curran

(2020). Defining and conceptualizing outcomes for de-implementation: Key distinctions from implementation outcomes. Implementation Science Communications, 1, 43. https://doi.org/10.1186/s43058-020-00035-3

39.

Reschovsky

J. D.

Rich

E. C.

Lake

T. K.

(2015). Factors contributing to variations in physicians’ use of evidence at the point of care: A conceptual model. Journal of General Internal Medicine, 30(Suppl 3), 555–561. https://doi.org/10.1007/s11606-015-3366-7

40.

Schlesinger

Grob

(2017). Treating, fast and slow: Americans’ understanding of and responses to low-value care. The Milbank Quarterly, 95, 70–116. https://doi.org/10.1111/1468-0009.12246

41.

Schmidt

D. E.

(2010). The development of a disinvestment framework to guide resource allocation decisions in health service delivery organizations. The University of British Columbia. https://open.library.ubc.ca/cIRcle/collections/ubctheses/24/items/1.0073252

42.

Sucharew

Macaluso

(2019). Methods for research evidence synthesis: The scoping review approach. Journal of Hospital Medicine, 14, 416–418. https://doi.org/10.12788/jhm.3248

43.

van Bodegom-Vos

Davidoff

Marang-van de Mheen

. (2017). Implementation and de-implementation: Two sides of the same coin? BMJ Quality & Safety, 26, 495–501. https://doi.org/10.1136/bmjqs-2016-005473

44.

Verkerk

E. W.

Tanke

M. A. C.

Kool

R. B.

Van Dulmen

S. A.

Westert

G. P.

(2018). Limit, lean or listen? A typology of low-value care that gives direction in de-implementation. International Journal for Quality in Health Care, 30, 736–739. https://doi.org/10.1093/intqhc/mzy100

45.

Voorn

V. M. A.

Marang-van de Mheen

P. J.

Wentink

M. M.

Kaptein

A. A.

Koopman-van Gemert

A. W. M. M.

So-Osman

Vlieland

T. P. M. V.

Nelissen

R. G. H. H.

van Bodegom-Vos

, & for the LISBOA Study Group. (2014). Perceived barriers among physicians for stopping non-cost-effective blood-saving measures in total hip and total knee arthroplasties. Transfusion, 54, 2598–2607. https://doi.org/10.1111/trf.12672

46.

Voorn

V. M. A.

van Bodegom-Vos

So-Osman

(2018). Towards a systematic approach for (de)implementation of patient blood management strategies. Transfusion Medicine, 28, 158–167. https://doi.org/10.1111/tme.12520

47.

Willson

(2015). The problem with eliminating “low-value care. .” BMJ Quality & Safety, 24, 611–614. https://doi.org/10.1136/bmjqs-2015-004518