Mechanism-Based Middle-Range Theories: Using Realist Syntheses to Reconcile Specificity to Context and Generalizability

Abstract

Realist synthesis is a recognized methodological approach to evidence synthesis to inform evidence-based health policy and practice. The implicit assumption behind research synthesis is that the evidence it generates should be generalizable––drawing broad inferences from specific observations. While this understanding is generally shared among social scientists, tensions exist between having generalizable evidence and how this evidence can be useful in specific contexts. This paper considers the role of mechanism-based middle-range theories obtained from realist synthesis in bridging specificity to context and generalizability. Retroductive theorizing in realist synthesis helps to identify ideas about mechanisms related to the phenomenon embedded in the social and organizational contexts that could, in principle, have a much broader application. Also, because mechanism-based middle-range theories are linked to contextual features, they capture contextual nuances to enhance evidence implementation. We conclude that middle-range mechanisms provide an opportunity to achieve generalizability and contextualization in implementation science.

Background

Implementation science aims to improve the adoption of evidence-based interventions into practice. To achieve this feat, evidence of successful and indeed failed implementation of evidence-based interventions is required to improve the implementation outcomes of various strategies. There is a consensus among implementation scientists that the evidence informing the successful implementation of an evidence-based intervention should generate insights that apply widely and are useful for prediction. Contrary to generalizability and predictability, implementation scientists also generally understand that different contexts offer varying contextual conditions, with unique insights about the effects of implementation strategies and unique barriers, facilitators, or even mechanisms (Irie et al., 2023). To this end, Brownson et al. (2022) conclude that while we need generalizable evidence, implementation scientists should maintain their commitment to context-based decision-making. The challenge of achieving generalizability and contextualization in tandem remains a source of tension; focus on the scientific endeavor of widely applicable findings or focus on the primacy of context. The question then is how do we achieve this dual goal of generalizability and contextualization to improve implementation practice?

Mechanism-based theorizing has been proposed as a possible solution to achieve the goals of contextualization and generalization in implementation science (Geng et al., 2022; Kislov et al., 2019). Recent efforts have even further elucidated the functional relationships between implementation determinants (including preconditions), causal mechanisms, and how contextual variables influence outcomes through causal pathway diagrams (Lewis et al., 2018; May, 2013). Despite various attempts to theorize how and why implementation strategies achieve various outcomes, a fundamental gap in understanding the causal mechanisms by which strategies for implementing evidence-based practices address local barriers to effective, appropriate service delivery remains (Geng et al., 2022). For example, Lewis et al. (2022) developed a causal pathway diagram to investigate the role of mechanisms in perpetuating the impact of implementation strategies (Figure 1). Causal pathway diagrams are visualization tools for depicting and theorizing about the causal process through which strategies operate, including the mechanisms they activate, the contextual conditions that influence them (i.e., preconditions and moderators), and the proximal and distal outcomes they produce. Using mechanism-focused causality-based approaches, implementation scientists can theorize how implementation strategies succeed or fail to engage key mechanism(s) or contextual factors leading to the observed outcomes, and to what extent the strategy’s causal process unfolds successfully.

Figure 1.

The mechanism-focused causal pathway diagram (Lewis et al., 2022).

Realist methodologies have equally gained prominence for their focus on mechanism-based theorizing in implementation science. One of such realist methodologies is realist synthesis or review––henceforth realist synthesis (Rycroft-Malone et al., 2012). A realist synthesis is a theory-driven, realist-informed interpretive approach to synthesizing secondary empirical data to inform evidence-based practice and explain social phenomena (Hunter et al., 2022; Mukumbang et al., 2025). Realist syntheses were originally introduced to improve evidence-based policymaking (Pawson, 2002; Pawson et al., 2005). Pawson (2018) also argued that evidence-based medicine is fundamentally realist in its methodological underpinnings. Realist methodologies including realist synthesis have become quickly adopted in evidence-based medicine and policymaking––the hallmark of implementation sciences. Owing to their potential to improve our understanding of how, why, for whom, and under what conditions intervention, implementation strategies, and policies work, the number of realist syntheses conducted in health-related fields in the last decade has increased (2013––42; 2018––104; 2023––194).

Contrary to conceptualizing a mechanism as a process or chain of events, a mechanism is not a particular program component or process from the realist perspective. Rather, they are the participants’ reactions to the resources (information, opportunities, and constraints) the implementation strategies offer (Dalkin et al., 2015; Pawson & Tilley, 1997). For example, a mechanism can be an “increased confidence” in healthcare workers (reaction) due to friendly coaching (resource). Therefore, the key explanatory element in realist evaluation is the generative mechanism, which elucidates the reasoning the actors attribute to the resources, opportunities, and/or restraints provided by the intervention that leads to action (Mukumbang et al., 2017a, 2017b).

Theory building in realist synthesis is achieved by formulating program theories––how and why implementation strategies work by hypothesizing the context, mechanism, and outcomes at play. While mechanisms are central to causal explanations in realist synthesis, formulating an initial program requires a “generative” approach to causation. Pawson and Tilley (1997) identified context, mechanisms, and outcomes as the key elements for explaining how and why programs succeed or fail. They proposed that an outcome (O) is generated when a mechanism (M) is triggered in a specific context (C) (Pawson & Tilley, 1997). A realist program theory as relates to implementation strategies should explicate how an implementation strategy (S) changes a particular context (C) and how it produces observed implementation outcomes (O) through the generative mechanisms (M) triggered (Figure 2). This paper will use strategy–context–mechanism–outcomes configurations (SCMOCs) as a heuristic tool for formulating mechanism-based program theories.

Figure 2.

Realist conceptualization of causation. Inspired by Pawson and Tilley‘s (1997).

In engaging with generalizability and contextualization in implementation science, Irie et al. suggest considerations of “theoretical perspectives on context and generalizability, including the mobilization of critical theory, transportability, and/or realist approaches” (Irie et al., 2023). Focusing on this proposition, we adopted a realist-informed perspective to theoretical generalizability as an opportunity to reconcile specificity to contexts and generalizability in implementation research. In this paper, theoretical generalization––the abstraction of content and concepts to obtain a unitary form of the occurrences to explain the emergence of an observation or outcome––is presented as a possible opportunity to address the generalizability-contextualization tension. It will be illustrated that theoretical generalizability achieved through retroductive theorizing (unearthing the underlying causal entities––i.e., mechanisms––to explain how an observation occurs) and the construction of middle-range theories provide elements that can play a crucial role in causal narratives about other concrete cases. We (1) discuss the different types of generalizability, identifying the type relevant to developing mechanism-based middle-range theories; (2) define and discuss the nature of middle-range theories and explain how and why they enable generalization in implementation research; and (3) use an example of a realist synthesis of the implementation of community-based interventions and strategies to illustrate how we developed middle-range mechanism-based theories and their potential applicability.

Main Text

Types of Generalizability

Informed primarily by pragmatism, health systems research commonly applies mixed-methods research approaches to understand the strategies and processes needed to implement an evidence-based change (Palinkas et al., 2011). Quantitative methods are fundamental used to explore the extent and variation of change (within and across units) caused by the implementation strategies and use generalization as a quality standard (Polit & Beck, 2010). Generalization of evidence obtained using quantitative methods is achieved mainly by applying statistical methods to extrapolate from a sample to a population. The pragmatic use of qualitative methods informed by interpretivism/constructivism in implementation science generates evidence that is not amenable to “generalizability” to a larger population using extrapolation.

Generalizability is a reasoning approach that involves drawing broad inferences from specific observations. Nevertheless, generalizability has different meanings when adopting different research methods or approaches. While adopting quantitative methods, the quantitative researcher strives to achieve external validity––the extent to which research findings can be applied to other settings. Although qualitative research has been lauded for its ability to capture contextual characteristics with better nuance than quantitative research, owing (for example) to the small and carefully selected samples––which are not necessarily meant to be representative of the population––generalizability in terms of extrapolations from sample to a population is not considered a strength of qualitative research (Smith, 2018). Because generalizability using extrapolation as applies to quantitative research is never the goal of qualitative research, there are arguments that this notion of generalizability should not be considered a weakness of qualitative research (Morse, 1999). Qualitative studies are not conducted to generalize findings but to provide a rich and contextualized understanding of some aspect of human experience and perception of an implementation strategy in particular cases (Polit & Beck, 2010).

To address the confusion regarding generalizability and contextualization, Firestone (1993) developed a typology of generalization, which we have adopted to guide our arguments. These are (1) classic sample-to-population generalization, (2) analytic or theoretical generalization, and (3) case-to-case transfer. In classic sample-to-population generalization, findings in a study are relevant elsewhere because of the close similarities between the sample characteristics and the targeted population. In case-to-case transfer, the warrant for transferability rests on judgments about how similar the research setting is to another. In analytic generalization, especially for mechanism-based theory-driven research methods or approaches like realist methodologies, the warrant is that the same causal forces theorized in the research setting are also found in other settings.

Classic Sample-to-Population (Statistical) Generalization

The classic understanding of generalization relates to statistical generalization, extrapolating from a sample to a population. First, comparability of the sample and the study population is ensured using demographic characteristics and randomly selecting a sample similar in characteristics to those in the population. This kind of generalization usually requires obtaining a representative sample through randomization, giving every member of the population an equal chance to be included in the study with a determinable probability of selection (Polit & Beck, 2010). Individual-level randomization is primarily possible and applied in drug trials dealing with efficacy. In individually randomized drug trials, judgments about differences in health status and morbidity between trial participants and other populations are generally more straightforward, and routine monitoring of drug use after licensing can facilitate a judgment of generalizability (Eldridge et al., 2008). While this kind of generalization is the aspiration of implementation scientists, it is hardly achieved in real-life situations, especially in implementation research dealing with intervention effectiveness. Consequently, implementation scientists mainly adopt other sampling forms, such as cluster randomization.

Since implementation strategies usually target organizations, such as clinics, hospitals, and hospital departments, cluster randomization is considered more appropriate than individual patient randomization (Foy et al., 2023). For example, by applying the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework, the concerns of external validity (generalizability) for the implementation scientist would be reach, and adoption, implementation, and maintenance. For example, to judge adoption (the extent to which the settings included are representative of a broader population of settings), eligibility criteria for clusters and numbers of clusters randomized and analyzed are essential considerations for discussing the generalizability of trial findings to clusters and individuals (Eldridge et al., 2008).

One key element creating uncertainty around the generalizability of findings obtained through cluster randomized trials is the lack of knowledge about how clusters with different characteristics respond to different health interventions. A better understanding of the factors affecting implementation in different circumstances and among different clusters can clarify implications for generalizability (Eldridge et al., 2008). This understanding leads to a second model of generalizability, the case-to-case transfer, transferability, or moderatum generalizations (Payne & Williams, 2005), which are mainly adopted by researchers who adopt qualitative research methods.

Case-to-Case Transfer (Transferability)

Transferability refers to the extent to which the findings and interpretations derived from a qualitative study can be applicable and relevant to other contexts beyond the specific research context. Thick description is described by Lincoln and Guba (1985) as a way of achieving a type of external validity. By describing a phenomenon in sufficient detail, we can evaluate the extent to which the conclusions drawn are transferable to other times, settings, situations, and people. To achieve transferability, the implementation researcher must provide the readers with evidence that the research study’s findings could apply to other contexts, situations, times, and populations, for example. Improving the possibility of transferable qualitative research findings requires a detailed description of the research context, including the participants, setting, and cultural nuances. A detailed contextual description allows readers to understand the unique features and characteristics of the study, facilitating the assessment of its relevance and applicability to other contexts.

Transferability can be achieved by contextualizing the research findings. Contextualization clearly illustrates the unique contextual factors that may have influenced the research findings, such as the broad range of background features, aspects of the implementation strategy studied, and implementation outcomes. Such articulation also includes describing specific cultural, social, or environmental elements that constitute the research context. Contextualizing research findings enables readers to assess the transferability of the findings by considering the similarities and differences between the study context from which the evidence is generated.

Qualitative research designs and applying qualitative methods in implementation science tend to be mainly deductively informed by implementation theories, models, or frameworks (Moullin et al., 2020). Qualitative methods in implementation science also tend to explain specific implementation-related phenomena generated using quantitative methods within mixed methods rather than qualitative methods alone (Hamilton & Finley, 2019). In this way, the findings are not sufficiently contextualized as there is less nuance capture of complexity and the dynamic nature of interventions, implementation processes, and intersecting implementation contexts and their interactive effects on outcomes. This is because, in mixed-methods research, qualitative findings typically explain the quantitative findings or provide an alternative perspective to understanding the phenomena.

Analytical or Theoretical Generalization

Firestone’s (1993) analytical generalization involves making projections about the likely transferability of findings obtained from a case study based on a theoretical analysis of the factors producing outcomes and the effect of context. Yin (2017) espoused analytical generalization by suggesting that it allows the researcher to compare (or “generalize”) case study data to existing theory. Theoretical generalization entails using data from particulars to develop broader constructs or theories. For instance, meta-ethnography can be used to achieve analytic generalization through its structured, interpretive approach to generate theories and novel ideas (Soundy & Heneghan, 2022). For the theory to be generalizable, it should apply beyond the immediate group and become applicable to all similar situations, questions, and problems, regardless of the comparability of the demographic composition of the groups. Staying on the track of theoretical generalizability, Pawson (2024) defines generalizability as “the process of extending a specific explanation of a specific instance to a wider or universal set” (p. 221). As applied to implementation science, theories need to incorporate elements associated with the implementation strategy, the process of evidence implementation, the contextual conditions under which implementation is taking place, and the expected implementation outcomes. Most importantly, these theories need to specify the focal level of interest for research (Verran, 1997), illustrating their potential contribution to generalizability.

Theorizing involves drawing on empirical data to develop, validate, modify, or expand theoretical explanations (Meza et al., 2023). Formulating generalizable theories, therefore, requires obtaining evidence from various sources (Morse, 2023). Such evidence can be obtained from qualitative, quantitative, or both sources. In applied qualitative inquiry, implementation scientists can arrive at insightful inductive generalizations regarding the phenomenon under study using rigorous inductive analysis and abductive approaches (Morse, 1999). These qualitative studies draw from the expertise of practitioners who understand the implementation challenges of the evidence-based intervention needing to be addressed and formulate informal theories from their repertoires of concrete examples, understandings, and actions gleaned from experience. Quantitative methods can be used to construct theories and provide the feasibility of measuring concepts or variables in a theory.

The construction of theory in implementation research follows two broad approaches: (1) starting from a validated theory to inform data collection and analysis (theory → data) and (2) starting from the data collected in the program context to generate theories (data → theory) that explain how and why different components of health intervention interacting with the implementation context can potentially improve implementation outcomes. Substantive and implementation theories employed in implementation research mostly reflect the first approach. These theoretically informed approaches (theory → data) are mostly applied in implementation science as fidelity assessment, process evaluation, and complex evaluation (Ridde et al., 2020) and, to a lesser extent, to foster the development of a valuable body of explanatory theories. Kislov et al. (2019) explained that these substantial theories are usually applied to design an intervention or to systematize or explain process evaluation findings. For example, Aunon et al. (2020) used the Information–Motivation–Behavioral Skills (IMB) theoretical model to develop a nurse-delivered mHealth intervention to support antiretroviral therapy adherence among female sex workers living with HIV in Mombasa, Kenya. Kislov et al. (2019) explained further that “implementation researchers tend to treat theoretical knowledge as ‘received wisdom’ to be applied with reverence rather than challenged, developed, and moved forward” (p. 5).

Kislov et al. (2019) confirm an inclination to theoretically informed (theory → data) research over theory-driven research approaches (data → theory) in health research. Kislov (2019) suggested that implementation researchers often apply theory to shape data collection and analysis, and little effort is made to use data collection and synthesis and the resulting empirical findings for theory development. Some approaches, like theory-driven research, use data to inform theory and theory to inform evaluation design and data collection (Verran, 1997). According to Cash (2018), “A ‘good’ theory-driven research allows researchers to build a corpus of robust scientific knowledge that is valuable internally and accessible externally” (p. 87). Therefore, theory-driven research can potentially develop and test (confirm, refute, or refine) generalizable theories.

The goal of generalizing through mechanism-based approaches is to look for ideas about mechanisms related to implementation embedded in the social and organizational contexts that could, in principle, have a much broader application in implementation science. Developing mechanism-based middle-range theories is an important and central part of realist-based retroductive theorizing. Drawing on realist concepts about mechanisms provides a more explicit and coherent warrant for why case findings may be transferable to, for example, other settings. Broderick et al. (2024) demonstrated developing mechanism maps through realist thematic analysis method, which provided a retrospective framework to understand causal relationships between factors driving intervention successes or failures. They argued that these mechanism maps can inform the generalizability of health programs by linking them to contextual factors that are central to implementation (Broderick et al., 2024). The key assumption here is that mechanisms are inherent and widely occurring causal processes that may be triggered when the necessary contexts are present. These mechanisms then go on to produce outcomes. For example, gravity is a widely occurring mechanism and, under the correct contexts, will act to pull two objects with mass toward each other. Therefore, mechanism-based theorizing provides a basis for generalization from case studies in which there exists evidence of a widely occurring mechanism.

Retroductive Theorizing and Contextualization

Generally, contextualization refers to “linking observations to a set of relevant facts, events, or points of view that make research possible and theory that form part of a larger whole Rousseau & Fried, (2001. p 1).” In qualitative research, “contextualization” involves deeply understanding and describing the environment, social factors, and circumstances that influence participants’ experiences and behaviors. This practice allows researchers to interpret data more meaningfully by considering the broader context in which it was collected. Essentially, contextualization places research findings within relevant social, cultural, and historical settings, leading to a richer interpretation of the data. According to Halme et al. (2024), researcher’s philosophical commitments also determine how context is operationalized. While contextualization in constructivism shares the same meaning in realist inquiries, they are conceptualized differently. Contextualization in realist methodology entails considering the relevant contextual features in relation to the mechanisms of action.

Formulating mechanism-based theories in realist-informed inquiries is achieved through retroduction. Retroductive theorizing is concerned with building mechanism-based theories or explanatory models. It is, therefore, a theory-building approach to uncovering the underlying causal mechanisms responsible for an outcome (Mukumbang, 2023). This theorizing process involves developing explanatory “models” based primarily on identifying the relevant context conditions that trigger or suppress the actions of mechanisms to cause an outcome. It is, therefore, a mechanism-based approach to theorizing, which involves unearthing and making explicit the mechanisms and other conditions that must exist to explain an observed phenomenon, such as the adoption, reach, or fidelity of an implementation strategy (Lawson, 1997). The researcher seeks to clarify the causal processes––that is, mechanisms essential for a phenomenon to occur (Meyer & Lunnay, 2013).

Retroductive theorizing is achieved by formulating context–mechanism–outcome configurations (CMOCs). These explicitly set out the relationship between what functions as context, the causal process (i.e., mechanism), and an outcome of interest for any case being studied. Context conditions or characteristics often considered when conceptualizing realist theories include: (1) the individual capacities of the key actors and stakeholders, such as interests, attitudes, knowledge, and skills; (2) the interpersonal relationships required to support the intervention, such as lines of communication, management, and administrative support, as well as professional relations and contracts; (3) the institutional setting in which the intervention is implemented, such as the culture and norms, leadership, and governance of the implementing body; or (4) the wider (infra-)structural and welfare system, such as political support, the availability of funding resources, and competing policy priorities and influences (Nielsen et al., 2022). These aspects of context are conceptualized in retroductive theorizing as: (1) observable features (space, place, people, things) that trigger or block the intervention, and (2) the relational and dynamic features that shape the mechanisms through which the intervention works (Greenhalgh & Manzano, 2021).

Middle-Range Theories and Generalizability

Implementation science and the extant literature describe three levels of theories: grand, mid-level or middle-range, and lower-level theories (Davidoff et al., 2015; Kislov et al., 2019). Grand theories focus on the functions, operations, and interrelatedness of social constructs relevant to implementation science. Although the implementation science field borrows theories from many other fields (Nilsen, 2015), empirical studies relying on grand theories are rare in the implementation science literature. For example, the theory of diffusion is considered one of the most influential theories in the broader knowledge utilization field, of which implementation science is a part (Nilsen, 2015), is considered a middle-range theory. The theory of diffusion emphasizes the crucial role of intermediary actors, such as opinion leaders, change agents, and gatekeepers, in ensuring the successful adoption and implementation of ideas or innovations. The normalization process theory (NPT), which conceptualizes evidence-based implementation, focusing on embedding and integrating innovation in healthcare settings, is also described in the implementation literature as a middle-range theory (Huddlestone et al., 2020). It deals with a generalizable empirical problem, what implementation processes are, and what specific mechanisms shape and motivate them––making no claims beyond this. Sonia Dalkin and colleagues’ recent realist review of NPT studies is helpful (Dalkin et al., 2021).

Program theories developed by formulating case-specific CMOCs from single case studies are considered lower-level theories (Kislov et al., 2019). A program theory comprises a collection of statements describing a specific program, clarifying the reasons, methods, and circumstances under which the program’s effects happen, anticipating the program’s outcomes, and outlining the necessary prerequisites to achieve the intended program effects (Sharpe, 2011). Due to its context-specific nature, a program theory will provide a conceptual basis for refining and improving existing programs and support inferences about new programs. Thus, it may still need to undergo further refinements to improve their generalizability.

In implementation science, empirical studies relying on grand theories may be at risk of becoming absorbed in the pre-existing all-encompassing master schemes, limiting our abilities to develop data-driven theories (Kislov, 2019). Grand theories predominantly favor highly abstract theorizing (Ponjaert, 2021), thus mainly reproducing prior theoretical understanding, distinct from concrete empirical concerns. Therefore, grand theories are too abstract to capture the nuances of contextual understanding to capture implementation, and program theories are considered too specific to the context in which they are developed. Middle-range theories have been proposed as a balance between context-specific (program) and grand theories with an improved potential for generalizability owing to the mid-level abstraction––conceptualization of implementation processes and outcomes. According to Kislov et al. (2019), middle-range theories “lie between the working hypotheses that evolve in abundance during day-to-day research and the all-encompassing speculations comprising a master conceptual scheme” (p. 2). Thus, they bridge theories and program theories (Brodie et al., 2011).

Robert K. Merton is credited for conceptualizing middle-range theories in his seminar work “Social Theory and Social Structure” (Merton, 1968). He described the middle-range theory as involving “abstractions, of course, but they are close enough to observed data to be incorporated in propositions that permit empirical testing” (Merton, 1968). The “Middle” in “middle range” concerns abstraction and generality (Cartwright, 2020). Merton’s goal for conceptualizing middle-range theory was to link non-substantive concepts to form explanations of diverse social phenomena––generalizability (Pawson, 2000). Merton’s “middle-range theory” was seen as the most appropriate alternative to grand theories to explain social phenomena. Hedström and Bearman (2011) confirmed that middle-range theories do not claim to explain all social processes but provide…

a clear, precise, and simple type of theory which can be used for partially explaining a range of different phenomena, but which makes no pretense of being able to explain all social phenomena … It is a … toolbox of semi-general theories, each adequate for explaining a limited range or type of phenomena. (Hedström & Bearman, 2011, p. 4)

Middle-range theories do not seek to explain the whole in a single theory but seek to develop explanations for the parts that make up the whole (Pawson, 2000). While middle-range theories are narrower in scope and less abstract than grand theories, they are also less specific to a particular context, although informed by specific contexts. Merton envisaged what middle-range theories should look like and how they can be applied in the greater scheme of theories, but he provided little guidance on developing them. Pawson (2000) found no framework for middle-range hypothesis generation and highlighted the need to frame initial research questions to capture middle-range explanations.

Middle-range theories can be formulated by consolidating otherwise segregated hypotheses and empirical regularities (Merton, 1968). Robert et al. (2017) used a realist approach to develop a middle-range theory on free public healthcare seeking in sub-Saharan Africa, building on Sen’s capability approach. Through an iterative process, they explored theoretical literature on healthcare access, services use, healthcare-seeking behavior, and user fee exemption policies. Applying retroductive theorizing, they formulated a middle-range theory on seeking free public healthcare in sub-Saharan Africa (Robert et al., 2017).

Of course, theoretical generalizability approaches like retroductive theorizing aim to develop broader constructs from specific cases or through the triangulation of evidence from different sources. In realist-informed research, moving from the specific program theories to a more abstract theory is known as analytical generalization (Mukumbang, 2018; Mukumbang et al., 2018). Analytic generalization, as applied to retroductive theorizing, entails applying counterfactual thinking (testing possible alternative explanations) to argue toward transfactual (mechanism-centered) conditions (Mukumbang, 2018). The process of analytic generalization in retroductive theorizing is known as retrodiction and entails investigating what mix of causal powers interacted in what way to produce any particular event (Elder-Vass, 2015). Mukumbang (2023) states, “Retrodiction is the systematic comparison of explanations obtained from different cases toward a more refined theory” (p. 107). Retrodiction is about abstracting and accentuating––highlighting the most prominent–– mechanisms identified from various sources of evidence and individual program theories. These abstracted mechanisms can be described as middle-range mechanisms. The causal mechanisms obtained from retrodiction inform the formulation of middle-range theories (Figure 3).

Figure 3.

SCMOCs of a realist-informed mechanism-based middle-range program theory Inspired by Mukumbang (2023).

Whatever approach is applied to develop them, middle-range theories should have three characteristics: (1) sufficient abstraction, (2) logical derivation, and (3) adaptive and cumulative explanations. Merton talked about a “sufficient” level of abstraction in his definition of middle-range theory. Owing to the uncertain description of “abstraction,” it is therefore considered a workaday or a heuristic tool of theory building. However, any causal account of an event (the behavior of a given entity at a given time) seen in level abstracted terms forms only part of a larger picture. A middle-range explanation can always be provided by re-integrating different but related events into the larger stratified picture of which they form a part. By doing so, we can capture causal configuration patterns across contexts, allowing us to explain demi-regularities. In realist syntheses, demi-regularities are “frequently reproduced behaviors or patterns that get seen in human activity and can emerge in the setting … as theme-type things seen across different studies” (Phillips, 2016). The explanations of each demi-regularity (independent but related event) must be linked to the other events in which it is inextricably implicated, either as a subset or superset (Elder-Vass, 2006).

Regarding adaptive and cumulative nature, the integration of explanations of different events makes middle-range theories sufficiently general to explain outcomes across settings and social activities. Thus, a fruitful middle-range theory will harness and elucidate many different empirical instances—for example, reference group theory. By logical derivation, Merton (1968) suggested that middle-range theories should have “limited sets of assumptions from which specific hypotheses are logically derived and confirmed by empirical investigation” (p. 68). Middle-range theories derived using realist-based retroductive theorizing deal with causal explanations by explaining the underlying mechanisms that give rise to observations (Lawson, 1997). These causal explanations take the form of SCMOCs and should be written at a middle level of abstraction: specific enough to clearly explain the phenomenon and general enough to apply across cases of the same type.

Mechanism-Based Middle-Range Theory: An Illustrative Example

To illustrate the development of middle-range theories through retroductive theorizing, we considered a recent realist synthesis conducted to understand how and why engagement or participatory practices enhance or hinder the adoption of non-communicable disease (NCD) research and interventions in low- and middle-income countries (Klingberg et al., 2021, 2024). Our review captured scientific and grey literature spanning 2013–2020 to align with the World Health Organization’s Global Action Plan for Prevention and Management, which emphasized the importance of involving communities in addressing NCDs (McCormack et al., 2013). Seventeen articles describing 15 studies from low- and middle-income countries were included in the realist synthesis. Our observation is that in most of the studies included in the reviews, which allowed us to identify context and mechanisms, the authors used predominantly qualitative research methods to explore and evaluate the community engagement strategies.

The realist synthesis drew on theoretical and empirical literature beyond specific diseases or settings while focusing on informing NCD research and interventions in low- and middle-income countries (Klingberg et al., 2021). The synthesis of existing literature was complemented using realist interviews––theory-driven interviews focused on eliciting, testing, and/or confirming mechanism-based theories (Mukumbang et al., 2020)––and stakeholder consultation. We drew on available realist review principles and reporting standards (Booth et al., 2020; Dada et al., 2023; Jagosh, 2019; Wong et al., 2014). Our review synthesis comprised iterative cycles of literature searches, screening, data extraction, theorizing, and stakeholder feedback through realist interviews, undergirded by retroductive theorizing––Figure 4 (Klingberg et al., 2021).

Figure 4.

Applied retroductive process to middle-range theory development Conceptualized from Klingberg et al. (2024)

Through the iterative realist synthesis process outlined in Figure 4, we formulated a middle-range theory showing why and how community engagement in NCD research and intervention can lead to tailored interventions for the community. We created a configurational middle-range theory map to illustrate how and why community engagement occurs. We accomplished this by extracting four middle-range mechanisms from the data through the process of abstraction and accentuation, which involves emphasizing the most significant mechanisms––middle-range mechanisms and the most significant outcomes observed (Figure 5). Mechanism accentuation follows that certain mechanisms take precedence over others and occur more frequently, becoming evident in demi-regularity phenomena (Hedström & Swedberg, 1996). In step 5 of Figure 4, an important part of our analysis was re-integrating different but related events using SCMOCs into the configurational map. By doing so, we captured causal configuration patterns across contexts, allowing us to explain the reported outcomes.

Figure 5.

Mechanism-based middle-range theories of NCD research and interventions adoption in LMIC settings (Klingberg et al., 2024).

Our middle-range theory represents five strategies implemented to improve community engagement in NCD-related research and interventions: (1) community needs focused intervention, (2) responsive and humble implementation team, (3) support, facilitation, and empowerment for community engagement, (4) inclusion of key community actors, and (4) community-led participation. According to our middle-range theory, these strategies will introduce four abstracted mechanisms: (1) solidarity and unity, (2) community agency, (3) community empowerment, and (4) financial and reputational motivation. These four mechanisms are each triggered by relevant context conditions, as illustrated in the middle-range model. In terms of generalizability, our findings show that for community engagement to occur in a particular context, these four mechanisms must be triggered. Identifying the relevant context conditions associated with the triggered mechanisms speaks to the contextualization component, achieved through retroductive theorizing. Our middle-range theory also illustrates that the accentuated mechanism causes three interrelated outcomes: community ownership of intervention, relevant and applicable findings and interventions, and tailored research to community needs.

Community agency is ubiquitous in every community and mobilizes communities to achieve a common goal when triggered. The theory suggests that community participation in research activities and adoption of the intervention increases when community agency is activated by context conditions such as their previous exposure and/or preference for the NCD promotion research and intervention. The value of identifying community agency as a mechanism and developing an understanding of its behavior under different contexts is that it enables us to make claims about generalizability. For example, if we were seeking to understand the success or failure of implementing an NCD promotion intervention in a particular setting, we can infer that the same mechanism and SCMOCs may be in operation. As the SCMOCs are expressed in the middle range, we can gather data to test if our inference is confirmed, refuted, or needs to be refined.

We also identified prior exposure or preference to research or intervention triggers community agency. This observation suggests that community members are only interested in contributing to research in specific ways. In that case, they might not participate if the approach adopted by the research team does not align with their preferences, which can lead to the mechanism of community agency not being triggered. Such information is helpful for the contextualization of the evidence in a new setting. For instance, knowing that the activation of community agency depends on the community’s preference concerning NCD promotion research endeavors and interventions can guide the research team to engage with the local community to establish their preferences. This information can help researcher teams to be realistic and transparent about what can be accommodated, pursuing institutional flexibility where possible. Power sharing should be pursued where feasible, welcoming views from as early as possible where the research agenda or intervention focus can still be adjusted (McCormack et al., 2013).

Discussion

While Yin (2017) and others believe that findings from studies conducted in certain settings or contexts can be generalizable, researchers such as Stake (1995) and Thomas (2010) argue that these studies are meant for “particularization, not generalization” (p. 8). Thomas (2010), who rejects the concept of theoretical generalizability in context-specific research, insists that “the goal of social scientific endeavor, particularly in the study of cases, should be exemplary knowledge…that can come from [the] case…rather than [from] its generalizability” (p. 576). Thomas (2010) further asserts that simply generalizing the findings from a specific setting can dampen the researcher’s “curiosity and interpretation” of the outcomes. This discussion on the role of a research setting typifies the tension between generalizability and specificity in the context of implementation science. The argument presented in this paper is that mechanism-based middle-range theories reconcile or strike some balance between retaining the nuances of context captured in the case study and generalizing the findings to other populations and contexts.

Implementation scientists recognize that evidence generated from a specific setting with contextual characteristics should ideally be applicable in other contexts from which the evidence was generated. On the other hand, attempting to implement evidence-based practices in contexts other than those from which the original evidence was generated is complex and challenging and most often requires contextual adaptation (Chambers & Norton, 2016; Eccles et al., 2009). This paper proposes that mechanism-based middle-range theories can serve to achieve generalization and contextualization. Achieving these two objectives simultaneously through mechanism-based middle-range theories is not only about the mid-level abstraction of the theories but also the mechanisms surfaced through retrodiction. The warrant for generalizability is based on the premise that the same health intervention may trigger the relevant and necessary mechanisms needed for desired outcomes. Contextualization would require considering those context conditions relevant to triggering or drowning out the postulated mechanisms. Mechanism-based middle-range theories can improve implementation science practice by increasing attention to surfacing mechanisms responsible for desirable outcomes while identifying critical context conditions required for such outcomes to occur––hence providing knowledge that helps to inform successful implementation.

According to Ylikoski (2019), “Mechanism-based theorizing provides a fruitful basis for understanding … social phenomena” (p. 14). We aimed to make a case that middle-range mechanisms developed through retroductive theorizing to explicate how and why health intervention with similar modalities works and under what health systems and contextual conditions. Next, we demonstrate how these actors assimilate the impact of the intervention components. Our middle-range mechanisms postulate how, through the reasoning of actors in reaction to similar implementation strategy modalities, they are likely to adopt the actions and interactions that generate the desired implementation outcomes. This property captures the potential of generalizable middle-range mechanisms. In the same way, identifying which context conditions are likely to trigger these middle-range mechanisms enhances the transferability or contextualization potential of the middle-range theories.

An observation we have made that may be of interest to the qualitative audience is that within realist methodology, information about mechanisms, context conditions, and their interactions to activate outcomes was mostly obtained using qualitative data collection methods. This finding is supported by a mapping review conducted by Renmans & Castellano Pleguezuelo (2023), which revealed that qualitative methods are the most used in realist evaluations with an estimated 97% of all realist evaluations using some kind of interviews. Qualitative methods such as grounded theory have a focus which is shared and overlaps with realist approach of theorizing as both approaches are interested in theory development (Oliver, 2012). Therefore, we argue that applying qualitative data collection methods can contribute to balancing contextualization and generalization through the realist philosophy of science. For this to be effective, these qualitative methods must be divorced from their constructivist ontology and aligned to the realist philosophy of science as several authors have attempted to do in recent times with interviews, focused groups discussions, and photovoice (Manzano, 2016, 2022; Mukumbang & Dada, 2024).

The implicit assumption behind research synthesis is that the evidence they generate should be generalizable—drawing broad inferences from specific observations and applying them to a specific context to inform practical application—contextualization. In terms of review ontology, systematic reviews with meta-analysis call for two things: careful classification of the programs under consideration and establishing their effects. Generalizability in this instance relates to achieving “heterogeneous replication” (Pawson, 2002). On the other hand, the ontology of narrative reviews leans on several factors about the program and the context within which it was implemented, along with information on outcomes and on the methodology employed in the original evaluations. Consequently, generalizability in narrative reviews aims to achieve “proximal similarity” (Pawson, 2002), including similarity in context. Evidence obtained from realist synthesis is quintessential of mechanism-based middle-range theories focused on providing causal explanations based on “generative mechanisms” and its “contiguous context.” This paper considers the role of realist synthesis in addressing the generalizability-contextualization divide through “transferable theory” formulated as mechanism-based middle-range theories in bridging specificity to context and generalizability.

Developing realist-informed mechanism-based theories in implementation research should provide robust strategy-context-mechanism-outcome linkages. The linkage should illustrate the connection between the health intervention modalities, its influence on context and the actors’ reasoning leading to their actions and implementation outcomes. In this way, mechanism-based middle-range theories can be used to design or understand health interventions and implementation strategies. The implementation strategy design can be enhanced using strategy–context–mechanism–outcome linkages and full causal pathway diagrams, resulting in more reliable evidence on the most effective strategies for different contexts. According to Huybrechts et al. (2024), organizational theories in implementation frameworks can engage with context conditions meaningfully beyond the notions of barriers and facilitators to explain how context conditions shape implementation processes. According to Broderick et al. (2024), using a retrospective approach to creating mechanism maps (retroduction) can improve our understanding of causal relationships between factors driving intervention successes or failures. We argue that through strategy–context–mechanism–outcome linkages, multi-component and multilevel strategies can be optimized by focusing on the relevant context conditions that effectively engage key mechanisms. Also, when implementation strategies fail, implementation scientists can investigate the reasons by examining whether the strategy failed to engage key mechanisms or if contextual factors failed to trigger the mechanisms.

Conclusion

Implementation scientists are torn between the generalizability of evidence generated through empirical research to populations or contexts other than where they were generated or focusing on achieving contextualization (transferability) through a detailed reporting of studied cases. This dilemma has led to the generalization–contextualization dualism. We proposed that formulating mechanism-based middle-range theories about causation in the form of SCMOCs in implementation science offers an opportunity to address this dualism. Middle-range mechanisms embedded within these theories can serve as building blocks of causal scenarios in settings and populations other than the studied cases. Simultaneously, because middle-range mechanisms are linked to contextual features, they capture contextual nuances to enhance evidence implementation through SCMOCs. We conclude that middle-range mechanisms provide an opportunity to achieve generalizability and contextualization in implementation science.