Sage Journals: Discover world-class research

Abstract

Realist evaluation submits that theories and models of how, why, for whom and under what circumstances programs work could be formulated by conceptualizing the relational links between the context within which programs are implemented, the generative mechanisms the programs trigger, and the outcomes of interest. Qualitative and quantitative data collection and analysis allow for the description of the relevant context, the generative mechanisms, and the emergent outcomes of programs and provide explanatory power to link these elements. The ‘realist interviewing technique’, whereby interviewees comment on a suggested ‘program theory’ to provide refinement, is proposed as a distinctive approach for conducting interviews in a realist-informed inquiry. However, the application of this interviewing strategy within the realist evaluation studies is underutilized. In this study, we demonstrate how the realist interview technique reinforces and maintains theoretical awareness and contributes to trustworthiness through three theory-building phases: theory gleaning, theory refining, and theory consolidation.

Keywords

realist evaluation realist interview theoretical awareness retroduction generative mechanisms

Introduction

Realist evaluation is a theory-driven approach to evaluation (Westhorp, 2014; Wong et al., 2016) drawn from the seminal work of Pawson and Tilley (1997). The use of realist evaluation to evaluate the implementation of policies, programs, and interventions in health services and other fields of research and evaluation has increased in the last two decades (Wong et al., 2016). This enhanced interest can be explained by the potential of realist evaluation to enable the evaluation of human service programs through a ‘clear box’ type of evaluation that investigates the effectiveness of the programs within an open system (Kazi, 2003).

According to Kazi and Spurling (2000: 4), ‘realist evaluation research is about improving the construction of [theories or] models, and therefore about improving the content of the practice itself’. Therefore, realist evaluation is fundamentally theory-driven. Therefore, maintaining a theoretical awareness—the systematic reflection on theory development and testing—during the entire realist inquiry is important. This aligns with Pawson and Sridharan’s (2009) assertion that ‘programs are theory incarnate’. According to Sharpe (2011), the evaluation of a program’s theory is an evaluation of the program rather than the evaluation of the program theory. Nevertheless, while refining the program theory involves evaluating the program, it is also true that evaluating the program involves both the theory and its implementation in that particular case.

The use of qualitative interviewing for theory-building (including proposing and testing hypotheses) has been challenged in the literature (Bendassolli, 2013). Nevertheless, the method-neutral nature of realist research opens up the possibility for using qualitative interviewing (as part or as the unique method) for theory-building in realist studies. Pawson (1996) not only suggested that qualitative interviewing could be an important technique to obtaining information from various respondents faithful to their thoughts and deeds, but could be a valuable approach to generate, validate, refute and modify theories in realist studies.

While a good interviewer could explore different angles to the research questions with the participant, or even reflect with the participant on the subject matter to inspire a theory, traditional qualitative interviews might not be the best way to verify the preliminary program theories. This is because traditional qualitative interviews are based on the principle that ‘if one puts a straight question, then most of the time one gets a straight answer’, an understanding inadvertently shared between the researcher and the respondent (Pawson and Tilley, 1997: 165). This principle is mostly shared in positivist, survey research. Interpretivists, on the other hand, do not always accept interview responses at face value, albeit they do not necessarily search for the underlying causal mechanisms or a theoretical expression of meaning.

Manzano (2016) found that most qualitative interview approaches used in the studies she reviewed aimed to evaluate the program effectiveness and/or describe barriers to using the program. Her findings suggest that the research methods employed in most realist studies gravitate toward data-driven approaches. By adopting a data-driven approach, the investigators may fail to offer the respondents the opportunity to examine and comment on the preliminary theories or hypotheses that the investigators formulated. The inclination to employ data-driven approaches to a realist inquiry could arguably be related to the influence of other paradigms and research approaches, such as the grounded theory or the investigator’s inclination to understand the process, which presupposes unstructured interviews.

The ‘realist(ic) interviewing’ approach was proposed by Pawson (1996: 313) to describe the management of the flow of information between the researcher and the study participants as an important methodological strategy to examine program theories in realist inquiries. The realist interview technique focuses on creating ‘a situation in which the theoretical postulates/conceptual structures under investigation are open for inspection in a way that allows the respondent to make an informed and critical account of them’ (Pawson, 1996: 313).

In this article, we report on our experiences of applying the realist interviewing technique within a realist evaluation study and to demonstrate how our approach contributed to the improved trustworthiness of the results. We elaborate on the application of the realist interviewing technique through the phases of developing a theory: theory gleaning, theory refinement, and theory consolidation. In particular, we demonstrate how this technique can be used to tease out the various components (intervention, context, actor, mechanism, and outcome) of a realist program theory and maintain theoretical awareness throughout the evaluation process.

Realism: ontological foundation

Critical realism is understood to originate as a scientific alternative to positivism and intepretivism, drawing elements from both methodological strains in its account of ontology and epistemology (Fletcher, 2017; Wynn and Williams, 2012). Critical realists agree with positivists that ‘knowledge should be positively applied, but reject the positivist method for doing this, arguing that causal explanations have to be based not on empirical regularities but on references to unobservable structures’ (Cruickshank, 2012: 212). On the other hand, critical realists and interpretivists recognize the importance of ideas, experiences, narratives, and discourses in understanding social phenomenon, but the realist goes further to explain that these forms of expression serve to explore causal explanations (McEvoy and Richards, 2006). Nevertheless, realism goes beyond these two paradigms in recognizing that the world is an open system with a constellation of structures, mechanisms, and contexts (Kazi, 2003).

Realism is a philosophy that offers an understanding of a social activity, whereby both social structure and agency find a place (Bhaskar, 1975). Realism (particularly critical realism) accepts the existence of independent structures that influence the actions of actors in a particular setting while acknowledging the role of the subjective knowledge of these actors (Sobh and Perry, 2006). Critical realism, therefore, suggests that effects or outcomes arise as a result of the interaction between social structures, mechanism, and human agency (McEvoy and Richards, 2006). Thus, critical realist philosophy places emphasis on the search for the causative or generative mechanisms that explain the social world (Williams et al., 2017).

Three ontological domains are identified in realist philosophies: the real, the actual, and the empirical (Bhaskar, 1975; McEvoy and Richards, 2006). The ‘real’ is the greater domain encompassing the ‘actual’, which in turn includes the ‘empirical’ (Dickinson, 2006). The real constitutes the realm of objects and their structures (Kazi, 2003) and relates to the existence of (usually) invisible mechanisms, with the generative power causing what is observed. Therefore, anything that can have real effects is itself real (Westhorp, 2014). For instance, we know that culture is real because it has real effects. Similarly, the implication is that social constructs and institutions are real and have real effects (Westhorp, 2014).

The ‘actual’ defines what happens when the existing powers are activated (Kazi, 2003)—events (and nonevents) independent of whether they are observed or not. This layer represents the portion of those events that take place in the ‘real’ that may or may not be experienced by the relevant actor (Clark et al., 2008; Schiller, 2016). The ‘actual’ domain is, therefore, a subset of the real and includes actual events generated by mechanisms (Wynn and Williams, 2012). These casual mechanisms cannot be seen operating directly but they can be inferred through a combination of empirical investigations and theory construction (McEvoy and Richards, 2006).

The third domain, the ‘empirical’, is a subset of the actual and relates to human perception and experiences of what actually happens—the day-to-day experience (Eastwood et al., 2014). It contains information that becomes known to human beings through direct and indirect experiences associated with the ‘actual’ domain. Through research endeavors, researchers can theorize about the ‘real’ by exploring event patterns and the experiences and perceptions of the actors of the phenomenon (Schiller, 2016). The relationship between these three domains is illustrated in Figure 1.

Figure 1.

Ontological positions of realist philosophy informing realist interviews.

It is worth mentioning some relevant criticisms that critical realism has received from both positivists and interpretivists. Regarding value-to-facts when conducting research, Bhaskar argued that we can move from values to facts with ‘descriptively adequate’ accounts of ‘value impregnated’ events (Cruickshank, 2010). Positivists contend that the use of values of human emancipation introduces bias in research endeavors because the use of values in any given situation is a judgment call. In response to this criticism, realists suggest that all research endeavors draw on the values of the participants but it acknowledged in other paradigms than others, which accentuates the importance of ascertaining generative mechanisms (Walsh and Evans, 2014).

Interpretivists, on the other hand, challenge that a layered ontology cannot be identified with certainty as reality is provisional and contestable, especially as our knowledge of reality is subjective and partial (DeForge and Shaw, 2012). In a similar vein, Cruickshank (2004) argued that critical realism has two mutually exclusive definitions of ontology: as a fallible interpretation of reality and as a definitive definition of a reality beyond our knowledge claims. Realists have responded to these criticisms by clarifying that deeper layers of ontology are real because their effects are real, so researchers to seek unravel their effects as comprehensive as possible, while acknowledging that these effects are generative and not definite (Walsh and Evans, 2014).

Scientific realism and realist evaluation

According to McEvoy and Richards (2003), ‘critical realism offers a coherent framework for evaluation research that is based on the understanding of causal mechanisms’ (p. 411). The primary purpose of scientific inquiry informed by the critical realist philosophy is to obtain knowledge about the underlying causal mechanisms—structures, powers, and relations that explain how things work beneath a surface (Scott et al., 2013).

Realist evaluation strategy is based on a scientific approach to the construction of theories and models (Kazi and Spurling, 2000). Realists elicit and test the program theory of a program, policy, or intervention to explain how, why, for whom and under what circumstance it works (Pawson and Tilley, 1997). This explanation is achieved by conceptualizing the causative links between the contexts (C) within which programs are implemented, the generative mechanisms (M) the programs trigger, and the outcomes (O) of interest. To obtain information regarding the context, mechanisms, and outcomes, realist evaluation is method-neutral. Often, a mixed-methods approach is used, involving the integration of quantitative and qualitative research findings for confirmation, completeness, and abductive inspiration (McEvoy and Richards, 2006). Multimethod strategies are applied to test the extent to which the program theory corresponds with reality and how data collection and analysis directly contribute to the further development of these theories or models (Kazi and Spurling, 2000). While advocating for a pluralist and pragmatic approach to the selection of methods in realist evaluation, it is advised that the choice made should be appropriate to the hypotheses generated (Salter and Kothari, 2014).

Quantitative and qualitative data collection and analysis approaches in realist evaluation can be used to identify and classify patterns attributed to the context, mechanisms, and program outcome (Westhorp, 2014). Qualitative approaches are often used to explore the context features, the underlining mechanism, and the intervention modalities. In other words, the qualitative methods allow for the identification of the constraints and opportunities the program offers, the relevant context elements, the generative mechanisms (reasoning and the choices of the actors), and the behaviors of the actors (emergent demi-regularities). According to Smith and Elger (2012), traditional interviewing techniques could be used to obtain information on ‘the social contexts, constraints and resources within which those informants act’ (p. 6) to formulate theories. Pawson and Tilley (1997) suggest that in addition to eliciting theories through various interviewing techniques, placing the formulated program theories before various program actors for examination could inspire, validate, falsify, and/or modify hypotheses about how programs and interventions work, which is an essential process for theory refinement.

Realists advocate for the use of explanation, abstraction, and interpretive logics to make causal inferences—drawing conclusions regarding causation by applying forms of reasoning (Eastwood et al., 2014). Realists predominantly adopt retroduction, which is a form of inference that seeks to identify and verify mechanisms that are theorized to have generated the phenomena under study. In addition to identifying and verifying causal mechanisms, retroduction allows the researcher to ‘identify the necessary contextual conditions for a particular causal mechanism to take effect and to result in the empirical trends observed’ (Fletcher, 2017: 189). Retroduction is an explanatory approach characterized by the use of causal mechanisms as the basis for this explanation, the possibility for multiple potential explanations, and the understanding that these causal mechanisms may or may not be observable empirically (Wynn and Williams, 2012).

Another important tool of reference-making commonly used by realist researchers is abduction. Originally coined by Charles Peirce (Psillos, 2011), abductive reasoning typically begins with an incomplete set of observations and proceeds to the likeliest possible explanation for the set. Abductive reasoning is commonly applied when there is lack of completeness, either in the evidence that the data provides, or in the explanation, or both. Also known as theoretical redescription, abduction is a process in which ‘empirical data are re-described using theoretical concepts’ (Fletcher, 2017: 188). Thus, abductive reasoning is very useful in forming and evaluating explanatory hypotheses (Thagard and Shelley, 1997), which are two important processes in realist evaluation.

The realist interviewing approach

While conducting a realist evaluation, the evaluator starts by eliciting an initial program theory—assumptions of how the program should be organized and why the program is expected to work (Westhorp, 2014). The next step entails testing the hypothesis in selected cases: the realist investigator is interested in investigating whether a program theory holds (Manzano, 2016). To this end, Pawson proposes the realist interview technique comprising the teaching-learning function and the conceptual focusing function (Pawson, 1996: 304). The realist interview incorporates the components of presenting the respondent with a formal description of (part of) the parameters of the initial program theory for examination (teacher-learning function) and offers the respondent the opportunity to explain and clarify the thinking of the researcher based on their (respondents) ideas (conceptual focusing function).

In realist research, the assumptions and expectations of the program designers (folk theories) are usually explored. In addition, the realist evaluator explores the sense and the experiences that the program actors use to construct, maintain, and negotiate the expected behavior. To obtain information from these program actors, the investigators use appropriate multimethod data collection and analysis approaches. These methods are usually applied to glean information on the program modalities, important context conditions, mechanisms provided and/or triggered by the program, and the potential outcomes of the program that could be used to formulate the initial program theory.

While there are no prescriptive methods for gleaning the initial program theory, exploratory or explanatory interview (discussion) approaches applied during the theory-gleaning phase could be very useful in understanding the content and context of programs. These interviews are described as theory-gleaning interviews. Connelly (2001) suggested that the realist interview could be a privileged method of data collection in the sense that ‘it is the primary source for both identifying and predicting the generative mechanisms at work in the specific context being studied’. An initial program theory is formulated based on the information gathered from the theory-gleaning interviews, but also document and systematic reviews, observations, and other relevant methods. Manzano (2016) suggests that the use of theory-refining and theory-consolidation interviews along with other methods is central to obtaining a more refined program theory.

The phrase ‘I’ll show you my theory if you show me yours’ coined by Pawson (1996: 307) and Pawson and Tilley (1997: 169) captures the essence of the teaching-learning function of realist interviewing. In applying the teacher-learner approach, the interviewer is advised against adopting the deliberate naiveté position (Kvale, 1996: 33), but rather the position of an expert. Pawson directs that the interviewer should adopt an active and explicit role in teaching the preliminary theory that has been developed in a previous exploratory study to the interviewee. In this case, the respondent gains an understanding of the conceptual structure of the investigation or the program and, in turn, the interviewee can make sense of the individual questions that will emanate during the interview process. Through this process, the interviewer will learn, in turn, how the respondent constructs meaning through the conceptual framework received from the interviewer. Pawson and Tilley suggest that in ‘teaching’ the interviewee, the intuition that the interviewee should get is ‘Yes, I understand the general theoretical ground you are exploring, this makes your concepts clear to me, and applying them to me gives the following answers . . .’ (p. 167). Therefore, the respondent becomes more of a participant in the meaning-making process than simply a source of information.

Pawson and Tilley (1997) suggest that the conceptual focusing function of the realist interview can be applied to obtain responses related to the CMO, although more about the program mechanisms relate to how the respondents take decisions and make choices in relation to a program. Conceptual focusing occurs when the respondents are given the opportunity to express their points of view based on their own thinking and decision-making process in the context of the interviewer’s own theory. The expertise related to the ‘generative mechanism’ of a program often lies with the informant as s/he describes the detailed way in which reasoning contributes to the observed outcomes (Pawson, 1996: 303).

Another important aspect relevant to conducting realist interviews is the notion of ‘knowledgeability’ of the interviewees. This notion relates to ‘who knows what’ with regard to the program. In this regard, Pawson and Tilley (1997: 160–161) identify two important categories of potential informants relevant to realist evaluation of a program, ‘practitioners’ and ‘subjects’. The identification of potential key informants for realist interviews is important for the selection of participants and for how the interviews will be focused and conducted in relation to teasing the elements such as intervention modalities, context, actors, mechanisms, and outcomes.

The context-mechanism-outcome configuration and the realist interview

The context-mechanism-outcome (CMO) configuration is a heuristic analytic tool used to construct program theories in realist evaluation studies. Therefore, in eliciting the program theory of a program or an intervention in realist evaluation, the investigator strives to configure the causal relationship between the context within which the program is implemented (individual, organizational, and environmental), the mechanisms that are provided by the program and the expected outcomes of the program. CMO configurations postulate how programs activate mechanisms (M) among whom and in what conditions (C), to bring about alterations in behavior or event or state regularities (O) (Pawson and Tilley, 2004: 9). For instance, providing free home-based glucose testing machines to diabetic patients who live far from the closest healthcare facility (C) could motivate (M) them to control their blood sugar levels (O). CMO propositions bring together mechanism variation and context variation to explain outcome pattern variation (Pawson and Tilley, 2004: 9).

Connelly (2001) suggested that the CMO heuristic tool requires expansion and elaboration for it to cope with the complexities that exist in other fields. With the field of health policy and systems offering such complexities, Marchal et al. (2018) and Mukumbang et al. (2018a) elaborated on the CMO configurational logic proposed by Pawson and Tilley (1997) to include components of the ‘Intervention’ and the ‘Actors’. This follows the understanding that people are not passive recipients of innovations (Greenhalgh et al., 2004) and programs can only work when the relevant actors adopt either all or parts of the intervention modalities. In this regard, generative mechanisms are used to explain how the intervention (or aspects of the intervention) unfolds in a particular context and in relation to the various actors to produce the observed outcomes. Thus, representing the intervention modalities (I) and the relevant actors (A) provides a comprehensive representation of how and why a program works (or not). Using the example provided above, the ICAMO elements will be identified as such. Providing free home-based glucose testing machines (I) to diabetic patients (A) who live far from the closest healthcare facility (C) could motivate (M) them to control their blood sugar levels (O). Following this, Marchal et al. (2018) and Mukumbang et al. (2018a) suggest that an intervention-context-actor-mechanism-outcome (ICAMO) configuration would provide a better analytical tool because aspects of the intervention that (supposedly) trigger the mechanisms and the actors through whom the intervention works are accounted for. In this article, we adopted the ICAMO heuristic (Figure 2).

Figure 2.

The conceptualized intervention-context-actor-mechanism-outcome analytic tool.

Pawson (2013: 21) cautioned that the CMO heuristic tool is an ‘ugly circumlocution’ with the parts dependent on the whole. Julnes et al. (1998) added that while the CMO framework offers a useful tool for theory-driven evaluation, it does not represent the complete range of evaluation exercises. Similarly, the ICAMO heuristic tool mainly offered case-specific and fundamental building blocks and was applied in this article as a model useful for our retroductive and abductive thinking processes. According Marchal et al. (2018), although the ICAMO heuristic tool is can be useful for within case analysis, they might not be very useful when comparing multiple cases or events.

Pawson and Tilley (1997) suggest that the selection of the potential interviewee should be based on their contributions toward clarifying the program theory. Different respondents might contribute to different components (Manzano, 2016: 350). Practitioners will have specific ideas on the intervention modalities, what is within the intervention that works, knowledge on the outcomes of the intervention (because they are likely to have experienced successes and failures), and some awareness of actors and places for whom and in which the intervention works (Pawson and Tilley, 1997: 161). On the other side, the program users are more likely to be sensitized about mechanisms and intervention modalities than to its contextual constraints and outcome patterns (Pawson and Tilley, 1997: 160).

Program theory and the phases in the realist interview

The program: adherence club intervention

The adherence club intervention—a group-based adherence-enhancing intervention—was implemented in the Western Cape Province of South Africa to address challenges of clinic congestion, poor retention in care, and suboptimal adherence to antiretroviral treatment (ART) in the context of rapidly growing HIV patient population on ART in South Africa (Wilkinson, 2013). It aims to (a) retain patients in ART care by providing a more efficient way to manage stable patients; (b) maintain good long-term adherence in PLWHA on ART through quick access to medication; and (c) decongest the health facility through group sessions (facilitated by trained non-clinical health care workers) (HAST, 2015).

The adherence club intervention is designed to streamline ART care for adult (18+ years), treatment-experienced patients with good clinic attendance record, and evidence of medication adherence (two most recent consecutive viral loads undetectable (<400 copies/mL)) (UNAIDS and MSF, 2012). Through group consultations, convenient medication pickup processes, and providing access to a clinician when needed, the adherence club drastically reduces the waiting times of the patients.

Any patient reporting with symptoms suggesting illness, adverse drug effects, or weight loss is referred by the club facilitator to the club nurse for further consultations. Based on the outcome of the consultation, they are either sent to collect their medication from the club facilitator or removed from the club (having uncontrolled comorbidities such as diabetes or hypertension). The intervention also provides a social environment that encourages patient interaction. The adherence club intervention has been described in greater detail elsewhere (Bateman, 2013; Mukumbang et al., 2016a; 2019; Wilkinson, 2013).

Program theory: developmental stages vis-à-vis realist interviews

Manzano (2016: 343) proposed three types of realist interviews: theory gleaning, theory refinement, and theory consolidation interviews to guide the process of eliciting and testing program theories.

Phases in the realist interview

Phase 1: theory gleaning interviews

The first step in conducting a realist evaluation is to elicit the initial program theory that explains how the intervention is expected to work according to the program designers and implementers (Pawson and Tilley, 1997: 88). This represents the theory gleaning phase or ‘theory elicitation’ where different sources are used to obtain relevant information to articulate the preliminary understanding of how, why, and under what conditions a program works.

The researcher, therefore, strives to make sense of the program’s original intention by conceptualizing, categorizing, and ordering (Andersen and Kragh, 2010) the experiences and assumptions of the program designers and implementers. According to Pawson and Tilley (1997: 161), program designers and people working to implement the program are a good source of information at this phase of the study. Leeuw (2003: 14) asserts that managers, stakeholders, and workers involved in a program have ‘cognitions’ (or ‘mental maps’) about the organization and the environment of the program. In accordance with this suggestion, we interviewed the program designers and senior and middle-level managers (Mukumbang et al., 2016b). We only included the facility-level managers in the second phase of theory refinement.

The questions we asked during the interviews were predominantly exploratory. At the initial phase of the study, we conducted a review of documents such as program descriptions, implementation guidelines, and a toolkit on the adherence club. The information obtained from the document review served as pointers to the aspects that required more probing during the interview process with the key informants. The goal was to explore the assumptions of the program designers and managers (folk theories) to improve our understanding of the theory or theories underpinning the functioning of the intervention (Mukumbang et al., 2016b).

Our approach at this point was to understand the nature of the adherence club program and how and why the program designers/managers envisaged it should work. For this reason, we adopted a standard semistructured in-depth interview method (Mukumbang et al., 2016b). We also used probing questions to obtain specific information. The excerpt in Table 1 illustrates the exploratory nature of the questions asked, focusing on obtaining details of relevant contexts, potential mechanisms, and main outcomes of interest.

Table 1.

Example of an exploratory interview with a program manager.

Participant	Contribution	ICAMO themes
Interviewer:	I am interested in formulating a program theory based on what was going on in the minds of the people who designed the adherence club and the people who are implementing it, what was driving it, what made them get into this program? So the first thing that I want to know is what you see as the purpose of the adherence club.
Participant:	Well initially, when it started, I was not part of the initial process. My understanding then, when it was started without me being involved, was that it was a way to get our stable patients into a club system where they could pick up medication quickly and they could get out of the clinic quickly because as it was before patients were sitting in clinics for hours just waiting for the meds pickup, and basically their consultation was 2 to 5 minutes because they were stable, they were ‘well’ patients.So we needed a program where these patients could be seen quicker and get out quicker. That was my understanding because remember our patients caused a lot of congestion in the facilities and medical officers and very experienced nurses were seeing these stable patients where you had a lot of sick patients still that needed to be attended to and they were waiting long in clinics.So it helped two things: (i) for systems to be improved in clinics in terms of triage and (ii) getting these patients out faster and reducing waiting times (number 1) and (number 2), reducing the burden on pharmacy.	Mechanism:Perceived benefitsContext:Facility organizationIntervention modality:Quick service provisionActors:Patients and healthcare workerOutcome:Reduced waiting times and workload

As illustrated in the excerpt above as well as the overall interview guide, the interviews were designed to obtain information on the accounts and viewpoints of the respondents on salient issues and events. We started each interview with the managers and program designers with general questions on adherence to medication, retention in care, and the interviewee’s role. Then we asked about the expected and observed (emergent) outcomes. We asked questions relating to how the program designers and managers thought the different resources they provide in the intervention would propagate a change in behavior to tease out possible mechanisms and relevant contexts. We applied a thematic content analysis method (Miles and Huberman, 1994) to analyze the data. The results from the document analysis and the analysis of the semistructured in-depth interviews were used to formulate ICAMO configurations (following the coding frame—Table 2) that informed the further development of the adherence club’s initial program theory.

Table 2.

Definition of relevant ICAMO terms and coding frame.

Category		Definition	Coding Rules
Intervention		An intervention is a combination of program elements or strategies designed to produce behavior changes or improve health status among individuals or a group	Modalities or program activities of the adherence club to improve retention in care or improve patients’ adherence to antiretroviral therapy
Context		Context refers to salient conditions that are likely to enable or constrain the activation of program mechanisms	Components of both the physical and the social environment that favor or disfavor the expected outcomes
Actors		These are the individuals, groups, and institutions who play a role in the implementation and outcomes of an intervention	This was coded as the actions or actual practices of an individual, group, or institution
Mechanisms		This refers to any underlying determinants or social behaviors generated in certain contexts	Any explanation or justification why a service or a resource was used by an actor to achieve an expected outcome, or considered as a constraint
Outcomes	Immediate outcome	Describes the immediate effect of the adherence club program activities	Immediate outcome typically refers to changes in knowledge, skills, or awareness, as these types of changes typically precede changes in behaviors or practices
	Intermediate outcome	Intermediate outcomes refer to behavioral changes that follow the immediate knowledge and awareness changes	Codes here define a move from direct outcomes to intermediate outcomes, identified through the indirect impact of the activity and accountability of the program
	Outcome Long-term outcome	Refer to change in the medium- and long-term, such as a patient’s health status, and impact on community and health system	The codes here represent the further indirect impact of the activity demonstrating the lesser accountability of the program

To further consolidate the preliminary ICAMO configuration, we carried out a systematic review of available studies on group-based ART adherence support models in sub-Saharan Africa (Mukumbang et al., 2017a) and a scoping review of social, cognitive, and behavioral theories that have been applied to explain adherence to ART (Mukumbang et al., 2017b). Applying retroduction, we developed the initial program theories of the adherence club program as an ICAMO configurational map—a logic in which outcomes are considered to follow from the alignment, within a case, of a specific combination of attributes—of the elements of the realist heuristic tool (Pawson and Tilley, 2004).

The configurational mapping was done by linking each active mechanism with the observed outcomes (M-O links); then we looked for the context(s) in which the mechanism was contingent and assessed the link with the actual intervention and the involved actors (retroduction). After obtaining conjectured ICAMO configurations, we applied counterfactual thinking and judgemental rationality (examining alternative explanations) to argue toward transfactual (mechanism-centered) conditions (Isaksen, 2016). We then applied abductive reasoning to identify the possible explanatory ICAMO configurations. By converting the ICAMO explanatory configuration to ‘if …, then . . . , because . . .’ phrases, we obtained testable hypotheses of the initial program theory (Box 1) (Mukumbang et al., 2018b).

Box 1.

Initial program theory of the adherence club intervention represented by two tentative theories (hypotheses).

Initial program theory 1 IF adult (18+ years) clinically ‘stable’ patients with evidence of good clinic attendance are group-managed, receive quick symptom checks, quick access to medication, consistent counseling and social support from the peer counselor,THEN patients are likely to adhere to medication and remain in care,BECAUSE they develop a group identity, which improves their perceived social, support, satisfaction and trust; and acquire knowledge, which helps them to understand their perceived threat and perceived benefits and improves their self-efficacy. As a result, they become encouraged, empowered, and motivated, thus, more likely to remain in care and adhere to the treatment. Initial program theory 2 IF operational staff receive goals and targets set to continuously enroll patients in the adherence club and strictly monitor their participation through strict standard operating practices (the promise of exclusion in the event of missed appointment and active patient tracing),THEN patients are likely to adhere to medication and remain in care,BECAUSE they fear (perceived fear) losing the benefits (easy access to medication, peer support, reduced waiting times, and two-month ART collection) of the club system and they are coerced through adhesive club rules. As a result, they become nudged to remain in care and adhere to the treatment, which might decongest the health facility.

Although each of the postulated theories has multiple mechanisms and multiple outcomes, during the theory testing phase, we shared individual ICAMOs with the respondent for a systematic approach to responding while maintaining a holistic feel of the theory.

Phase 2: theory refinement interviews

Once the initial program theory was elicited, the next step was to verify it through an empirical study. Following Koenig’s (2009) confirmation that the case study research design aligns with the realist evaluation approach, we conducted refinement of theory through a multi–case study design (Yin, 2013). We identified cases based on the following classification (Creswell and Plano Clark, 2011; Gerring, 2009):

Typical case—the most representative of the phenomenon being explored;

Deviant case—most likely case to be negative with regard to the phenomenon under consideration;

Crucial case—a case that is critical in understanding the phenomenon.

Based on the retention in care rates from monitoring reports of the adherence club program from 2014 to 2017, we classified our cases accordingly. The ‘typical’ case exemplified by a facility that showed a steady improvement in retention in the care of patients on ART in the facility (Mukumbang et al., 2019a). The ‘deviant’ case illustrated a poor-performing facility in terms of retention in care (Mukumbang et al., 2019b), while the ‘crucial’ case represented a facility that started off with very good retention in care rates, but then saw a steep drop following a change in the way the ART program was run (Mukumbang et al., 2018c). The role of the realist interviewing technique in this phase was to use the initial program theory as the basis for obtaining further information that could clarify, modify, approve, or discredit the initial program theory. In addition to this, other methods were applied to triangulate or support the information obtained through the realist interview techniques, notably nonparticipant observations and descriptive retrospective cohort analysis.

Two qualitative data collection methods were used: nonparticipant observations (Patton, 2015) and semistructured realist interviews (Manzano, 2016; Pawson, 1996). We conducted four nonparticipant observations of the adherence club meetings, where we observed club sessions without interfering in any of the processes. These included two sessions of exclusive medication collection and two sessions of blood sample collection/medication collection. The goal of the nonparticipant observation was to obtain insights into events and activities. We captured the dynamics of interactions of the group members with each other and with care providers in our field notes.

Our focus, once again, is on the role of the interview technique applied. In essence, this was the point at which the teacher-learner and the conceptual refining function become apparent. During this phase, ‘the researcher’s theory is the subject matter of the interview, and the subject is there to confirm or falsify and, above all, to refine that theory’ (Pawson, 1996: 299). First, the interviewer explained the theories that were obtained during the exploratory or theory-gleaning phase to the respondent. Then, the respondent, having understood the proposition of the interviewer, commented on the theory based on their experiences and offered their opinions on how and why they think the program works (or not).

During the theory refinement phase of our study, we interviewed three categories of actors: clinical staff (doctors and nurses), club facilitators (lay counselors), and the users (patients). These different groups of actors were purposively selected on the basis of their knowledge of the adherence club’s intervention. The purpose was to obtain their interpretations of their social contexts and their reasoning regarding the resources and constraints offered by the adherence club intervention. We explained our initial program theory to the program managers and the adherence club facilitators and used their responses to refine the initial program theory. Table 3 shows an excerpt from an interview with a club nurse to illustrate the teacher-learner conceptualization of the realist interview method.

Table 3.

Excerpts from an interview with a club nurse to illustrate the teacher-learner conceptualization of the realist interview method.

Participant	Contribution	ICAMO themes
Interviewer:	So this is how we think the adherence club works. We think the adherence club motivates patients to take their medication. It empowers them because the whole goal is self-management. People are trying to make them capable of managing their disease themselves. That is the ultimate goal. So we think that it motivates, we think it empowers them. In addition, on the other hand, because of the club rules and regulations . . . we think that you people also nudge them . . . guide them in a little bit strict way to do that. So this supposed understanding that we have is: on the one side you are trying to encourage them ‘Do your own thing, we give you the resources . . . Use these resources to help yourself’ and on the other hand, there are rules ‘You need to do this, do that’ so that everything should work. Therefore, that is the summary of how we think the adherence club works. So in your opinion, how does it work?
Participant:	So there are a lot of patients, our criteria for patients to be put into the club: you need to be at least 6 months on ARVs [antiretroviral medication] with a low viral load (LDL). And if it’s females, then we want them to have a Pap smear done, right? So then we place all these patients into clubs. And then it is a good motivation for them. They like it because number one, the waiting time in this waiting area, in the ARV unit is reduced.We [health care workers] also tell them [patients] they do not need to go and fetch their folder because in this facility you need to wait in a pre-waiting area, wait for a clerk to come to get your card according to your waiting time. Then you go into the main waiting area and you wait for your folder to be taken out by the clerk, then, they wait until they get a few folders and then they send you through to our ARV department. And then from here, you have to wait until another clerk puts your ARV stationary into your folder and then wait for a nurse to weigh you and then wait for a Sister to call you. So it is a whole long procedure. So we tell patients, we encourage them that ‘if you take your ARVs and you are compliant with your ARVs, we can put you in a club if you are 6 months and longer but on condition your, your viral load has to be lower than detected and if you are a female, a pap smear must be done, then we will put you into a club. Then, you just come straight into the facility, straight to the counselor. You avoid all those waiting periods and medication collection. You do not have to go to the pharmacy where you have to sit with the mainstream and your medication is here because our counselors collect the medication the day before.’ And so our counselors come in their own time at 7 and they are ready for our patients, they explain to them the rules, and they get their medication.	Context:Patient criteriaMechanism:MotivationOutcome:Reduced waiting timesActor:PatientsContext:Facility organizationContext: PreparationActors:Patients and Health care workerContext:Patient criteriaMechanism:Perceived benefitsIntervention modality: Use of ‘treatment buddy’Actors: Counselors
	So by 8, half past 8 for the latest they [patients] are out of the facility. As a result, they can still go to work, they do not have to sign a day’s leave, and all those things. While they are in the club they form a bond because our counselors open up a group chat so in that it is also a support group. So they would remind each other ‘do not forget your club date is tomorrow’.One of the rules [of the adherence club] is that you need to attend your club sessions. You need to be complaint with your dates. If you are unable to come you send someone or you can phone but you’ve got 5 days grace period to collect the medication. If you do not communicate with us or you do not collect your tablets within 5 days, you are going to be placed out of the club and back into the mainstream where you have to go through that whole waiting procedure.That is why I say it is a combination of your theories.What we have also done now is, all the patients, because we provide a holistic, integrated service in this [ARV] department, we have made a chronic club – we have 3 chronic clubs. So if you have hypertension or diabetes then we will put you together in one group – in one club. So we know when those patients come we will take your blood pressure, we will do your blood sugar test, we will send you on your yearly eye testing and we have a feet exam. So that you are also not disadvantaged.The other thing is that our patients’ blood gets taken twice a year, every 6 months, therefore, they also know that if their viral load is not lower than detected, they get placed back into the mainstream. So that is also a way of them also being encouraged to be a complaint, taking their medication.	Mechanism:Perceived benefitsMechanism:Bonding (group identity formation)Intervention modality:Club rulesMechanism:Nudging through rulesActors:Patients and Health care workerContext:Service organizationIntervention modality:Quick medication pickupMechanism:Perceived supportIntervention modality:Adherence monitoringMechanism:Perceived fear of losing club benefits

Notably, we described the two theories explicating how and why we thought the adherence club hypothetically would work. First, we wanted to know if based on their understanding and experience of working on the program, they could clarify our understanding of the preliminary theories. Then, we invited them to provide more information to back up their choice of theory and substantiate their points by providing examples. Therefore, in addition to ‘teaching’ the respondent the initial program theories, we asked structured qualitative questions. These questions fell within the ambit of the conceptual focusing component of the realist interviewing technique whereby the respondent clarifies the thinking of the researcher based on their own ideas, knowledge, and experiences. Table 4 illustrates an example of structured qualitative interview.

Table 4.

Example of structured qualitative questions with the club nurse.

Participant	Contribution	ICAMO themes
Interviewer:	How is the adherence club program organized at the facility?
Participant: Interviewer: Respondent:	We have a separate space at the back [of the clinic] where we allow the club people to gather. So they have their own privacy, their own space . . . So they have that freedom.How important is having their own space and privacy?I think it plays a big role because like you can say it is 35 patients that we have per club yet it is also intimate and they are able to form bonds . . . and they know it is that group and they try as far as possible to remain within that group because they have that support.	Context:Availability of spaceIntervention modality:Grouping patientsActors:PatientsMechanism:BondingOutcome:Retention in club

The excerpt above illustrates how respondents, having understood the program theory with which the researcher/interviewer is working, framed their responses with the goal of providing clarity to the researchers’ theories. The respondent identified the two theories that we elicited as being applicable to a certain extent. The respondent also provided further information (examples) to back up the choice of theory or theories. We used some exploratory questions to ‘guide’ the reflections of the respondents (Table 4). This captures the potential conceptual function of the realist interview process whereby the participants have the opportunity to recount their own decision-making process.

As mentioned previously, the questions to the program users did not require them to comment on the entire initial program theories, but only on how they made decisions in response to the resources, constraints, and opportunities provided by the intervention. Therefore, while interviewing the program users, our focus was on collecting information about (or related to) the reasoning of the patients leading to their decision to adhere (or not) to their medication or to remain (or not) in care. This represents the conceptual focusing component of the realist interviewing technique. An example of the conceptual focusing questions with a program user is illustrated in Table 5.

Table 5.

Example of structured qualitative questions with a program user (patient).

Participant	Contribution	ICAMO themes
Interviewer:	So now, what was important about you people sitting and discussing?
Participant:	Especially for me, what is very important is when you have a problem, you may think that the problem is affecting only you, but once you come to the club, you can hear other people also speaking about the same problem. You will be sharing that problem, that is why I like the club, because you can share problems and you do not have to stress yourself because when you are at home, . . . you can see may be your problem is better than some others problems	Intervention modality:Grouping patientsActors:Patients and Health care workerMechanism:Perceived benefitsOutcome:Reduced stress

During the theory refining phase of the study, both quantitative and qualitative data were collected in relation to the initial program theory. The use of a multimethod evidence base was meant to ensure good documentation of the implementation of the program (Sharpe, 2011). We employed quantitative data collection and analysis methods to identify and classify the outcome patterns and qualitative data to explore implementation features related to the context (observation) and the mechanism (exploratory and realist interviews). The retrospective cohort analysis was conducted to describe the primary outcomes of the adherence club intervention (retention in care and adherence to medication) and the qualitative explanatory design provided evidence regarding the ICAMO configuration links in the implementation chain. We applied this multimethod approach to data collection and analysis in our three contrastive cases (Mukumbang et al., 2018c, 2019a, 2019b). After conducting the analyses in the different contexts (cases), we constructed ICAMO matrix tables and models in relation to the different adherence club intervention modalities (supplemental file 1).

Phase 3: theory consolidation interviews

Our aim at this point was twofold: to strengthen or reduce support for the tested theories, and to determine how the two theories best explain (or not) how, why and in what circumstance the adherence club intervention works.

Two approaches were applied representing the two components of the realist interview technique. In the first approach, attributed to the conceptual focusing component of the realist interview technique toward theory consolidation, we had ‘conversations’ with the relevant stakeholders. According to Manzano (2016: 356) these theory consolidation interviews ‘should be guided with the help of the specificities of the individual cases, and from there, they can be directed into the general program’. The theory obtained after the theory consolidation interviews should ‘. . . reflect the primary actors’ and researchers’ interpretations of meanings and intentionality [mechanisms], and the reciprocal influences of social action and context’ (Wynn and Williams, 2012: 789). The process of fine-tuning theories could involve repeating interviews with some key participants to probe, confirm, or receive clarifications on aspects of the program theory. This is typically done during the data analysis process when the investigators identify gaps or issues that require further clarification.

The excerpt in Table 6 represents part of an interview that was conducted for the purposes of obtaining clarification on the program theory. The goal was to obtain missing information related to the role of the health talks that are provided as part of the adherence club intervention. To this end, we scheduled an appointment with the supervisor of the club facilitators at the subdistrict level.

Table 6.

An interview demonstrating the use of the conceptual focusing approach to theory verification.

Participant	Contribution	ICAMO themes
Interviewer:	So this question is how important are the talks that patients receive? Do you think the talk really has an impact in terms of encouraging and motivating them to be adherent to their medication?
Participant:	Yes, it is a very big impact you know, because it keeps on reminding them the do’s and the don’ts. Because if you do not, do the talks, they will forget the rules, they will forget that in order for me to belong what I need to do you know. So, because in our talks we talk about condom use, we talk about STIs [sexually transmitted infections], we talk about things that can interact with your medication . . . meaning that the amount of the virus will go up and then you will be failing from the line that you are. Then, we will have to take you to the other line meaning that we have to take you out of the club and then blood and other things will need to start afresh you know.Sometimes we [health care workers] give them [patients] opportunity to come up with topics and they discuss whatever among themselves, or sometimes we say ‘guys tell us, on the news there was this story, what do you think about what happened’, because you do not want each and every time when they come here we talk about HIV . . .	Mechanism:Knowledge acquisition/learningIntervention modality:Health talksOutcome:Reduced waiting timesActors:Patients and Health care workerContext:Program organization

In the second approach, representing the teacher-learner component of the realist interview technique, we conducted a working meeting with the adherence club program designers and managers to present them with the initial program theory and the ICAMO configurations we obtained after testing of the initial program theory in three contrastive sites.

After presenting our findings in a one-hour session, we divided the attendees into two groups and asked them to discuss on the two program theories and the mechanisms of motivation, empowerment, and self-efficacy or nudging by club rules. Attendees were given 15 minutes to discuss and represent how and why the adherence club works in light of the theories presented. Both groups agreed that both theories provided potential explications of how the adherence club intervention works. This begged the question which of the theories provided a stronger explanation (Judgemental rationality). Judgemental rationality is applied to evaluate and compare the explanatory power of different theoretical explanations and to select theories which most accurately represent the domain of “real” abductively (Hu, 2018). To this end, we asked the discussants to identify enabling and disabling mechanisms provided by the various modalities of the adherence club intervention in relation to the different contexts (three cases) that they were provided. This exercise took another 30 minutes.

After some discussion, a consensus was reached that both theories should be combined to explain the working of the adherence club’s intervention. This decision was backed by the argument that different patients have different attitudes and responses to the intervention. That is, some patients would perceive the resources and opportunities offered by the adherence club modalities as a source of motivation while others might respond more to the restrictive nature of the club rules and they prefer to be nudged rather than given the opportunity to make the decisions.

After these consultations with the relevant stakeholders, we systematically applied a counterfactual thinking (positing alternative circumstances) to each of the possibilities that emanated from discussions. The counterfactual conditions were meant to foster our judgments of greater explanatory power (Bhaskar, 2009) of greater explanatory power (Bhaskar, 2009) of causality in the functional theory that was developing (Roese, 1997). This thinking toward a functional theory was instrumental in addressing the alternative explanations that we had and gave us more confidence in our findings (Yin, 2013). Figure 3 captures how we navigated the research phases with regard to the different realist interviewing types.

Figure 3.

Alignment of realist interview types along the realist research phases.

Byng et al. (2005) argue that while it is important to have the CMO configurations of the different units of the program, policy, or intervention, it adds value to see how these units come together as a whole. They suggested constructing a configurational map (model) to represent the bigger picture. This was achieved through the process of retrodiction – placing the different within-case theories or models in a juxtaposition allowing for the differences and similarities to become clear (McAvoy and Butler, 2018). Retrodiction was used to examine the similarities and differences between the various cases through abstraction and accentuation,—highlighting the most prominent mechanisms (Hedstrom and Swedberg, 1998). This follows the logic that certain mechanisms dominate others and occur more frequently and thus become apparent at the level of the ‘actual’ phenomena in the form of partial regularities or demi-regularities. The following mechanisms were identified: ‘motivation’, ‘empowerment’ and ‘being nudged’. These mechanisms were used to construct a configurational map for the adherence club’s intervention as an entire intervention with its modalities as illustrated in Figure 4.

Figure 4.

Refined configurational map of the adherence club intervention.

Discussion

Pawson and Tilley (1997) proposed that although realist evaluation is method-neutral, the use of traditional interviewing techniques is not sufficient for mining the theories that would explain what works, for whom, and under what circumstances with regard to an intervention. In response to this challenge, Pawson (1996) and Pawson and Tilley (1997: chap 6) proposed a theory-driven approach to interviewing—the realist interview. Although qualitative interviewing is the most common method of data collection in realist evaluations, the application of the realist interviewing technique within these studies is conspicuously absent (Manzano, 2016). In this paper, we sought to demonstrate the practical application of the realist interview technique through our project of evaluating the adherence club intervention using the realist evaluation approach.

We applied the overall interviewing process in three phases as proposed by Manzano (2016): theory gleaning, theory refinement, and theory consolidation. In the theory-gleaning phase, we applied explanatory and exploratory interview questions to elicit the initial program theory of the adherence club intervention (Box 1). Following this phase, we applied the two components of the realist interviewing process: teacher-learner function and conceptual focusing within the theory-refinement and theory-consolidation phases.

In the theory refinement phase, the teacher-leaner function was the dominant component of the realist interviewing technique. This is because our goal was to obtain a reflection from the participant on the initial program theory formulated. In the third phase, the conceptual focusing function was the dominant component as it was based on seeking precise information (identifying enabling and disabling mechanisms located within different context) to confirm, validate, or disproof other information. Figure 5 represents our conceptualization of the interview management process incorporating the realist interviewing technique.

Figure 5.

Conceptualized information management process in a realist evaluation study—the realist interview technique.

Following the notion that the realist evaluation approach is in essence theory-driven, the realist interviewing approach provided a scaffold that maintained theoretical awareness throughout the evaluation process. This scaffolding role was particularly useful as we had to navigate from the emic (the perspective of the actors) to the etic (the perspective of the researcher). McEvoy and Richards (2003) confirmed that ‘theory development is an iterative process, as researchers need to combine the results of multiple, well-conducted studies in order to tease out the generative mechanisms that explain not only why an intervention works or does not work, but also for whom the intervention works and under what circumstances’ (p. 415). Following the exploratory and explanatory interviews with program designers and managers, we formulated hypotheses of how the intervention is intended to work supported by evidence from other sources. Armed with our tentative theories, we returned to the field to test our theories in three contrastive sites where we shared our preliminary theories with various stakeholders for commenting.

Based on the information from the theory-testing interviews and from other sources, we formulated theories representing how and why the adherence club intervention worked or not in the different cases. We returned with these case-based theories to the program designers and managers in theory consolidation interviews and a discussion forum, and presented our tested theories for possible refinement—modification, verification, and dismissal. After the critical examination of the modified theories by the program designers and managers, we applied counterfactual thinking to foster our judgment of causality to the functional theory that was developing. To this end, the realist interviewing technique was instrumental in maintaining our focus on identifying and conceptualizing the components of the ICAMO heuristic tool.

We found that the realist interview approach bolsters theory development and refinement through its iterative nature (repeated movement between data analysis and collection). According to Manzano (2016: 157), the realist analysis process ‘is an ongoing iterative process of placing nuggets of information within a wider configurational explanation’. After sourcing information from various sources (nonparticipant observation and retrospective analysis of the retention in care and adherence to medication behavior of the patients in the adherence club) and conceptualizing the nuggets of information to formulate the ICAMO configurations, applying the realist interview technique made it easy for us to go back and forth between the data and the program theory that was emerging. Therefore, the realist interview technique as applied through the various phases of theory development, testing, and consolidation offered a systematic approach to address the aims of each phase of the realist evaluation cycle while grounding the researchers within the theoretical framework.

Because the realist interviewing technique allowed us to navigate between the perspectives of the actors and those of the researchers and simultaneously adopt the iterative process of data collection and analysis, we could fine-tune the program theory on the basis of relevant emerging data. We thus argue, along with other authors (Manzano, 2016; Pawson, 1996; Pawson and Tilley, 1997), that the realist interviewing approach enhances the process of theory development and refinement and that it is most appropriate when two or more theories are being tested. To this end, we encourage its application in other realist evaluation studies where suitable.

During the application of the realist interview technique, we encountered some challenges. First, we noticed when we piloted the interview guide that the respondents would tend simply to agree to what we presented. Because the realist interview technique requires the interviewer to ‘teach’ the respondent their theory, there was the tendency that the respondent would simply agree to what the interviewer presented, a phenomenon described as acquiescence. To minimize the chances of acquiescence, we asked the respondents to give instances where the adherence club worked and instances where it did not work according to the theory.

Another challenge emerged in the interviews with the program users. The initial program theories should be explained in a comprehensible way so that the respondent understands clearly the logic of thinking of the interviewer. As Pawson and Tilley put it, ‘Yes, I understand the general theoretical ground you are exploring, this makes your concepts clear to me, and applying them to me gives the following answers . . .’ (Pawson and Tilley 1997: 167). In addition, with regard to the various components (intervention-context-actor-mechanism-outcome) of the program theory, Pawson (1996) suggests that the program users are not conversant with aspects of context and to an extent the outcome but would be a good source of information with regard to the mechanisms as they can explain their thinking and decision-making processes vis-à-vis the intervention.

In practice, it was not easy to present our overall initial program theory to the patients. We therefore fed them microelements of the initial program theory within the questions as advised by Pawson and Tilley (1997). This approach is in line with the conceptual focusing technique whereby we explored how the respondents take decisions and make choices regarding the adherence club. While we were unable to apply the teacher-learner approach with regard to the patients receiving care in the adherence club intervention, it was possible to use conceptual focusing. For this reason, the phases that have been identified here are not prescriptive.

Different methodologies take different approaches to validity, trustworthiness, and rigor (Porter, 2007). The trustworthiness of using the realist interview technique pertains to the accounts and conclusions reached by the researcher(s). According to Maxwell (2012), a realist approach to trustworthiness should ensure that the models or theories developed based on what data technique used should allow the researcher to collect relevant information. In this way, Maxwell (2012) suggests that methods used in realist studies should be ‘assessed for the purpose for which they are used, the context of this use, the data, conclusions, and understandings that are drawn from them, and, in particular, the ways that these understandings and conclusions could be wrong’ (p. 132). We discuss the trustworthiness of the realist interviewing technique in terms of descriptive trustworthiness and theoretical trustworthiness.

According to Maxwell (2012), ‘meanings and constructions of the actors are part of the reality that an account must be tested against in order to be interpretively as well as descriptively valid’ (p. 139). After eliciting the initial program theory based on the above-mentioned triangulation approaches, we applied the realist interviewing technique to verify the preliminary theories. The realist interviewing of the various actors of the adherence club intervention provided both descriptive and interpretive validity of the program theory of the adherence club intervention.

The theoretical trustworthiness of the realist interviewing technique relates to the extent to which the realist interviewing technique helps the interviewer to identify aspects or concepts that the theory employs and the relationships that are hypothesized to exist among these concepts. The realist interviewing approach supports the other data collection approaches to identify the various aspects of the ICAMO heuristic explanatory model and also to strengthen the links in the ICAMO configurations. In this case, the interviewees provided responses that strengthened the link by identifying mechanism-outcome (M-O) links and the various contextual elements on which the mechanisms are contingent. Jackson and Kolla (2012) also suggest collecting and analyzing data in realist studies in a way that linked dyads (M-Os, C-Ms, C-Os) and triads (CMOs) are identified.

Additionally, by holding discussions with the adherence club program designers and evaluators in the theory consolidation phase, we invoked the notion of judgmental rationality, which implies that arguments can be found and these arguments could provide for the validity of a judgment about truth (Bhaskar, 2009). By applying judgmental rationality, Archer et al. (2016) suggest that ‘we can publicly discuss our claims about reality, as we think it is, and marshal better or worse arguments on behalf of those claims. By comparatively evaluating the existing arguments, we can arrive at reason, though provisional judgements about what reality is objectively like’. The authors are accentuating the importance of exposing one’s tentative theories for examination by others familiar with the phenomenon. In addition, they are making the argument that we can only lay claim to a reality independent of our knowing it, provisionally and full in the knowledge that the real is a transitive object of our enquiry. This approach was very useful regarding examining the theories that we had formulated and how the program designers and implementers could identify with the theories. This was an important step in consolidating our program theory.

Conclusion

While Pawson and Tilley (1997) developed the realist evaluation approach as a theory-based approach to evaluating complex programs showing varying outcomes when implemented, they proposed the realist interviewing approach as a distinctive approach to verify theories developed through the evaluation process. Nevertheless, the realist interviewing technique is generally underutilized within realist evaluation studies. We showed how the realist interviewing approach reinforces and maintains theoretical awareness throughout the theory-driven evaluation process and we brought to light the trustworthiness that the realist interviewing technique affords to a realist evaluation study. We, therefore, recommend the use of the realist interview approach to suitable realist studies.

Supplemental Material

Supplemental_Material – Supplemental material for Using the realist interview approach to maintain theoretical awareness in realist studies

Supplemental material, Supplemental_Material for Using the realist interview approach to maintain theoretical awareness in realist studies by Ferdinand C Mukumbang, Bruno Marchal, Sara Van Belle and Brian van Wyk in Qualitative Research

Footnotes

Acknowledgements

We would like to acknowledge Dr. Ana Manzano for her guidance and expertise toward refining of this article. We also acknowledge the contributions of the two unknown reviews who reviewed the work. Their comments and queries provided great insights toward improving the manuscript.

Declaration of Conflicting Interests

The authors declare that there is no conflict of interest.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Medical Research Council of South Africa [National Health Scholars Program]. This research was also funded by an African Doctoral Dissertation Research Fellowship (ADDRF) award offered by the African Population and Health Research Center (APHRC) in partnership with the International Development Research Centre (IDRC). The work was also partly funded by the Framework 4 Agreement between the Belgian Directorate General for Development Cooperation and the University of the Western Cape (Grant No.: BBD 0344023272).

ORCID iD

Ferdinand C Mukumbang

Supplemental material

Supplemental material for this article is available online.

Author biographies

Ferdinand C Mukumbang is a postdoctoral fellow at the School of Public Health, University of the Western Cape, South Africa. His work is focused on differentiated models of care and community-based interventions for managing people living with HIV. He also works on social theory-building and the development of research and evaluation methodology for public health.

Bruno Marchal is a professor and the Head of the Health Services Organization unit at the Department of Public Health, Institute of Tropical Medicine, Antwerp (Belgium) and visiting researcher at the School of Public Health, University of the Western Cape, Cape Town. His current research focuses on the stewardship of local health systems, research approaches for complex issues in health (including realist research) and life-long conditions.

Sara Van Belle is a political scientist and anthropologist, with a PhD in public health (LSHTM, 2014) and is a post-doctoral fellow at the Institute of Tropical Medicine. She has a keen interest and expertise in governance, accountability and public sector reform, policy analysis, health systems and sexual and reproductive health, and (social) theory-building, and the development of research and evaluation methodology for public health.

Brian van Wyk is an associate professor at the School of Public Health, University of the Western Cape. He is a behavioral scientist with interests in HIV/AIDS and health systems, with particular emphasis on treatment adherence and adolescents.

References

Andersen

Kragh

(2010) Sense and sensibility: two approaches for using existing theory in theory-building qualitative research. Industrial Marketing Management 39(1): 49–55. DOI: 10.1016/j.indmarman.2009.02.008

Archer

Decoteau

Gorski

, et al. (2016) What is Critical Realism? Available at: http://www.asatheory.org/current-newsletter-online/what-is-critical-realism (accessed 10 October 2019).

Bateman

(2013) MSF again paves the way with ART. South African Medical Journal 103(2): 71–73. DOI: 10.7196/SAMJ.6666

Bendassolli

(2013) Theory building in qualitative research: reconsidering the problem of induction. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research, 14(1). DOI: 10.17169/fqs-14.1.1851

Bhaskar

(1975) Forms of realism. Philosophica 15(1): 99–127.

Bhaskar

(2009) Scientific Realism and human emancipation. London: Routledge. DOI: 10.4324/9780203879849

Byng

Norman

Redfern

(2005) Using realistic evaluation to evaluate a practice-level intervention to improve primary healthcare for patients with long-term mental illness. Evaluation 11(1): 69–93. DOI: 10.1177/1356389005053198

Clark

Lissel

Davis

(2008) Complex critical realism. Advances in Nursing Science 31(4): E67–E79. DOI: 10.1097/01.ANS.0000341421.34457.2a

Connelly

(2001) Critical realism and health promotion: effective practice needs an effective theory. Health Education Research 16(2): 115–120. DOI: 10.1093/her/16.2.115

10.

Creswell

Plano Clark

(2011) Designing and Conducting Mixed Methods Research. Los Angeles, CA: Sage Publications. Available at: https://uk.sagepub.com/en-gb/afr/designing-and-conducting-mixed-methods-research/book233508

11.

Cruickshank

(2004) A tale of two ontologies: an immanent critique of critical realism. The Sociological Review 52(4): 567–585. DOI: 10.1111/j.1467-954X.2004.00496.x

12.

Cruickshank

(2010) Knowing social reality: a critique of Bhaskar and Archer’s attempt to derive a social ontology from lay knowledge. Philosophy of the Social Sciences 40(4): 579–602. DOI: 10.1177/0048393109340664

13.

Cruickshank

(2012) Positioning positivism, critical realism and social constructionism in the health sciences: a philosophical orientation. Nursing Inquiry 19(1): 71–82. DOI: 10.1111/j.1440-1800.2011.00558.x

14.

DeForge

Shaw

(2012) Back- and fore-grounding ontology: exploring the linkages between critical realism, pragmatism, and methodologies in health & rehabilitation sciences. Nursing Inquiry 19(1): 83–95. DOI: 10.1111/j.1440-1800.2011.00550.x

15.

Dickinson

(2006) The evaluation of health and social care partnerships: an analysis of approaches and synthesis for the future. Health and Social Care in the Community 14(5): 375–383. DOI: 10.1111/j.1365-2524.2006.00650.x

16.

Eastwood

Jalaludin

Kemp

(2014) Realist explanatory theory building method for social epidemiology: a protocol for a mixed method multilevel study of neighbourhood context and postnatal depression. SpringerPlus 3(1): 12. DOI: 10.1186/2193-1801-3-12

17.

Fletcher

(2017) Applying critical realism in qualitative research: methodology meets method. International Journal of Social Research Methodology 20(2): 181–194. DOI: 10.1080/13645579.2016.1144401

18.

Gerring

(2009) Box-Steffensmeier

Brady

Collier

(eds) Case Selection for Case-Study Analysis: Qualitative and Quantitative Techniques. (vol. 1). Oxford: Oxford University Press. DOI: 9780199286546.003.0028

19.

Greenhalgh

Robert

Macfarlane

, et al. (2004) Diffusion of innovations in service organizations: systematic review and recommendations. The Milbank Quarterly 82(4): 581–629. DOI: 10.1111/j.0887-378X.2004.00325.x

20.

HAST (2015) Guidelines for ART clubs. Availabe at: http://www.differentiatedcare.org/Portals/0/adam/Content/RN8-EedHTEq0MpQtRd2jDA/File/GuidelineforARTClubsMarch2015.pdf

21.

Hedstrom

Swedberg

(1998) Social Mechanisms. Melbourne: Cambridge University Press. Availabe at: http://people.soc.cornell.edu/swedberg/1998SocialMechanisms.pdf

22.

(2018) Methodological implications of critical realism for entrepreneurship research. Journal of Critical Realism 17(2): 118–139. DOI: 10.1080/14767430.2018.1454705

23.

Isaksen

(2016) A critique of methodological applications of critical realism. Journal of Critical Realism 15(3): 245–262. DOI: 10.1080/14767430.2016.1169369

24.

Jackson

Kolla

(2012) A new realistic evaluation analysis method: linked coding of context, mechanism, and outcome relationships. American Journal of Evaluation 33(3): 339–349. DOI: 10.1177/1098214012440030

25.

Julnes

Mark

Henry

(1998) Review: promoting realism in evaluation: realistic evaluation and the broader context. Evaluation 4(4): 483–504. DOI: 10.1177/13563899822208743

26.

Kazi

MAF

(2003) The contribution of realist evaluation for practice. In: Kazi

MAF

(ed) Realist Evaluation in Practice. DOI: 10.4135/9781849209762

27.

Kazi

MAF

Spurling

(2000) Realist evaluation for evidence-based practice. Paper Presented at the European Evaluation Society’s Fourth Conference, Lausanne, Switzerland, October 2000. Available at: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.533.8182&rep=rep1&type=pdf

28.

Koenig

(2009) Realistic evaluation and case studies: stretching the potential. Evaluation 15(1): 9–30. DOI: 10.1177/1356389008097869

29.

Kvale

(1996) Interviews : An Introduction To Qualitative Research Interviewing. Thousand Oaks, CA: Sage Publications.

30.

Leeuw

(2003) Reconstruction program theories: methods available and problems to be solved. American Journal of Evaluation 24(1): 5–20. DOI: 10.1016/S1098-2140(02)00271-0

31.

Manzano

(2016) The craft of interviewing in realist evaluation. Evaluation 22(3): 342–360.

32.

Marchal

Kegels

Van Belle

(2018) Theory and realist methods. In: Emmel

Greenhalgh

Manzano

Monaghan

Dalkin

(eds) Doing Realist Research. Los Angeles: Sage Publications, pp. 80–89.

33.

Maxwell

(2012) A Realist Approach to Qualitative Research. Thousand Oaks, CA: Sage Publication, 222. DOI: 10.1017/CBO9781107415324.004

34.

McAvoy

Butler

(2018) A critical realist method for applied business research. Journal of Critical Realism 17(2): 160–175. DOI: 10.1080/14767430.2018.1455477

35.

McEvoy

Richards

(2003) Critical realism: a way forward for evaluation research in nursing? Journal of Advanced Nursing 43(4): 411–420.

36.

McEvoy

Richards

(2006) A critical realist rationale for using a combination of quantitative and qualitative methods. Journal of Research in Nursing 11(1): 66–78. DOI: 10.1177/1744987106060192

37.

Miles

Huberman

(1994) Qualitative data analysis (2nd Edition). California: Sage Publications, Inc.

38.

Mukumbang

Van Belle

Marchal

, et al. (2016a) Realist evaluation of the antiretroviral treatment adherence club programme in selected primary healthcare facilities in the metropolitan area of Western Cape Province, South Africa: a study protocol. BMJ Open 6(4): e009977. DOI: 10.1136/bmjopen-2015-009977

39.

Mukumbang

Van Belle

Marchal

, et al. (2016b) Towards developing an initial programme theory: programme designers and managers assumptions on the antiretroviral treatment adherence club programme in primary health care facilities in the metropolitan area of western cape province, South Africa. PLoS One 11(8): e0161790. DOI: 10.1371/journal.pone.0161790

40.

Mukumbang

Van Belle

Marchal

, et al. (2017a) An exploration of group-based HIV/AIDS treatment and care models in Sub-Saharan Africa using a realist evaluation (Intervention-Context-Actor-Mechanism-Outcome) heuristic tool: a systematic review. Implementation Sciences 12: 107. doi: 10.1186/s13012-017-0638-0

41.

Mukumbang

Van Belle

Marchal

, et al. (2017b) Exploring “generative mechanisms” of the antiretroviral adherence club intervention using the realist approach: a scoping review of research-based antiretroviral treatment adherence theories. BMC Public Health 17(1): 385. doi: 10.1186/s12889-017-4322-8

42.

Mukumbang

Marchal

Van Belle

, et al. (2018a) Unearthing how, why, for whom and under what health system conditions the antiretroviral treatment adherence club intervention in South Africa works: a realist theory refining approach. BMC Health Services Research 18(1): 343. doi: 10.1186/s12913-018-3150-6

43.

Mukumbang

Marchal

Van Belle

, et al. (2018b) A realist approach to eliciting the initial programme theory of the antiretroviral treatment adherence club intervention in the western cape province, South Africa. BMC Medical Research Methodology 18(47). doi: 10.1186/s12874-018-0503-0

44.

Mukumbang

Marchal

Van Belle

, et al. (2018c) ‘Patients are not following the [Adherence] club rules anymore’: a realist case study of the antiretroviral treatment adherence club, South Africa. Qualitative Health Research 18(12): 1839–1857. doi: 10.1177/1049732318784883

45.

Mukumbang

Orth

Van Wyk

(2019) What do the implementation outcome variables tell us about the scaling-up of the antiretroviral treatment adherence clubs in South Africa? Health Research Policy and Systems 17(1): 28. doi: 10.1186/s12961-019-0428-z

46.

Mukumbang

van Wyk

Van Belle

, et al. (2019a) Unravelling how and why the Antiretroviral Adherence Club Intervention works (or not) in a public health facility: a realist explanatory theory-building case study. Plos One: 1–26. DOI: 10.1371/journal.pone.0210565

47.

Mukumbang

van Wyk

Van Belle

, et al. (2019b) ‘At this [adherence] club, we are a family now’: a realist theory-testing case study of the antiretroviral treatment adherence club, South Africa. Southern African Journal of HIV Medicine 20(1): 14. DOI: 10.4102/sajhivmed.v20i1.922

48.

Patton

(2015) Qualitative Research & Evaluation Methods: Integrating Theory and Practice (4th edition). Thousand Oaks, California: Sage Publications, Inc.

49.

Pawson

(1996) Theorizing the interview. The British Journal of Sociology 47(2): 295–314. Available at: http://www.jstor.org/stable/591728

50.

Pawson

(2013) The Science of Evaluation: A Realist Manifesto. Los Angeles, CA and London: Sage Publication.

51.

Pawson

Sridharan

(2009) Theory-driven evaluation of public health programmes. In Killoran

Kelly

(eds) Evidence-Based Public Health. Oxford: Oxford University Press, 43–62. Available at: http://doi.org/10.1093/acprof:oso/9780199563623.003.04

52.

Pawson

Tilley

(1997) Realistic Evaluation. London: Sage Publication.

53.

Pawson

Tilley

(2004) Realist Evaluation. Available at: http://www.communitymatters.com.au/RE_chapter.pdf

54.

Psillos

(2011) Choosing the realist framework. Synthese 180: 301–316.

55.

Porter

(2007) Validity, trustworthiness and rigour: reasserting realism in qualitative research. Journal of Advanced Nursing 60(1): 79–86. DOI: 10.1111/j.1365-2648.2007.04360.x

56.

Roese

(1997) Counterfactual thinking. Psychology Bulletin 121(1): 133–148.

57.

Salter

Kothari

(2014) Using realist evaluation to open the black box of knowledge translation: a state-of-the-art review. Implementation Science 9(1): 1–14. DOI: 10.1186/s13012-014-0115-y

58.

Schiller

(2016) Critical realism in nursing: an emerging approach. Nursing Philosophy 17(2): 88–102. DOI: 10.1111/nup.12107

59.

Scott

Zachariadis

Barrett

(2013) Methodological implications of critical realism for mixed-methods research. MIS Quarterly: Management Information Systems 37(3): 855–879.

60.

Sharpe

(2011) A review of program theory and theory-based evaluations. American International Journal of Contemporary Research 1(3): 72–75.

61.

Smith

Elger

(2012) Critical Realism and Interviewing Subjects. Working Paper Series. SoMWP-1208, Studying Organizations Using Critical Realism: A Practical Guide. London.

62.

Sobh

Perry

(2006) Research design and data analysis in realism research. European Journal of Marketing 40(11–12): 1194–1209. DOI: 10.1108/03090560610702777

63.

Thagard

Shelley

(1997) Abductive reasoning: logic visual thinking, and coherence. in: Dalla Chiara

M-L

, et al. (eds) Logic and scientific methods. Dordrecht: Kluwer, 413–427.

64.

UNAIDS and MSF (2012) Strategies, Tools and Policies to Get the Best HIV Treatment to More People, Sooner (pp. 1–26). Available at: https://www.hst.org.za/publications/NonHST%20Publications/MSF_Speed_up_Scale-up_report-webres.pdf

65.

Walsh

Evans

(2014) Critical realism: an important theoretical perspective for midwifery research. Midwifery 30(1): e1–e6. DOI: 10.1016/j.midw.2013.09.002

66.

Westhorp

(2014) Realist impact evaluation: an introduction. Research and Policy Development (September). London: Overseas Development Institute.

67.

Wilkinson

(2013) ART adherence clubs: a long-term retention strategy for clinically stable patients receiving antiretroviral therapy. Southern African Journal of HIV Medicine 14(2): 48. DOI: 10.4102/hivmed.v14i2.77

68.

Williams

Rycroft-Malone

Burton

(2017) Bringing critical realism to nursing practice: Roy Bhaskar’s contribution. Nursing Philosophy 18(2): e12130. DOI: 10.1111/nup.12130

69.

Wong

Westhorp

Manzano

, et al. (2016) RAMESES II reporting standards for realist evaluations. BMC Medicine 14(1): 96. DOI: 10.1186/s12916-016-0643-1

70.

Wynn

Williams

(2012) Principles for conducting critical realist case study research in information systems. MIS Quarterly 36(3): 787–810. DOI: 10.1016/j.ijproman.2012.11.012

71.

Yin

(2013) Case Study Research: Design and Methods. California: Sage Publications.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.69 MB