Sage Journals: Discover world-class research

Abstract

Although realist evaluation (RE) requires multiple data collection methods, qualitative interviews are considered most valuable and are most frequently used. The guiding principles of RE may limit the emergence of new Context–Mechanism–Outcome (CMO) configurations by evoking particular underlying mechanisms. This paper proposes a new method for conducting semi-structured interviews in the RE context by drawing on the literature and examining the ability of vignettes to explore perceptions about specific situations. Vignettes are developed based on researchers’ knowledge of the setting and program theory and are updated through an iterative process throughout data collection. Interviews focus on situations illustrated in the vignette to capture variations in interviewees’ perceptions. This method constrains interviewees to using retroduction to identify the hidden underlying mechanisms that link contextual elements to outcomes based on their experiences. This method allows researchers to focus on CMO configurations without evoking mechanisms, which contributes to the rigor of the method.

Keywords

vignette realist evaluation interview

Background

Realist evaluation (RE) is a theory-driven evaluative approach that has gained popularity in the past decade (G. Wong et al., 2016). RE aims to identify “what works under what circumstances and for whom” (Pawson & Sanjeev, 2009; Pawson & Tilley, 1997) and is a valuable approach to evaluate complex and constantly evolving interventions. The recent development of guides, recommendations, and tools has made the approach more accessible and contributed to its dissemination and utilization (e.g., Fick & Muhajarine, 2019; Manzano, 2016; Wong et al., 2016).

The realist approach aims to refine a program theory by identifying relationships between contexts, underlying mechanisms, and outcomes (Pawson & Tilley, 1997). However, although these are key concepts in RE, conceptual ambiguities about their definitions persist (Lemire et al., 2020; Nielsen et al., 2021). In this article, context refers to all pre-existing elements (de Souza, 2013; Hewitt et al., 2012). Depending on the focus, context can include observable features or relational and dynamic features (Greenhalgh & Manzano, 2022). The mechanism is an invisible element that can be triggered in a context to cause the outcome (Astbury & Leeuw, 2010; Hewitt et al., 2012; Lacouture et al., 2015; Westhorp, 2018). Mechanisms can include a specific program element, reactions to the program, or reactions to the context or a change in context (Lemire et al., 2020). Finally, the outcome is the result of one or more mechanisms activated following an intervention in a context (Lacouture et al., 2015; Pawson & Sanjeev, 2009; Ridde et al., 2012; Robert & Ridde, 2014). Context–Mechanism–Outcome (CMO) configurations conceptually link a context, mechanism, and outcome (or combination) by emphasizing the causal link between these elements (Ridde et al., 2012; Westhorp, 2018). Recurrent patterns of CMO configurations allow for the identification of regularity or demi-regularity to refine a program theory (Wong et al., 2016).

Realist evaluation often requires several types of data, but the qualitative interview remains the most common mode of data collection allowing for the emergence of mechanisms (Mukumbang et al., 2019). Qualitative interview techniques for RE have been specifically developed to improve usefulness and ensure rigor (Manzano, 2016; Wong et al., 2016). However, effectively eliciting CMO configurations in realist interviews without influencing interviewees’ perceptions remains a challenge. Traditional realist interviewing usually requires the directive wording of mechanism questions and a great deal of interview control (Bonell et al., 2022; Manzano, 2016; Pawson, 1996). In addition, realist methodology and interviewing techniques are complex and require a thorough understanding, which can overwhelm novice users and create barriers to utilization (Emmel, 2013; Pawson & Tilley, 1997).

The use of vignettes promotes the democratization and rigor of realist interviewing. Vignettes have been used in at least two REs, with the authors reporting that the method allowed interviewees some distance from their personal situations. This allowed interviewees to take a step back in order to stimulate in-depth discussions that explicitly identified mechanisms (Ohly et al., 2019; Sriranjan et al., 2020). Thus, the use of vignettes makes it possible to combine both the direct interview method proposed by Manzano (2016) and Pawson (1996) with the more indirect method proposed by Sayer (2000). The vignette can reproduce similar emotions and reflections as being in an actual situation. Therefore, this method may reduce memory and anticipation biases by having interviewees respond to the vignette context rather than try to remember their thinking at a specific moment in the past. This is particularly valuable in the context of a retrospective study. Clear guidelines will help democratize this method and increase the uptake of RE.

This article proposes a step-by-step method to use vignettes in realist interviews (example in Box 1). Before exploring the method, the precepts of qualitative interviews for RE are briefly described, followed by a description of vignettes and guidance for qualitative interviewing.

Box 1. Case Example

RE of the champion to implement organizational change in primary healthcare clinics

The vignette was used as part of a qualitative, multiple case study using an RE approach to describe how champions can promote quality improvement projects. In recent years, significant investments have been made to support various organizational changes in primary care. Studies suggest that the presence of a champion, a member of the organization who promotes innovation, facilitates the implementation of change. However, a better understanding of the underlying characteristics and mechanisms of champions to understand why and how they facilitate the integration of organizational change is needed.

A multiple case study was conducted in primary care practices engaged in an organizational change process. To describe the role of the champion in adopting organizational change and explore the factors that make champions effective, semi-structured interviews were planned with champions and change stakeholders to assess the initial theory.

Data analysis revealed regularities in the relationships between contextual elements, the champion intervention, the underlying mechanisms, and champion-related outcomes. The relationships between these three elements were represented in the form of CMO configurations. Validation with stakeholders also took place. This process, used iteratively but not linearly, allowed for the development of a final theory explaining the champion’s role in adopting organizational changes in primary care.

Realist Interview Principles

RE almost always involves qualitative methods such as individual interviews or focus groups (Palm & Hochmuth, 2020; Renmans & Castellano Pleguezuelo, 2022). However, realist interviews can be challenging due to their focus on understanding how and why an intervention or program works, as well as the underlying mechanisms driving the observed outcomes (Manzano, 2016; Mukumbang et al., 2019; Pawson, 1996). To uncover these mechanisms, researchers must be flexible, adaptable, and open to unexpected findings. Specific challenges of conducting realist interviews, documented in several studies, include selecting appropriate participants, developing interview questions, and analyzing interview data (Salter & Kothari, 2014; Wong et al., 2016). Overall, conducting realist interviews requires a high level of skill and expertise, as well as a commitment to understanding the complex nature of interventions or programs and the mechanisms driving their outcomes.

Pawson (1996) first proposed a realist interview technique comprising the teaching–learning and conceptual focusing function. However, Goicolea et al. (2015) proposed that the question must be inspired by the initial program theory, without naming the theory explicitly. Manzano (2016) developed a more detailed and operationalized method for realist interviews based on Pawson’s recommendations, suggesting tips and proposing fundamental principles, but did not address the critiques of Pawson’s proposition. As no other guidelines have been proposed, these remain the current standard.

Following two particular principles makes an interview relevant to RE (Manzano, 2016): (1) plan the study based on the program theory and (2) ask questions according to a realist approach. To plan a study based on the program theory, researchers start by interviewing participants who know the program of interest well. They can provide a big picture of the program theory and fill large knowledge gaps by presenting a broad vision. Then, the researchers interview participants with more specific experiences to help refine the program theory. Realist interviews are divided into three phases: theory gleaning, theory refinement, and theory consolidation interviews (Manzano, 2016).

Phase 1: Theory Gleaning Interviews

This phase aims to explore how the intervention is expected to work (Manzano, 2016; Pawson & Tilley, 1997, 2001). According to the initial theory, researchers use this first phase to obtain relevant information to articulate a preliminary understanding of the case context and outcome. Then, they formulate hypotheses of how, why, and under what conditions the program will work. Interviewees include program designers or people with a general understanding of the case or of the program’s functioning (Leeuw, 2003).

Phase 2: Theory Refinement Interviews

Once the researcher has a general view of the case and has applied a program theory, the next step is to obtain empirical data to confirm, falsify, or refine that theory (Pawson, 1996). During this step, data collection and interviews aim to explore the related observed context and implementation features. For example, the researcher could explore the mechanism using retroduction.

Phase 3: Theory Consolidation Interviews

These interviews aim to clarify the program theory or obtain missing information (Mukumbang et al., 2019). They are guided by the specificities of individual cases, and from there, they can be applied to the general program (Manzano, 2016). The consolidated theory must reflect the perceptions of actors involved in the program, their experiences, and the relationships between the program and its context (Wynn & Williams, 2012). This final phase of interviewing may involve reinterviewing key actors to probe, confirm, or clarify targeted aspects of the theory (Mukumbang et al., 2019).

To follow the second principle, that questions be asked according to a realist approach, the interviewer should understand what is happening in the natural setting to ensure a conversational feel to the interview. The aim of the interview is to understand the program’s story, not the participant’s story. It is also a key tool to explore micro-events that contribute to refinement of the program theory. The exploration of these elements allows fragments of theories to emerge within CMO configurations, which can be compared to other fragments to refine the program theory. Experts suggest presenting the theory to interviewees, who then confirm, falsify, or improve the theory according to their perceptions. To avoid “directing the interview,” Manzano (2016) proposed testing multiple theories with participants. In this way, the interview guide addresses the program theory and rival explanations. Similarly, Mukumbang et al. (2019) suggested that respondents may simply accept a proposed theory, which may interfere with the emergence of new CMO configurations by evoking particular underlying mechanisms. The method proposed here avoids this issue.

Retroduction in Realist Evaluation

Retroduction is a key concept in RE. It is the most suitable form of reasoning in realist research and evaluation, as it encourages theorizing and enables testing of hidden mechanisms (Jagosh, 2020; The RAMSES II Project, 2017b). “Retroduction entails the idea of going back from, below, or behind observed patterns or regularities to discover what produces them” (Lewis-Beck et al., 2004). Retroduction allows for a focus on the causal links in CMO configurations. Retroductive logic involves understanding the explanatory causes of observations to produce a theory (Jagosh, 2020; Manzano, 2016; The RAMSES II Project, 2017b). In RE, data must be collected and analyzed using retroductive logic combining inductive and deductive elements with creativity and intuition (The RAMSES II Project, 2017b). The deductive element aims to test the initial theory, whereas the inductive element is used to formulate a modified version of the program theory based on available data (DeCarlo, 2018). Program theory allows for the development of emerging hypotheses (in the form of CMO configurations) that are evaluated through a deductive process (Justin et al., 2022). Analyses require the researcher to use their judgment, experience, imagination, and creativity to construct a theory (The RAMSES II Project, 2017a).

The Use of Vignettes in Qualitative Research

A vignette is a short description of a person, an object, or a hypothetical scenario that can be used to prompt detailed reflections and examine cognitive processes (Atzmüller & Steiner, 2010; Herskovits, 1950; Rossi & Nock, 1982). Vignettes have been used in many disciplines to collect data on individuals or teams through individual or group interviews or questionnaires (Barter & Renold, 1999; Hughes, 1998; Hughes & Huby, 2002). In qualitative research, vignettes have shown potential to deeply understand healthcare professionals’ experiences (Sheringham et al., 2021; Tremblay et al., 2022). Vignettes allow qualitative researchers to focus on specific elements and help reveal interviewees’ true thoughts about a situation by depersonalizing the focus (Finch, 1987; Gourlay et al., 2014; Tremblay et al., 2022). We propose to use vignettes in qualitative interviews while respecting the standards for semi-structured interviews described by Kvale (2007) and Kvale and Brinkmann (2015): preparing questions carefully, establishing a relationship of trust with participants, active listening, and clarifying answers during the interview.

Qualitative Vignettes in Realist Evaluation

Based on the case example presented in Box 1, we describe the steps involved in using a vignette in qualitative interviews: (1) research problem and program theory; (2) construct and write vignette; (3) sample and adapt vignette; and (4) interview using the vignette.

Step 1: Research Problem and Initial Program Theory

The first step in designing an RE is to identify or develop an initial theory. The RAMESES II Project outlined six questions to choose, develop, construct, or refine the initial program theory (Wong et al., 2013). The theory should describe how and why mechanisms generate an outcome according to the context and include at least one CMO configuration (Wong et al., 2017).

In the example in Box 2, critical interpretive synthesis was used to determine the potential contextual elements and underlying mechanisms of a champion who may affect the implementation of organizational changes (Dixon-Woods et al., 2006; Martin et al., 2023). The initial program theory included many contexts to consider, interventions made by the champion that seemed to influence change adoption in the past, and mechanisms that may explain the champion outcome. The initial program also included CMO configurations based on previous analytical work and the authors’ retroduction.

Box 2. Examples of Initial Program Theory

Here are a few examples of the initial program theories we tested as part of the project and the associated contexts (C), mechanisms (M), and outcomes (O):

1. Having a champion provide feedback during the change (C) enables the quick adjustment of actions (M), which promotes the adoption of new behavior (O).

2. A champion who shares their experiences with peers (C) fosters the team’s confidence in the change proposal (M), which reassures clinicians and increases their commitment to change (O).

3. A champion who mentors and responds to queries quickly (C) helps maintain momentum during a change (M), making it easier to adopt new behaviors (O).

Step 2: Vignette Writing and Construction

Vignette development and utilization in qualitative research requires robust methodological guidance to ensure the vignette’s relevance (Tremblay et al., 2022). The vignette must promote cognitive and emotional engagement and stimulate real-life contextual reasoning (Farmer et al., 2006; Finch, 1987). This should provoke participants’ intuition and instinctive answers, as a well-constructed vignette allows access to interviewees’ natural reactions and supports deep sincerity (Hughes & Huby, 2002). In realist interviews, vignettes should be developed based on the program theory without including any underlying mechanisms to reflect the system-level reality. Thus, vignettes represent the researchers’ knowledge about the context, program elements, and expected outcomes. When creating, using, analyzing, and interpreting the vignette, particular care must be taken to ensure validity. Validity is enhanced by considering multiple perspectives on a situation (Porter, 2007). Therefore, it is essential to ensure that the perspectives of different types of participants are included and to enhance the vignette with these different perspectives throughout the data collection process.

In the example, the vignette was developed using the initial program theory and known contextual elements and outcomes. Because vignettes differ from case to case depending on the context and outcomes, they evolve over time to represent the evolution of the initial program theory and the case. The vignette shown in Box 3 describes contextual elements (setting characteristics, change characteristics, setting elements, and the champion’s actions) and the outcomes (intermediate and final outcomes).

Box 3.

Vignette

Step 3: Sampling of Participants and Vignette Adaptation

For participant selection, it is advisable to follow the principles outlined by Manzano (2016) to interview those with a comprehensive understanding of the program or case under study. The more the researcher learns about the program theory and the specific case, the clearer and more supportive the vignette will be for the next interviewee. Further interviews with individuals who have directly experienced the program then provide insights into their reasoning and reactions to explain the information in the vignette. Ultimately, interviewees use the vignette to identify the mechanism linking the context and the outcome to finalize the CMO configuration. For example, the champion interview informed the researcher about the link between the intervention and context: “Which action did you take to facilitate the change implementation? Why did you take this action?” Then, peers made links between the champion’s actions and the outcomes using their own rationale and identified the causal mechanisms behind the link.

Step 4: Interviewing Using the Vignette

Depending on the data collection phase (theory gleaning, refinement, and consolidation), the nature of the interviews is slightly different. Initially, interviews focus on better understanding the case. The researcher should begin by asking open-ended questions to discover, corroborate, or clarify relevant contextual elements, interventions, and outcomes. The researcher should actively listen to the interviewee and add relevant elements to the vignette. Then, the vignette, or parts of the vignette, is presented to the interviewee, with the interview guide focusing on specific situations illustrated by the vignette. Interviewees are then invited to add details they perceive as improving the vignette. This captures subtle variations among interviewee perceptions of the context and outcomes and gleans further knowledge about the case. Interviewees can then refine, validate, or correct the researcher’s analysis. The interviewer should then invite interviewees to link an outcome to a contextual or intervention element and explain their rationale, leading to the CMO configuration. In some cases, a cascade of mechanisms may be described, and the interviewer may need to ask for more information about the rationale. The interviewer can explicitly target a CMO configuration by asking interviewees to describe examples of linked elements and to complete the configuration by asking about the favorable context for that link. By exploring each micro-event, the interviewer can explore, gain more knowledge, and challenge elements.

In the case example (Box 1), interviewees clarified the context of specific elements of the organization and the change initiative (e.g., relations between members of the organization, perceptions of their peers, and the change initiative itself). They also described and added details about their feelings in general and during the change process. These elements are essential to build a holistic view of the context, intervention, and outcome. An example of an interview using the vignette is provided in Box 4. Probing helps to directly ask for mechanisms without specifying what the mechanism is. It can also provide some linked elements to help researchers understand or target specific configurations. The mechanism is the critical element of the RE approach. The vignette can also support case comparisons in a multiple case study performed using a realist approach. The interviewer could ask: “What do you think the outcome would be if [the champion] took this action in a larger organization?” Rather than presenting the theory or a rival theory, the mechanism is intentionally left to the interviewee. This method imposes the use of retroduction by the interviewee, thereby increasing the rigor of the data collection.

Box 4. Examples of Questions and Prompts

Interviewer: “Now I’d like to show you a vignette with three columns. In the first column, there is the context, in the second, the champion’s actions, and the last presents some potential effects of the champion’s actions. Each of these lists can be enhanced. The exercise is to link a champion’s action to an impact on implementation. Try to explain your rationale behind the link. I’ll leave you a few minutes to read and reflect upon them.”

Interviewee: “The champion shows authority. Clearly, I’m convinced that it reduces clinician receptiveness to the objective. If something is imposed, there’s less of a sense of belonging, less of a desire to ... In short, clearly, for me, authority reduces responsiveness.”

Interviewer: “And why?”

Interviewee: “Ah, but it’s clear that when something is more imposed there’s no sense of equality, and if someone is superior and asks you to do something it creates a bit of a break effect, to want to continue to implant yourself in something. That’s not what’s concerned you about it. And it’s clearly not soliciting a desire to prove yourself and then collaborate.”

In this example, the interviewer can also use other prompts to explore rival theories or nuances or to identify a more appropriate theory. For example, “Can you give me any contexts in which the use of authority might be helpful?” or “How does a sense of equality promote behavioral change?”

Conclusion

An RE approach fosters better understanding of complex interventions and contributes a causal explanation of the observed outcome. However, this relatively new approach presents several challenges in its application. Tools and methods such as the vignette help democratize RE and improve its scientific rigor.

The method proposed here is based on the literature on vignettes and RE interview principles. It should contribute rich data on the causal explanations underlying observations made at the system level. Vignettes point out situations of interest in the same way Manzano (2016) has proposed. The difference is that, rather than presenting an underlying mechanism and asking participants what they think about it, the researcher asks participants to use reasoning to propose a mechanism. Vignettes encourage interviewees to make interconnections between contextual elements and outcomes, thereby improving the credibility of results and facilitating data analysis. However, as the number of interconnections increases, this method may require more interviews to saturate CMO configurations.

The realist interview remains challenging, but vignettes help to structure interviews and ensure rigor throughout the process.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethical Statement

ORCID iD

Elisabeth Martin

References

Astbury

Leeuw

F. L.

(2010). Unpacking black boxes: Mechanisms and theory building in evaluation. American Journal of Evaluation, 31(3), 363–381. https://doi.org/10.1177/1098214010371972

Atzmüller

Steiner

P. M.

(2010). Experimental vignette studies in survey research. Methodology, 6(3), 128–138. https://doi.org/10.1027/1614-2241/a000014

Barter

Renold

(1999). The use of vignettes in qualitative research. Social Research Update, 25(9), 1–6. https://sru.soc.surrey.ac.uk/SRU25.html

Bonell

Warren

Melendez-Torres

G. J.

(2022). Methodological reflections on using qualitative research to explore the causal mechanisms of complex health interventions. Evaluation, 28(2), 166–181. https://doi.org/10.1177/13563890221086309

DeCarlo

(2018). Scientific inquiry in social work. https://pressbooks.pub/scientificinquiryinsocialwork/front-matter/version-information

de Souza

D. E.

(2013). Elaborating the Context-Mechanism-Outcome configuration (CMOc) in realist evaluation: A critical realist perspective. Evaluation, 19(2), 141–154. https://doi.org/10.1177/1356389013485194

Dixon-Woods

Cavers

Agarwal

Annandale

Arthur

Harvey

Hsu

Katbamna

Olsen

Smith

Riley

Sutton

A. J.

(2006). Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Medical Research Methodology, 6(35), 1–13. https://doi.org/10.1186/1471-2288-6-35. https://bmcmedresmethodol.biomedcentral.com/counter/pdf/10.1186/1471-2288-6-35.pdf

Emmel

(2013). Sampling and choosing cases in qualitative research: A realist approach. Sage.

Farmer

Iversen

Campbell

N. C.

Guest

Chesson

Deans

MacDonald

(2006). Rural/urban differences in accounts of patients' initial decisions to consult primary care. Health & Place, 12(2), 210–221. https://doi.org/10.1016/j.healthplace.2004.11.007

10.

Fick

Muhajarine

(2019). First steps: Creating an initial program theory for a realist evaluation of Healthy Start-Départ Santé intervention in childcare centres. International Journal of Social Research Methodology, 22(6), 545–556. https://doi.org/10.1080/13645579.2019.1595375

11.

Finch

(1987). The vignette technique in survey research. Sociology, 21(1), 105–114. https://doi.org/10.1177/0038038587021001008

12.

Goicolea

Hurtig

A. K.

San Sebastian

Vives-Cases

Marchal

(2015). Developing a programme theory to explain how primary health care teams learn to respond to intimate partner violence: A realist case-study. BMC Health Services Research, 15. https://doi.org/10.1186/s12913-015-0899-8

13.

Gourlay

Mshana

Birdthistle

Bulugu

Zaba

Urassa

(2014). Using vignettes in qualitative research to explore barriers and facilitating factors to the uptake of prevention of mother-to-child transmission services in rural Tanzania: A critical analysis. BMC Medical Research Methodology, 14(1), 21. https://doi.org/10.1186/1471-2288-14-21

14.

Greenhalgh

Manzano

(2022). Understanding ‘context’ in realist evaluation and synthesis. International Journal of Social Research Methodology, 25(5), 583–595. https://doi.org/10.1080/13645579.2021.1918484

15.

Herskovits

M. J.

(1950). The hypothetical situation: A technique of field research. Southwestern Journal of Anthropology, 6(1), 32–40. https://doi.org/10.1086/soutjanth.6.1.3628688

16.

Hewitt

Sims

Harris

(2012). The realist approach to evaluation research: An introduction. International Journal of Therapy and Rehabilitation, 19(5), 250–259. https://doi.org/10.12968/ijtr.2012.19.5.250

17.

Hughes

(1998). Considering the vignette technique and its application to a study of drug injecting and HIV risk and safer behaviour. Sociology of Health and Illness, 20(3), 381–400. https://doi.org/10.1111/1467-9566.00107

18.

Hughes

Huby

(2002). The application of vignettes in social and nursing research. Journal of Advanced Nursing, 37(4), 382–386. https://doi.org/10.1046/j.1365-2648.2002.02100.x

19.

Jagosh

(2020). Retroductive theorizing in Pawson and Tilley's applied scientific realism. Journal of Critical Realism, 19(2), 121–130. https://doi.org/10.1080/14767430.2020.1723301

20.

Justin

Hannah

Serena

Rachel

Cathy

Margaret

Fiona

Paula

Dave

Nicola

E. W.

(2022). Benefits of realist evaluation for rapidly changing health service delivery. BMJ Open, 12(7), Article e060347. https://doi.org/10.1136/bmjopen-2021-060347

21.

Kvale

(2007). Doing interviews. https://methods.sagepub.com/book/doing-interviews

22.

Kvale

Brinkmann

(2015). InterViews. Sage Publications. https://books.google.ca/books?id=EgsvnwEACAAJ

23.

Lacouture

Breton

Guichard

Ridde

(2015). The concept of mechanism from a realist approach: A scoping review to facilitate its operationalization in public health program evaluation. Implementation Science: IS, 10(153), 1–10. https://doi.org/10.1186/s13012-015-0345-7

24.

Leeuw

F. L.

(2003). Reconstructing program theories: Methods available and problems to be solved. American Journal of Evaluation, 24(1), 5–20. https://doi.org/10.1177/109821400302400102

25.

Lemire

Kwako

Nielsen

S. B.

Christie

Donaldson

Leeuw

(2020). What is this thing called a mechanism? Findings from a review of realist evaluations. New Directions for Evaluation, 2020(167), 73–86. https://doi.org/10.1002/ev.20428

26.

Lewis-Beck

Bryman

Futing Liao

(2004). The SAGE encyclopedia of social science research methods. Sage.

27.

Manzano

(2016). The craft of interviewing in realist evaluation. Evaluation, 22(3), 342–360. https://doi.org/10.1177/1356389016638615

28.

Martin

Bergeron

Kusion

P.-Y.

Gaboury

(2023). The champion to promote organizational change: Theoretical causal relationships. Submitted.

29.

Mukumbang

F. C.

Marchal

Van Belle

van Wyk

(2019). Using the realist interview approach to maintain theoretical awareness in realist studies. Qualitative Research, 20(4), 485–515. https://doi.org/10.1177/1468794119881985

30.

Nielsen

S. B.

Lemire

Tangsig

(2021). Unpacking context in realist evaluations: Findings from a comprehensive review. Evaluation, 28(1), 91–112. https://doi.org/10.1177/13563890211053032

31.

Ohly

Crossland

Dykes

Lowe

Moran

V. H.

(2019). A realist qualitative study to explore how low-income pregnant women use Healthy Start food vouchers. Maternal and Child Nutrition, 15(1), Article e12632. https://doi.org/10.1111/mcn.12632

32.

Palm

Hochmuth

(2020). What works, for whom and under what circumstances? Using realist methodology to evaluate complex interventions in nursing: A scoping review. International Journal of Nursing Studies, 109. https://doi.org/10.1016/j.ijnurstu.2020.103601

33.

Pawson

(1996). Theorizing the interview. British Journal of Sociology, 47(2), 295–314. https://doi.org/10.2307/591728

34.

Pawson

Sanjeev

(2009) Theory-driven evaluation of public health programmes. In Amanda

Mike

P. K.

(Eds.), Evidence-based Public Health: Effectiveness and efficiency. Oxford; (Oxford Academic, 1 Feb. 2010).

35.

Pawson

Tilley

(1997). Realistic evaluation. Sage.

36.

Pawson

Tilley

(2001). Realistic evaluation bloodlines. American Journal of Evaluation, 22(3), 317–324. https://doi.org/10.1177/109821400102200305

37.

Porter

(2007). Validity, trustworthiness and rigour: Reasserting realism in qualitative research. Journal of Advanced Nursing, 60(1), 79–86. https://doi.org/10.1111/j.1365-2648.2007.04360.x

38.

Renmans

Castellano Pleguezuelo

(2022). Methods in realist evaluation: A mapping review. Evaluation and Program Planning, 97 102209. https://doi.org/10.1016/j.evalprogplan.2022.102209

39.

Ridde

Robert

É.

Guichard

Blaise

Olmen

J. V.

(2012). Théorie et pratique de l’approche Realist pour l’évaluation des programmes. In Dagenais

Ridde

(Eds.), Approches et pratiques en évaluation de programmes. Presses de l’Université de Montréal. https://doi.org/10.4000/books.pum.5999

40.

Robert

É.

Ridde

(2014). L’approche réaliste pour l’évaluation de programmes et la revue systématique: De la théorie à la pratique. Mesure et évaluation en éducation, 36(3), 79–108. https://doi.org/10.7202/1025741ar

41.

Rossi

P. H.

Nock

S. L.

(1982). Measuring social judgments: The factorial survey approach. Sage. https://books.google.ca/books?id=pai6AAAAIAAJ

42.

Salter

K. L.

Kothari

(2014). Using realist evaluation to open the black box of knowledge translation: A state-of-the-art review. Implementation Science: IS, 9(1), 115. https://doi.org/10.1186/s13012-014-0115-y

43.

Sayer

(2000). Realism and social science. Sage Publications Ltd. https://sk.sagepub.com/books/realism-and-social-science

44.

Sheringham

Kuhn

Burt

(2021). The use of experimental vignette studies to identify drivers of variations in the delivery of health care: A scoping review. BMC Medical Research Methodology, 21(1), 81. https://doi.org/10.1186/s12874-021-01247-4

45.

Sriranjan

A. N.

Abrams

Wong

Park

(2020). Towards an understanding of GPs’ viewpoint on diagnosing postnatal depression in general practice: A small-scale realist evaluation. Primary Health Care Research and Development, 21(e42), 1–8. https://doi.org/10.1017/S1463423620000316

46.

The RAMSES II Project . (2017a). Quality standards for realist evaluation. https://ramesesproject.org/media/RE_Quality_Standards_for_evaluators_and_peer_reviewers.pdf

47.

The RAMSES II Project . (2017b). Retroduction in realist evaluation. https://www.ramesesproject.org/media/RAMESES_II_Retroduction.pdf

48.

Tremblay

Turcotte

Touati

Poder

T. G.

Kilpatrick

Bilodeau

Roy

Richard

P. O.

Lessard

Giordano

É.

(2022). Development and use of research vignettes to collect qualitative data from healthcare professionals: A scoping review. BMJ Open, 12(1), Article e057095. https://doi.org/10.1136/bmjopen-2021-057095

49.

Westhorp

(2018). Doing realist research. Sage.

50.

Wong

Westhorp

Manzano

Greenhalgh

Jagosh

Greenhalgh

(2016). RAMESES II reporting standards for realist evaluations. BMC Medicine, 14(1), 96. https://doi.org/10.1186/s12916-016-0643-1

51.

Wong

Westhorp

Greenhalgh

Manzano

Jagosh

Greenhalgh

(2017). Quality and reporting standards, resources, training materials and information for realist evaluation: The RAMESES II project. https://doi.org/10.3310/hsdr05280

52.

Wong

Westhorp

Pawson

Greenhalgh

(2013). Realist synthesis: RAMESES training materials. https://www.ramesesproject.org/index.php?pr

53.

Wynn

Williams

C. K.

(2012). Principles for conducting critical realist case study research in information systems. MIS Quarterly, 36(3), 787–810. https://doi.org/10.2307/41703481

The Use of Vignettes to Improve the Validity of Qualitative Interviews for Realist Evaluation

Abstract

Keywords

Background

Realist Interview Principles

Phase 1: Theory Gleaning Interviews

Phase 2: Theory Refinement Interviews

Phase 3: Theory Consolidation Interviews

Retroduction in Realist Evaluation

The Use of Vignettes in Qualitative Research

Qualitative Vignettes in Realist Evaluation

Step 1: Research Problem and Initial Program Theory

Step 2: Vignette Writing and Construction

Step 3: Sampling of Participants and Vignette Adaptation

Step 4: Interviewing Using the Vignette

Conclusion

Footnotes

Declaration of Conflicting Interests

Funding

Ethical Statement

ORCID iD

References