Sage Journals: Discover world-class research

Abstract

Realist evaluation methodology aims to understand social programmes by revealing what works, for whom, in what circumstances, and how and why. Realist evaluation starts with generating initial programme theories (IPTs), which are subsequently tested and refined systematically using a multi-methods approach. This article describes a case study of the utilisation of vignettes, or short hypothetical stories, as part of the teacher-learner cycles recommended within realist evaluation. First, we explore the methodological alignment between vignettes and realist evaluation. We then present a specific case example of the application of vignettes as a data collection tool and discuss the potential advantages and the challenges of using vignettes within realist evaluation. Finally, we offer recommendations for researchers who wish to employ vignettes as a powerful instrument that can be used to better explain IPTs to participants and, in turn, enrich their participation in theory refinement within the realist evaluation framework.

Keywords

realist evaluation methodology theory refinement vignettes teacher-learner cycles realist interview

Introduction

Realist Evaluation – What is it and Why is it Important?

Developed by Pawson and Tilley (1997), realist evaluation is a theory-driven evaluation methodology used to understand social programmes and produce policy-relevant recommendations. Whereas traditional experimental questions ask, ‘does it work?‘, realist evaluation is an applied research methodology which aims to answer the question ‘what works, for whom, in what circumstances, and how and why?’ across various contexts. In this way, realist evaluation offers an explanatory focus for academics and programme stakeholders (Emmel et al., 2018; Pawson, 2013; Pawson & Tilley, 1997; Wong et al., 2017).

Realist evaluation attributes the changes associated with social interventions to ‘generative causation’, described as the triggering of underlying mechanisms within certain contexts, and what outcomes this interaction of mechanisms and contexts generates (Pawson, 2013). Generative causation is summarised by Pawson and Tilly using the representative heuristic: Context-Mechanism-Outcome configurations (CMOCs). Context, the pre-existing conditions within which the intervention is implemented (Jagosh, 2019), can be defined as the social rules, values and sets of relationships that either support or constrain the activation of programme mechanisms (Pawson & Tilley, 1997). Mechanisms describe people’s reactions and reasoning, including beliefs, values, desires and cognitive processes, towards what the social programme offers (Jagosh et al., 2015). In the realist explanation, a mechanism is sometimes described as a ‘weaving process’ between the resources offered to the programme’s stakeholders and their reasoning for those resources (Pawson & Tilley, 1997). Mechanisms are often hidden, operate on a different level of the system, and are sensitive to variations in contexts (Astbury & Leeuw, 2010). Outcomes can be intended or unintended, apparent or obscure, and they are the generative products of the context-triggered mechanisms. Realist evaluation looks at this pattern from a theoretical perspective, whereby the analysis brings about the real causes of outcomes (Jagosh et al., 2015; Pawson & Tilley, 1997).

To answer ‘what works, for whom under what circumstances’, the realist evaluator starts with one or more initial theory (ies) on how the programme is expected to work. These initial programme theories are subsequently refined throughout the evaluation process by uncovering generative causation present within case studies of programmes. Data collection methods and tools are thus purposefully designed to test and refine the preceding theories using ‘retroductive reasoning’ to identify generative causation.

Retroduction is a mode of interference that allows theorising and testing for hidden causal mechanisms responsible for manifesting the observed pattern (Jagosh, 2020). Within realist evaluation, retroduction is the process of developing theories about conditions through observing a phenomenon of interest. In this process, the researcher traces the fundamental conditions of phenomena backwards from the empirical (i.e., what is observed) to the actual (i.e., the events/actions) to the real (e.g., the mechanisms) (Mukumbang, 2021) by using personal hunches and experiences, alongside inductive, deductive, and abductive thinking to reproductively theorise (Greenhalgh et al., 2017a, Greenhalgh et al., 2017c; Pawson & Tilley, 1997). Ultimately, the refined theories lay at the level of Middle-Range (MRT) (Merton, 1968). As MRTs represent a higher level of abstraction, they allow for the transferability of realist evaluation results across different programmes (Pawson & Tilley, 1997, p. 123).

Data Collection and Interviewing in Realist Evaluation

Realist evaluations lend themselves to multi-method approaches: quantitative data are helpful in observing outcomes patterns and qualitative data are considered useful for discovering the underlying mechanisms and comparisons that explain context variations (Pawson, 2013). While not all realist evaluations use a mixed methods approach, most realist evaluations, if not all, include some qualitative research components, with interviews being the most widely used data collection tools (Manzano, 2016).

Pawson and Tilley have recommended a process of ‘teacher-learner cycle’ for theory-driven interviewing with a realist evaluation. To accomplish this, the researcher and the interviewee engage in cycles of exchanges using a teacher-learner approach. This approach introduces theories and theoretical thinking to the research participants. This is followed by a conceptual refinement function, through which interviewees give feedback on their thoughts and refine the researcher’s hypothesis (Pawson & Tilley, 1997). Here, the roles of the teacher and learner are not constant but alternate during the thinking process to help understand the complexities of the programme: Evaluators are expected to ‘teach’ interviewees about their theories and then be ‘taught’ back by the interviewee’s own theories. Therefore, the conceptual refinement process happens in between (Manzano, 2016).

Recent reflections have noted several challenges inherent to interviewing within realist evaluation. Challenges include a lack of guidance and few available methodological supports (Brönnimann, 2022) and the applicability of traditional didactic theory-driven interviewing styles that can be challenging across some participant groups (Gilmore, 2019). Moreover, the literature on interviewing within realist evaluation is often limited to one-to-one approaches (Manzano, 2016), with fewer examples of realist interviewing that use group (i.e., focus group) interviews, despite their common use within qualitative research (Manzano, 2022). In addition to specific interview challenges, difficulties within realist evaluation can also influence the interview process. Notably, researchers have noted difficulties distinguishing between Mechanisms and Contexts Fields (Pedersen, et al., 2012) and uncovering either throughout data collection (Marchal et al., 2012). This challenge is compounded by the inconsistent conceptualisation and operationalisation of these key realist evaluation concepts within the literature (Greenhalgh & Manzano, 2021).

Alternative methods, such as photovoice (Mukumbang & van Wyk, 2020), Wordclouds and artifacts (Teeling et al., 2022), and vignettes (Cooper et al., 2020), typically used within wider social and/or participatory research fields, may offer solutions to address the aforementioned methodological challenges with theory-driven interviewing. Specifically, and while not originally designed to conduct realist interviews, applying these methods within realist evaluations could promote the engagement of participants, either one-on-one or as a group, such that programme theories can be presented and subsequently refined.

Methodological Insights/Contribution

Calls for more guidance on conducting realist interviewing, particularly with participant groups that might require different approaches to properly elicit and refine theories (Gilmore, 2019), are prevalent throughout the literature (Wong et al., 2017). This paper uses a detailed case example to present guidance and recommendations on using vignettes to support realist data collection within focus group discussions. The utility of vignettes to support realist interviews and theory refinement is demonstrated, firstly, (i) by describing what vignettes are and how they align with realist methodology; secondly, (ii) by using a case study to describe how to develop and present vignettes for theory refinement; and finally, (iii) by reflecting on the use of vignettes within realist evaluations and providing recommendations for when and for whom the use of vignettes may best support theory refinement realist evaluation research.

Vignettes

Vignettes are short stories or situations about hypothetical events in specified circumstances (Gourlay et al., 2014). They are often presented in written format but can also be presented as pictures or videos. First used to evaluate the normative responses of people in survey research, especially concerning sensitive situations (Finch, 1987), vignettes can help to contextualise abstract thoughts. The development of vignettes has since led to their broader use in both quantitative (i.e., survey tools with fixed answers) and qualitative (i.e., framing out topics for more accessibility) research (Kandemir & Budd, 2018, March). Vignettes can also be used to interpret actions in specific situations, clarify judgements on complicated topics, such as moral dilemmas, discuss sensitive experiences, or can serve as introductory ice-breakers or wind down an interview session (Barter & Renold, 1999) and are often used to explore more sensitive topics (Gourlay et al., 2014) and for research with children and youth (Teachman, 2018). Vignettes have been successfully utilised across different cultures, with documented advantages in facilitating understanding and transformation of concepts (Crafter et al., 2015; Soydan & Stål, 1994; Torres, 2009), and for the sake of evaluating the comprehension within humanitarian global settings (Jordans et al., 2012). Their flexible nature (Schoenberg & Ravdal, 2000) thus allows one to use vignettes as a data collection tool and adapt their use depending on the iterative refinement process.

Vignettes within Realist Evaluation and the Realist Interview

The approach and purpose of vignettes are well aligned with key realist evaluation tenets, and their use offers a promising approach to address the aforementioned challenges within realist research. Firstly, and as vignettes are commonly used to elicit and understand participants’ responses to specific situations and contexts (Wilks, 2004), vignettes could assist with the uncovering of generative causation within realist evaluation. Specifically, the simplistic representation of situations within vignettes can help to understand real-world complexities and processes, providing important insights into an individual’s social world, including their values and beliefs. Moreover, as vignettes facilitate respondents to explore and discuss social phenomena ‘in their own terms’ (Gourlay et al., 2014), vignettes can help understand the empirical, actual, and real layers within retroductive thinking, especially as a way to ask about situations that we cannot directly observe. As they work to translate an experience, reasoning, or thought into a ‘real’ world example, vignettes have the potential to support the elucidation of generative causation of a particular phenomenon.

Second, vignettes have the potential to facilitate the “I’ll show-you-my-theory-if-you’ll-show-me-yours” (Pawson, 1996, p. 307) approach, as a core concept of the teacher-learner cycle, whereby an organised narrative of an assumption (e.g., an IPT) can be presented in such a way that allows the respondent(s) to reflect on and clarify that assumption (i.e., refining) (Pawson, 1996). Third, vignettes may enhance participation and increase accessibility and understanding of the research topic and questions. Fourth, vignettes may be particularly important for cross-cultural interviewing, where insights into the context or responses may be challenging to uncover. Finally, as vignettes are typically hypothetical, vignettes presented within a focus group could lend themselves to debate while limiting the risk of judgement, thereby facilitating openness. In the same vein, vignettes can be helpful to overcome the threats of superficially agreeing with the researcher’s hypotheses, as a noted risk of using the teacher-learners approach (Greenhalgh et al., 2017b), given their neutral distance from both researcher and the participant. This can trigger deeper levels of discussion and higher engagement of participants. Taken together, the use of vignettes has the potential to contribute significantly to the theory refinement processes within realist evaluation.

The use of vignettes within a realist evaluation may also present challenges. For instance, ensuring vignettes represent the participants and are properly understood is essential and may require additional time and resources compared to other techniques. Additionally, given their in-depth nature, the number of vignettes used within an interview may need to be limited to ensure sufficient time for discourse. As such, researchers may need to either limit the theories or generative causation that are presented or present theories at a higher level of abstraction, such that vignettes do not present too many ideas but still give sufficient attention to individual elements. While this may not allow for the collection of detailed information on all aspects of generative causation for theory refinement, participants will likely focus on what is more relevant and important.

A Case Study of Vignettes in Practice

Background: Purpose and Objectives of the Original Study

While humanitarian workers and volunteers are highly susceptible to adverse mental health outcomes, including burnout, post-traumatic stress disorder, depression and anxiety (Cardozo et al., 2005, 2012; Connorton et al., 2012; Corey et al., 2021; Thormar et al., 2010), there is growing evidence suggesting that organisational and managerial factors within humanitarian organisations can mitigate those adverse outcomes (Ager et al., 2012; Aldamman et al., 2019; Cardozo et al., 2013; Thormar et al., 2013). In light of this, a realist evaluation was conducted in 2018 aiming to develop an MRT for ‘how, why, and for whom organisational factors impact the well-being of humanitarian workers’ (Aldamman, 2020) using a case study of humanitarian volunteers located in the White Nile Branch of the Sudanese Red Crescent Society (SRCS). These volunteers regularly respond to seasonal floods, ongoing political instability, and a high influx of forcibly displaced persons from neighbouring countries.¹ The overall study design was a mixed-methods realist evaluation with four phases (see Figure 1).

Figure 1.

Overall realist study design. The orange box denotes where Vignettes were used.

In Phase 1, a desk review of SRCS volunteering documents, a field visit, informal observations and meetings, and current evidence and theories on volunteer mental health were all utilised to formulate rough theories. Phase 2 included the development of initial programme theories (i.e., IPTs), whereby, and in line with mixed method approaches, the rough theories generated in Phase 1 benefited from the findings of a quantitative study of N = 409 SRCS volunteers (Aldamman et al., 2019), as well as five formal interviews with experts in volunteering within the context of the SRCS. The resulting IPTs were tested and systematically refined in Phase 3, which involved a field case study of the SRCS White Nile branch, including eight interviews with SRCS volunteer leaders and six Focus Group Discussions (FGDs) with volunteers. In total, 51 individuals participated in these interviews and FGDs. During this third phase of the research, vignettes were brought in to support participatory theory refinement during the FGDs. Lastly, Phase 4 of the research included synthesising the results of Phase 3 into a middle-range theory for how, why, for whom, and in which contexts volunteer management practices impact volunteer mental health.

The Use of Vignettes during Focus Group Discussions – A Case Study Example

Departing from the one-to-one interviews, FGDs help capture the agreements and disagreements between participants, allowing for context variations and offering a more realistic account of how people think and experience a phenomenon. FGDs also help build a collective sense-making among participants (Bryman, 2012), which is particularly vital for theory refinement. FGDs are a commonly used method in RE as they arguably offer the group reasoning on how a given programme works (Manzano, 2022). FGDs were also a familiar tool for SRCS volunteers, where they are commonly used for need assessments and training. Hence, they facilitated conducting RE in a non-Western context and helped overcome any unintended power imbalances between the researcher and participants.

Aligned with Lune and Berg’s (2017) recommendations for FGDs, efforts were also implemented to ensure all participants’ confidentiality, safety, respect and engagement. To encourage participant engagement in the discussions, the FGD was preceded by the setting of mutually agreed upon co-created ground rules. These rules included building on each other’s contributions, refraining from interrupting or speaking over others to ensure smooth voice recording, embracing diverse opinions even if they differ from our own, maintaining the confidentiality of the discussions, seeking clarification through questions when something is unclear, and respecting the designated time for the FGD and breaks. Initially, the first set of questions was related to stressors volunteers face in their work and life to better understand their experiences and to be able to refer to these stressors when testing the IPTs using the vignettes.

Departing from the traditional FGDs setting and in adherence with realist evaluation interviewing processes, FGDs were designed and utilised in a theory-driven way. In this way, the questions asked and methods used to moderate the FGD reflected the ‘teacher-learner’ or “I’ll show-you-my-theory-if-you’ll-show-me-yours” (Pawson, 1996, p. 307) approach to theory refinement. Specifically, to accommodate the challenge of introducing IPTs to the volunteers in an engaging and accessible way, vignettes were designed and used as a simplified method of ‘teaching’ FGD participants about the IPTs. Aligned to the ‘teacher’ component of the realist interview, clearly putting forward stories (which represented programme theories) was thought to provide participants with a digestible and contextually informed situation representing the theory to be explored. Doing so was thus intended to address two specific challenges: (i) the difficulty in getting participants to openly and actively discuss the IPTs and (ii) the difficulty in getting participants to understand the process of refining the IPTs. Before proceeding with vignettes, support was also received from an online realist community of practice (RAMESES), who further cautioned about the importance of volunteers discussing their real-life situations rather than focusing on fictitious ones.

Vignettes were also designed as one vignette per IPT. Each vignette thus reflected a story about a volunteer. Stories were simple and contained the IPT, with core words, such as those referring to a potential Context, Mechanism, or Outcome, emphasised by writing them in a different colour. The names and context in vignettes were also adapted for Sudanese culture. For example, the vignette presented in Figure 2 uses traditional Sudanese names, and the incident (i.e. a road traffic accident) is one of the most prevalent risks in Sudan. The situations discussed were aimed at the level of middle-range. The stories were specific enough for the case study participants but could be extrapolated to other humanitarian volunteer contexts.

Figure 2.

Example of a Vignette used as part of the FGD with White Nile branch SRCS volunteers.

The same vignettes were used across all FGDs. In total, 6 FGDs occurred, with 43 participants. Several steps were taken to utilise the vignette for the realist interview thoroughly:

First, before presenting the vignettes in a FGD, a description of vignettes was presented to the participants. It was explained that vignettes are stories used to help focus on a topic and that these did not reflect a story about an individual(s) in the group. A discussion on etiquette within the FGD was held, where participants proposed appropriate ground rules for the discussion. Next, vignettes were presented on a flipchart and read aloud to participants. After the vignette was read and participants were given considerable time to reflect on and digest the scenario, participants were presented with two general questions designed to elicit personal opinions about the story. These general questions were broad and used to encourage the thinking process among volunteers (e.g., How can leaders help volunteers facing adversity?).

Next, group discussions and reflections occurred. This often led to participants discussing their own situations. As the emphasis was placed on ensuring that the discussions focused on real, lived experiences rather than fictitious accounts, participants’ own experiences were further explored. The researcher allowed the discussion among the participants but asked questions to clarify or dive deeper into topics. At this time, the researcher took detailed notes and began the process of CMOc elicitation.

The researcher then asked questions related to the topic. These questioning methods were applied to create a deeper level of discussion, which helped the participants refine their thoughts and identify generative causation. These included questions such as: “why does this matter…? “does that happen differently for…” or “why do you think … occurs”? Responses differing across volunteers were also highlighted, asking them to identify or hypothesise as to “why this difference responses might happen?“, employing the Retroductive theorising. Questions were specifically asked for both clarifying purposes and to draw participants’ attention to components of the IPT that had received less attention during the discussion and were not yet clear or addressed. In addition, key points from previous FGDs were presented to participants for feedback in the form of “other volunteers have said that… what do you think?“. Lastly, participants were asked to list examples and experiences after agreeing on refining or supporting an IPT. This worked to provide a level of plausibility to the refined theories.

Refinement of the IPTs was not iterative across the FGDs, as each FGD presented the original vignettes. As such, at the end of each FGD, there were FGD-specific potential refinements. After completing all FGDs, a systematic analysis of the FGD recordings led to refined theories. These were then synthesised to present an overarching middle-range theory. Throughout the FGDs, the researcher was focused on being attentive and allowing for individuals' opinions and experiences to emerge, reminding the group to respect these differences, where applicable. In the current example, the vignettes were utilised solely during the FGDs and to explain the IPTs. Figure 3 presents an overall schematic of the steps used during the Focus Group Discussions when presenting the vignettes.

Figure 3.

The steps followed in the FGDs while presenting vignettes.

Discussion and Reflection

Vignettes in this study were used as a data collection tool to help introduce participants to the theory to be refined and to help facilitate the theory-driven interview. The vignettes were successful in helping volunteers be more engaged in the study, encouraging them to participate and, more importantly, to understand the programme theories such that they could reflect on them in light of their experiences within the SRCS. This approach is particularly important given the recognised challenges realist researchers face when explaining realist methodology principles, including the need to ensure theories are not overly complex and easily understandable (Gilmore, 2019).

While this may not have been the first attempt to use vignettes under the realist evaluation framework, to our knowledge, this paper is the first to explain the methods used in detail. Since then, we have noted a few other examples of using vignettes within the realist evaluation framework. Kohli et al. (2021) used vignettes within a realist evaluation to examine the shifts in social norms and behaviours to improve the health and well-being of adolescent girls in Senegal. In contrast, however, they used the vignettes to quantitatively measure changes in social norms among programme participants, not using a theory-driven approach. Sriranjan et al. (2020) used them as part of exploring the GPs’ perspectives on diagnosing postnatal depression in their patients. While those vignettes were built based on an implicit theoretical framework, they were not explicitly incorporated into teacher-learner cycles. Belrhiti et al. (2020) used vignettes to guide and trigger discussions around the role of leadership in the motivation of health workers in public hospitals. As a sort of theory-implicit-situations, they used short texts to exemplify drivers of performance and retention among participants. Galazka et al. (2021) used vignettes to present their refined theories on social outcomes of community care interventions for persons with lower limb wounds, but with fewer details on how they utilised those tools. Cooper et al. (2020) used vignettes as part of a realist evaluation of risky behaviours among adolescents to ensure neutral and balanced feedback on the programme theories. Similar to our approach, they used hypothetical situations to encourage participants to reflect on their experiences. They found that using vignettes for theory teaching was helpful, particularly in its visual form (Cooper, 2018).

Taken together, vignettes offer a participatory, innovative approach that can potentially address some documented challenges within the realist literature. What is lacking is more guidance, and examples on utilising vignettes in realist interviewing, whereby accumulating evidence may further improve their usage and applicability to realist evaluation. We contribute to this process by offering several recommendations based on our experience to help researchers develop and use the vignettes.

Recommendations

Reflecting on our experience using vignettes within a realist evaluation, we put forward the following recommendations. First, researchers need to understand as much as possible about the population and the context before developing the vignettes. Being able to speak the same language, having organisational and volunteering experience, and having prior experience in training and facilitating focus groups were instrumental in drafting the core ideas of each vignette but still insufficient to ensure their compatibility with the participant’s lived experiences. Familiarising oneself with culturally relevant names and understanding the volunteers’ living conditions and day-to-day practices in the current case study was crucial to vignette development.

Specific recommendations aligned to different vignette development and use are presented in Table 1. Overall, IPTs should be developed into short, straightforward stories. This means it may be impossible to include a complete CMO should the vignette become too long. Before presenting vignettes to participants, which we recommend being read aloud, one should explain the purpose of the vignettes and how they will be used in the session. The questions posed after each vignette should be written as open-endedly as possible. The discussion should also be based on the participant’s experiences, not those of the people in the vignette. Different versions of the stories should also be developed if participants do not understand. Notably, vignettes are only one tool and should be considered an introductory tool for teaching-learning. In this way, vignettes should not be used as a stand-alone data collection method. Instead, vignettes should be followed by realist interviewing questions. In sum, using vignettes can be useful to overcome several recognised challenges within realist approaches by supporting realist interviewing. Future realist evaluations might therefore benefit from adopting this method.

Table 1.

Key Considerations for the Use of Vignettes in Realist Evaluations.

Steps	Questions/Considerations
Exploring the use of vignettes	• Is my participant group one that will benefit from a tool to support theory refinement? • What I am expecting from using vignettes? And for what reason(s) should they be included in my methods? • What types of methods/tools are acceptable and aligned to local traditions? • Do I have the skill-set, or can I develop the skill-set, to use vignettes? • Do I need to use the vignettes in a group or an individual setting?
Understanding participants and contexts	• Are there specific language, literacy and resource considerations? • What are culturally relevant stories, including names and examples? • Are there any traditional ways stories are presented and shared? • Who are my cultural references to check whether the scenarios are understood and culturally sensitive?
Developing the vignettes	• Level of abstraction should allow participants to situate their own experience(s) within the examples • Open-ended questions should trigger discussions and help in the sense-making process • Not too long, but length to discuss needs to be considered • Distinguishing key CMO components • More than one vignette for each IPT could be considered
Presenting the vignette	• Explain the purpose of the vignette exercise beforehand • Emphasise that the discussions should be grounded in the group’s experiences, not the scenario’s characters • Read aloud • Allow time for participants to consider/reflect on the vignette before discussing
Discussion	• Discuss alongside theory-driven questions • Participant example led • Record the refinement suggestions within an interview setting and consider how are you going to bring those to the upcoming settings

Conclusion

Interviewing within realist evaluation requires the researcher and participant to engage in theory refinement through a theory-driven interview. In addition to little guidance on conducting a ‘realist interview’, challenges with the realist theory-driven interview have been noted, leaving many facing obstacles when collecting qualitative data or using techniques that may not always provide the best opportunity for identifying generative causation. The use of vignettes to support the realist interview is aligned with realist epistemology and methodology. Its application can support a more accessible and in-depth exploration of presented programme theories within certain population groups. Considerations when proposing, developing and utilising vignettes are put forward to ensure vignettes are relevant to the population, context, and theories to be refined and appropriately used to best support theory-driven interviewing.

Footnotes

Acknowledgements

The authors thank the IFRC reference centre for psychosocial support, the Danish Red Cross, the Swedish Red Cross, and the Sudanese Red Crescent Society (SRCS) for facilitating the research. The authors are most grateful to the SRCS volunteers who participated in the research activities.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

This study was conducted as a part of the CONTEXT programme. CONTEXT has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie Grant Agreement No. 722523.

ORCID iDs

Kinan Aldamman

Frédérique Vallières

Brynne Gilmore

Note

References

Ager

Pasha

Duke

Eriksson

Cardozo

B. L.

(2012). Stress, mental health, and burnout in national humanitarian aid workers in gulu, northern Uganda [article]. Journal of Traumatic Stress, 25(6), 713–720. https://doi.org/10.1002/jts.21764

Aldamman

(2020). Managerial practices to ensure the well-being of humanitarian volunteers: A realist evaluation trinity college dublin]. Trinity College Dublin. School of Psychology.

Aldamman

Tamrakar

Dinesen

Wiedemann

Murphy

Hansen

Badr

E. E.

Reid

Vallieres

(2019). Caring for the mental health of humanitarian volunteers in traumatic contexts: The importance of Organisational Support.

Astbury

Leeuw

F. L.

(2010). Unpacking black boxes: Mechanisms and theory building in evaluation. American Journal of Evaluation, 31(3), 363–381. https://doi.org/10.1177/1098214010371972

Barter

Renold

(1999). The use of vignettes in qualitative research. Social Research Update, 25(9), 1–6. https://sru.soc.surrey.ac.uk/SRU25.html

Belrhiti

Van Damme

Belalia

Marchal

(2020). Unravelling the role of leadership in motivation of health workers in a Moroccan public hospital: A realist evaluation. BMJ Open, 10(1), e031160.

Brönnimann

(2022). How to phrase critical realist interview questions in applied social science research. Journal of Critical Realism, 21(1), 1–24. https://doi.org/10.1080/14767430.2021.1966719

Bryman

(2012). Chapter 21: Focus groups. In Bryman

(Ed), Social research methods (4th ed.). Oxford University Press.

Cardozo

Crawford

T. I.

Petit

Ghitis

W. F.

Sivilli

Scholte

Ager

Eriksson

(2013). Factors affecting mental health of local staff working in the Vanni region, Sri Lanka. Psychological Trauma: Theory, Research, Practice, and Policy, 5(6), 581–590. https://doi.org/10.1037/a0030969

10.

Cardozo

B. L.

Crawford

C. G.

Eriksson

Zhu

Sabin

Ager

Foy

Snider

Scholte

Kaiser

(2012). Psychological distress, depression, anxiety, and burnout among international humanitarian aid workers: A longitudinal study. PLoS One, 7(9), e44948. https://doi.org/10.1371/journal.pone.0044948

11.

Cardozo

B. L.

Holtz

T. H.

Kaiser

Gotway

C. A.

Ghitis

Toomey

Salama

(2005). The mental health of expatriate and Kosovar Albanian humanitarian aid workers [article]. Disasters, 29(2), 152–170. https://doi.org/10.1111/j.0361-3666.2005.00278.x

12.

Connorton

Perry

M. J.

Hemenway

Miller

(2012). Humanitarian relief workers and trauma-related mental illness. Epidemiologic Reviews, 34(1), 145–155. https://doi.org/10.1093/epirev/mxr026

13.

Cooper

(2018). In Aldamman

(Ed.), Personal communication.

14.

Cooper

Lhussier

Carr

(2020). Blurring the boundaries between synthesis and evaluation. A customized realist evaluative synthesis into adolescent risk behavior prevention. Research Synthesis Methods, 11(3), 457–470. https://doi.org/10.1002/jrsm.1407

15.

Corey

Vallières

Frawley

De Brún

Davidson

Gilmore

(2021). A rapid realist review of group psychological first aid for humanitarian workers and volunteers. International Journal of Environmental Research and Public Health, 18(4), 1452. https://doi.org/10.3390/ijerph18041452

16.

Crafter

de Abreu

Cline

O’Dell

(2015). Using vignette methodology as a tool for exploring cultural identity positions of language brokers. Journal of Constructivist Psychology, 28(1), 83–96. https://doi.org/10.1080/10720537.2014.923354

17.

Emmel

Greenhalgh

Manzano

Monaghan

Dalkin

(2018). Doing realist research. Sage.

18.

Finch

(1987). The vignette technique in survey research. Sociology, 21(1), 105–114. https://doi.org/10.1177/0038038587021001008

19.

Galazka

A. M.

Edwards

Harding

(2021). Realist evaluation of social outcomes in community care: The application of affordance theory to the lindsay leg clubs. Journal of Critical Realism, 20(3), 280–299. https://doi.org/10.1080/14767430.2021.1918969

20.

Gilmore

(2019). Realist evaluations in low- and middle-income countries: Reflections and recommendations from the experiences of a foreign researcher. BMJ Global Health, 4(5), e001638. https://doi.org/10.1136/bmjgh-2019-001638

21.

Gourlay

Mshana

Birdthistle

Bulugu

Zaba

Urassa

(2014). Using vignettes in qualitative research to explore barriers and facilitating factors to the uptake of prevention of mother-to-child transmission services in rural Tanzania: A critical analysis. BMC Medical Research Methodology, 14(1), 11–21. https://doi.org/10.1186/1471-2288-14-21

22.

Greenhalgh

Manzano

(2021). Understanding ‘context’in realist evaluation and synthesis. International Journal of Social Research Methodology, 25(5), 1–13. https://doi.org/10.1080/13645579.2021.1918484

23.

Greenhalgh

Pawson

Wong

Westhorp

Greenhalgh

Manzano

Jagosh

(2017a). Developing realist programme theories. The RAMESES II Project.

24.

Greenhalgh

Pawson

Wong

Westhorp

Greenhalgh

Manzano

Jagosh

(2017b). The realist interview. The RAMESES II project. http://www.ramesesproject.org/media/RAMESES_II_Realist_interviewing.pdf

25.

Greenhalgh

Pawson

Wong

Westhorp

Greenhalgh

Manzano

Jagosh

(2017c). Retroduction in realist evaluation. The RAMESES II project. http://www.ramesesproject.org/media/RAMESES_II_Retroduction.pdf

26.

Jagosh

(2019). Realist synthesis for public health: Building an ontologically deep understanding of how programs work, for whom, and in which contexts. Annual Review of Public Health, 40, 361–372. https://doi.org/10.1146/annurev-publhealth-031816-044451

27.

Jagosh

(2020). Retroductive theorizing in Pawson and Tilley's applied scientific realism. Journal of Critical Realism, 19(2), 121–130. https://doi.org/10.1080/14767430.2020.1723301

28.

Jagosh

Bush

P. L.

Salsberg

Macaulay

A. C.

Greenhalgh

Wong

Cargo

Green

L. W.

Herbert

C. P.

Pluye

(2015). A realist evaluation of community-based participatory research: Partnership synergy, trust building and related ripple effects [article]. BMC Public Health, 15(1), 725–811. https://doi.org/10.1186/s12889-015-1949-1

29.

Jordans

M. J. D.

Luitel

N. P.

Poudyal

Tol

W. A.

Komproe

I. H.

(2012). Evaluation of a brief training on mental health and psychosocial support in emergencies: A pre-and post-assessment in Nepal. Prehospital and Disaster Medicine, 27(3), 235–238. https://doi.org/10.1017/S1049023X12000738

30.

Kandemir

Budd

(2018). Using vignettes to explore reality and values with young people. Qualitative social research

31.

Kohli

Shaw

Guntzberger

Aubel

Coulibaly

Igras

(2021). Transforming social norms to improve girl-child health and well-being: a realist evaluation of the Girls’ Holistic Development program in rural Senegal. Reproductive Health, 18(1), 1–14. https://doi.org/10.1186/s12978-021-01295-5

32.

Lune

Berg

B. L.

(2017). Focus group interviewing. In Lune

Berg

B. L.

(Eds), Qualitative research methods for the social sciences (9th, global edition (pp. 94–106). Pearson Education Ltd

33.

Manzano

(2016). The craft of interviewing in realist evaluation. Evaluation, 22(3), 342–360. https://doi.org/10.1177/1356389016638615

34.

Manzano

(2022). Conducting focus groups in realist evaluation. Evaluation, 28(4), 406–425. https://doi.org/10.1177/13563890221124637

35.

Marchal

van Belle

van Olmen

Hoerée

Kegels

(2012). Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research. Evaluation, 18(2), 192–212. https://doi.org/10.1177/1356389012442444

36.

Merton

R. K.

(1968). Social theory and social structure: Free Press.

37.

Mukumbang

F. C.

(2021). Retroductive theorizing: A contribution of critical realism to mixed methods research. Journal of Mixed Methods Research, 17(1), 15586898211049847. https://doi.org/10.1177/15586898211049847

38.

Mukumbang

F. C.

van Wyk

(2020). Leveraging the Photovoice methodology for critical realist theorizing. International Journal of Qualitative Methods, 19, 1609406920958981. https://doi.org/10.1177/1609406920958981

39.

Pawson

(1996). Theorizing the interview. British Journal of Sociology, 47(2), 295–314. https://doi.org/10.2307/591728

40.

Pawson

(2013). The science of evaluation: A realist manifesto. Sage.

41.

Pawson

Tilley

(1997). Realistic evaluation. Sage Publications, Inc. http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=1997-36931-000&site=ehost-live

42.

Pedersen

L. M.

Nielsen

K. J.

Nielsen

(2012). Realistic evaluation as a new way to design and evaluate occupational safety interventions. Safety science, 50(1), 48–54. https://doi.org/10.1016/j.ssci.2011.06.010

43.

Schoenberg

N. E.

Ravdal

(2000). Using vignettes in awareness and attitudinal research. International Journal of Social Research Methodology, 3(1), 63–74. https://doi.org/10.1080/136455700294932

44.

Soydan

Stal

(1994). How to use the vignette technique in cross cultural social work research. Scandinavian Journal of Social Welfare, 3(2), 75–80. https://doi.org/10.1111/j.1468-2397.1994.tb00060.x

45.

Sriranjan

A. N.

Abrams

Wong

Park

(2020). Towards an understanding of GPs’ viewpoint on diagnosing postnatal depression in general practice: A small-scale realist evaluation. Primary Health Care Research and Development, 21, e42. https://doi.org/10.1017/S1463423620000316

46.

Teachman

(2018). Optimizing interviews with children and youth with disability. In Liamputtong

(Ed), Handbook of research methods in health social sciences (pp. 1–18). Springer.

47.

Teeling

S. P.

Dewing

Baldie

(2022). Developing new methods for person-centred approaches to adjudicate context–mechanism–outcome configurations in realist evaluation. International Journal of Environmental Research and Public Health, 19(4), 2370. https://doi.org/10.3390/ijerph19042370

48.

Thormar

S. B.

Gersons

B. P. R.

Juen

Djakababa

M. N.

Karlsson

Olff

(2013). Organizational factors and mental health in community volunteers. The role of exposure, preparation, training, tasks assigned, and support. Anxiety, Stress & Coping, 26(6), 624–642. https://elib.tcd.ie/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=edswss&AN=000325959800003&site=eds-live&scope=site

49.

Thormar

S. B.

Gersons

B. P. R.

Juen

Marschang

Djakababa

M. N.

Olff

(2010). The mental health impact of volunteering in a disaster setting: A review. The Journal of Nervous and Mental Disease, 198(8), 529–538. https://doi.org/10.1097/NMD.0b013e3181ea1fa9

50.

Torres

(2009). Vignette methodology and culture-relevance: Lessons learned through a project on successful aging with Iranian immigrants to Sweden. Journal of Cross-Cultural Gerontology, 24(1), 93–114. https://doi.org/10.1007/s10823-009-9095-9

51.

Wilks

(2004). The use of vignettes in qualitative research into social work values. Qualitative Social Work, 3(1), 78–87. https://doi.org/10.1177/1473325004041133

52.

Wong

Westhorp

Greenhalgh

Manzano

Jagosh

Greenhalgh

(2017). Quality and reporting standards, resources, training materials and information for realist evaluation: The RAMESES II project. Health Services and Delivery Research, 5(28). https://doi.org/10.3310/hsdr05280

Vignettes to Support Theory Refinement: Methodological Insights From a Realist Evaluation

Abstract

Keywords

Introduction

Realist Evaluation – What is it and Why is it Important?

Data Collection and Interviewing in Realist Evaluation

Methodological Insights/Contribution

Vignettes

Vignettes within Realist Evaluation and the Realist Interview

A Case Study of Vignettes in Practice

Background: Purpose and Objectives of the Original Study

The Use of Vignettes during Focus Group Discussions – A Case Study Example

Discussion and Reflection

Recommendations

Conclusion

Footnotes

Acknowledgements

Declaration of conflicting interests

Funding

ORCID iDs

Note

References