Abstract
Qualitative research is increasingly being integrated within the development and evaluation phases of trials of complex health interventions. Qualitative research can complement effectiveness data and provide insights around how context and implementation impact the results of the trial and to what extent interventions fit implementation contexts after evaluation. Several qualitative studies have been conducted at different stages of trials of mental health and psychosocial support interventions for populations affected by adversity such as armed conflict and disasters. However, these qualitative components are usually implemented as disjointed components within the trial. The current protocol aims to provide a shared framework detailing the qualitative components of the CHANGE project trial: a program of work to address alcohol misuse and associated mental health comorbidities among conflict-affected populations in Uganda and Ukraine. In particular, the objectives of the current protocol are (i) to identify the specific qualitative questions and methods that will be undertaken in CHANGE; (ii) characterize the different methodological approaches to analyzing the data; and (iii) explain how each qualitative component within the different work-packages will cumulatively add value to each other over the duration of the CHANGE project. The current protocol will represent a useful template for the integration of serial qualitative research components within complex health interventions in humanitarian settings. Each qualitative components described will undergo formal ethics approval by ethics boards in the United Kingdom, Uganda, and Ukraine. Multiple mechanisms will be in place to ensure rigor and trustworthiness of the research by meeting the criteria of credibility, transferability, dependability, and confirmability.
Keywords
Background
Trials of healthcare interventions in humanitarian settings are difficult to conduct because of the contextual constraints related to insecurity, instability, population movement, the need to culturally adapt interventions to local circumstances, and limited resources. Against this background, mixed methods are required to successfully develop, evaluate, and implement an intervention in a complex setting. It is well established that the application of mixed methods, particularly qualitative components incorporated in randomized controlled trials (RCTs), can elicit important information for the development of the intervention and can assist in explaining the dynamics between context and intervention on the primary outcomes of trials (Medical Research Council, 2008; O’Cathain et al., 2013; Rapport et al., 2013). This is particularly relevant for the development and evaluation of mental health interventions for which heterogenous outcomes at the population level can be explained by complex interplays between the characteristics of the participant, the person delivering a psychological intervention, specific components of the intervention, and external factors which may influence the therapeutic process (Cuijpers et al., 2019). Additionally, qualitative research is particularly appropriate to humanitarian settings which are, by definition, complex contexts where external factors are more likely to influence the implementation of the intervention and of the trial (Kohrt et al., 2019).
Qualitative studies have been conducted as part of RCTs of various mental health and psychosocial (MHPSS) 1 interventions in contexts affected by adversity (Greene, Rees, et al., 2019; Heim et al., 2021; Tol et al., 2018). One of the MHPSS interventions with demonstrated effectiveness in contexts of adversity is Problem Management Plus (PM+) (Bryant et al., 2017; de Graaff, Cuijpers, Acarturk, et al., 2020; Jordans et al., 2021; Rahman et al., 2019). Problem Management Plus is a brief, transdiagnostic, scalable intervention developed by the World Health Organization (Dawson et al., 2015). It is composed of five sessions and can be delivered by lay workers such as teachers and peers (World Health Organization, 2017).
Qualitative methods have been used at various stages in trials of PM+. For example, in the formative research phase, a rapid qualitative assessment to adapt PM+ for Syrian refugees in Jordan and Turkey led to 82 changes to the intervention manual, as well as to training, supervision, and implementation protocols (Akhtar et al., 2021). These changes included both changes to how material was presented (e.g., use of metaphors or images) as well as changes to the content of the intervention (e.g., inclusion of an additional session to enhance family engagement). At the process evaluation stage, some studies used interviews and focus groups with different stakeholders to assess the feasibility, acceptability, and other implementation outcomes of PM+ in Nepal (Sangraula et al., 2020), Pakistan (Khan et al., 2019), and the Netherlands (de Graaff et al., 2020) at the pilot RCT stage. Findings from these studies confirmed the acceptability and feasibility of delivering PM+ in these settings. Qualitative methods have also explored the potential scale-up of PM+ (Fuhr, Acarturk, Sijbrandij, et al., 2020). Beyond PM+, qualitative methods have also been used at the formative stage (Abi Ramia et al., 2018; Chiumento et al., 2020; Murray et al., 2018; Singh et al., 2021) and at the process evaluation stage (Greene, Rees, et al., 2019; Heim et al., 2021; Tol et al., 2018) of various other MHPSS interventions.
Although the use of qualitative methods during MHPSS RCTs is quite common, these qualitative components are usually implemented as disjointed units. There tends to be little attention to connecting qualitative research methodology across the different phases of a RCT, with little discussion of continuity between findings. This paper describes the protocol for the qualitative components of the CHANGE trial, a multi-site study which seeks to develop and implement a complex mental healthcare intervention among refugees and internally displaced persons (IDPs) in Uganda and Ukraine (LSHTM, 2020).
Populations affected by armed conflict are at higher risk for mental health problems (Charlson et al., 2019), and preliminary evidence indicates that this might be the case for alcohol use problems as well (Lo et al., 2017). Although psychological interventions for mental health problems exist, there are no open-access evidence-based MHPSS interventions available that also target comorbid alcohol misuse among populations living in humanitarian settings (Greene, Ventevogel, et al., 2019). To address this gap, the CHANGE project seeks to further develop PM+ by complementing it with psychosocial strategies addressing alcohol misuse (with the new intervention called PM+A (Problem Management Plus Alcohol)) (Fuhr et al., 2021). Problem Management Plus Alcohol will be evaluated in two settings: amongst IDPs, veterans, and persons living in proximity to the conflict line in Ukraine, and amongst South Sudanese refugees residing in a refugee camp in northern Uganda. A focus on these different population groups and settings means that the qualitative components of the CHANGE study have important comparative as well as developmental elements.
In the current paper, we aim to do the following: 1. Identify the specific qualitative questions and methods that will be undertaken in CHANGE. 2. Characterize the different methodological approaches to analyzing the data. 3. Explain how each qualitative component within the different work-packages will cumulatively add value to each other over the different phases of the project.
The current protocol will represent a useful template for other trials of complex health interventions in humanitarian settings that plan on including serial qualitative components.
Methods, Sampling, and Analysis
The CHANGE study consists of five phases of work. We describe the qualitative components during these five phases below. The different components are also summarized in a table in Appendix A.
Phase 1. Intervention development: This phase is concerned with adapting PM+ into an intervention responding to alcohol problems amongst men experiencing psychological distress and alcohol misuse in settings affected by conflict-related humanitarian crises: PM+A. Qualitative work in CHANGE will support the development of PM+A in the following ways. 1. Participant perspectives on alcohol in context: To inform the development of the PM+A intervention, we will undertake semi-structured interviews to explore key factors shaping alcohol use and misuse amongst men affected by conflict. We will recruit: (i) 20 refugees, 20 family caregivers, 10 MHPSS providers, and 10 community and religious leaders in Rhino Refugee Camp, northern Uganda; and (ii) 20 conflict-affected men, 20 family caregivers, and 20 service providers located across Ukraine. In Uganda, participants will be identified by refugee camp block leaders, whereas in Ukraine, participants will be identified by outreach through community members and organizations. Semi-structured interviews (in Juba Arabic or English in Uganda, and Ukrainian or Russian in Ukraine) will focus on: (a) participants’ definitions and causal explanations of alcohol problems and their relationship with conflict and displacement; (b) responses of significant others and the cultural contexts of alcohol misuse; (c) patterns of help-seeking, coping strategies, and barriers to care, and (d) the availability of services. English language versions of the interview topic guides are provided in Appendix B. Interviews will be transcribed, and thematic analysis will be applied (Braun & Clarke, 2006). Analysis will be done by site using NVivo 12 software (or locally available software such as Dedoose) and will focus on identifying, characterizing, and explaining social and psychological factors that may be important for the content of the PM+A intervention in its local contexts. 2. Understanding how PM+A should be implemented in the local contexts: Theory of Change (ToC) is a conceptual model that can aid in understanding “how and why an initiative works and how it should be implemented” (De Silva et al., 2014). The output of a ToC workshop is a map that presents a causal pathway of implementation that “can be empirically tested by measuring indicators for every step of the hypothesized causal pathway to impact” (De Silva et al., 2014). Three ToC workshops will be conducted in Uganda (in Kampala, Arua, and the Rhino refugee camp) and two in Ukraine. They will aim to develop a causal map describing the mechanisms by which the PM+A intervention will achieve its ultimate impact and outline barriers and facilitators toward implementation. To further explore the process of developing the ToC map, we will recruit a purposive sample of approximately five ToC participants per site to participate in semi-structured interviews that will explore (i) their understanding of the ToC process and its outcomes; (ii) the perceived value and utility of ToCs; (iii) the role of participants in determining the ToC map; and (iv) the perceived strengths and limitations of the resulting ToC map. Interviews will be transcribed, and thematic analysis will be applied to transcripts at each site (Braun & Clarke, 2006). Analysis will be assisted by NVivo 12 software. The topic guide to be used in this study is provided in Appendix C. 3. Treatment development workshops: To assemble the evidence drawn from systematic reviews and expert consensus into a coherent model of treatment, we will hold three treatment development workshops with an international group of expert contributors (n ≈ 25), including experts from Uganda and Ukraine. These workshops will be held virtually, and the transcripts of the recordings will constitute the formal data subjected to analysis. We will use two approaches to qualitative content analysis to explore these data: (i) within-site analysis of participant attributions (Miles & Huberman, 1994) will enable us to identify contending constructions of causal processes and explanations for treatment components and (ii) decision-tree modeling (Gladwin, 1989) to understand the ways in which participants construct preferences and choices about the pathways through which treatment components are likely to be applied. An important component of this analysis is understanding the shaping effects of social and cultural context as these are considered in relation to PM+A.
Phase 2. Pilot testing: Pilot testing will ensure that the initial model of PM+A developed in Phase 1 is adaptable to local circumstances, and that it is feasible, acceptable, and perceived as effective. We will undertake an uncontrolled before and after treatment cohort study in which PM+A will be delivered to participants in Uganda and Ukraine. This process will be characterized by iterative loops of evaluation and refinement of the intervention. Following this treatment cohort, a pilot RCT will be conducted to evaluate the methodological procedures prior of applying those in fully powered RCTs (in Ukraine and Uganda, respectively). Qualitative sub-studies will be nested in both these pilot studies to understand the dynamics of the intervention in use. 4. Treatment cohort: Qualitative end-of-session interviews with participants and facilitators: To assess the relevance and acceptability of the PM+A intervention in the local context, and to identify components of the intervention that might require modification, we will explore the views of both participants and facilitators of PM+A. During the treatment cohort, we will schedule brief 20–30 min interviews with a subset of participants of each sequential cohort at the end of each session (or at different time intervals depending on feasibility) focusing on (i) the relevance, (ii) cultural acceptability, (iii) comprehensibility, and (iv) perceived effectiveness of the session content and materials. Similar interviews will also be conducted with the facilitators. Interviews will play an important role in identifying barriers and facilitators to content and delivery of PM+A, as well as possible solutions to them. This data will inform any further content adaptations of PM+A that are necessary prior of testing the intervention in a pilot RCT. 5. Pilot RCT: End of treatment interviews: A pilot RCT will be conducted to assess the methodological procedures such as recruitment, randomization, and retention. It will also explore if PM+A versus enhanced usual care is acceptable, feasible, and safe. To achieve these objectives, we will conduct semi-structured interviews with intervention recipients (approximately five intervention completers, five drop-outs, and five participants in the control group, n = 15 in each site), facilitators (n = 5 in each site), supervisors (n = 2 in each site), and family members (n = 5–10 in each site). Interviews will be conducted no longer than 2 weeks after the final session of the intervention in each arm. Interviews will focus on (i) the experience of participating/facilitating the interventions, (ii) on perceptions around treatment adherence including barriers and facilitators to attendance and implementation of skills, (iii) perceived effectiveness and helpfulness, (iv) perceived acceptability and appropriateness of the intervention content and materials, and (v) opportunity costs. Additionally, interviews (approximately five in each site) will be conducted with the research team to investigate issues related to the trial procedure (e.g., issues around recruitment or screening).
In Phase 2, interviews will be transcribed, and thematic analysis will be applied to transcripts at each site (Braun & Clarke, 2006). Analysis will be assisted by NVivo 12 software or locally available software (e.g., Dedoose). Analysis will focus on identifying, characterizing, and explaining activities that promote or inhibit the effective delivery of the PM+A intervention in its local contexts, and factors that affect the effective operation of the broader set of trial procedures.
Phase 3. Evaluation: Effectiveness and cost-effectiveness of PM+A will be evaluated through fully powered RCTs in Uganda and Ukraine, respectively. 6. Fully powered RCTs: End of treatment interviews: As in the pilot RCT stage, semi-structured interviews will be conducted with intervention recipients, facilitators, family members, and supervisors to investigate intervention delivery, fidelity, dose, perceived effectiveness, feasibility, acceptability of the intervention, and opportunity costs for participants and facilitators. Additionally, we will investigate additional topics that were not explored at the pilot RCT stage such as potential sustainability of the intervention as well as potential for scalability. Interviews will be conducted with approximately the same number and type of participants as those included in the pilot RCTs and the same procedures will be followed.
Phase 4. Implementation: Little is known about the dynamics of implementation processes in interventions aimed at improving mental health and healthcare delivery in settings affected by forced migration and conflict. A robust and rigorous implementation analysis is therefore likely to be of value in understanding how such interventions can be effectively implemented in the future. Phase 4 will consist of two main components: 7. Understanding implementation as a temporal process: Qualitative sub-studies in Phases 1–3 will produce information about factors that promote or inhibit the implementation of PM+A. Interview transcripts and data summaries collected in sub-studies through the life of CHANGE will be explored using a prescheduled theoretical framework (May et al., under review) drawing on Normalization Process Theory (May & Finch, 2009) to perform comparative qualitative content analysis of implementation processes within and between sites (Hsieh & Shannon, 2005). These analyses will contribute to our understanding of the mechanisms that shape implementation of PM+A in practice. 8. Explaining differences in implementation between settings: Comparative analyses of qualitative data will help us understand differences in implementation and outcomes of PM+A between Ukraine and Uganda. We will undertake up to 10 semi-structured interviews with researchers at each site. We will present them with results from the longitudinal qualitative analysis noted above, and explore their accounts of specific differences in contexts, mechanisms, and outcomes of implementation, and present a robust conceptual model that identifies, characterizes, and explains factors that promote or inhibit the delivery and take-up of PM+A in both settings. This model will provide information useful to scaling up workshops to be held in Phase 5 (scale-up) of CHANGE.
Phase 5. Scale-up: The overall aim of this phase will be to examine the potential for scaling up PM+A in Uganda and Ukraine. We will investigate how PM+A could be scaled up through the health system (both government and humanitarian) and integrated within other humanitarian sector activities such as protection programs for survivors of gender-based violence. We will explore elements required to support vertical and horizontal scaling up of PM+A for refugees in Uganda and conflict-affected men in Ukraine, the potential barriers and facilitators for scaling up, and how scaling up for PM+A can best support health system responsiveness to the mental health and alcohol misuse needs of refugees and IDP. 9. Document and policy scoping review: The aim of the current component will be to conduct a document and policy scoping review to examine the risk environment (Rhodes, 2009) as well as other health system, political, socio-cultural, and economic factors that are likely to influence the potential scale-up of PM+A in Ukraine and Uganda. The scoping review will include publicly available documents from relevant governmental, inter-governmental, and non-governmental agencies related to relevant health policies, programs, and services. 10. Semi-structured interviews with key stakeholders on scaling up: The aim of this qualitative component will be to assess potential for scaling up PM+A, as well as potential barriers and facilitators, by conducting several semi-structured interviews with purposively selected key stakeholders. These will include, but will not be limited to, government officials at national and district levels, health system/service managers, donor agencies, NGO, academia, and service providers (n ≈ 30 per site). These interviews will further inform the ToC workshops on scaling up. Thematic analysis will follow inductive and deductive approaches to explore horizontal and vertical expansion of PM+A. 11. Interviews following ToC workshops on scaling up PM+A: Based on previous experience from the authors (Fuhr, Acarturk, Sijbrandij, et al., 2020; Fuhr, Acarturk, Uygun, et al., 2020), we will hold two ToC workshops in Uganda and Ukraine focusing on how to scale-up PM+A through the health systems and other humanitarian sector activities. Semi-structured interviews will be conducted with a subset of ToC participants. As for the interviews conducted following the ToCs in the development phase, the aim of the current qualitative components will be to investigate the decision-making process that led to the final ToC map detailing the causal pathways leading to PM+A scale-up in Uganda and Ukraine.
Cumulative Qualitative Analysis
Our protocol describes a set of discrete sub-studies within the CHANGE trial. Each of these has a specific value and impact, but importantly it is the cumulative analysis of these sub-studies and their data that is likely to have the greatest impact (e.g., investigating during the pilot testing phase whether the intervention is perceived to address drivers of alcohol misuse identified during the formative stage interviews). An overarching approach to these data involves thinking through their analysis and interpretation of commonalities and differences between settings and over time. The qualitative data itself is challenging here: collected in two country settings (Ukraine and Uganda); two social contexts (South Sudanese participants in a Ugandan refugee camp and Ukrainian participants dispersed across urban and rural areas in Ukraine); and speaking up to four languages (Juba Arabic, English, Ukrainian, and Russian).
To build a cumulative and longitudinal analysis of these data, online “data clinics” will take place through the duration of the project. These data clinics will constitute a structured way of working collaboratively to identify common features and discrepancies between the data produced in different settings and at different stages of the project. During these qualitative clinics, data summaries will be shared to identify elements of interest, and then discussed to think through possible implications, as well as to characterize significance and relations between settings and over time. Analytical outcomes will follow from interpretative agreements among the data clinic participants.
Ethics
Each qualitative component described will undergo formal ethics approval by ethics boards in the United Kingdom (London School of Hygiene and Tropical Medicine), Uganda (Mildmay Uganda Research Ethics Committee), and Ukraine (National University of Kyiv-Mohyla Academy). The components described in Phase 1 (intervention development) have already received ethical approval by the respective ethics country boards (LSHTM Ethics Reference: 2279) and we are in the process of submitting ethical approvals for Phase 2 (pilot testing). We will adhere to key ethical principles for qualitative research following international guidance (e.g., Declaration of Helsinki) and specific guidance for humanitarian settings (Elrha, 2017). We will also follow the recommendations for conducting ethical MHPSS research in emergency settings as suggested by the Inter-Agency Standing Committee Reference Group for MHPSS in Emergency Settings (Chiumento et al., 2017; Inter-Agency Standing Committee for Mental Health and Psychosocial Support in Emergency Settings, 2014).
Rigor
We will ensure rigor in the different qualitative components through several mechanisms. At the offset of the project, training was provided on qualitative data collection (e.g., semi-structured interviews, focus groups, and online and telephone interviewing) and on qualitative data analysis. The regular qualitative data clinics also provide a space for discussion of methodological and analytical challenges and of ways to ensure quality throughout the process. These clinics will also contribute to rigor by allowing for triangulation of data and findings from different contexts. The cumulative nature of our different qualitative components will also represent a form of triangulation by having subsequent components build upon and confirm findings from previous components. Additionally, our team includes dedicated social scientists and experts in qualitative methods which will be available to support at each stage of the qualitative studies.
The principle of trustworthiness will be used as a criterion to ensure rigor by meeting the criteria of credibility, transferability, dependability, and confirmability (Lincoln & Guba, 1986). The criterion of credibility will be ensured by prolonged engagement of our team in the local contexts. Transferability will be established by exploring similarities and differences between the two study settings using similar data collection methods and by precise contextualization of the study findings. This protocol contributes itself to increasing the dependability of the findings by providing a detailed description of the aims and methods of data collection and analysis for each qualitative component. Finally, we will aim to achieve confirmability by taking a reflexive approach and by maintaining a sense of awareness and openness around how our preconceptions might inform the study findings (e.g., by discussing this during the regular qualitative data clinics and by having different members of the team contribute to the interpretation of the data).
Conclusion
There is a need for high-quality qualitative studies conducted as part of complex health interventions in humanitarian settings. This protocol sets out the rationale and plan of investigation for the qualitative components of the CHANGE project that seeks to develop, test, and implement PM+A. As we have described above, the various qualitative elements of this work will, over time, contribute to both the design and delivery of the trial intervention and its associated procedures, to understanding of their implications, and will offer information about the potential for scaling up this intervention in “real world” implementation settings. Together with the quantitative findings, these results will ultimately contribute to improving the well-being of populations affected by conflict through the development of an evidence-based open-access intervention for alcohol misuse and associated mental health comorbidities.
We expect that the qualitative work set out in this protocol will shape the trial in two ways. First, it will contribute to the development of a locally appropriate intervention for alcohol misuse in settings affected by conflict which reflects the needs and perspectives of local stakeholders. Second, it will contribute to assessing important process elements of the trial including feasibility, acceptability, and fidelity. This protocol also offers a template for the integration of qualitative research on and in complex health interventions in humanitarian settings. Additionally, this protocol provides a framework on how to integrate multiple serial qualitative components into a coherent whole to ensure that different components build upon each other and to reduce disjointedness of qualitative components in similar trials. Given the paucity of systematic implementation science research within the MHPSS field as a whole (Cohen & Yaeger, 2021; Dickson & Bangpan, 2018), improving our understanding of implementation and scaling up processes through the proposed qualitative components of CHANGE is crucial.
Supplemental Material
sj-pdf-1-ijq-10.1177_16094069211063031 – Supplemental Material for A Complex Intervention for Alcohol Misuse Among Conflict-Affected Populations in Uganda and Ukraine: Study Protocol for the Qualitative Components in the CHANGE Trial
Supplemental Material, sj-pdf-1-ijq-10.1177_16094069211063031 for A Complex Intervention for Alcohol Misuse Among Conflict-Affected Populations in Uganda and Ukraine: Study Protocol for the Qualitative Components in the CHANGE Trial by Alessandro Massazza, Daniela C. Fuhr, Sergiy Bogdanov, Wietse A. Tol, Bayard Roberts, Abhijit Nadkarni, Geoffrey Akudrabo, Lena Skovgaard Andersen, Kostyantyn Dumchev, Andriy Karachevskyy, Eugene Kinyanda, Kateryna Koss, Quincy Moore and Carl R. May in International Journal of Qualitative Methods
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Institute for Health Research (NIHR) (using the UK’s Official Development Assistance (ODA) Funding) and Wellcome (grant reference number 219468/Z/19/Z) under the NIHR-Wellcome Partnership for Global Health Research. CRM’s contribution was also supported by NIHR Applied Research Collaborative, North Thames. The views expressed are those of the authors and not necessarily those of Wellcome, the NIHR, or the Department of Health and Social Care.
Supplemental Material
Supplemental material for this article is available online.
Note
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
