Sage Journals: Discover world-class research

Abstract

The initial program theory (IPT) is a core element of realist evaluation. An IPT presents a hypothesis about how, for whom, why, and in what contexts an intervention is effective. Despite their importance within realist studies, methodological guidance on how to develop an IPT is relatively scarce. We describe the methodological process of eliciting an IPT of facility-based maternal death reviews (FbMDRs), a multicomponent intervention to improve the quality of maternal care. We used a four-step approach to elicit the IPT through a combination of literature review, document review, and in-depth interviews with key stakeholders while drawing on the experience of the research team. The literature and document reviews enabled us to identify facilitators and barriers influencing the functioning of FbMDRs. We used the causal pathway approach to deconstruct the maternal death review procedure and we elicited a causal explanation for each component on the basis of the results of the document review, the literature review and key respondent interviews. We organised stakeholder consultations that provided insights of key actors and validated its real-world relevance. We discuss the challenges encountered during its development and the lessons learned. The challenges include the fact that maternal death reviews are multicomponent interventions, for which a ‘simple’ programme theory cannot be defined, the positionality of the research team, and the visual representation of a programme theory with multiple components.

Keywords

facility-based maternal death reviews initial programme theory realist evaluation complex and multicomponent intervention quality improvement maternal health Benin

Background

Over the last two decades, maternal survival has been high on the global health agenda. Reducing maternal mortality is crucial to the health-related goals of the 2030 Sustainable Development Goals (World Health Organization, 2016). This has led to the large-scale promotion of effective and affordable interventions to improve maternal mortality outcomes, such as increased access to facility-based care. Across Sub-Saharan Africa (SSA), the number of facility-based births surged (Black et al., 2016). For instance, in Rwanda, the percentage of births that took place in health facilities increased from 26.6% between 2000–2007 to 90.7% between 2008–2016 (Doctor et al., 2019). Similar trends were seen in Malawi (from 55.5% to 91.4%), Uganda (from 36.8% to 73.4%) and Ghana (from 45.9% to 73.9%) (Doctor et al., 2019). Such improvements may contribute to decreasing preventable maternal deaths, but only if they are accompanied by the provision of respectful, high-quality care during childbirth and in the postnatal period to every woman (World Health Organization, 2016a). Yet, with an average maternal mortality rate in SSA estimated at 545 maternal deaths per 100,000 live births in 2020, it seems clear that the recent increase in facility-based deliveries has not been translated into a commensurate reduction in maternal mortality (Gabrysch et al., 2019; World Health Organization, 2023). Poor quality of care is a likely explanation (World Health Organization, 2016a). Improving the quality of care in late pregnancy, during labour and birth, and in the immediate postnatal period is an essential step towards ending preventable maternal mortality (Jolivet et al., 2018; Kruk & Pate, 2020).

Improving the quality of maternal care demands precise information that allows the identification of the factors contributing to poor care outcomes, including the most extreme - maternal deaths (World Health Organization, 2004). Such information can be generated through Maternal Death Surveillance and Response, which is a form of continuous surveillance that links the health information system and quality improvement processes from local to national levels. It includes routine identification and notification, the determination of causes and avoidability of all maternal deaths through reviews, and the use of this information to respond with actions that will prevent future deaths. The maternal death review action cycle typically includes five components: (i) identification of maternal deaths, (ii) data collection, (iii) analysis of the findings, (iv) recommendations and action, and (v) evaluation and refinement (World Health Organization, 2004). This strategy can be defined as a systematic investigation aimed at identifying the causes of maternal deaths and developing actionable recommendations to improve maternal care and prevent future deaths (World Health Organization, 2004). Maternal death review action cycles have contributed to reducing maternal mortality in several low- and middle-income countries (LMICs) (World Health Organization Regional Office for South-East Asia, 2014), such as Sri Lanka (Ariyarathna & Hulathduwa, 2019). If implemented in optimal conditions, this strategy has the potential to reduce maternal mortality by up to 35% (Willcox et al., 2020). The World Health Organization (WHO) recommended maternal death reviews “to reduce maternal mortality and morbidity by improving the quality of care provided” (World Health Organization, 2004, p. 11) and revised its technical guidance in 2013 and 2021 (World Health Organization, 2013, 2021). In 2016, most LMICs (85%) reported having a policy requiring the review of all maternal deaths (World Health Organization, 2016b).

Benin scaled up Facility-based Maternal Death Reviews (FbMDRs) in 2013 to address its high maternal death ratio (Ministère de la Santé du Bénin, 2013), which in 2020 was estimated at 523 maternal deaths per 100,000 live births (World Health Organization, 2023). An FbMDR committee was set up in the 34 health districts, six regional and two national tertiary hospitals, and at the national level. Essentially, it was hoped that these committees would review all maternal deaths within 30 days after their occurrence and propose specific, measurable, achievable, realistic, and time-sensitive recommendations for improvements in care quality (Ministère de la Santé du Bénin, 2013). However, evaluations of FbMDRs in Benin from 2016 to 2022 revealed significant challenges hindering them from achieving their objectives. This included time constraints, unreliable data, strained relationships between providers, a fear of blame, a lack of multidisciplinary committees, and limitations in data collection and analysis tools. As a result, few actionable data for quality care improvement were generated, and the causal pathways underlying maternal mortality were not properly assessed (Ministère de la santé du Benin & Centre de Recherche en Reproduction Humaine et en Démographie, 2017, 2018, 2019, 2020, 2021, 2022, 2023; Boyi Hounsou et al., 2021; Boyi Hounsou et al., 2024). Critical issues affecting the quality of care, such as leadership and governance, were not addressed (Boyi Hounsou et al., 2024).

Other LMICs have reported similar challenges (Kinney et al., 2021; Lusambili et al., 2019; Willcox et al., 2020). A systematic review conducted by Willcox et al. (2020) found that this strategy has mixed results in lowering maternal mortality. Hypotheses explaining the suboptimal outcomes include poor quality of FbMDR processes, insufficient human and financial resources to conduct the reviews or implement the recommendations, misalignment between the organisational culture of the health system and FbMDR’s no-name, no-blame principle, as well as lack of motivation and engagement among key actors (Willcox, Okello, Maidwell-Smith et al., 2023; Willcox et al., 2023). Willcox et al. (2020) called for new research on the implementation of FbMDRs, especially in low-resource settings. Few studies have comprehensively assessed why FbMDRs are effective and how, and few authors address explicitly the factors which enable their effectiveness in improving care and reducing maternal mortality (Willcox et al., 2020;Willcox, Okello, Maidwell-Smith Alice Maidwell-Smith, Tura, van den Akker, Knight et al., 2023; Willcox et al., 2023).

Quality of care is complex in nature (Hanefeld et al., 2017), and there is no universally accepted definition (Donabedian, 1988; Institute of Medicine, 1990a; Roemer & Montoya-Aguilar, 1988; Wilson & Goldsmith, 1995; World Health Organization, 2006). The Institute of Medicine defines it as “the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge” (Institute of Medicine, 1990b, pp. 128-129). Its complex nature challenges the ontological, epistemological, and methodological foundations of the quasi-experimental study design usually applied to evaluate quality improvement strategies such as FbMDRs (Groene & Sunol, 2019; Merali & Allen, 2011; Stern et al., 2012). Theory-driven evaluations, like realist evaluation, help understand complex issues by exploring how outcomes result from the interplay between intervention, actors and context and by identifying the mechanisms contributing to the observed outcomes (Breuer et al., 2016; Mathison, 2005).

A central element of realist evaluation is the initial programme theory (IPT), defined by Pawson and Tilley (1997) as “a working hypothesis about how a programme is intended to work and for whom”. The PT is empirically tested and refined to more accurately explain how and why interventions work, in what circumstances, and for whom. Realist evaluation is increasingly used to study complex health interventions and policies (Booth et al., 2020). The principles of realist evaluation and, more specifically, the methods that can be used to elicit the initial programme theory have been discussed by a number of authors, for instance, Pawson and Tilley (1997), Wong et al. (2016), and Marchal et al. (2012). (Pawson & Tilley, 2004) characterised realist evaluation as a ‘logic of enquiry’ rather than a prescribed research technique or method. They advocated for flexibility, encouraging researchers and evaluators to use their preferred methods creatively adapted to their projects rather than adhering to rigid procedures. Other authors wrote about the challenges and pitfalls of realist evaluation (Flynn et al., 2020; Pawson & Manzano-Santaella, 2012; Salter & Kothari, 2014). Fick and Muhajarine (2019) highlighted how the absence of clear guidance on the ‘how to’ aspect can engender ambiguity and uncertainty, particularly for newcomers to realist evaluation. Worked examples of the elicitation of the initial programme theory are scarce. Only recently have papers presenting practical examples been published (see, for instance, Bosongo et al., 2024; Mukumbang et al., 2018; Smeets et al., 2022). This study aims at contributing to the methodological development of realist research by presenting the process of developing an initial programme theory through a worked example that focuses on facility-based maternal death reviews in Benin.

Methods

Realist Evaluation

We adopted the realist evaluation approach (Pawson & Tilley, 1997). The notion of generative causation is central to realist evaluation (Pawson, 2013). This assumes that an action is causal if and only if its effects are triggered by a mechanism that operates in a specific context (Pawson, 2013). Causes not only precede effects in time but also contain within themselves the capacity for bringing about the effect. Outcomes are considered to arise from mechanisms triggered in specific environments. These mechanisms are not necessarily directly observable but can be inferred through empirical investigation and theoretical reasoning (Pawson, 2013). In realist evaluation, the outcomes (O) occur because an intervention (I) triggers mechanisms (M) in specific contexts (C), due to which actors (A) do something which leads to the observed outcome (Marchal et al., 2018; Pawson et al., 2005). Pawson and Tilley described how mechanisms relate to the programme: the latter “introduce the appropriate ideas and opportunities (‘mechanisms’) to groups in the appropriate social and cultural conditions (‘contexts’)”. (Pawson & Tilley, 1997, p. 57). In the same seminal work, they write how “the evaluation of social programs will deploy identical explanatory forms, reaching ‘down’ to the layers of individual reasoning (what is the desirability of the ideas promoted by a program?) and ‘up’ to the collective resources on offer (does the program provide the means for subjects to change their minds?)” (Pawson & Tilley, 1997, p. 66). Context refers to anything external to the intervention that is necessary for the mechanism to be triggered (Pawson & Tilley, 1997, p. 2004).

The realist evaluation cycle (Figure 1) starts with the formulation of the research question, which is followed by the development of the initial programme theory of the programme or situation under study. Realist evaluation is method-neutral: the study design and data collection methods are chosen in function of their capacity to allow testing the initial programme theory. The data analysis takes a retroductive approach (Pawson, 2013; Pawson & Tilley, 1997). Retroduction is a process of reasoning backwards from the observed outcomes to uncover the mechanisms and the context elements that contribute to the outcomes (Wong et al., 2016). Retroduction uses inductive and deductive reasoning and can be compared to the way detectives work in that the analyst seeks to explain the observed outcomes by identifying plausible explanations and refuting alternative explanations (Pawson, 2013). Retroduction “is underpinned by a belief that an understanding of causation cannot be achieved using only observable evidence. Retroductive theorising requires that inquirers use their common sense, intelligence, expertise, and informed imagination to build and test theories about underpinning causal processes” (Wong et al., 2016). The researcher uses the Context-Mechanism-Outcome configuration as a heuristic in the data analysis. (Pawson & Tilley, 1997). We used the ICAMO configuration (Marchal et al., 2018), which is an adaptation of the CMO configuration. The results of the analysis are then compared to the initial programme theory, which is adapted where needed.

Figure 1.

The Realist evaluation cycle, adapted from Marchal et al., (2012).

The Process of Eliciting the IPT

IPTs can be elicited on the basis of interviews with the different actors involved in the design and implementation of the intervention to elicit their knowledge, assumptions and experience, reviews of the literature and documents, and empirical research, or a combination thereof (Wong et al., 2016). In this study, we elicited our IPT through a combination of the results of a scoping review, a document review and in-depth interviews with key stakeholders, interwoven with the experience of the research team. This phase took nine months, spread across the timeframe from November 2021 to October 2023. In the Results section, we present this process in detail. We followed the RAMESES II reporting standards for realist evaluations (Wong et al., 2016).

Results

In this section, we present first the methodological process of eliciting the IPT. We then discuss the challenges we met and the solutions we found to deal with them.

Steps Taken to Elicit the Initial Programme Theory

We elicited our IPT in four steps, as summarised in Figure 2.

Figure 2.

Steps employed to elicit the initial programme theory of FbMDRs.

Step One: Scoping Review

We started with a scoping review (Arksey & O’Malley, 2005) to identify programme theories of how and why quality improvement interventions, including FbMDRs, work (or not). We used the BeHEMoTh — Behaviour of interest; Health context; Exclusions; Models or Theories— procedure, conceived by Booth and Carroll (2015), to structure the search and identification of theories within the literature. Our MeSH terms and search strategies are presented in Supplemental Material 1. We found 160 papers in PubMed, 77 in Cochrane, and 377 in Web of Science. We read all the titles and abstracts and found none of the included studies used a relevant programme theory that could be adapted to FbMDRs.

Step Two: Document and Literature Reviews

Document Review

We started with an extensive search for documents on quality improvement strategies for maternal care in general and FbMDR in particular. We included national ministerial decree(s), international and local standard operating procedures, evaluation reports (periodic and annual, national, regional, or health district levels), and the reports of the national health system and a health facility context assessment for implementing a multifaceted intervention to improve the quality of perinatal care. We searched for these documents on official websites and physical libraries of the Ministry of Health and its key financial and technical partners (e.g., the United States Agency for International Development, United Nations Population Fund and World Health Organization). We also collected (un)published reports and PhD theses from participants in the in-depth interviews. All collected and analysed documents are listed in Supplemental Material 2.

Literature Review

Because the first scoping review did not yield any result, we decided to use the berry-picking approach as an alternative search strategy (Figure 3) (Bates, 1989). This type of review is appropriate for identifying elements to generate an IPT as it is an iterative, adaptive and flexible process (Finfgeld‐Connett & Johnson, 2013). It starts with a literature search, followed by selecting the first set of articles relevant to the research question. Electronic bibliographic database searches are combined with identifying relevant references in retrieved papers, journal scanning, and author, subject, and citation searching. Each new piece of information uncovered may introduce new ideas and directions to follow, which consequently helps refine the query. Unlike conventional searches, which are characterised by a static query, fixed search terms and overarching concern for reproducibility, berry picking allows continuous modification of the query and search terms. In our study, we conducted two searches (search terms are shown in Supplemental Material 3). We performed the first literature search looking for facilitators and barriers to the effective implementation of FbMDR. We identified one scoping review and one systematic realist review (Hut-Mossel et al., 2021; Kinney et al., 2021). We conducted the second literature search to identify factors which influence the main facilitators and barriers. This second search found two systematic reviews of qualitative studies, one of which used a realist approach (Willcox, Okello, Maidwell-Smith et al., 2023; Willcox et al., 2023).

Figure 3.

The berry-picking search strategy adapted from Bates (1989).

The causal pathway approach proposed by Lewis et al. (2018) describes the cascade of outcomes from proximal to distal levels, helping to clarify how the components of a strategy lead to intermediate outcomes, which in turn affect distal outcomes. It highlights variations in outcomes based on different circumstances and facilitates a visual representation of the relationships among interventions, mechanisms, and outcomes (Klasnja et al., 2024; Lewis et al., 2018). We applied this to the FbMDRs to structure the analysis of the document review. We divided the FbMDR process in 3 phases: (1) Adoption of FbMDR by key stakeholders (managers and providers), (2) Completion of the FbMDR procedure and (3) Implementation of recommendations.

Process of Analysing the Documents and the Peer-Reviewed Papers

First, we defined our key concept by adopting WHO’s definition of FbMDR as a “qualitative, in-depth investigation of the causes of, and circumstances surrounding, maternal deaths which occur in health care facilities” (World Health Organization, 2004, p. 57). We described its key process and principles: FbMDR is, in essence, an in-depth discussion within the multidisciplinary team that cared for the woman about the circumstances that led to her death. These in-depth discussions should be conducted with respect to the principles of ‘no name’ and ‘no blame’. Participants should adopt self-reflection and accept constructive criticism. The dysfunctions that led to the death and their root causes should be identified, and relevant recommendations should be formulated. It is expected that if these recommendations are implemented, the dysfunctions and their root causes will be corrected, and similar deaths will be avoided through an improvement in the quality of maternal care. Finally, the findings of FbMDRs should not be used to sanction providers, even if the death is due to a professional fault, to encourage an open discussion of sensitive details about their practices without fear of punishment, fostering a culture of learning and improvement in patient care.

Second, we defined the key categories of facilitators and barriers of FbMDRs following the Theoretical Framework for Studying Maternal and Perinatal Death Surveillance and Response Implementation of Kinney et al. (2021). We extracted facilitators and barriers from the four papers selected in the berry-picking review and from the records of the document review. We discussed these findings and their relevance to the context of Benin within the research team (Supplemental Material 4).

Third, we inductively identified the main facilitators and barriers that can impact each of the three phases of FbMDRs. These included, for instance, a learning environment, effective leadership and a culture of blame. We then identified the factors that influence these main facilitators and barriers.

Fourth, we looked for mechanisms related to the different actors (managers, providers) and elements of the implementation context within which they operate.

Finally, we use the “if …, then …, because …” statements to formulate the elements of the first draft of the initial programme theory (Supplemental Material 5).

Step Three: In-Depth Interviews and Observations and Synthesis

In a third step, we carried out in-depth interviews with key respondents to elicit their views on how FbMDR works (or not) – in the realist literature, also called the folk theories. Folk theories are the implicit assumptions, arguments and logic of policymakers who design programmes, as well as the models and reasoning that stakeholders use to understand how a programme is expected to work (Pawson & Tilley, 1997). These folk theories are shaped by personal experiences, cultural norms and common sense rather than scientific evidence (Pawson & Tilley, 1997).

In practice, we used a maximum variation purposive sampling strategy to identify and select respondents for the interviews. We conducted nine in-depth interviews with policymakers, academics, representatives of financial and technical partners, and implementers involved in designing and implementing maternal care quality improvement strategies, including FbMDRs. The first author (CBH) conducted these interviews using an interview guide between June and October 2022. She also conducted four non-participant observations of FbMDR sessions using an observation guide. We explored (i) how and why the participants think FbMDRs were developed and (ii) how, why, for whom, and in which conditions the participants think the reviews work (or not).

The in-depth interviews were recorded after the consent of the participants and transcribed verbatim. Transcripts were imported into NVivo 12 for coding and analysis. Multiple readings of the transcripts and their memos while coding led to new emerging codes. The coded data were deductively analysed starting from the first draft of the IPT to see how far its themes and sub-themes could be confirmed. Inductive analysis was conducted to identify any new themes. We triangulated data from the two sources and synthesized the findings through retroductive inference, moving between empirical observations and theoretical reflection to uncover the underlying mechanisms driving the outcomes. Then, we reformulated the IPT based on these insights. We specifically added two components to the causal pathway

1) Adoption of FbMDRs by managers and by providers

2) Correct identification of root causes

3) Formulation of recommendations

4) Implementation of recommendations

5) Reduction of in-facility maternal death rate

For each component, we differentiated between proximal and distal outcomes. We connected mechanisms to the actors and the contextual conditions operating at each level. We developed the graphical representation of the second draft of the IPT using concept mapping, which helps to “make explicit what is normally implicit” (Jonassen & Marra, 1994). Through concept mapping, researchers organise and represent knowledge by creating diagrams that connect concepts with labelled lines to show their relationships. It can help in understanding complex information more deeply by clarifying how ideas relate to one another (Jonassen & Marra, 1994). This step ended with a refined IPT.

Step Four: Finalisation of the IPT

We organised two consultation and co-creation meetings with policymakers of the Ministry of Health (November and December 2022) and two with maternal health actors of a regional hospital in Benin (July and December 2022). This hospital was purposively chosen based on the experience of the hospital leadership team with FbMDRs and their commitment to reducing maternal mortality. In these meetings, the findings of this study were presented to the participants. Through structured discussions and breakout sessions, they were given the opportunity to systematically discuss those findings and to provide feedback. The main insights emphasised the theory’s contextual relevance of the causal explanations. Although there were differing opinions during the meetings with the hospital managers and providers regarding some mechanisms, such as blaming, these differences were addressed through effective facilitation, ensuring that all voices were heard and integrated into the discussions. This led to additional insights that were fed into the IPT. The final version of the IPT is presented in Figure 4.

Figure 4.

ICAMO map representing our initial programme theory of FbMDRs.

Discussion

In this paper, we illustrated how we elicited the initial programme theory of FbMDRs in Benin. We encountered several challenges and presented our reflections here.

The first challenge was linked to the fact that facility-based maternal death reviews are, in essence, multicomponent interventions, which by their nature are not easily captured by one programme theory. The development of the initial programme theory took indeed quite some time, within the already narrow timeframe of the PhD study of the first author. The scoping review we carried out to identify established theories took four months and was not fruitful. Other authors reported excessive use of time and funding on literature reviews to formulate the initial programme theory (Chen & Rossi, 1989; Flynn et al., 2020; Marchal et al., 2010). To counter this, Flynn et al. (2020) propose that researchers “live within their means” as the IPT itself is not the final product of realist evaluation but just a detailed hypothesis that needs to be tested. To deal with this difficulty, some authors purposively choose an established theory from the literature. This is aligned with Lipsey and Pollard’s (1989) theory development approach. However, numerous theories can suit a given programme (Lipsey & Pollard, 1989; Pawson & Tilley, 2004). Similar to theory adjudication during the analysis phase of a realist study (Pawson & Tilley, 2004), a careful choice is thus required.

In our case, deconstructing the reviews into five components allowed us to zoom in on the causal pathway of each component and of the link between the components. The causal pathway approach Lewis et al. (2018) helped us be systematic and not ignore mechanisms that may be deeply embedded within cultural norms, power dynamics, or historical factors. This way, we could identify ‘nuggets’ within data extracted from the literature and formulate the first draft of IPT despite the absence of an established theory (Lipsey & Pollard, 1989). The causal pathway approach also helped us to look more systematically for unexpected factors and alternative explanations. We suggest being flexible and creative, as recommended by Pawson and Tilley (1997, 2004), while being as pragmatic and efficient as possible to avoid any waste of time during this first period and use the needed time for theory refinement.

The second difficulty we faced was identifying the causal relationships between the intertwined elements of FbMDR. Identifying the most likely explanation while dealing with the personal bias and preconceptions of the first author, who has worked as an FbMDRs trainer of trainers and implementation supervisor since 2017, was challenging. At the same time, her insider status contributed insights that the other authors could never have obtained. Bygstad and Munkvold (2011) similarly raised the issue of the interpretive perspective, which implies that the researcher has insight beyond the knowledge of key informants. Olmos-Vega et al. (2023) suggested constructing a reflexivity plan that includes personal and collaborative reflexivity and allows using the power of subjectivity through meaningful reflexivity practices. Such plans may help as reflexivity often gets lost in the pressing issues of intensive data generation and pressure to complete analyses.

We addressed this challenge by being as reflexive as possible throughout the process and organising team reflexivity sessions, where we critically examined our preliminary results. The iterative process of realist analysis involves constant refinement and adjustment based on the accumulating empirical evidence. Numerous discussions within the multidisciplinary research team and with the national- and hospital-level stakeholders were needed, but these allowed for checking the plausibility of the causal explanations checks. Mukumbang et al. (2018), Fick and Muhajarine (2019), and Griffiths et al. (2022) reported similar stakeholder involvement in developing their IPTs. Yet, iterative stakeholder consultation to improve the IPT may introduce a confirmation bias, as stakeholders tend to agree with what the researchers presented. We limited this bias by asking the stakeholders to think of ways to improve the theories.

The third difficulty was the visual representation of the IPT. The amount of information related to the IPT of a multicomponent intervention is large, which makes creating a clear visual representation challenging and time-consuming. To our knowledge, no authors have compared methods of visual presentation of programme theories nor recommended the best methods. We used the concept mapping technique, which enabled us to make the information in the initial programme theory more explicit. As a result, the participants of the workshops could navigate through it autonomously. Working with a visual designer from the beginning could help identify the most suitable method of visual representation for the intervention in question and potentially save time.

Conclusion

In this paper, we present a worked example of eliciting the initial programme theory of a multicomponent intervention. We found that using the causal pathway approach helped to elicit the IPT by allowing us to define causal explanations for each component and for the linkages between the components. It also stimulated the exploration of unexpected factors and alternative explanations. Allocating time for team reflexivity to critically examine the entire process helps in mitigating personal biases and preconceptions and fosters more rigorous research findings. Finally, collaborating with a visual designer from the beginning can help in communicating the complexity of the IPT.

Supplemental Material

Supplemental Material - Eliciting the Initial Programme Theory in a Realist Evaluation of Facility-Based Maternal Death Reviews in Benin: Methodological Process, Challenges and Lessons Learned

Supplemental Material for Eliciting the Initial Programme Theory in a Realist Evaluation of Facility-Based Maternal Death Reviews in Benin: Methodological Process, Challenges and Lessons Learned by Christelle Boyi Hounsou, Jean-Paul Dossou, Thérèse Delvaux, Lenka Benova, Edgard-Marius Ouendo, Marjolein Zweekhorst, and Bruno Marchal in International Journal of Qualitative Methods.

Footnotes

Acknowledgements

We acknowledge the contribution from our colleagues of the Centre de Recherche en Reproduction Humaine et en Démographie: Mr Christian Agossou and Dr. Gottfried Agballa. We sincerely thank all the participants of this study.

Author Contributions

CBH: Conceptualisation, Data collection, Analysis, Writing first draft and Editing; JDP: Conceptualisation, Analysis, Reviewing and Editing; LB: Reviewing and Editing; TD: Reviewing and Editing; EMO: Reviewing and Editing; MZ: Reviewing and Editing; BM: Conceptualisation, Analysis, Reviewing and Editing.

Declaration of Conflicting Interest

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study is supported by Directorate-General for Development Cooperation and Humanitarian Aid, Belgium, The European Union; The European Union (Horizon 2020, No 847824).

Ethical Statements

ORCID iDs

Christelle Boyi Hounsou

Jean-Paul Dossou

Thérèse Delvaux

Lenka Beňová

Edgard-Marius Ouendo

Marjolein Zweekhorst

Bruno Marchal

Supplemental Material

Supplemental material for this article is available online.

References

Akuze

Annerstedt

K. S.

Benova

Chipeta

Dossou

J. P.

Gross

M. M.

Kidanto

Marchal

Alvesson

H. M.

Pembe

A. B.

van Damme

Waiswa

Hanson

ALERT Study Team . (2021). Action leveraging evidence to reduce perinatal mortality and morbidity (ALERT): study protocol for a stepped-wedge cluster-randomised trial in Benin, Malawi, Tanzania and Uganda. BMC Health Serv Res, 21(1), 1324. https://doi.org/10.1186/s12913-021-07155-z

Ariyarathna

Hulathduwa

(2019). A missed opportunity of a preventable maternal death: A case report from Sri Lanka. Forensic Res Criminol Int J, 7(6):307–311.

Arksey

O’Malley

(2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19–32. https://doi.org/10.1080/1364557032000119616

Bates

M. J.

(1989). The design of browsing and berrypicking techniques for the online search interface. Online Review, 13(5), 407–424. https://doi.org/10.1108/eb024320

Black

R. E.

Laxminarayan

Temmerman

Walker

(Eds.), (2016). Reproductive, maternal, newborn, and Child health: Disease control priorities 3rd ed. (Volume 2): The International Bank for Reconstruction and Development / The World Bank.

Booth

Briscoe

Wright

J. M.

(2020). The “realist search”: A systematic scoping review of current practice and reporting. Research Synthesis Methods, 11(1), 14–35. https://doi.org/10.1002/jrsm.1386

Booth

Carroll

(2015). Systematic searching for theory to inform systematic reviews: Is it feasible? Is it desirable? Health Information and Libraries Journal, 32(3), 220–235. https://doi.org/10.1111/hir.12108

Bosongo

Belrhiti

Chenge

Criel

Coppieters

Marchal

(2024). The role of provincial health administration in supporting district health management teams in the Democratic Republic of Congo: Eliciting an initial programme theory of a realist evaluation. Health Research Policy and Systems, 22(1), 29. https://doi.org/10.1186/s12961-024-01115-9

Boyi Hounsou

Agossou

M. C. U.

Bello

Delvaux

Benova

Vigan Guézodjè

Hounkpatin

Dossou

J. P.

(2021). “So hard not to feel blamed!”: Assessment of implementation of Benin’s Maternal and Perinatal Death Surveillance and Response strategy from 2016–2018. International Journal of Gynecology & Obstetrics, 158(Suppl 2), 6–14. https://doi.org/10.1002/ijgo.14041

10.

Boyi Hounsou

Dossou

J.-P.

Delvaux

Benova

Ouendo

E. M.

Lokossou

Zweekhorst

Marchal

(2024). “We do these audits, but in the end, it’s useless.” How can Facility-based maternal death reviews improve the quality of care in Benin? SSM - Health Systems, 3, Article 100032. https://doi.org/10.1016/j.ssmhs.2024.100032

11.

Breuer

Lee

De Silva

Lund

(2016). Using theory of change to design and evaluate public health interventions: A systematic review. Implement Sci [Internet], 11(1), 63. Available from: https://doi.org/10.1186/s13012-016-0422-6

12.

Bygstad

Munkvold

(2011). Search of mechanisms conducting a critical realist data analysis. In International conference on information systems 2011, ICIS 2011, Shanghai, China, 4-7 December 2011. (p. 3).

13.

Chen

H.-T.

Rossi

P. H.

(1989). Issues in the theory-driven perspective. Evaluation and Program Planning, 12(4), 299–306. https://doi.org/10.1016/0149-7189(89)90046-3

14.

Doctor

H. V.

Radovich

Benova

(2019). Time trends in facility-based and private-sector childbirth care: Analysis of Demographic and Health Surveys from 25 sub-Saharan African countries from 2000 to 2016. J Glob Health, 9(2), Article 020406. https://doi.org/10.7189/jogh.09.020406

15.

Donabedian

(1988). The quality of care. How can it be assessed? JAMA, 260(12), 1743–1748. https://doi.org/10.1001/jama.260.12.1743

16.

Fick

Muhajarine

(2019). First steps: Creating an initial program theory for a realist evaluation of Healthy Start-Départ Santé intervention in childcare centres. International Journal of Social Research Methodology, 22(6), 545–556. https://doi.org/10.1080/13645579.2019.1595375

17.

Finfgeld‐Connett

Johnson

E. D.

(2013). Literature search strategies for conducting knowledge‐building and theory‐generating qualitative systematic reviews. Journal of Advanced Nursing, 69(1), 194–204. https://doi.org/10.1111/j.1365-2648.2012.06037.x

18.

Flynn

Schick-Makaroff

Levay

Greenhalgh

(2020). Developing an initial program theory to explain how patient-reported outcomes are used in health care settings: Methodological process and lessons learned. International Journal of Qualitative Methods, 19(49), 1–11. https://doi.org/10.1177/1609406920916299

19.

Gabrysch

Nesbitt

R. C.

Schoeps

Hurt

Soremekun

Edmond

Manu

Lohela

T. J.

Danso

Tomlin

Kirkwood

Campbell

O. M. R.

Campbell

O. M.

(2019). Does facility birth reduce maternal and perinatal mortality in brong ahafo, Ghana? A secondary analysis using data on 119 244 pregnancies from two cluster-randomised controlled trials. Lancet Global Health, 7(8), e1074–e1087. https://doi.org/10.1016/S2214-109X(19)30165-2

20.

Griffiths

Weston

Morgan-Trimmer

Wheat

Gude

Manger

T. M.

Clarkson

Quinn

Sheaff

Clark

Sherriff

Byng

(2022). Engaging stakeholders in realist programme theory building: Insights from the prospective phase of a primary care dementia support study. International Journal of Qualitative Methods, 21(3), 160940692210775. https://doi.org/10.1177/16094069221077521

21.

Groene

Sunol

(2019). Quality improvement is complex and contextual. BMJ British Medical Journal, 367, l6155. https://doi.org/10.1136/bmj.l6155

22.

Hanefeld

Powell-Jackson

Balabanova

(2017). Understanding and measuring quality of care: Dealing with complexity. Bulletin of the World Health Organization, 95(5), 368–374. https://doi.org/10.2471/BLT.16.179309

23.

Hut-Mossel

Ahaus

Welker

Gans

(2021). Understanding how and why audits work in improving the quality of hospital care: A systematic realist review. PLoS One, 16(3), Article e0248677. https://doi.org/10.1371/journal.pone.0248677

24.

Institute of Medicine . (1990a). Medicare: A strategy for quality assurance, volume II: Sources and methods. The National Academies Press. https://doi.org/10.17226/1548

25.

Institute of Medicine . (1990b). A strategy for quality assurance. National Academy Press.

26.

Jolivet

R. R.

Moran

A. C.

O’Connor

Chou

Bhardwaj

Newby

Requejo

Schaaf

Say

Langer

(2018). Ending preventable maternal mortality: Phase II of a multi-step process to develop a monitoring framework, 2016-2030. BMC Pregnancy and Childbirth, 18(1), 258. https://doi.org/10.1186/s12884-018-1763-8

27.

Jonassen

Marra

(1994). Concept mapping and other formalisms as mindtools for representing knowledge. ALT-J, 2(1), 50–56. https://doi.org/10.1080/0968776940020107

28.

Kinney

Walugembe

D. R.

Wanduru

Waiswa

George

(2021). Maternal and perinatal death surveillance and response in low- and middle-income countries: A scoping review of implementation factors. Health Policy and Planning, 36(6), 955–973. https://doi.org/10.1093/heapol/czab011

29.

Klasnja

Meza

R. D.

Pullmann

M. D.

Mettert

K. D.

Hawkes

Palazzo

Weiner

B. J.

Lewis

C. C.

(2024). Getting cozy with causality: Advances to the causal pathway diagramming method to enhance implementation precision. Implementation Research and Practice, 5(4), Article 26334895241248851. https://doi.org/10.1177/26334895241248851

30.

Kruk

Pate

(2020). The lancet global health commission on high quality health systems 1 year on: Progress on a global imperative. Lancet Global Health, 8(1), E30-e32. https://doi.org/10.1016/S2214-109X(19)30485-1

31.

Lewis

C. C.

Klasnja

Powell

B. J.

Lyon

A. R.

Tuzzio

Jones

Walsh-Bailey

Weiner

(2018). From classification to causality: Advancing understanding of mechanisms of change in implementation science. Frontiers in Public Health, 6, 136. https://doi.org/10.3389/fpubh.2018.00136

32.

Lipsey

M. W.

Pollard

J. A.

(1989). Driving toward theory in program evaluation: More models to choose from. Evaluation and Program Planning, 12(4), 317–328. https://doi.org/10.1016/0149-7189(89)90048-7

33.

Lusambili

Jepkosgei

Nzinga

English

(2019). What do we know about maternal and perinatal mortality and morbidity audits in sub-saharan Africa? A scoping literature review. International Journal of Human Rights in Healthcare, 12(3), 192–207. https://doi.org/10.1108/IJHRH-07-2018-0052

34.

Marchal

Dedzo

Kegels

(2010). A realist evaluation of the management of a well- performing regional hospital in Ghana. BMC Health Services Research, 10(1), 24. https://doi.org/10.1186/1472-6963-10-24

35.

Marchal

Kegels

Van Belle

(2018). Theory and realist methods. In Emmel

Greenhalgh

Manzano

Monaghan

Dalkin

(Eds.), Doing realist research (pp. 80–89): Sage Publications.

36.

Marchal

van Belle

van Olmen

Hoerée

Kegels

(2012). Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research. Evaluation, 18(2), 192–212. https://doi.org/10.1177/1356389012442444

37.

Mathison

(2005). Encyclopedia of evaluation (Vols. 1-0). Sage Publications, Inc. https://doi.org/10.4135/9781412950558

38.

Merali

Allen

(2011). Complexity and systems thinking. In Allen

Maguire

McKelvey

(Eds.), The Sage handbook of complexity and management (pp. 31–52). Sage.

39.

Ministère de la Santé du Bénin (2013). Arrêté n°096/MS/DC/SGM/CTJ/DPP/DRFM/DNEHS/DNSP/DSME/SP-C portant obligation de notification des cas de décès maternels, néonatals, institutionnalisation des audits des décès maternels, néonatals des cas de near-miss. Ministère de la Santé du Bénin.

40.

Ministère de la santé du Benin & Centre de Recherche en Reproduction Humaine et en Démographie (2017). Rapport annuel de la Surveillance des Décès Maternels/néonatals et RIposte (SDMR) en 2016. Ministère de la santé du Bénin.

41.

Ministère de la santé du Benin & Centre de Recherche en Reproduction Humaine et en Démographie (2018). Rapport annuel de la Surveillance des Décès Maternels/néonatals et RIposte (SDMR) en 2017. Ministère de la santé du Benin.

42.

Ministère de la santé du Benin & Centre de Recherche en Reproduction Humaine et en Démographie (2019). Rapport annuel de la Surveillance des Décès Maternels/néonatals et RIposte (SDMR) en 2018. Ministère de la santé du Benin.

43.

Ministère de la santé du Benin & Centre de Recherche en Reproduction Humaine et en Démographie (2020). Rapport annuel de la Surveillance des Décès Maternels/néonatals et RIposte (SDMR) en 2019. Ministère de la santé du Benin.

44.

Ministère de la santé du Benin & Centre de Recherche en Reproduction Humaine et en Démographie (2021). Rapport annuel de la Surveillance des Décès Maternels/néonatals et RIposte (SDMR) en 2020. Ministère de la santé du Benin.

45.

Ministère de la santé du Benin & Centre de Recherche en Reproduction Humaine et en Démographie (2022). Rapport annuel de la Surveillance des Décès Maternels/néonatals et RIposte (SDMR) en 2021. Ministère de la santé du Benin.

46.

Ministère de la santé du Benin & Centre de Recherche en Reproduction Humaine et en Démographie (2023). Rapport annuel de la Surveillance des Décès Maternels/néonatals et RIposte (SDMR) en 2022. Ministère de la santé du Benin.

47.

Mukumbang

F. C.

Marchal

Van Belle

van Wyk

(2018). A realist approach to eliciting the initial programme theory of the antiretroviral treatment adherence club intervention in the Western Cape Province, South Africa. BMC Medical Research Methodology, 18(1), 47. https://doi.org/10.1186/s12874-018-0503-0

48.

Olmos-Vega

F. M.

Stalmeijer

R. E.

Varpio

Kahlke

(2023). A practical guide to reflexivity in qualitative research: AMEE guide No. 149. Medical Teacher, 45(3), 241–311. https://doi.org/10.1080/0142159X.2022.2057287

49.

Pawson

(2013). The science of evaluation: A realist manifesto. Sage Publications.

50.

Pawson

Greenhalgh

Harvey

Walshe

(2005). Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy, 10(Suppl 1), 21–34. https://doi.org/10.1258/1355819054308530

51.

Pawson

Manzano-Santaella

(2012). A realist diagnostic workshop. Evaluation, 18(2), 176–191. https://doi.org/10.1177/1356389012440912

52.

Pawson

Tilley

(1997). Realistic evaluation. Sage Publications Ltd.

53.

Pawson

Tilley

(2004). Realist Evaluation. Sage. https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=e092393652456869d7ccf6fa57b5a5a4d2d9e10a.

54.

Roemer

M. I.

Montoya-Aguilar

(1988). Quality assessment and assurance in primary health-care. World Health Organization.

55.

Salter

K. L.

Kothari

(2014). Using realist evaluation to open the black box of knowledge translation: A state-of-the-art review. Implementation Science, 9(1), 115. https://doi.org/10.1186/s13012-014-0115-y

56.

Smeets

R. G. M.

Hertroijs

D. F. L.

Mukumbang

F. C.

Kroese

M. E. A. L.

Ruwaard

Elissen

A. M. J.

(2022). First things first: How to elicit the initial program theory for a realist evaluation of complex integrated care programs. The Milbank Quarterly, 100(1), 151–189. https://doi.org/10.1111/1468-0009.12543

57.

Stern

Stame

Mayne

Forss

Davies

Befani

(2012). Broadening the range of designs and methods for impact evaluations. Department for International Development.

58.

Willcox

M. L.

Okello

I. A.

Maidwell-Smith

Tura

A. K.

van den Akker

Knight

(2023). Maternal and perinatal death surveillance and response: A systematic review of qualitative studies. Bulletin of the World Health Organization, 101(1), 62–75G. https://doi.org/10.2471/BLT.22.288703

59.

Willcox

M. L.

Okello

I. A.

Maidwell-Smith

Tura

A. K.

van den Akker

Knight

Dumont

Muller

(2023). Determinants of behaviors influencing implementation of maternal and perinatal death surveillance and response in low- and middle-income countries: A systematic review of qualitative studies. International Journal of Gynecology & Obstetrics, 165(2), 586–600. https://doi.org/10.1002/ijgo.15132

60.

Willcox

M. L.

Price

Scott

Nicholson

B. D.

Stuart

Roberts

N. W.

Allott

Mubangizi

Dumont

Harnden

(2020). Death audits and reviews for reducing maternal, perinatal and child mortality. Cochrane Database of Systematic Reviews, 3(3), Article 12982. https://doi.org/10.1002/14651858.CD012982.pub2

61.

Wilson

Goldsmith

(1995). Quality and its measurements. In Wilson

Goldsmith

(Eds.), Quality management in health care (pp. 229–258): McGraw-Hill.

62.

Wong

Westhorp

Manzano

Greenhalgh

Jagosh

Greenhalgh

(2016). RAMESES II reporting standards for realist evaluations. BMC Medicine, 14, 1–18.

63.

Wong

Westhorp

Manzano

Greenhalgh

Jagosh

Greenhalgh

(2016). RAMESES II reporting standards for realist evaluations. BMC Medicine, 14(96), 1–18. https://doi.org/10.1186/s12916-016-0643-1

64.

World Health Organization . (2004). Beyond the numbers: Reviewing maternal deaths and complications to make pregnancy safer. WHO.

65.

World Health Organization . (2006). Quality of care: A process for making strategic choices in health systems. World Health Organization. https://iris.who.int/bitstream/handle/10665/43470/?sequence=1.

66.

World Health Organization . (2013). Maternal death surveillance and response: Technical guidance information for action to prevent maternal death. WHO.

67.

World Health Organization . (2016). The global strategy for women’s, children’s and adolescents’ health (2016-2030): Adolescents’ health: Report by the Secretariat (p. 6). Every Woman Every Child. https://www.unicef.org/zambia/media/5886/file/Global-Strategy-for-Womens-Childrens-and-Adolescents-Health-2016%E2%80%932030.pdf.

68.

World Health Organization , Standards for improving quality of maternal and newborn care in health facilities, (2016a). Retrieved February 25, 2024, from https://apps.who.int/iris/bitstream/handle/10665/249155/9789241511216-eng.pdf

69.

World Health Organization . (2016b). Time to respond: A report on the global implementation of maternal death surveillance and response (MDSR). WHO [Internet].

70.

World Health Organization , (2021). Maternal and perinatal death surveillance and response: Materials to support implementation. World Health Organization.

71.

World Health Organization . (2023). Trends in maternal mortality 2000 to 2020: Estimates by WHO, UNICEF, UNFPA, world bank group and UNDESA/population division. World Health Organization.

72.

World Health Organization Regional Office for South-East Asia , (2014). Study on the implementation of maternal death review in five countries in the South-East Asia region of the world health organization. Accessed August 9, 2022. https://iris.who.int/bitstream/handle/10665/205952/B5115.pdf?sequence=1&isAllowed=

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.96 MB

Eliciting the Initial Programme Theory in a Realist Evaluation of Facility-Based Maternal Death Reviews in Benin: Methodological Process,Challenges and Lessons Learned

Abstract

Keywords

Background

Methods

Realist Evaluation

The Process of Eliciting the IPT

Results

Steps Taken to Elicit the Initial Programme Theory

Step One: Scoping Review

Step Two: Document and Literature Reviews

Document Review

Literature Review

Process of Analysing the Documents and the Peer-Reviewed Papers

Step Three: In-Depth Interviews and Observations and Synthesis

Step Four: Finalisation of the IPT

Discussion

Conclusion

Supplemental Material

Supplemental Material - Eliciting the Initial Programme Theory in a Realist Evaluation of Facility-Based Maternal Death Reviews in Benin: Methodological Process, Challenges and Lessons Learned

Footnotes

Acknowledgements

Author Contributions

Declaration of Conflicting Interest

Funding

Ethical Statements

ORCID iDs

Supplemental Material

References

Supplementary Material