Abstract
Background
Process evaluations of randomised controlled trials (RCTs) can provide insight and inform us on the intervention implementation, the causal mechanisms and the contextual factors. This will inform about interventions’ success or failure due to their implementation or the interventions themselves. We aim to consolidate the methodology from previous process evaluations of complex interventions upon their findings on facilitators and barriers to address the prevention of type 2 diabetes mellitus among women with gestational diabetes mellitus (GDM).
Methods
Comprehensive search will be conducted on electronic databases and reference lists of recent reviews for RCTs of complex interventions which address process evaluations of diabetes prevention intervention (DPI) for women with GDM in healthcare settings. There is no restriction on the language of the papers and year of publication until December 2020. Data from each study will be extracted by two reviewers independently using standardised forms. Data extracted include descriptive items on the study design and the outcomes of process evaluations from the three dimensions: (1) implementation; (2) mechanism of impact and (3) context. The quality of the studies will be assessed using mixed methods appraisal tool which is designed for the appraisal of mixed studies in systematic reviews. A narrative and framework analysis of the findings will be presented to inform the contents of a new DPI for women with GDM.
Discussion
The findings from this process evaluation findings are valuable in determining whether a complex intervention should be scaled up or modified for other contexts in future plan. It will give deeper understanding of potential challenges and solutions to aid in the implementation of effective DPIs for GDM in Malaysia.
Keywords
Background
Gestational diabetes mellitus (GDM) is a condition that occurs when there is impairment in glucose tolerance during pregnancy, with 84% of diabetes in pregnancy being GDM (International Diabetes Atlas, 2019). The worldwide prevalence of GDM ranged from 7.5% to 27% in 2019 with the incidence of GDM continuing to increase, especially in low to middle income countries (International Diabetes Atlas, 2019). For example, 1 in 4 live births in South East Africa were affected by GDM compared to 1 in 6 live births in Europe. In Malaysia, the prevalence of GDM ranges 7.71%–27.9% (4th Report of National Obstetric Registry, 2013–2015; International Diabetes Atlas, 2019; Logakodie et al, 2017). The maternal consequences of GDM are wide ranging including obstetric complications such as pre-eclampsia and a 2–7 fold increased risk of having future episodes of GDM and developing type 2 diabetes (T2D) (Bellamy et al., 2009; Reece, 2010) and increased risk of cardiovascular diseases (Lekva et al., 2015). For instance, the International Hyperglycaemia and Adverse Pregnancy Outcomes study found that around half of the women with GDM developed T2D or prediabetes 11 years later (Hapo Study Cooperative Research Group, 2010). The risk factors for this conversion include maternal age, glucose levels in pregnancy, family history of diabetes, pre-pregnancy and postpartum body mass index, dietary patterns, physical activity, breastfeeding and genetic predisposition (Davis et al., 2017; Gunderson et al., 2015; Liu et al., 2014). Women with GDM are also more likely to have antenatal and postnatal depression than those without GDM (Napoli et al., 2020; Wilson et al., 2020).
There are also potential adverse outcomes for the infant such as shoulder dystocia, neonatal hyperbilirubinaemia, hypoglycemia and respiratory distress syndrome (Bhandiwad et al., 2015) in the short term and increased risk of T2D in the long term (Lowe et al., 2018).
As women receive more medical attention during the antenatal and postpartum period than any other stage in their life course, this represents an important window of opportunity for prevention of T2D post-GDM. While randomised controlled trials (RCTs) have shown that intensive diabetes prevention intervention (DPI) is more effective in reducing T2D risk than usual care in the general population (Diabetes Prevention Program Research Group, 2015; Gong et al., 2019; Tuomilehto et al., 2001), RCTs of DPIs for women with GDM have to date been less effective in preventing future GDM episodes or T2D. There have been approximately 30 such RCTs in the past 10 years. A recent meta-analysis in women with GDM reported mixed results; of the 15 RCTs included in the review, in 8 studies that could be pooled, there was a non-significant reduction in risk for T2D (risk ratio = 0.75, 95% confidence interval (CI) 0.55–1.03 (Goveia et al., 2018). More recent RCTs have reported a small non-significant mean reduction in weight over 12 months (Liu et al., 2018) and a non-significant reduction in T2D at 3 years (Shek et al., 2014).
There is a need to understand why current models of DPI in GDM women are of limited effectiveness. As DPIs are, in effect, complex interventions made up of multiple components or active ingredients (Craig et al., 2008), one approach could be is to critique the process evaluation (Moore et al., 2015). This is an umbrella term for a range of theoretical constructs and methodologies that aim to identify the key components of a complex intervention that help explain how and why they are (or are not) effective.
The Medical Research Council (MRC) by Moore et al. (2015) has produced a theoretical framework that describes three dimensions of a process evaluation (see Figure 1). The first dimension is implementation where it identifies the process through which the interventions are delivered and what is delivered in real practice. In the implementation dimension, it does include the theoretical framework and behaviour change technique used, the dosage to define how much intervention delivered and received, the reach to which the audience comes into contact with the intervention and the fidelity such as the consistency of the planned and real intervention. Second dimension of the process evaluation is the mechanism of impact which aims to explain the pathway through which intervention activities produce intended (or unintended) effects. The study of mechanisms may include participant or therapist responses to interaction with a complex intervention and examining mediating or intermediate processes which explain subsequent changes in outcomes, including those unintended. The last dimension is the context in which the intervention is set. This includes moderating factors external to the intervention but that may influence its implementation, such as health policy, organisational factors or changes in delivery of medical care, socio-cultural factors such as stigma or cultural values, lack of support to attend the complex intervention and psychological factors such as shame or health beliefs. Conceptual framework (adapted from Medical Research Council guidance on process evaluations by Moore et al. (2015).
In summary, process evaluations use quantitative and/or qualitative methods to examine these three dimensions (implementation, mechanisms of action and contextual factors) will help to identify the active ingredients in the RCTs (Anderson et al., 2013).
There has been no systematic review of process evaluation in RCTs of DPI in the healthcare settings and general population. However, processes that are most commonly reported in these DPIs often include nutritional advice and/or physical activity; behaviour change techniques such as goal setting and motivational interviewing; target setting such as aiming for 5–7% weight loss; specialist healthcare providers such as health trainers; ‘dose’ in terms of number of sessions and duration of the intervention and pacing of sessions such as initial high frequency of sessions followed by less frequent maintenance phases. In contrast, the majority of the DPI processes for GDM included diet and physical advice (Goveia et al., 2018). There was tentative suggestion that those DPIs which started soon after delivery were more effective (risk ratio = 0.61, 95% CI 0.40–0.94). There have been less than a handful of RCTs that have used breastfeeding as an intervention.
This study therefore first aims to conduct a systematic review of the processes evaluated in RCTs of DPIs in GDM; second to describe the range of processes assessed and the methods used to do so; third to identify processes that overall appear to be a) more effective and b) not effective in reducing the risk of T2D post GDM. These findings will inform the content of a DPI for women with GDM in Malaysia.
Methodology
Design and Registration
We will conduct the systematic review in accordance with Preferred Reporting Items for Systematic reviews and Meta-Analyses Protocol (PRISMA) guidelines from Moher et al. (2009) (see Figure 2) and checklist (see appendix). The systematic review is prospectively registered with PROSPERO (Registration number: CRD42020208212) and amendments will be documented with version description. Preferred Reporting Items for Systematic Review and Meta-Analyses: RCT randomised controlled trial.
Eligibility Criteria
Types of studies
Only RCTs will be included; cohort and uncontrolled before-and-after studies, case series, reviews, non-journal article, non-report based on empirical research and non-human research will be excluded. We will aim to extract process evaluation findings from all RCTs whether or not these have been explicitly described as some RCTs do include some aspects of process evaluation.
Types of patients
The patient group will consist of women who have current or a history of GDM. Women at high risk of GDM such as those with obesity, family history of T2D, menopausal and post-menopausal women will be excluded.
Types of intervention
The interventions included will be complex interventions consisting of lifestyle change delivered either in the antenatal and/or postnatal period through digitalised, face-to-face or mixed methods intervention during the antenatal and/or postnatal period. It includes dietary management, physical activity, psychological aspect and others. Interventions that test a single ingredient, for example, a pharmacological agent will be excluded.
The comparators
It may include treatment as usual, active control or placebo control. Placebo controlled is a condition where one that appears in all respects to be identical to the treatment condition and lacking on the critical ingredient of the treatment. Meanwhile, an active control is one in which participants engage in some task during the intervention period.
Types of outcome
The primary outcome will be the strengths and limitations of each process evaluation as well as the identification of implementation barriers and facilitators for the complex intervention following MRC guidance as a reference point. Quantitative and qualitative data will be extracted, coded and grouped into categories of implementation, mechanism of impact and context dimensions and their related constructs. In the implementation dimension, the following constructs will be assessed: the description on the type of intervention and control; the process through which the intervention delivery is achieved; the theoretical framework and behaviour change technique used to inform the intervention; the reach to the extent the target audience comes into contact with the intervention; dosage (number of sessions or total time) delivered and received and the fidelity which include the constancy of delivering according to the protocol and competency in delivering them. In the mechanism of impact dimension, the following will be extracted: format of delivery (digital, non-digital or hybrid); participant and healthcare provider experience towards the complex intervention as well as any intended or unintended process which explain subsequent changes in the outcomes. Whereas in the context dimension, we will extract data on the socio-cultural; organisational, local and national policy factors (such as barriers, facilitators, contamination and adoption) which may influence the intervention implementation, intervention mechanisms and its outcomes. Inductive derivation of the key themes done through constant comparison between the findings from the papers within each category and examining the relationships between them. The secondary outcomes we will extract include maternal rates of T2D and of subsequent episodes of GDM; risk factor for T2D (weight, blood sugar profile and lipid profile); psychological issue; mother–infant relationship; maternal complications and infant complications.
Information Sources
We will search from the following databases: OVID (EMBASE, MEDLINE, PsycINFO and Global Health), Cochrane Library, PUBMED and hand-searching of reference lists of recent reviews (Goveia et al., 2018; Miyazaki et al., 2017). There is no restriction on the language of the papers and year of publication until December 2020. For each RCT included in this review, we will specifically search for duplicates and secondary publications, and contacting all the authors to request any published or unpublished process evaluation. Forward and backward citation will also be used to screen for eligible articles.
Search Strategy
The keywords will be developed and iteratively refined from the research questions and definitions of key concepts by using Cochrane search terms for diabetes prevention, GDM and lifestyle interventions including antenatal, postnatal, breastfeeding, T2D, randomised control trial and process evaluations. The main key words used will be GDM, randomised controlled trial and process evaluation. Boolean search will be used to combine the keywords with operators such as AND, NOT and OR to further produce more relevant results. For example: ((GDM OR gestational diabetes OR pregnancy-induced diabetes OR diabetes in pregnancy) AND ((RCT OR controlled clinical trial OR pragmatic control trial OR clinical trial) AND (process evaluation OR programme evaluation OR process assessment OR process acceptance OR outcome measures)); ((GDMs OR gestational diabetes OR pregnancy-induced diabetes)) AND ((RCT OR controlled clinical trial OR pragmatic clinical trial OR clinical trial) OR (process evaluation OR programme assessment OR process acceptance OR outcome measures).
Data Management
Rayyan QCRI will be used for title and abstract screening. Duplicate papers will be identified and removed accordingly from search strategy database and EndNote. Two authors will independently assess the eligibility of the publications based first on the titles and abstracts of records identified by the searches. After excluding papers that do not meet eligibility criteria based on titles and abstracts, the remaining publications will be downloaded for further assessment of eligibility. Data will be stored in a common file which will be password protected. Non-English language papers will be translated into English by bilingual professionals or academics. The reviewers will resolve disagreements as to eligibility through a consensus-based discussion, and if necessary, with a third reviewer.
Standardised Data Extraction
The following data about the RCT will be extracted: general information of the study (author, title of the study, year and type of publication, aim of the studies, studies country, economy level, funding sources and conflict of interest); methodology (study population description, sample size, type of study, source and setting of the population, sampling population, inclusion and exclusion criteria, sampling technique, study measurement and instrument as well as instruments validity and reliability); study duration (study start date and study end date); results (primary findings, secondary findings, response rate, attrition rate, effect sizes, participant’s characteristic results such as age, ethnicity, marital status, parity, type of birth, education and employment) and summary (discussion, conclusion, strength, limitation, future recommendation and risk of bias assessment). The outcomes we will extract include: maternal rates of T2D and of subsequent episodes of GDM; risk factor for T2D (weight, blood sugar profile and lipids profile); psychological issue (depression); mother–infant relationship and other maternal and infant complications.
In terms of the process evaluation aspects, the MRC provides many examples of processes but there is no consensus for a gold standard or a minimum checklist for process evaluation. Therefore, we have derived a checklist of process constructs within the three dimensions. Qualitative and quantitative data will be extracted for each construct. In the implementation dimension, the following constructs will be assessed: the theoretical framework used to inform the intervention; an assessment of reach; dose (number of sessions or total time) delivered; dose received; fidelity of delivering protocol as stated; description of behaviour change techniques and competency in delivering them. In the mechanistic dimension, the following will be extracted: format of delivery (digital, non-digital or hybrid); participant experience and health provider experience. Digital interventions will include the use of telephone, video-conferencing, mobile short message service (SMS), apps or web-based interfaces and other remote wireless relay systems (Moore et al., 2015). In the contextual factors, we will extract data on external factors such as socio-cultural, organisational, local and national policy factors (such as barriers, facilitators, contamination and adoption) which may influence the implementation and mechanism of the DPI. The data extraction forms will be pilot tested by the two reviewers and iteratively revised. Two reviewers will independently extract data from the rest of the included list of articles.
Quality Assessment
Data to assess quality will be extracted from the included studies. Quality assessment tool or risk of bias tool uses scale or checklist which provides a list of criteria or domain in order to appraise quality of the study (Pace et al., 2012; Pluye & Hong, 2014). Hence, a unique tool known as Mixed Method Appraisal Tool by Hong et al. (2018) will be used to assess the quality of different study designs (see Figure 3). This tool is about judgement making by the reviewers. For each included study, it is necessary for the reviewer to choose appropriate category of studies to appraise. Then, rate the criteria of the chosen category accordingly. For each criterion, a yes or no (Pace et al., 2012; Pluye & Hong, 2014) may be scored as 1 or 0, respectively. The ‘cannot tell’ response category indicates that the paper do not report appropriate information to answer ‘yes’ or ‘no’ and the reviewer would look for companion papers or contact the authors for more information or clarification. In terms of scoring (https://bit.ly/2VwFzkA), the overall score can be presented using descriptors such as * or %. The number of * will reflect the number of quality criteria a study has met. For quantitative and qualitative studies, an * (20%) is for meeting one criterion, and ***** (100%) are for meeting all five criteria. For mixed methods studies, the same scoring is applied to each of the qualitative, quantitative and mixed methods component. The component with the lowest score will reflect the overall quality of a mixed methods study. The quality rating of each study and criterion will be supported with explanatory notes to better inform the quality of the included studies (Hong et al. 2018). This may lead to a sensitivity analysis such as comparing the quality of studies by contrasting their results *, **, ***, and ****. Mixed methods appraisal tools, 2018.
Data Synthesis and Outputs
The extracted qualitative and quantitative data for each of the three MRC process evaluation dimensions will be summarised. In quantitative part, descriptive items (i.e., number of positive RCTs) will be tallied and synthesised into three tables: (1) Overall characteristics of the study; (2) methodology: stages of the process evaluation will be gathered and grouped among the studies grouping (i.e., feasibility/piloting, effectiveness and post-evaluation) and (3) quality assessment: using a tool developed by the Effective Public Health Practice Project for knowledge synthesis which provides a standardised means to assess study quality and overall methodological rating of either strong, moderate or weak category for evidence support practice. Meanwhile for the qualitative part, framework analysis will be used to extract the data from all of the dimensions. It is a matrix-based method involving the construction of thematic categories into which data can be coded (Ritchie et al., 1994). Framework analysis can be flexible during the analysis process where it allows the user to both collect all the data and then analyse it or do data analysis during the collection process. In the analysis stage, the gathered data are sifted, charted and sorted in accordance with key issues and themes. Hence, the extracted data will be coded and grouped accordingly into sub-components for each dimension of implementation, mechanism of impact and implementation. The key themes derivation will be identified through reasoning and constant comparison between the findings from the collective papers within each category by examining the relationships between them. One practical feature of framework analysis is that it enables questions or issues identified in advance by various stakeholders (such as policymakers, practitioners or user groups) to be explicitly and systematically considered in the analysis, while also facilitating enough flexibility to detect and characterise issues that emerge from the data. Using a modified MRC framework, we will map our methodological and implementation findings to triangulate and synthesise the data. Triangulation takes part as a method to develop comprehensive understanding and to act as a strategy to test the validity through the convergence of information from different sources of the data. These categories of data information will then be subjected to thematic synthesis in order to produce a comprehensive set of synthesised findings (Tong et al., 2012). Throughout the review, patient and public perspectives are synthesised from published papers, and hence, no public and patients were directly involved in this study. The findings will be synthesised according to each aim and will be used to inform the contents of a new DPI for women with GDM in Malaysia.
Discussion
There have been approximately 30 RCTs of complex lifestyle interventions to reduce the risk of T2D in women with GDM yet collectively the evidence for the effectiveness of these interventions remains limited. Before conducting another RCT of DPI in this high-risk group, it is important to understand why previous RCTs have either had negative or weak effects. Process evaluation integrated into RCTs of complex interventions capture an umbrella of concepts to help explain the underlying mechanisms of action. They vary in their objectives, methodology and the selection of outcome and there is no consensus as yet as to how best to conduct them. We have proposed a framework for a systematic review that will extract and summarise what processes have been evaluated under the three key MRC dimensions of implementation, mechanism and context. We will use the findings to inform the development of a logic model, the contents of a digital app and derive a process evaluation framework for our future feasibility study testing of a refined DPI.
The strengths of this protocol include that it sets out a priori a systematic review using a comprehensive approach to include all RCTs regardless of whether the term process evaluation is explicitly described. As there is no validated consensus checklist to conduct a process evaluation, we have adapted other models (Grant et al., 2013; Saunders, 2006). We have proposed a pragmatic checklist of key processes using widely accepted current theories and national guidance (Moore et al., 2015).
The limitations are that we may have unintentionally omitted one or more relevant processes, but we aim to reduce this risk by piloting the data extraction schedule with several RCTs and a priori revising the protocol if new processes are identified during the review. There are some frameworks we could have used instead such as the Wagner Chronic Care model which six interdependent elements of a complex intervention, namely, the Health System: The Community: Self-Management Support: Decision Support: Delivery System Design: and Clinical Information Systems, but they do not give guidance on the methods for assessing the processes (Barr et al., 2003).
Conclusion
In summary, we have devised a protocol for a systematic review of process evaluations for DPI for GDM, and the findings are potentially informative for other lifestyle interventions to reduce chronic diseases such as obesity, cardiovascular diseases and depression.
Supplemental Material
sj-pdf-1-ijq-10.1177_16094069211034010 – Supplemental Material for A Protocol of Process Evaluations of Interventions for the Prevention of Type 2 Diabetes in Women With Gestational Diabetes Mellitus: A Systematic Review
Supplemental Material, sj-pdf-1-ijq-10.1177_16094069211034010 for A Protocol of Process Evaluations of Interventions for the Prevention of Type 2 Diabetes in Women With Gestational Diabetes Mellitus: A Systematic Review by Iklil Iman Mohd Sa’id, Iliatha Papachristou Nadal, Angus Forbes, Kimberley Goldsmith, Irmi Zarina Ismail, Faezah Hassan, Siew Mooi Ching, Nicola Guess, Helen Murphy, Matthew Prina, Barakatun Nisak Mohd Yusoff, Nurul Iftida Basri, Mazatulfazura SF Binti Salim, Nur Hafizah Mahamad Sobri, Pamela Phui Har Yap, Khalida Ismail, Boon How Chew and On behalf of MYGODDESS Project team in International Journal of Qualitative Methods
Footnotes
Acknowledgements
We would like to thank the Ministry of Education Malaysia through the MyPAiR (Malaysia Partnership & Alliances in Research) program and the UK’s Medical Research Council that have jointly created the MyPAiR-Non-communicable Disease Year 2019 research grantfor providing the funding for this systemic review, a part of the Malaysian GestatiOnal Diabetes and prevention of DiabtES Study (MYGODDESS) (
). We would like to thank our Knowledge User Advisory Panel members for providing guidance and feedback. Their contributions represent independent research part funded by the NIHR Biomedical Research Centre (South London and Maudsley NHS Foundation Trust and KCL) and the NIHR Applied Research Collaboration South London (King’s College Hospital NHS Foundation Trust). The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. MYGODDESS Project team members and collaborators include Dr. Hanifatiyah Ali from the Department of Family Medicine Universiti Putra Malaysia who helped in the early protocol drafting by IIMS, Dr. Ziti Akthar Supian at Seri Kembangan Health Clinic, Dr. Haslinda Hassan at Puchong Health Clinic, Dr. Fuziah Paimin at Putrajaya Presint 9 Health Clinic, Dr. Nurain Mohd. Noor and Dr. Wan Ahmad Hazim Wan Ghazali from the Departments of Medicine and Obstetrics and Gynaecology in Hospital Putrajaya, respectively.
Author Contributions
KI and BHC designed the review. IIMS lead and the drafting of the protocol. IPN, AF, KG and MP provided input on the search strategies. BHC, KI, IPN, AF, KG, IZI and FH provided substantive feedback on the manuscript. All authors contributed to the manuscript drafts and reviewed the final manuscript. All authors approved the final protocol.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work is supported by MYPAIR Grant UK-Malaysia: Joint Partnership Call on Non-Communicable Diseases (Malaysia: JPT.S (BPKl) 2000/011/06/05 (27); UK: MR/T018240/1).
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
