Protocol for a process and implementation evaluation of the SMARThealth pregnancy hybrid type 2 cluster randomised controlled trial

Abstract

Background:

This protocol outlines the process and implementation evaluation of the SMARThealth Pregnancy (SHP) pragmatic, type 2 hybrid cluster randomised trial conducted in two states in India (Haryana/Telangana). The SHP trial aims to improve the community-level identification, diagnosis, referral and management of women with anaemia, diabetes, and hypertension during pregnancy and in the year after birth.

Objectives:

The process and implementation evaluation aims to understand how, why, and for whom the SHP intervention may be effective (or not). It aims to identify contextual factors, barriers, and facilitators relevant to the implementation of the intervention, and understand mechanisms and strategies employed during its implementation.

Design:

The study utilised a process evaluation design.

Method:

A mixed methods evaluation drawing from realist evaluation, normalisation process theory, the Medical Research Council framework, reach, effectiveness, adoption, implementation and maintenance framework, and Proctor’s typology will be employed for understanding the implementation process. The evaluation will involve focus group discussions and semi-structured interviews with healthcare providers (Accredited Social Health Activist, primary care doctors, and auxiliary nurse midwife), women, and field staff. Quantitative process data describing reach, fidelity, dose, and adoption of intervention will be collected. Observations of trial setup and implementation will be conducted. Both qualitative and quantitative data will be analysed iteratively before the effectiveness outcomes of the SHP trial are available and will subsequently be triangulated with the trial primary outcome evaluation data.

Discussion:

The findings from this process evaluation will provide an understanding of how the intervention works in practice, its potential to detect and manage anaemia, diabetes and hypertension during pregnancy and in the year after birth, and its scalability as an integrated model for the management of non-communicable diseases in pregnancy/postnatal care.

Keywords

maternal health mHealth non-communicable diseases anaemia gestational diabetes preeclampsia community health workers rural India complex intervention clinical decision support

Introduction

Perinatal health and non-communicable diseases in India

Non-communicable diseases (NCDs) are women’s leading causes of death globally and in India.¹ Women who develop complications during pregnancy are at increased risk for NCDs in the months and years following birth.² Women with hypertensive disorders of pregnancy (HDP), such as gestational hypertension or preeclampsia, are at risk of later chronic hypertension and premature cardiovascular diseases,³ and those with gestational diabetes mellitus (GDM) are at a significantly increased risk of type 2 diabetes in years after birth.⁴ Although it has been recommended that pregnancy care should be part of an integrated approach to NCDs,^5–7 clear evidence and guidance is missing on how this can best be performed.

In India, in addition to NCDs, anaemia rates amongst pregnant women are amongst the highest in the world. Anaemia is linked to poor pregnancy outcomes, such as post-partum haemorrhage, maternal death, low birth weight, preterm birth, and perinatal and neonatal mortality.⁸ The Indian Government has declared anaemia a health priority and launched the comprehensive national anaemia reduction programme (Anaemia Mukt Bharat) in 2018.⁹ Substantial progress has been made in maternal health; however, gaps in coverage and service delivery remain.¹⁰

Both NCDs and anaemia are leading challenges in maternal health in India, and their high prevalence has severe, lasting consequences for both mothers and their children. In addition, NCDs and anaemia are often interconnected and exacerbate one another, and both anaemia, HDP, and GDM have been identified as priority areas for improving maternal and child health in rural India.¹¹

Perinatal mobile health interventions

Mobile health interventions including clinical decision systems are increasingly being used in maternity care. Recent evaluations found improvements in safety and health outcomes; however, design of interventions, quality, complexity of systems, and evaluations were very heterogeneous.^12–14 In particular, interventions that are working across ante- and postnatal stages, high-quality large-scale randomised controlled trials (RCTs), and evaluations that capture implementation outcomes of the intervention in routine primary care settings are rare.^15–17

The SMARThealth Pregnancy intervention

To fill this gap and in response to locally identified challenges around the timely detection and referral of high-risk conditions during and after pregnancy in rural India, we developed the SMARThealth Pregnancy (SHP) intervention.

SHP leverages the Systematic Medical Appraisal, Referral and Treatment (SMARThealth) platform, initially developed by the George Institute for Global Health for cardiovascular disease prevention and management.¹⁸ This platform is a mobile health clinical decision-support system designed to assist community health workers in screening various conditions using point of care testing and an app functioning via a low-cost Android tablet with clinical decision-support algorithms based on local guidelines. The app enables Accredited Social Health Activists (ASHAs) to electronically refer high-risk cases to the Primary Health Centre (PHC) for medical review. The primary care doctor also uses a connected tablet app for more complex decision support, such as medication management suggestions, based on national guidelines.^19–23 The follow-up module of the platform supports the ASHAs to conduct timely follow-up visits and discuss relevant topics like visit to the doctor, adhering to healthy lifestyle and medications, or achieving target levels for BP and diabetes during the visit. SHP has adapted the SMARThealth approach for pregnant women to improve the identification, diagnosis, referral, follow-up and management of women with anaemia, diabetes, and hypertension during and after pregnancy.²⁴ Acceptability and feasibility of SHP have previously been established in a pilot cluster randomised trial in four sites in rural India from 2019 to 2020.²⁵

The SHP intervention comprises five main components (see Table 1).

Table 1.

SHP intervention components.

SHP component	Details
1. Healthcare worker education programme	For community health workers known as ASHA’s, and primary care doctors.
2. SMARThealth Pregnancy (SHP) app	Contains a clinical decision-support algorithm to support the screening, referral and management of anaemia, hypertension, and diabetes.
3. SMARThealth Pregnancy dashboard	Provides analytics, oversight and tracking.
4. Supply chain and logistics support	Monitored via the SHP app and feed back to PHCs and district officials.
5. Stakeholder engagement	With community stakeholders to increase awareness of the programme and high-risk pregnancy conditions, and with primary care doctors and district officials to ensure the programme relevance for local priority needs and community ownership.

SHP: SMARThealth Pregnancy; ASHA: Accredited Social Health Activist; PHC: Primary Health Centre.

The SHP trial

The SHP trial is a pragmatic, type 2 hybrid effectiveness and implementation, parallel-group cluster randomised trial across two states in India (Haryana and Telangana). The trial aims to improve the community-level identification, diagnosis, referral, follow-up and management of women with anaemia, diabetes, and hypertension during pregnancy and in the year after birth.²⁶ Specific objectives of SHP are to (i) determine if the SHP intervention can improve women’s health in the year after pregnancy, specifically by decreasing the prevalence of anaemia by 9%; (ii) to determine if SHP can improve follow-up care for women with GDM or HDP; and (iii) to identify the key areas to support implementation of the intervention beyond the trial phase.²⁶ Recruitment for the SHP trial started on 4 June 2022, and was completed on 19 December 2023. The last date of participant follow-up is scheduled for 15 April 2025. The full trial protocol, including a detailed logic model, of the SHP complex intervention has been published elsewhere.²⁶

Process and implementation evaluation

Various mechanisms and processes contribute to effectiveness in a complex intervention. Process evaluations accompanying RCTs help to evaluate the effectiveness of an intervention by understanding how it was implemented, the mechanisms by which it achieved the effect, and in which ways the intervention interacted with the context in which it was implemented.^27,28 An implementation evaluation, on the other hand, aims to understand how the different components of an intervention were adopted, followed, and sustained, providing insights into post-trial sustainability and adoption.²⁹

For the SHP trial outlined above, a process and implementation evaluation will be conducted alongside the trial. In order to capture both process and implementation outcomes and to understand how the SHP intervention has been embedded and normalised in practice, a combination of frameworks will be applied. We will utilise the Medical Research Council (MRC) guidance,³⁰ Realist evaluation,³¹ normalisation process theory (NPT),³² the reach, effectiveness, adoption, implementation and maintenance (RE-AIM) framework,³³ and Proctor’s typology³⁴ to understand how the intervention was delivered and implemented and to inform strategies for its broader adoption and long-term sustainability.

Aims

Through evaluation of the process and implementation of the SHP trial, we aim to (i) assess whether the intervention was delivered as designed and (ii) understand how the SHP intervention was implemented. This evaluation will help clarify assumptions around causal mechanisms, and enhance understanding about the generalisability of the intervention components for future implementation. The specific objectives of this process evaluation are to:

Assess whether the SHP trial has been implemented as intended, focusing on the degree of adherence to the planned protocol (fidelity) and the extent to which the intervention was delivered (dose).

Evaluate how, why, and to what extent the SHP intervention has been effective considering factors such as RE-AIM.

Identify and analyse the barriers, facilitators, and strategies that influenced the implementation of the SHP intervention.

Understand the necessary conditions and strategies required for the SHP intervention to be sustained and scaled up in clinical settings, in contexts similar to the trial sites.

Understand how contextual factors and mechanisms as well as unexpected consequences impacted the trial outcomes and the overall success of the intervention.

Methods

Study setting

The study will be community-based and conducted in rural and semi-rural districts in India: Siddipet district in Telangana State and Jhajjar and Rohtak districts in Haryana State, India. Across India, the maternal mortality rate has significantly dropped in the last decades to an average 113 per 100 000 live births in 2018, according to estimates from the Sample Registration System (SRS).³⁵ Recent estimates from the Health Management Information System (HMIS) indicate that maternal mortality in Telangana has dropped to 53 maternal deaths per 100,000 live births in 2017–2019³⁵ and has been maintained at less than 70 in 2019–2021.³⁶ In comparison, depending on estimates, Haryana had a higher maternal mortality rate of 101-91 for 2014–2018 (SRS) and 90 for 2017–2019 (HMIS), for 214 maternal deaths per 100 000 live births before 2018.³⁷ As reported in the recent Comptroller and Auditor General of India’s Performance Audit, contrary to national rates, MMR in Haryana has increased from 98 (2015–2017) to 110 (2018–2020).³⁸

The study will be conducted in both rural and semi-rural areas of India, specifically in the Siddipet district of Telangana and the Jhajjar and Rohtak districts of Haryana. These locations were strategically chosen to represent two distinct geographical, linguistic, and cultural populations. Each of the selected PHCs serves approximately 30,000 people across multiple villages. From each PHC’s catchment area, two villages will be selected to serve as the study sites for the trial. All community health workers and PHC staff from these sites will be eligible to participate in the study.

Study design

A sequential, mixed methods study is designed to assess each of the objectives of the process evaluation. The process evaluation will run concurrently with the SHP trial, which will include both the intervention and active control arms. The Standards for Reporting Implementation Studies (StaRI) checklist³⁹ was utilised supporting transparent and accurate reporting of this process and implementation evaluation (Supplemental Material).

Theoretical frameworks

The process and implementation evaluations of the SHP trial are informed by a number of theoretical frameworks adopting a pragmatic approach to support the translation of research evidence into practice. As no individual framework met the different requirements of the implementation and process analysis, a number of frameworks were combined to guide the analysis. These frameworks provide structure and insight into the complex nature of the intervention and its components.

Integration of frameworks in data collection and analysis

To avoid conceptual overlap, the frameworks are applied sequentially and hierarchically, with clearly delineated roles across study phases. The MRC framework provides the overarching structure for the process evaluation, defining the domains of implementation, mechanisms of impact, and context. Within this structure, RE-AIM and Proctor/Weiner constructs are used operationally to guide data collection and descriptive analysis of implementation outcomes (e.g. reach, fidelity, acceptability, and sustainability). Realist Evaluation underpins theory-driven analysis by linking these implementation outcomes to context–mechanism–outcome (CMO) configurations, enabling explanation of how and why outcomes occur across settings. NPT is applied analytically to selected qualitative data to explain mechanisms related to intervention embedding and routinisation. Together, the frameworks are not used in parallel but are integrated across stages of data collection, analysis, and interpretation, ensuring theoretical complementarity rather than redundancy.

The overall development and evaluation of the intervention follow the U.K. MRC guidance on implementation and evaluation of complex interventions,⁴⁰ using theories in order to understand and explain the process, components, and strategies of the intervention, which may or may not have been effective.⁴¹ The MRC framework highlights the need to unveil causal assumptions and defines three broad areas for evaluation: implementation (what is implemented, and how), mechanism of impact (how the intervention leads to change), and context (how context affects implementation and outcomes).⁴⁰ This process evaluation has been conceptually designed along those three areas.

Within this overarching structure, the implementation evaluation will be guided by the RE-AIM framework, an established and widely used framework for planning and evaluating health interventions.³³ The RE-AIM framework focuses on five implementation dimensions: reach, effectiveness, adoption, implementation fidelity, and maintenance.⁴² In order to assess acceptability and sustainability of the implementation, Proctor’s typology³⁴ and Weiner’s implementation measures⁴³ will also be applied.

A realist evaluation approach will be used at the analytical stage to test the initial programme theory (process evaluation framework) to explain how the context and mechanisms of the SHP intervention work, for whom, and under which circumstances.³¹ In the context of complex interventions and RCTs, such as SHP trial, realist evaluation is an effective method for understanding and clarifying key components.³⁶ Realist evaluation focuses on the interplay between CMOs, allowing interpretation of observed implementation outcomes across different settings and participant groups. This evaluation method will help clarify how the intervention produces both intended and unintended effects, as well as the contextual factors that contribute to its success or failure.

In addition, NPT will be used as a focused analytical lens to examine how the SHP intervention becomes embedded in the routine practices of healthcare workers and communities. NPT is widely used to identify, characterise, and explain mechanisms that affect the implementation of new health processes, technologies, and other complex interventions.⁴⁴ It focuses on four key constructs: coherence (e.g. what is the purpose of the SHP intervention), cognitive participation (what promotes participation in the SHP intervention), collective action (how do women and ASHAs interact with the SHP app to make them work for them), and reflexive monitoring (how do ASHAs, women and PCPs, appraise the intervention’s effectiveness).³² NPT will be applied to selected qualitative data to elucidate mechanisms related to participation, enactment, and appraisal rather than to assess implementation outcomes per se.

Table 2 summarises the purpose and application of frameworks applied.

Table 2.

Overview of frameworks applied to the SHP study.

Framework/theory	Purpose	Application
UK MRC guidance on implementation and evaluation of complex interventions⁴⁰	To understand and explain the process, components, and strategies of the intervention to unveil causal assumptions	Overall SHP study implementation (what is implemented, and how), mechanism of impact (how the intervention leads to change), and context (how context affects implementation and outcomes)
RE-AIM framework³³	To test the initial programme theory (process evaluation framework) To understand how the context and mechanisms of the SHP intervention work, for whom, under which circumstances	Implementation evaluation of the SHP intervention: reach, effectiveness, adoption, implementation fidelity, and maintenance
Proctor’s typology³⁴ and Weiner’s implementation measures⁴³	To assess acceptability and sustainability of the implementation	Implementation evaluation of the SHP intervention: Acceptability, sustainability
Realist evaluation	To test the initial programme theory (process evaluation framework) and to understand how the context and mechanisms of the SHP intervention work, for whom, and under which circumstances	Applied to the programme theory of the overall SHP study: context, mechanisms, and outcomes
NPT⁴⁴	To analyse and explain how the SHP intervention becomes embedded in the routine practices of healthcare workers and communities	Applied to understand the embeddedness of the SHP intervention into routine practices

MRC: Medical Research Council; NPT: normalisation process theory; SHP: SMARThealth Pregnancy; RE-AIM: reach, effectiveness, adoption, implementation and maintenance.

Figure 1 outlines how the different theories and frameworks are being combined.

Figure 1.

Integrated conceptual approach to the SHP2 process and implementation evaluation.

The design and reporting of the process evaluation will adhere to the Quality Standards for Realist Evaluation⁴⁵ and the StaRI checklist.⁴⁶

Process evaluation framework

The process evaluation framework (initial programme theory) outlines how and under which conditions the SHP intervention is expected to lead to its desired effects (see Figure 2). Specifically, the framework helps to capture how we expect the SHP intervention in rural India to work for the stakeholders (women, ASHAs, primary care doctors), how the context may influence the implementation, mechanisms, and outcomes, and how and which components of the implementation and mechanisms are leading to the expected outcomes, and contribute to the overall aim of improving women’s health outcomes in the perinatal phase, for conditions like anaemia, diabetes, and hypertension in rural India.

Figure 2.

The SHP process evaluation framework.

The process evaluation components (highlighted in green) are exploring contextual factors, the implementation of the SHP trial, and mechanisms of impact from the intervention. Questions were developed based on the RE-AIM and Realist frameworks to fit within these components. The causal assumptions of the SHP intervention inform the process evaluation components and the interpretation of the primary and secondary outcomes.

During the process evaluation, we will continue to refine the overall SHP programme theory, as well as the assumptions about how the intervention works, and develop implementation strategies, to address barriers and use facilitators to implementation. These strategies will be derived from the data evaluation, fidelity checklists, and oral/written reports, as well as iteratively informed by a continuous feedback loop from end users (women) and implementers (ASHAs/primary care doctors/field supervisors) on what works (and what does not), and any strategies applied will be captured in a log.

Figure 3 provides an overview of the analysis phases planned in our realist evaluation.

Figure 3.

Analysis phases applied in the SHP2 realist evaluation.

Implementation strategies

During the trial, a number of strategies will be applied to support implementation success and overcome any barriers that may arise.²⁶ The implementation strategies were designed based on findings of the pilot trial,²⁵ in consultation with the study team and local experts, and informed by the Expert Recommendations for Implementing Change (ERIC) compendium.⁴⁷ The ERIC compendium is a list of discrete strategies that can serve as building blocks for constructing multifaceted, multilevel implementation strategies. Our identified strategies for SHP trial include (i) strategies to improve the implementation process (e.g. staff support for ASHAs), (ii) strategies to improve adherence to the intervention (e.g. regular quality control), (iii) strategies to build capacity (e.g. through regular refresher training), or (iv) strategies for dissemination (e.g. community events). A list of planned implementation approaches for each of these strategies is shown in Table 3, which was developed considering the recommendations for specifying and reporting implementation strategies by Proctor et al.⁴⁸ From the start of the SHP study, the strategies will be continuously reviewed and adapted as needed. The process will ensure that any identified barriers to implementation are addressed promptly and effectively. Strategies and any modifications will be documented throughout in a log.

Table 3.

Planned strategies to support implementation of the SHP trial (intervention arm).

Category	Target	Implementation strategy/action	Temporality	Dose	Implementation outcome targeted
I. Strategies to improve the implementation process	Women	Information collected from health system on mortality and morbidity of pregnant mothers and their newborn. This information collected by study team and will help in addressing the major drivers of maternal and newborn mortality. This will help achieve successful programme implementation and outcomes.	Periodic	Regularly	Implementation
	Women	For women who move back to their mother’s village for the baby’s birth visits will be replaced with phone calls, resuming face-to-face visits when the woman returns to her original village.	Continuous	Over the period of the woman’s stay at her mother’s house	Effectiveness, adoption, implementation, maintenance
	ASHAs	Study staff will support ASHAs by attending visits (with participant permission) and providing support for using the technology until the ASHA feels confident of performing assessments independently.	Continuous	Initially for the visits, then as needed/identified in the fidelity checklist	Effectiveness, fidelity, adoption, acceptability, implementation
	Women, ASHAs, primary care doctors, district officials, community, field staff	Continuous feedback loops to enable corrections and changes that are necessary to achieve successful programme implementation and outcomes	Continuous	Regularly	Effectiveness, fidelity, adoption, acceptability, implementation, maintenance
II. Strategies to improve adherence to the intervention	ASHAs	Quality control visits will be made to observe the intervention delivered by ASHAs and assessed by the study team.	Periodic	1 visit	Fidelity, effectiveness
	ASHAs	Compliance of the ASHAs with the intervention activities will be assessed with a fidelity checklist, administered by the study team	Before and after refresher training	Twice	Fidelity
	ASHA, primary care doctors	Compliance with the intervention visits will be monitored centrally by tracking the visits recorded in the SHP app. If trends of non-engagement across multiple participants from the same ASHA worker or PHC are noted, the study team will arrange a site visit.	Continuous	Regularly	Fidelity
III. Strategies to build capacity	ASHAs	Every 6 months, the study team will conduct a 1-day refresher training session for the ASHAs.	One day every 6 months	1 day	Reach, effectiveness, implementation
IV. Strategies for dissemination	Community	Community stakeholder engagement events will be held to increase awareness of the programme and the importance of high-risk pregnancy conditions, and co-develop pathways to implementation and sustainability after completion of the trial.	Before start, and after completion of the trial	Regularly	Implementation, maintenance, acceptability
	Women	Stakeholder engagement events will be held to increase awareness of the programme and the importance of high-risk pregnancy conditions, and co-develop pathways to implementation and sustainability after completion of the trial.	Before start, during, and after completion of the trial	Regularly	Implementation, maintenance, acceptability
	District officials, primary care doctors	We will regularly update and engage with district officials and primary care doctors to ensure the programme addresses local priority needs and is seen as being owned by the communities.	Continuous	Regularly	Implementation, maintenance, acceptability

SHP: SMARThealth Pregnancy; ASHA: Accredited Social Health Activist; PHC: Primary Health Centre.

Throughout the project we will be applying responsive feedback loops, an approach that is effective in providing agile, flexible, adaptive, iterative, and actionable feedback.⁴⁹ Feedback loops provide timely assessments and actionable feedback by creating a communication circle between the project leads, implementers, researchers, healthcare workers, and decision-makers which enable corrections and changes that are necessary to achieve successful programme implementation and outcomes. Responsive feedback loops have more recently also been incorporated in process evaluations of RCTs using digital interventions.⁵⁰

Participants

The participants for the qualitative components of this process and implementation evaluation, including focus group discussions (FGDs) and in-depth interviews (IDIs), will include a range of individuals involved in the SHP intervention. Both FGDs and IDIs are used to capture the perspectives of different stakeholders. FGDs will be conducted with ASHAs who participate in the training and are using the SHP app and diagnostic devices during their visits to explore both their shared and individual experiences in balancing their existing routine responsibilities and SHP activities, as well as their interactions with primary care doctors, auxiliary nurse midwives (ANMs), and women. IDIs will be conducted with women, primary care doctors, ANMs, health officials, women, and field supervisors in order to provide a space for participants to share sensitive details, personal reflections, and a secure environment to discuss their experiences (women). In addition, practical considerations around feasibility determined the mixed use of FGDs and interviews.

Sampling will be guided by thematic saturation, defined as the point at which no new themes or relevant insights emerge from successive interviews or FGDs. Based on the planned study design and practical feasibility, qualitative data collection will be conducted in 2 rounds (midline and endline), with an anticipated total of 32 IDIs and 12 FGDs across both field sites, as detailed in Table 3 (see below, under qualitative data collection plan). This will include 8 IDIs with pregnant and postpartum women, 8 IDIs with primary care doctors, 4 IDIs with ANMs, 8 IDIs with field supervisors, 4 IDIs with policy or health officials, and 12 FGDs with ASHAs. Final sample sizes may vary depending on achievement of thematic saturation.

Participants will be selected from those already involved in the SHP2 trial. Policy makers will be identified with the assistance of the district and state administrative heads. For healthcare workers, all healthcare workers who participated in the SHP study were eligible. For women, all participants of the study are eligible to participate, regardless of whether they have completed all assessments or not. In addition, women who have rejected participation are eligible. Participants who decline participation in the process evaluation are ineligible.

The study team will also identify independent experts from these areas to facilitate qualitative interviews. These interviews and FGDs will be conducted at convenient locations for participants. All sessions will be audio-recorded, with informed consent of the participants.

Informed consent process

Participants will provide consent before qualitative interviews or focus groups by personally signing the Informed Consent Form (ICF) after reviewing the participant information sheet. Both documents, outlining the interview process, participant role, and any potential risks, will be available in written and verbal formats in the local language.

The consent process will be managed by a qualified and experienced member of the research team, who will be authorised by the Chief Investigator. A copy of the signed ICF will be given to the participant. The original signed form will be retained at the study site. Also, where desirable and acceptable to participant groups, a summary of the discussion proceedings will be written and countersigned by all participating in the discussion meeting. The data collected during these interviews and focus groups will be handled in accordance with the data protection policy for the SHP trial. All personal and sensitive information will be kept confidential and stored securely to protect participants’ privacy. The policy will also ensure that participants’ rights are respected, including their ability to withdraw from the study at any point without penalty.

During data collection, all study participants will be allocated a unique study identifier. No personal data (names, addresses, contact details) will be stored in the study database. A password protected database containing personal information needed for participant follow-up will be maintained for use by designated study field investigators only for the duration of the period of participation of an individual in the study. All information collected on participants will be kept confidential and used only for the purposes outlined in the ICF by study staff and clinicians involved in the study.

Data collection

The data collection for this mixed methods study includes both quantitative and qualitative data, collected from a variety of sources such as SHP app analytics, trial data, interviews and FGDs.

Assessment measures and planned forms of measurement has been previously published in the protocol for the SHP main trial.²⁶

The sample size for this process and implementation evaluation is determined by the objectives of the evaluation, the types of data to be collected (quantitative and qualitative), and practical considerations including resources and time. For the quantitative study, we aimed to include either the full sample, and/or from all staff and a representative sample of women. For the qualitative evaluation, we included a smaller, more purposive sample.

Quantitative data collection

Quantitative data will be sourced from the SHP trial database.²⁶ Data will be extracted from the SHP app, which tracks ASHAs’ screening and follow-up activities and the number of primary care doctor visits. At the intervention sites, the ASHA worker will conduct a health assessment of the women during home visits, gather data on pregnancy, childbirth, and treatment (if any) received during or after childbirth, and enter this data in the SHP application. Data will be collected in the SHP app on the consumption of iron and folic acid tablets for improving blood haemoglobin, health examinations performed, haemoglobin levels, and blood pressure readings, as well as any other specific diseases or treatments received during pregnancy and after delivery. This information will help understand how the health status of the women has changed during the project’s duration when compared to women receiving usual care in the control sites. In addition, quantitative data from trial records, district records, surveys, checklists, and costing proforma will be collected, as outlined in the SHP trial protocol paper.²⁶

Qualitative data collection

The process evaluation framework will guide the qualitative data collection. Data collection will be done using pre-/ mid-/ post-implementation questionnaires in two rounds: round 1 (planned April to May 2023 and September to December 2023) and round 2 (planned November 2024 to January 2025), audio analysis of interviews and focus groups, observation of daily routines and interactions, fidelity checklists, journey mapping, and diary/project log analysis.

IDIs and FGDs will be conducted with representatives from beneficiaries (women participating in SHP trial), healthcare providers (ASHAs, ANMs, medical officers, healthcare providers), and field supervisors/staff in each field sites involved in implementing and influencing policy decisions concerning maternal and child health and NCDs in the state. A representative sample of participants from the study sites and clusters will be purposely selected to capture differences in PHC qualities. The questionnaires will be developed using the RE-AIM framework and Proctor et al.’s implementation outcomes.³⁴ Interviews and focus groups will be conducted in Telugu/Hindi, audio-recorded, transcribed verbatim in the original language, and translated into English for analysis. To ensure accuracy and preserve meaning, a subset of transcripts will be independently reviewed against the original recordings, and discrepancies will be resolved through discussion within the research team. Key terms and culturally specific expressions will be discussed among the team and meanings verified with bilingual speakers and the data collectors to ensure conceptual equivalence.

Additional qualitative methods applied will be observation and patient journey mapping of high-risk women to understand how they enter, navigate, experience, and exit the SHP programme. Furthermore, we will explore their experiences within existing private and public healthcare. These methods will capture patient behaviours, feelings, motivations, and attitudes across care episodes.

Data collection was designed according to the objectives of the process evaluation. To understand how and why the SHP intervention may be effective, a relatively larger representation from healthcare providers is planned, as these are directly involved in using and delivering the SHP trial components. To understand for whom the implementation is useful, IDIs with women as the end beneficiaries will be conducted, and to gain in-depth accounts of their experiences. The number of IDIs and FGDs is also guided by feasibility. It is expected that in particular the interviews will provide a rich qualitative understanding and will complement the larger quantitative data of end beneficiaries conducted in the trial.

An iterative approach to data collection and analysis will be used until data saturation (i.e. when no new themes are emerging). Throughout the trial, the research team will regularly organise meetings with stakeholders and debriefing sessions with field supervisors to review emerging findings. If needed, additional interviews will be conducted.

An overview of the overall data collection plan for the SHP process evaluation is shown in Table 4.

Table 4.

Overview of process evaluation data collection plan.

Stakeholders (intervention group)	n IDIs/FGDs in both fieldsites	n IDIs/FGDs in both fieldsites
Stakeholders (intervention group)	Round 1: 12 months	Round 2: endline	Total n IDIs/FGDs
Women
Pregnant and postpartum	4 IDIs	4 IDIs	8 IDIs
Healthcare providers
ASHAs	6 FGDs	6 FGDs	12 FGDs
Primary care doctors	4 IDIs	4 IDIs	8 IDIs
ANMs	4 IDIs	0	4 IDIs
Project field team/managing the supply chain
Field supervisors	4 IDIs	4 IDIs	8 IDIs
Policy officials
Healthcare officials	0	4 IDIs	4 IDIs
Total intervention	16 IDIs and 6 FGDs	16 IDIs and 6 FGDs	32 IDIs and 12FGDs

ASHA: Accredited Social Health Activist; FGD: focus group discussion; IDI: in-depth interview; ANM: auxiliary nurse midwife.

Process data collection

In addition to data gathered from interviews and SHP electronic questionnaire, a range of process data will be gathered through other mechanisms to capture additional context and individual-level information that might influence the study outcomes. Project supervisors and project managers will maintain a diary to gather information about the different activities, events or interventions by the government or other agencies in the study sites that can influence the study outcomes. In addition to this health system level data, the field team will also collect and record individual level information about the participants which is not captured by the SHP application; for example, any special diet taken by the women to improve their blood haemoglobin during, after pregnancy or details of treatment received during pregnancy or postpartum due to maternal or child complication, any socio-economic factor that can influence the general well-being of the women participating in the study.

Mixed methods data collection for process and implementation evaluation

To assess whether the SHP study has been implemented as planned (implementation fidelity, dose)

To assess whether, and to what extent, the study has been implemented as intended, we will assess fidelity and dose (number of visits) of the trial procedures themselves. This will be collected using process data records, including frequency of training, completion of electronic case record files, and activity logs, drawing on SHP app-based data sources described above. Explain whether, how, and why the intervention has been effective (Reach, Effectiveness, Adoption, Acceptability, Implementation, Maintenance).

We will collect effectiveness and implementation outcomes.

Effectiveness outcomes: This will be based on the primary and secondary outcomes of the SHP trial.²⁶

Implementation outcomes: This will be assessed using the RE-AIM framework and Proctor et al.’s typology. Reach will assess the potential reach of the SHP intervention, that is, whether it is used in all eligible participants (proportion of the target population and its representativeness). Effectiveness will assess the impact on health outcomes, including positive, negative, and unintended consequences. Adoption will assess how well and to what extent the intervention has been taken up by the participants of the SHP trial (including how representative this is for the target population in the setting). Implementation will explore the extent to which each of the SHP components were implemented and/or adapted, including an assessment of the costs of delivery. Maintenance will assess the degree to which SHP was sustained at both setting level (in sites) and individual level (among participants). Acceptability will assess to what degree the intervention was acceptable for the participants of the trial (women, ASHAs, primary care doctors). Feasibility will assess to what extent the SHP intervention can be adopted, implemented, and sustained as intended. To assess this, a range of data will be used, including trial data, checklists, surveys, and costing and administrative data, with app-based process data drawn from the SHP platform as described above. Topic guides for qualitative interviews and focus groups will be informed by the process evaluation framework, Proctors’ framework³⁴ and Weiner’s Implementation Measures,⁴³ translated into local languages, and piloted before administration.

Examine how barriers, facilitators, and applied strategies affected the implementation of the intervention

To understand the impact of barriers, facilitators, and applied strategies on the implementation process, qualitative methods (interviews, FGDs and surveys) will be conducted during and after the implementation of the study. The data collection will explore perceptions of barriers and facilitators of the intervention, by women, ASHAs, ANMs, primary care doctors, district officials, and study team. In addition, data from the process evaluation framework on maintenance and sustainability (see above) will feed into the analysis.

Throughout the trial implementation and evaluation process, we will incorporate feedback loops to assess the effectiveness of the strategies, and to improve the quality of the process implementation. Regular quantitative and qualitative monitoring data from field supervisor reports/notes, logistics chain surveillance, and mid-line assessments, together with SHP app-based monitoring data described above will be triangulated and revised. Additional intervention strategies will be defined by the operations group and the delivery team to ensure implementation success.

Understand what and how contextual factors and mechanisms impacted the outcomes

The realist evaluation approach will be used to understand contextual factors and mechanisms that contributed to the implementation and outcomes. As defined by the realist approach, in order to analyse the effectiveness of an intervention, it is essential to understand mechanisms (M) and the contexts in which they operate (C) and how they affect an outcome (O) (C + M = O).³¹ Assumed mechanisms will be tested and refined through both quantitative data and qualitative insights from interviews and FGDs. Stakeholder input will be essential in developing these configurations and refining the programme theory.

Understand what might be required for SHP to be sustained and widely implemented in similar clinical settings (sustainability)

Qualitative interviews and FGDs will be conducted during and after the implementation of the study to better understand what may be required to sustain the intervention. The data collection will explore perceptions of the intervention, and any barriers or opportunities to wider implementation by women, ASHAs, ANMs, primary care doctors, district officials, and study team. In addition, perspectives of decision-makers in government will be consulted to explore financial and technical sustainability and policy implementation, including early insights to integration to existing platforms or other opportunities moving from project to programme which would be fully run and administered by the local authority. In order to understand the extent to which the intervention can be maintained and continued within an ongoing service setting, including its durability, we will administer the Program Sustainability Assessment Tool⁵¹ with women, ASHAs, ANMs, primary care doctors, district officials, and study team after completion of the trial.

Data analysis

The process evaluation will be analysed using a mixed methods approach. The analysis will aim to seek possible explanations for implementation and process outcomes achieved in the SHP trial.

Quantitative and qualitative data will be triangulated and reported together to provide a holistic understanding of the implementation process and outcomes. Data from multiple stakeholders namely women, ASHAs, ANMs, primary care doctors, district officials, and study team will be triangulated to ensure that multiple perspectives are captured. The process evaluation framework (Figure 2) will guide the data analysis for process evaluation data. This framework will ensure that all aspects of the intervention, context, mechanisms, and outcomes are analysed comprehensively.

The quantitative data will be analysed as part of the overall SHP trial statistical analysis plan and will be published online before the SHP database will be locked. Contextual variables captured through process data (e.g. health system factors, implementation intensity, site-level characteristics, and participant socio-demographic factors) will be operationalised as covariates or stratification variables in the quantitative analysis. Where appropriate, multivariable regression models and subgroup analyses will be used to assess the influence of contextual factors on primary and secondary outcomes. Qualitative exploration through interviews and FGDs will inform interpretation of all major process and implementation themes.

Qualitative data analysis will be done in several stages, using a mixed deductive and inductive thematic analysis.⁴⁵ Audio recorded data from interviews and FGDs will be translated and transcribed in English. Thematic analysis of the data will be conducted using Atlas.ti software. Coding will be conducted by two trained qualitative researchers from the study team. An initial subset of transcripts will be independently coded to develop and refine the coding framework. Inter-coder agreement will be discussed and discrepancies resolved through consensus, with iterative refinement of the codebook before coding of the full dataset. Data will be coded using the constructs of the process evaluation framework (Figure 2, above). A combination of deductive and inductive coding will allow for generation of new themes outside these frameworks. Themes will be generated, and subsequently common and distinctive key themes and constructs will be identified based on frequency (e.g. elicited in >60% participants) and expressed relevance of participant groups (patients, healthcare workers, primary care doctors, field staff). The themes and findings will be triangulated with findings from observations, patient journey mapping, surveys, notes from field staff and SHP app data, and other information. Reporting of qualitative data will follow the COREQ guidelines.⁵² To investigate the influence of context and mechanisms on outcomes, the findings from the CMO configurations will be analysed using thematic analysis using Atlas.ti, descriptive statistics, and multivariate analysis of variance to assess differences in mechanisms. Figure 4 outlines the flow of data collection, analysis, and triangulation points.

Figure 4.

Flowchart of data collection, analysis, and triangulation points.

Ethical approval

The SHP trial received ethical approval from OXTREC (reference 27-21) on 24 November 2021, and the George Institute for Global Health ethics committee (reference 22/2021) on 1 December 2021, with version 2.0 approved in November 2022. Version 2.0 included details on the process analysis and the addition of a mid-line study visit in both study arms to facilitate accurate collection of birth outcomes. Written informed consent to participate will be obtained from all participants (PHCs, health workers, and individual women) prior to commencement of study. Women will be free to withdraw their consent at any time during the study without affecting their access to services.

To protect confidentiality in small rural communities where anonymity may be challenging, all data will be de-identified at the point of transcription and analysis, removing personal identifiers and using unique study codes. Access to identifiable data will be restricted to authorised members of the research team, and findings will be reported in aggregate or using anonymised quotations that avoid potentially identifying details (e.g. specific locations or roles).

In the event that incidental findings or medical emergencies are identified during study visits or interactions, participants will be informed accordingly and referred to appropriate local health services following standard clinical and referral protocols. Where required, study staff will notify supervising clinicians or local health authorities to ensure timely follow-up, in line with ethical approvals and participant consent.

Patient and public involvement

In the earlier pilot study, community feedback was collected through FGDs and IDIs. During the main SHP trial, continuous engagement with trial participants and other relevant stakeholders is being maintained. The SHP process evaluation study findings will be shared with the public. Findings will be disseminated through publication in peer-reviewed journals, meetings, digital and social media platforms.

Trial status

The SHP trial’s first patient was recruited in June 2021 and recruitment of patients exceeded the required sample size of 3240 in December 2023. The trial is expected to conclude in April 2025. The SHP process evaluation data collection started in June 2022 with observational visits in the sites, interviews and FGDs started in November 2022, the fidelity checklist has been administered in August 2024. Data collection will be ongoing until the follow-up, and endline data collection of the main trial has been completed. Data analysis is ongoing in iterative stages, with preliminary findings supporting the intervention strategies.

Conclusion

This process evaluation is an integral component of the SHP hybrid type-2 effectiveness-implementation trial and will be conducted concurrently with the main SHP trial. It aims to provide valuable and actionable insights into the implementation outcomes, underlying mechanisms, and contextual factors, as well as the barriers and facilitators influencing the intervention delivery. These insights will inform iterative refinement of the SHP intervention and identify the core components and implementation strategies required for successful scale-up and wider adoption.

In line with MRC guidance on process evaluations,³⁰ this study will contribute empirical evidence on how complex digital and task-sharing interventions can be implemented in low-resource health systems and low- and middle-income country settings.⁵³ The findings will directly inform policy-relevant questions related to feasibility, acceptability, integration within existing primary care and community health worker platforms, and resource requirements for delivery at scale. The findings from this process evaluation will provide valuable evidence for improving long-term health outcomes in perinatal women in rural India. Specifically, the results will provide decision-makers with evidence to guide adaptation of the SHP model within public health systems, inform scale-up frameworks for maternal health interventions, and support implementation planning at district and state levels. By clarifying which contextual and implementation factors are critical for sustainability, this evaluation will strengthen the translation of effective interventions into routine practice and contribute to improving maternal and perinatal health outcomes in rural India and comparable settings.

Supplemental Material

sj-docx-1-whe-10.1177_17455057261425789 – Supplemental material for Protocol for a process and implementation evaluation of the SMARThealth pregnancy hybrid type 2 cluster randomised controlled trial

Supplemental material, sj-docx-1-whe-10.1177_17455057261425789 for Protocol for a process and implementation evaluation of the SMARThealth pregnancy hybrid type 2 cluster randomised controlled trial by Nicole Votruba, Sreya Majumdar, Sudhir Thout Raj, Vaaruni Nayak, Ankita Sharma, David Peiris, Hueiming Liu, Varun Arora, Minakshi Verma, Mohammad Abdul Ameer, Devarsetty Praveen and Jane E. Hirst in Women's Health

Footnotes

Acknowledgements

We would like to thank all participants of this study.

ORCID iDs

Nicole Votruba

Jane E. Hirst

Ethical considerations

The SHP trial received ethical approval from OXTREC (reference 27-21) on 24 November 2021, and the George Institute for Global Health ethics committee (reference 22/2021) on 1 December 2021, with version 2.0 approved in November 2022. Version 2.0 included details on the process analysis and the addition of a mid-line study visit in both study arms to facilitate accurate collection of birth outcomes. Written informed consent to participate will be obtained from all participants (primary health centres, health workers, and individual women) prior to commencement of study. Women will be free to withdraw their consent at any time during the study.

Consent to participate

Not applicable.

Consent for publication

Not applicable.

Author contributions

Nicole Votruba: Conceptualisation; Investigation; Writing – original draft; Methodology; Writing – review & editing; Supervision; Formal analysis.

Sreya Majumdar: Writing – review & editing; Formal analysis; Project administration; Data curation.

Sudhir Thout Raj: Project administration; Formal analysis; Methodology; Writing – review & editing.

Vaaruni Nayak: Formal analysis.

Ankita Sharma: Formal analysis.

David Peiris: Writing – review & editing.

Hueiming Liu: Writing – review & editing.

Varun Arora: Writing – review & editing; Formal analysis; Investigation.

Minakshi Verma: Writing – review & editing; Project administration.

Mohammad Abdul Ameer: Project administration.

Devarsetty Praveen: Writing – review & editing; Conceptualisation; Supervision.

Jane E. Hirst: Writing – review & editing; Supervision; Conceptualisation.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research has been funded by the UKRI Future Leaders Fellowship grant (PI Prof. Hirst).

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data availability statement

Data and materials for the main research will be published along with publication of results.

Trial registration number

clinicaltrials.gov NCT05752955. Date of registration 3 March 2023.

Name and contact information for the trial sponsor

The George Institute for Global Health India, 308–309, Third Floor, Elegance Tower, Plot No. 8, Jasola District Centre, New Delhi 110025, Tel: +91 11 415 880 91-93 Email: info@georgeinstitute.org.in. The study funder has no role in the research design, conduct, or analysis. The study sponsor is responsible for operationalisation of the study.

Supplemental material

Supplemental material for this article is available online.

References

Wang

Abbas

Abbasifard

, et al. Global age-sex-specific fertility, mortality, healthy life expectancy (HALE), and population estimates in 204 countries and territories, 1950–2019: a comprehensive demographic analysis for the Global Burden of Disease Study 2019. Lancet 2020; 396(10258): 1160–1203.

Hirst

Nagraj

Henry

, et al. The obstetrician’s role in preventing cardiometabolic disease. Obstet Gynaecol 2019; 21(4): 247–254.

Behrens

Basit

Melbye

, et al. Risk of post-pregnancy hypertension in women with a history of hypertensive disorders of pregnancy: nationwide cohort study. BMJ 2017; 358: j3078.

Bhavadharini

Anjana

Mahalakshmi

, et al. Glucose tolerance status of Asian Indian women with gestational diabetes at 6weeks to 1 year postpartum (WINGS-7). Diabetes Res Clin Pract 2016; 117: 22–27.

Nguyen-Hoang

Smith

Bergman

, et al. FIGO pregnancy passport: a useful tool for women and their healthcare providers on health risks following pregnancy complications. Int J Gynecol Obstet 2023; 162(3): 787–791.

The Lancet. Non-communicable diseases in reproductive care. Lancet 2024; 404(10448): 95.

Jung

Karwal

McDonald

, et al. Prevention and control of non-communicable diseases in antenatal, intrapartum, and postnatal care: a systematic scoping review of clinical practice guidelines since 2011. BMC Med 2022; 20(1): 305.

Finkelstein

Kurpad

Bose

, et al. Anaemia and iron deficiency in pregnancy and adverse perinatal outcomes in Southern India. Eur J Clin Nutr 2020; 74(1): 112–125.

National Health Mission, Ministry of Health and Family Welfare and Government of India. Anaemia Mukt Bharat, https://nhm.gov.in/index1.php?lang=1&level=3&sublinkid=1448&lid=797 (10 March 2025).

10.

Longchar

Kodali

Hense

Trends and determinants of maternal health services utilization in India from 2015 to 2021. Sci Rep 2025; 15(1): 3711.

11.

Nagraj

Kennedy

Norton

, et al. Cardiometabolic risk factors in pregnancy and implications for long-term health: identifying the research priorities for low-resource settings. Front Cardiovasc Med 2020; 7: 40.

12.

Cockburn

Osborne

Withana

, et al. Clinical decision support systems for maternity care: a systematic review and meta-analysis. EClinicalMedicine 2024; 76: 102822.

13.

Mishra

Parida

Murmu

, et al. Effectiveness of mHealth interventions for monitoring antenatal care among pregnant women in low- and middle-income countries: a systematic review and meta-analysis. Healthcare (Basel) 2023; 11(19): 2635.

14.

Abraham

Melendez-Torres

A realist review of interventions targeting maternal health in low- and middle-income countries. Womens Health 2023; 19: 17455057231205687.

15.

Rahman

Yamaji

Nagamatsu

, et al. Effects of mHealth interventions on improving antenatal care visits and skilled delivery care in low- and middle-income countries: systematic review and meta-analysis. J Med Internet Res 2022; 24(4): e34061.

16.

Yadav

Kant

Kishore

, et al. The impact of mobile health interventions on antenatal and postnatal care utilization in low-and middle-income countries: a meta-analysis. Cureus 2022; 14(1): e21256.

17.

Knop

Nagashima-Hayashi

Lin

, et al. Impact of mHealth interventions on maternal, newborn, and child health from conception to 24 months postpartum in low- and middle-income countries: a systematic review. BMC Med 2024; 22(1): 196.

18.

Praveen

Patel

McMahon

, et al. A multifaceted strategy using mobile technology to assist rural primary healthcare doctors and frontline health workers in cardiovascular disease risk management: protocol for the SMARTHealth India cluster randomised controlled trial. Implement Sci 2013; 8: 137.

19.

Peiris

Praveen

Mogulluru

, et al. SMARThealth India: a stepped-wedge, cluster randomised controlled trial of a community health worker managed mobile health intervention for people assessed at high cardiovascular disease risk in rural India. PLoS One 2019; 14(3): e0213708.

20.

Praveen

Patel

Raghu

, et al. SMARTHealth India: development and field evaluation of a mobile clinical decision support system for cardiovascular diseases in rural India. JMIR Mhealth Uhealth 2014; 2(4): e54.

21.

Patel

Praveen

Maharani

, et al. Association of multifaceted mobile technology-enabled primary care intervention with cardiovascular disease risk management in rural Indonesia. JAMA Cardiol 2019; 4(10): 978–986.

22.

Mukherjee

Daniel

Kallakuri

, et al. Protocol for process evaluation of SMART Mental Health cluster randomised control trial: an intervention for management of common mental disorders in India. BMJ Open 2022; 12(6): e058669.

23.

Daniel

Maulik

Kallakuri

, et al. An integrated community and primary healthcare worker intervention to reduce stigma and improve management of common mental disorders in rural India: protocol for the SMART Mental Health programme. Trials 2021; 22(1): 179.

24.

Nagraj

Kennedy

Jha

, et al. SMARThealth pregnancy: feasibility and acceptability of a complex intervention for high-risk pregnant women in rural India: protocol for a pilot cluster randomised controlled trial. Front Glob Womens Health 2021; 2: 620759.

25.

Nagraj

Kennedy

Jha

, et al. A mobile clinical decision support system for high-risk pregnant women in rural India (SMARThealth Pregnancy): pilot cluster randomized controlled trial. JMIR Form Res 2023; 7: e44362.

26.

Hirst

Votruba

Billot

, et al. A community-based intervention to improve screening, referral, and follow-up of non-communicable diseases and anaemia amongst pregnant and postpartum women in rural India: study protocol for a cluster randomised trial. Trials 2023; 24: 510.

27.

Mann

Shaw

Guthrie

, et al. Protocol for a process evaluation of a cluster randomised controlled trial to improve management of multimorbidity in general practice: the 3D study. BMJ Open 2016; 6(5): e011260.

28.

Schierhout

Praveen

Patel

, et al. Why do strategies to strengthen primary health care succeed in some places and fail in others? Exploring local variation in the effectiveness of a community health worker managed digital health intervention in rural India. BMJ Glob Health 2021; 6(Suppl 5): e005003.

29.

Nilsen

Making sense of implementation theories, models and frameworks. Implement Sci 2015; 10(53): 53.

30.

Skivington

Matthews

Simpson

, et al. A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance. BMJ 2021; 374: n2061.

31.

Pawson

Tilley

Realist Evaluation. London: Sage, http://www.communitymatters.com.au/RE_chapter.pdf (1997, accessed 6 August 2025).

32.

Murray

Treweek

Pope

, et al. Normalisation process theory: a framework for developing, evaluating and implementing complex interventions. BMC Med 2010; 8(1): 63.

33.

Glasgow

Harden

Gaglio

, et al. RE-AIM planning and evaluation framework: adapting to new science and practice with a 20-year review. Front Public Health 2019; 7: 64.

34.

Proctor

Silmere

Raghavan

, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health 2011; 38(2): 65–76.

35.

Goli

Puri

Salve

, et al. Estimates and correlates of district-level maternal mortality ratio in India. PLOS Glob Public Health 2022; 2(7): e0000441.

36.

Government of Telangana. Performance audit on public health infrastructure and management of health services in Telangana state for the year ended March 2022. Government of Telangana, 2024.

37.

Meh

Sharma

Ram

, et al. Trends in maternal mortality in India over two decades in nationally representative surveys. BJOG 2022; 129(4): 550–561.

38.

Government of Harayana. Performance audit on public health infrastructure and management of health services-Government of Haryana (Vol 2). Government of Harayana, 2024.

39.

Pinnock

Barwick

Carpenter

, et al. Standards for Reporting Implementation Studies (StaRI): explanation and elaboration document. BMJ Open 2017; 7(4): e013318.

40.

Moore

Audrey

Barker

, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ 2015; 350: h1258.

41.

Craig

Dieppe

Macintyre

, et al. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 2008; 337: a1655.

42.

Glasgow

Vogt

Boles

SM.

Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health 1999; 89(9): 1322–1327.

43.

Weiner

Lewis

Stanick

, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci 2017; 12(1): 108.

44.

May

Cummings

Girling

, et al. Using Normalization Process Theory in feasibility studies and process evaluations of complex healthcare interventions: a systematic review. Implement Sci 2018; 13(1): 80.

45.

Wong

Westhorp

Manzano

, et al. RAMESES II reporting standards for realist evaluations. BMC Med 2016; 14(1): 96.

46.

Pinnock

Barwick

Carpenter

, et al. Standards for Reporting Implementation Studies (StaRI) statement. BMJ 2017; 356: i6795.

47.

Powell

Waltz

Chinman

, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci 2015; 10(1): 21.

48.

Proctor

Powell

McMillen

JC.

Implementation strategies: recommendations for specifying and reporting. Implement Sci 2013; 8(1): 139.

49.

Viswanath

Synowiec

Agha

Responsive feedback: towards a new paradigm to enhance intervention effectiveness. Gates Open Res 2019; 3: 781.

50.

Pant

Patro

Sedlander

, et al. Monitor to innovate with feedback loops: process evaluation protocol for an anemia prevention intervention. Gates Open Res 2022; 6: 13.

51.

Luke

Calhoun

Robichaux

, et al. The Program Sustainability Assessment Tool: a new instrument for public health Programs. Prev Chronic Dis 2014; 11: E12.

52.

Tong

Sainsbury

Craig

Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care 2007; 19(6): 349–357.

53.

Limbani

Goudge

Joshi

, et al. Process evaluation in the field: global learnings from seven implementation research hypertension projects in low-and middle-income countries. BMC Public Health 2019; 19(1): 953.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.08 MB