Abstract
Maternal mortality and morbidity do not adequately represent women's healthcare experiences. Traditional clinical outcomes must be complemented with tools that assess patients’ subjective experiences to provide a comprehensive view of care effectiveness. The proposed Antenatal Patient-Reported Experience Measures (PREM) tool, developed with input from women who have experienced maternity care, highlights relevant aspects for pregnant mothers and ensures reliability and cross-cultural validation. A multiphase study conducted in 15 Indian hospitals employed purposive sampling. Phase 1 involved forming an expert group, conducting focus group discussions, establishing a standard operating procedure, and creating the PREM Questionnaire. Phases 2 and 3 included a pilot study with 220 participants and a field study with 518 respondents. A PREM tool with 22 expert-sourced questions achieved an internal consistency value of 0.7205, indicating acceptable reliability. The mean content validity index was 0.998, demonstrating high validity, and the content validity ratio was 0.95. Experts deemed 21 out of 22 questions essential. The PREM tool displayed good reliability and strong content validity. The validated PREM enhances antenatal care outcomes by prioritizing women's perspectives.
Introduction
Patient safety embodies a vital principle of healthcare and is increasingly recognized as a significant and growing global public health concern. 1 It is a framework of organized activities aimed at sustainably reducing risks, minimizing avoidable harm, lowering error likelihood, and mitigating their impact. 1 The World Health Assembly established World Patient Safety Day in 2019. The World Patient Safety Day theme, observed on September 17, 2021, was “Safe Maternal and Newborn Care.” This theme highlights the importance of prioritizing safety in healthcare practices for these vulnerable populations.
The Sustainable Development Goal established by UN Member States in 2015 aims to reduce maternal mortality to below 70 deaths per 100 000 live births by 2030, but this target remains unmet. A lack of focus on unsafe and low-quality healthcare practices significantly hampers progress. Ensuring patient safety is vital for enhancing healthcare quality and achieving universal health coverage, necessitating the involvement of pregnant women, their partners, and families. 2
Maternal mortality and morbidity are considered to be indicators of clinical outcomes that help in improving obstetric care. 3 These indicators do not provide insight into the women's experiences, which is required to evaluate the quality of care provided by the healthcare system. 3 Traditional clinical outcome measures, such as maternal mortality rates, must be complemented with tools that assess patients’ subjective experiences, providing a more holistic view of healthcare effectiveness. Consequently, tools such as Patient-Reported Experience Measures (PREMs) and Patient-Reported Outcome Measures (PROMs) serve as questionnaire-based surveys that can enhance traditional clinical outcome measures, contributing to a more comprehensive understanding of healthcare effectiveness. 4 PROMS focuses on specific outcomes of care, measuring clinical effectiveness. On the other hand, PREMs gather information on women's experiences while receiving care, measuring the care provision process. 5 PREMs are surveys that allow patients to provide feedback on medical interventions. The insights from PREMs help identify adverse incidents and enable healthcare organizations to make necessary improvements. Additionally, they enhance communication among healthcare providers, patients, and families, leading to better patient care. 6
To meet the needs of service users, PREMs should be developed rigorously to align with the preferences of women seeking maternity care. Policies and protocols in maternity units must be based on patient-centric principles. 7
Women's perception of maternal care is subjective and influenced by their interpretation of the care received within maternity services. Notably, a woman's unique needs, preferences, and expectations significantly influence her assessment of maternal care quality. 8 These subjective assessments can serve as indicators of the quality of healthcare. It's essential to recognize the potential gap between women's preferences and those of healthcare professionals. To evaluate how well services meet women's needs, PREMs should be carefully designed to assess the performance of the healthcare system. 7
Most PREMs developed for maternity care have not actively engaged women in determining the relevant aspects of their maternity care experiences. 8 It is imperative that the content of the PREMs administered to women accurately reflects the services provided from their perspective. The PREMs should be relevant, comprehensive, and easily understandable for the women receiving them. It should also encompass critical aspects such as antenatal safety, effective communication, and patient confidentiality. A PREM tool specifically designed for the Indian subcontinent, focusing on antenatal care, is currently lacking in the literature. Other PREMs developed in various languages globally for healthcare settings serving pregnant women do not show adequate content validity for practical use application. 7
Irrelevant inquiries in tools may bias maternity care, hindering service improvement. The lack of an optimal PREM has obstructed its clinical integration. 7 Consequently, the proposal for an Antenatal PREM tool, developed in collaboration with women who have experienced maternity care, seeks to identify relevant, comprehensive, and easily understandable aspects for pregnant mothers, while incorporating test-retest reliability and cross-cultural validation parameters.
Materials and Methods
This multiphase prospective study on questionnaire development and validation was conducted from September 2023 to April 2024 and approved by the Ethics Committee (BU/2024/Ex.11/33/2024). The approval involved representation from all participating centers. Then, ethical approval was obtained from all participating centers.
The study aimed to develop and establish the face and content validity of a PREM tool for assessing antenatal care provided to women and benchmarking practices in the field.
The study was conducted in 15 hospitals across India, selected through purposive sampling, including small facilities (less than 100 beds), medium-sized facilities (100-300 beds), and large facilities (more than 300 beds) from North, South, East, West, and Central India. Participants were antenatal women visiting the antenatal outpatient department or admitted to the antenatal ward between 36 and 42 weeks of gestation, who had at least one visit per trimester and three visits in the same unit. Informed consent was obtained from all participants, ensuring that confidentiality was maintained throughout the study. Participants with learning difficulties, critically ill conditions, and mental health issues were excluded from the study. For participants who encountered problems understanding English, interpreter-mediated assistance was provided. While formal translation was not utilized, the implementation of standardized verbal support effectively minimized variability in comprehension. Trained bilingual research staff employed a pre-tested script to ensure consistency. This protocol was designed to mitigate interpreter bias and promote uniform comprehension.
The study had three phases.
Phase 1: PREMs Tool Design
A team comprising eight subject matter experts, encompassing obstetricians, nurses, quality professionals, and hospital administrators, was assembled from diverse private, government, and teaching hospitals, ensuring nationwide representation. This team participated in two video conference (VC) focus group discussions to formalize the standard operating procedure (SOP) for ensuring the safety of pregnant women during the antenatal period. A senior obstetrician moderated the VC focus group, providing non-directive guidance and ensuring focus on the set topics. After the discussion, the recorded video was analyzed, revealing twenty-two relevant items. A 22-item self-administered questionnaire, accompanied by English instructions, was developed from the identified SOP for patients or their caregivers to complete based on their care experiences. It covered antenatal safety, patient identification, communication, financial awareness, clinical care, and patient privacy, intended for administration during antenatal visits. The sequence of questions and response options was meticulously established. An online survey engaged 30 subject matter experts and seven patients from diverse small and large private, government, and teaching hospitals to provide feedback on the comprehensibility and relevance of each item in the questionnaire, utilizing a binary response format (“yes” or “no”) and offering open-ended comments and suggestions for tool enhancement. Face and content validation procedures were employed to evaluate the pertinence and clarity of the items.
Phase 2: Pilot Study
A pilot study encompassing 15 hospitals was executed with a sample size of 220, adhering to established guidelines that specify this as the upper threshold for sample size. 9 Subsequent evaluation of the pilot study results indicated that no modifications to the questionnaire were required.
Phase 3: Field Study
The field study required at least 160 participants to evaluate the questionnaire's reliability and validity, maintaining a 1:10 item-per-participant ratio. 10 Each hospital needed at least 50 subjects, resulting in a total of 518 participants from 15 hospitals who completed the questionnaire after providing informed consent.
Face validity was evaluated through the expertise of professionals, as outlined in the methodology. The content validity index (CVI) was determined using three distinct measures: item-level (I-CVI), scale-level (S-CVI), and content validity ratio (CVR), each of which assessed the relevance of individual items.9-11
The I-CVI definitively measures the proportion of experts who recognize the relevance. The S-CVI is calculated using two distinct and effective methods, 12 namely S-CVI/universal agreement (UA), the proportion of the items stated as relevant by all the experts (UA by experts) divided by the total number of items and S-CVI/Ave, the sum the I-CVI scores of all items divided by the total number of items (average of the I-CVI scores for all items across all experts). The CVR is determined by calculating the difference between the number of experts who consider an item relevant and half the total number of experts. This difference is then divided by half the total number of experts, expressed mathematically as CVR = (ne − N/2) / (N/2).13,14
Flow chart
Phase 1 (PREMs Tool design)
Phase 2 (Pilot study)
Phase 3 (Field study)
Results
Following extensive discussions in Phase 1, a PREM tool comprising 22 questions was developed, covering the eight domains outlined in the SOP. The average content validity, S-CVI/Avg, is 0.998, indicating a thorough design to cover all constructs. The scale-level CVI (SCV1/UA) is 0.95, indicating excellent validity. The CVR for the 21 questions is 1, indicating that all experts deemed them essential. One question, however, received a CVR of 0, meaning only half of the experts found it necessary (Table 1). The question with CVR of 0 was retained based on qualitative feedback from participants and its perceived importance in stakeholder discussions.
The Relevance Ratings on the Item Scale by 31 Experts—Antenatal Safety Study.
aItem-level content validity index. bScale-level content validity index, average I-CVI value calculation method. cContent Validity Ratio. dUniversal agreement.
CVI, content validity index; CVR, content validity ratio; UA, universal agreement.
The Phase 2 pilot study assessed the necessity of questionnaire modifications. It included 220 patients, 15 from 14 hospitals and 10 from one hospital. Kuder-Richardson Formula 20 (KR-20) Reliability Values of Dichotomous Item Questionnaires were calculated and found to be 0.7205, regarded as acceptable. This value signifies satisfactory reliability and internal consistency, demonstrating that the test items consistently measure the same construct (Table 2).
Kuder-Richardson Formula 20 (KR-20) Reliability Values of Dichotomous Item Questionnaires.
The mean age of participants was 28.5 years (Table S1). In the preliminary assessment, 99.1% of female participants verified their names, and 99.5% were satisfied with privacy during examinations. Hand hygiene was observed in 97.3% of cases, blood pressure was checked in 97.7% of participants, while counseling on diet and exercise was provided in 94.1% and 87.7% of cases, respectively. Adequate information on urgent symptoms was provided to 96.8% of women. Healthcare workers recognized the necessity of ultrasound consent and understanding the PCPNDT Act, with participants reporting 98%-99% compliance from workers. However, initial verification of names was not consistently maintained during lab and ultrasound procedures (Table 3).
Pilot Study: Antenatal (n = 220).
The study revealed that 90.9% of patients discussed HIV testing, while information on the subsequent visit, immunization, and iron tablets was shared with 99.1%, 97.7%, and 96.4%, respectively. Meanwhile, discussions about birth companions and delivery costs received limited attention, with awareness at 80.9% and 75.9%. Although the mode of delivery was noted in 95.9% of cases, concerns about delivery were not adequately addressed. Pain management had the lowest awareness at 78.6%, and only 89.1% of participants received breastfeeding counseling during antenatal visits (Table 3).
In a comprehensive field study, 518 participants from 15 distinct hospitals participated, with an average age of 28.7 years (Table S2). It was ascertained upon broader surveying that 97.5% of participants had their identities and hospital registration numbers duly verified. Furthermore, the study indicated a decrease in privacy during consultations compared to the pilot study, with a rate of 97.1% (Table 4).
Main Study: Antenatal.
Healthcare workers highlighted the importance of hand hygiene and blood pressure checks in antenatal care. However, only 93.1% received nutritional advice, and 81.7% received exercise guidance. Most participants (95.6%) were informed about danger signs, with 97.1% receiving explanations of lab reports and 93.4% discussing the PCPNDT Act. Although 96.7% had their details verified, only 87.5% provided informed consent for HIV testing. Counseling on immunization, iron intake, and doctor consultations was prioritized, achieving rates of 96.1%, 95.8%, and 97.1%, respectively. In contrast, only 84.2% received counseling on birth companions and 81.3% on delivery costs, even though 92.9% knew when to seek admission for delivery. Management of labor pain and education about the birthing process were less prioritized, with only 89% and 75.5% discussing these topics, respectively. Immediate breastfeeding counseling was provided to 86.7% of women (Table 4). It was also noted that there was missing data in the main study concerning questions about HIV, breastfeeding, and birth companions.
Domain-specific scores were calculated by organizing related questions, with mean and standard deviation values reported. Among the eight domains—Patient Identification, Financial Awareness, Patient Privacy, Infection Prevention, Clinical Care, Patient Safety, Patient Rights, and Patient Communication—Patient Communication achieved the highest score. At the same time, Financial Awareness received the lowest score (Table S3).
Discussion
The decisive item content validity, scale content validity, and CVR indicate that the tool effectively measures its intended constructs and has been designed with meticulous attention to detail. Additionally, the KR-20 reliability values demonstrate satisfactory internal consistency.
The overall response rate was 86%. The mean ages of participants in the pilot and primary field studies were 25.5 and 28.7 years, respectively, aligning with comparable studies that assess the quality of antenatal services. 3 A survey by Berehe et al (2017-18) in Ethiopia found that only 45.5% of participants’ names were verified, while blood pressure checks occurred for 84%-90% of women during antenatal visits. Only 23% of healthcare workers practiced hand washing. Approximately 50% of women were counseled on the importance of iron and folic acid intake, and 43% were counseled on the benefits of a healthy diet during pregnancy. Counseling rates for immunization, complication preparedness, and recognizing danger signs were 50%-55%, 43%, and 70%, respectively. 15
A 2019 study in Ethiopia found that 71% of women experienced adequate privacy during care, 82% received counseling on danger signs in pregnancy, and 65% discussed delivery plans. Additionally, 77% were informed about immunization, and 79% were informed about nutrition, while only 50% received information on breastfeeding and HIV testing. Overall, the quality of care from healthcare workers showed a positive trend. 16
Cross-sectional surveys conducted in Kenya, India, and Ghana from 2016 to 2017 revealed inadequate communication between healthcare workers and antenatal women. 17 Various studies consistently reported low-quality antenatal care,18,19 with select studies demonstrating better quality of care ranging from 50%-98%.20,21 Notably, all analyzed antenatal care studies originated from underdeveloped or developing countries. Developed countries prioritize care for pregnant women largely post-delivery and during the postpartum period.22,23
Our research indicates that over 90% compliance with antenatal care standards exists. In the preliminary assessment, an impressive 99.1% of female participants confirmed their names, while the main study reported a verification rate of 97.5%. Satisfaction regarding privacy during examinations was notably high, with 99.5% in the pilot study compared to 97.1% in the main study. However, perceived privacy during consultations declined, dropping to 97.1% in the main study. Healthcare professionals emphasized the importance of hand hygiene and blood pressure monitoring in both studies. Nutritional counseling was provided to 94.1% of participants in the pilot study and to 93.1% in the main study, whereas exercise guidance was offered to 81.7% of the participants. The communication of urgent symptom recognition was effectively conveyed to 96.8% and 95.6% of participants in the pilot and main studies, respectively. Compliance with ultrasound consent and understanding of the PCPNDT Act reached an impressive 98%-99% in the pilot study, but decreased to 93.4% in the main study. Information regarding HIV testing was made available to 90.9% and 87.5% of patients in the respective studies. Information on iron supplements was provided to 96.4%, while 99.1% were informed about the subsequent visit, and 97.7% received information on immunization during the pilot study. In the main study, immunization and iron intake counseling were successfully delivered to 96.1% and 95.8% of participants, respectively. Discussions concerning birth companions were documented at rates of 80.9% in the pilot study and 84.2% in the main study. Knowledge regarding delivery costs increased from 75.9% in the pilot study to 81.3% in the main study. Counseling on breastfeeding was reported at rates of 89.1% in the pilot study and 86.7% in the main study. Awareness of pain management improved significantly, rising from 78.6% in the pilot study to 89% in the main study. Additionally, concerns about the mode of delivery were addressed in 95.9% of cases in the pilot study, but only in 75.5% of cases in the main study.
We observed a divergence between the high rates of identity verification and the lower rates for HIV testing. Notably, healthcare workers have placed less emphasis on pain management, financial considerations, and counseling regarding birth companions. There appears to be a reluctance among healthcare workers to engage in discussions about HIV with pregnant women, which has resulted in data gaps in these critical areas. Factors such as cultural sensitivity and stigma likely contribute to the lack of awareness among women, leading to the absence of relevant data in the primary study. Additionally, operational bottlenecks may have contributed to the missing data regarding breastfeeding and birth companions. Areas identified as underperforming, such as pain management and financial counseling, emerge as critical targets for system-level improvement.
A broader field study involving participants from both the public and private sectors, including hospitals with small, medium, and large patient loads, confirmed the observations. This insight is essential for improving service quality at each hospital. Patient awareness carried the most weight among the domains addressed by the questions, while financial awareness received the least.
This study is notable for its originality in context and scope. Although PREMs tools have been developed for maternity care in various low- and middle-income countries (LMICs), there are few, if any, that have been specifically designed and validated in a stratified Indian hospital setting, incorporating structured input from both patients and healthcare professionals. The integration of public, private, small, and large facilities further enhances its uniqueness. This PREM tool distinctly reflects India's diverse healthcare infrastructure and patient demographics, and it has been validated across both low-resource and high-throughput hospital environments.
This research has the potential to significantly impact the quality assessment of maternal healthcare in India and similar LMICs. By providing a validated tool, healthcare systems can integrate patient perspectives into their quality improvement efforts. Its relevance for policymakers, hospital administrators, and frontline workers enhances its practical application, particularly in identifying and addressing gaps in antenatal care, including communication, counseling, and patient autonomy. Its user-friendly design and domain-specific focus allow for regular quality monitoring.
Strengths: The PREM tool was developed carefully, incorporating expert opinions from various fields. Key areas, including antenatal safety, patient identification, communication, financial awareness, clinical care, and patient privacy, were carefully addressed. A group of doctors and patients evaluated the clarity and relevance of each item. The validation process included a pilot study followed by implementation in routine settings, confirming the tool's strong internal consistency. Its main strength is validation in low-resource settings and in hospitals with varying patient loads. We emphasize the rigorous development and validation of the tool across different hospital settings, as well as its adaptability for low-resource settings, including simplified administration formats and eventual use in community-based care.
Limitations: The tool was developed in English, and assistance was provided to women who encountered language barriers and lacked fluency, which may have resulted in biased observations. The study included hospitals with varied socioeconomic populations, so questions about cost-effectiveness may not apply to low-cost facilities. The feasibility and acceptability of daily practice and routine implementation by healthcare professionals were not assessed. We acknowledge the linguistic constraints and absence of psychometric modeling and the necessity of factor analysis for structural validation.
Recommendations
Conducting surveys in local languages within the community, while ensuring cultural relevance and addressing literacy concerns, can effectively validate the tool (using WHO/ISPOR guidelines) and promote phased implementation and integration into existing quality assurance programs in both the public and private sectors, ultimately aiming to enhance patient care. We propose actionable steps for community validation, institutional adoption, and future formal linguistic validation, specifically highlighting the importance of translating information into major Indian languages and integrating it into routine hospital quality improvement initiatives and community outreach programs. It is suggested that the question with a CVR of 0 be re-evaluated in the planned factor analysis. We plan to conduct exploratory and confirmatory factor analyses in a follow-up study with a larger sample.
Conclusion
The concept of patient experience, a crucial component of patient-centered care, has gained significant importance in evaluating the performance and quality enhancement of healthcare systems. This PREM tool, which has undergone rigorous scientific validation and emphasizes women's participation in enhancing content validity, aims to improve patient outcomes within antenatal care-focused healthcare settings.
Supplemental Material
sj-docx-1-jpx-10.1177_23743735251371769 - Supplemental material for Validity and Reliability of the Newly Developed Patient-Reported Experience Measures Tool for Effectively Assessing the Quality of Care and Patient Safety During Antenatal Services
Supplemental material, sj-docx-1-jpx-10.1177_23743735251371769 for Validity and Reliability of the Newly Developed Patient-Reported Experience Measures Tool for Effectively Assessing the Quality of Care and Patient Safety During Antenatal Services by Anuradha Pichumani, Deepthy Balakrishnan, Lallu Joseph, Nitin Raithatha and Anju Indran in Journal of Patient Experience
Supplemental Material
sj-docx-2-jpx-10.1177_23743735251371769 - Supplemental material for Validity and Reliability of the Newly Developed Patient-Reported Experience Measures Tool for Effectively Assessing the Quality of Care and Patient Safety During Antenatal Services
Supplemental material, sj-docx-2-jpx-10.1177_23743735251371769 for Validity and Reliability of the Newly Developed Patient-Reported Experience Measures Tool for Effectively Assessing the Quality of Care and Patient Safety During Antenatal Services by Anuradha Pichumani, Deepthy Balakrishnan, Lallu Joseph, Nitin Raithatha and Anju Indran in Journal of Patient Experience
Supplemental Material
sj-docx-3-jpx-10.1177_23743735251371769 - Supplemental material for Validity and Reliability of the Newly Developed Patient-Reported Experience Measures Tool for Effectively Assessing the Quality of Care and Patient Safety During Antenatal Services
Supplemental material, sj-docx-3-jpx-10.1177_23743735251371769 for Validity and Reliability of the Newly Developed Patient-Reported Experience Measures Tool for Effectively Assessing the Quality of Care and Patient Safety During Antenatal Services by Anuradha Pichumani, Deepthy Balakrishnan, Lallu Joseph, Nitin Raithatha and Anju Indran in Journal of Patient Experience
Footnotes
Acknowledgements
The authors acknowledge Anusha, Devisree, Dr N. Indira, teams from Aster Mims Hospital, Baby Memorial Hospital, Bangalore Baptist Hospital, Bansal Hospital, Civil Hospital, Eras Lucknow Medical College, GBH American Hospital, Kiran Multi Super Speciality Hospital, Nazareth Hospital, Pramukh Swami Medical College, Sir Ganga Ram Hospital, Sree Renga Hospital, Triveni Hospital Pvt. Ltd And Upasani Super Speciality Hospital.
Ethical Statement
Ethical clearance obtained-(Reference number: Pramukhswami Medical College, Karamsad, Gujarat (Reference number: BU/2024/Ex.11/33/2024).
Informed Consent
Informed consent was taken from all the participants.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
