Abstract
Background
Patient- and family-centered care is an approach to the planning, delivery, and evaluation of health care that is grounded in mutually beneficial partnerships among health care providers, patients, and families. It refers to medical care that respects the patient’s values and needs, empowers them and their families in every aspect of their care, is easily navigable, and provides physical comfort and emotional support (1). Patient experience surveys sent after a care episode allow health organizations to measure their ability to provide quality patient-centered care (2,3).
In the United States, the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) is a standardized and validated survey used nationally to measure adult inpatient experience, allowing comparison between multiple hospitals, and influencing reimbursement under the Affordable Care Act (2). The survey contains 19 core questions about critical aspects of patients' experiences and asks for an overall rating of inpatient hospital experience (2). Information about the development, validity, and results from the original domains of HCAHPS are publicly available at www.hcahpsonline.org. The HCAHPS survey has been extensively administered in other countries, using translations available from the HCAHPS or newly created versions (3–7). To our knowledge, no formal comparison has ever been conducted between American and international data. A study conducted in Europe did, however, conclude that the results generated by the HCAHPS survey could be used for international comparison (5), giving greater legitimacy for its international use.
Literature Review
Correlation analyses between domains of patient experience and patients' overall rating of care allow for identifying priority interventions for improving the overall experience of care. The latest correlation analysis done by the Centers for Medicare & Medicaid Services (CMS) demonstrated that, similar to studies performed in China and Canada, communication and respect from nurses and doctors were best associated with high overall patient experience ratings (6,8–10). These results seem to transcend boundaries and types of care, as studies done in Mexico and the USA, within primary care or post-operative surgical care, attained similar results (9,11). The World Health Organization (WHO), through a study measuring the relative importance of non-clinical aspects of care, demonstrated that across regions, population surveys emphasized the importance of prompt attention (access to care), dignity, and communication as part of the medical care they receive (12). There have not been any studies specific to Central America published on this topic yet.
Our organization, Pacífica Salud: Hospital Punta Pacífica, is a private, for-profit hospital in Panama City, Panama. It is a hospital accredited by the Joint Commission International and affiliated with Johns Hopkins Medicine International. It has 76 beds and offers tertiary-care hospital services in adult medicine, surgery, obstetrics, and neonatology. Since its opening in 2006, it achieved ISO certification for its laboratory and radiology departments in 2014 and 2016 respectively, and received the JCI Primary Stroke Center certification in 2020. Patient experience survey data are collected since 2018.
The objectives of our study were to validate the use of the HCAHPS survey and to identify which domains of our patient experience survey have the strongest association with the overall inhospital patient rating of our institution. It also aimed to validate new, local questions added to our survey and to confirm any correlation with the overall hospital rating, to point out additional aspects of patient experience than those measured by the HCAHPS survey that could deserve consideration in our organization.
Methods
Survey
Pacífica Salud: Hospital Punta Pacífica started administering the HCAHPS survey in March 2018. Following the CMS guidelines, we used the standardized and validated Spanish version with 19 core questions (2). While some questions were modified over time to assess specific aspects of our services better, we maintained the questions on Communication from nurses, Communication from doctors, Responsiveness of hospital staff, Cleanliness of the hospital environment, Quietness of the hospital, and Overall patient experience. We added questions to evaluate certain additional characteristics of our organization which we predicted as also important for the patient experience. These questions concerned food service, medical care from hospitalists (a new program at the time), and the administrative processes of admission and discharge. All response options were ordinal, Likert-type choices, except for Overall rating of the hospital and Willingness to recommend, per HCAHPS format (for information about the complete survey, please consult the Supplemental material). Our survey results were solely for internal use.
Survey Collection
The Pacífica Salud Patient Experience survey was sent electronically 3 or 4 days after hospital discharge to all patients aged 18 years and older who had provided an email address at admission. The web tool E-encuestas® (Webtools S.L.) was used. Electronic reminders were sent every 3 to 4 days until survey closure on the tenth day of the following month, or when the overall response rate for the monthly period reached at least 10% of the people surveyed.
Analysis
The characteristics of the study sample were generated using descriptive statistics. Mean and standard deviation calculations were obtained for each question, and Cronbach's alpha tests were performed to validate the results' reliability. We conducted an exploratory factor analysis (EFA) using principal axis factor extraction and varimax rotation with the IBM SPSS software version 27. The EFA allowed us to extract the survey's underlying constructs for comparison with HCAHPS' underlying groupings and to explore whether other latent factors related to our local questions were present (13). Finally, using Pearson correlations, we analyzed each domain with the overall rating of hospital experience.
Results
A total of 1790 surveys were filled by eligible adults during the study period, from March 2018 to December 2019, for an overall response rate of 23.04%. 171 incomplete surveys were excluded from the study data. Prior to the analysis, we excluded the screening-type questions with a binary response option (Yes/No) and the questions modified during the collection period. Our final sample size totalled 1619 surveys of 14 questions. Most respondents were women (76.7%) and were younger than 45 years old (64%), as explained by the hospital's high volume of obstetrics activities. These results were comparable to the hospitalized population during the same period. The sample characteristics are reported in Table 1.
General Characteristics of Respondents.
The Cronbach's alpha confirmed the internal reliability of the overall scale, with an α of 0.760. We used the Kaiser–Meyer–Olkin test (KMO = 0.858) and the Bartlett's test of Sphericity (chi-square = 7450.771) to examine the adequacy of the sample for the exploratory factor analysis (EFA). The EFA suggested proper aggregation of the 14 items into 6 theoretical factors, categorized as domains (see Table 2). The EFA explained 72.39% of the total variance, as seen in Table 3. We analyzed item loading by factor to confirm aggregation into domains similar to what is encountered with the HCAHPS; all communalities were greater than 0.4, the accepted threshold for the proportion of variances shared among a set of items (14). Finally, the Cronbach's alphas for each domain, ranging from α = 0.32 to 0.85, are also displayed in Table 3.
Extracted Domains from the Exploratory Factor Analysis (EFA).
Results of the Exploratory Factor Analysis.
The patient's overall rating of their hospital experience of care showed the strongest correlation with the domains of Communication from nurses (r = 0.670, p < 0.001) and Administrative processes (r = 0.618, p <0.001). Communication from doctors (r = 0.378, p <0.001) and Responsiveness of hospital staff (r = 0.304, p < 0.001) were the domains with the lowest correlation. The correlations between Overall hospital rating and Food service (r = 0.407, p < 0.001) or Hospital environment (r = 0.424, p < 0.001) were of intermediate magnitude. These associations are shown in Table 4
Pearson Correlations between survey domains and the overall inpatient hospital experience rating.
**Correlation is significant at the 0.01 level (2-tailed).
Discussion
To our knowledge, this is the first study on patient experience and the use of HCAHPS surveys in Central America. Although the exploratory factor analyses showed a strong grouping under the same domains as the HCAHPS English version, some questions did not reach the accepted threshold to confirm correlation under the set factor (14). These questions also had low Cronbach's α reliabitlity results. A 2005 study comparing the equivalence between the English and the Spanish versions of the HCAHPS questionnaire also demonstrated a slightly different Cronbach's α reliability measurement with Hospital responsiveness (called nursing services, α = 0.65) and Physical environment (α = 0.44) (15). In our study, these values were even lower. It could be accounted for by the cultural differences in care expectations and language usage, as was encountered in studies by Hurtado et al and Zun et al (15,16).
The Pearson correlation analysis showed that the domain of Communication from nurses had the highest correlation with the overall rating of inpatient hospital experience. This is consistent with similar studies (5,6,9) constantly showing that communication from nurses carries the greatest weight in the overall evaluation of a patient's hospital stay and thus should be prioritized in any work to improve patient experience.
The administrative processes of admission and discharge had the second highest correlation with the overall rating of patient experience. Although these questions are notpart of the formal HCAHPS surveys, our exploratory factor analyses showed sufficient strength in their construct validity to allow us to consider them as elements that merit scrutiny in the evaluation of patient experience in our organization. These results are important and deserve further explanation considering our hospital's discharge procedures. After a doctor's discharge order, the patient cannot leave the hospital before payment of the episode of care or presentation of the bill to an insurance company. All medical expenses must thus be registered in the patient's medical bill before hospital departure. Each medication, item, or service received during the hospitalization is entered manually (because of limitations of our computer system) by the pharmacy, nursing team, procurement department, and physicians. The discharge process ends after manual revision of the bill by a secretary and its presentation for payment. Consequently, there is a clear distinction between the clinical care and the administrative processes (considered non-clinical), so we deemed it important to include this step in our patient experience survey.
Although administrative non-clinical processes, as delineated here, are not included in the CMS's criteria of patient experience, current trends suggest that they should deserve further consideration. The King's Fund (England) has led an extensive study about the patients' opinions of the administrative processes in the National Health Services (NHS) and demonstrated that “people care about their experience in administrative aspects of the NHS services” (17). It also highlighted the possible impacts administrative experiences can have on key factors, such as time and money; emotional wellbeing, and trust and satisfaction with the health services (17). Another study on patient experience for cancer care identified care administration and care coordination as having the strongest correlation with the overall satisfaction of care (18). A recent framework intending to add to the comprehension of the patient experience proposed that it should be viewed through a multifaceted continuum, from the person to the patient, to the user of healthcare services (19). As such, our inclusion of the non-clinical administrative aspects of admission and discharge as an important consideration in the evaluation of the patient experience is well-founded.
It is interesting to note that in our hospital, communication from doctors does not seem to correlate strongly to the overall rating of the hospital experience, as this domain ranked 5th out of 6. This is lower than what has been found in the study by Kemp et al (6), where the communication from doctors' domain was 4th out of 9 domains. We could make the hypothesis that since, in our environment, patients often ask for a specific doctor and seldom end up with the on-call doctor, a relationship, independent and separate from the hospital, was present.
Our study presents limitations that might prevent its reproducibility to other settings. All surveys were sent electronically via email, which does not correspond to the standard way dictated by the CMS (e.g., mail questionnaire or phone interview) to conduct a patient experience survey. However, the high costs of phone interviews and the absence of reliable postal services in Panama contributed to this decision. Various studies have tried to compare traditional forms of surveying (e.g., telephone or paper) to online surveys regarding data quality, response rate, and contestants’ demographics. Results have varied depending on the population, topic surveyed, and study design (20–22). Nevertheless, online surveying is definitively the way of the future. It has been advocated for in a joint report by various patient experience leaders from many hospital associations in the United States (23).
Surveys were sent manually 3 or 4 days after discharge, with an electronic reminder sent every 3 or 4 days until survey closure on the 10th day of the following month or when the response rate reached 10% for the month. The monthly response rate averaged 23.04%, with monthly variation from 15.2% to 69.6%, which falls within a similar range to what was achieved by the American hospitals using the HCAHPS survey during the same period (24). We cannot establish the time interval between hospital discharge and survey completion, nor can we attest that all sent emails were correctly received rather than lost due to incorrect electronic addresses. Our data collection system did not allow us to determine if trends in answers varied based on the time elapsed between discharge and survey completion, nor can we determine if the proportion changed over time. This same protocol was used during the entire survey collection period to ensure a uniform and constant way of collecting data.
The study design might have influenced the data quality for analysis. This retrospective study was conducted using answers from a questionnaire that was modified over time, so we restricted our analysis to answers collected throughout the entire data collection period. Some questions were thus not included in the analyses, notably those about food quality and taste or pain management. In addition, the response options for the questions on administrative processes were formatted as Excellent/Good/Fair/Poor, and this could be argued as relating more to patient satisfaction rather than patient experience, although several studies do not differentiate between both (25). Now that “admission and discharge administrative processes” was confirmed to be of great importance for the patients, further work will be needed to explore which components of these processes could be extracted and transformed into more typical questions of patient experience.
In its original design, the survey did not collect any information regarding many common sampling characteristics (race, ethnicity, socioeconomic status, medical co-morbidities, etc) that could impact survey results (26–28), thus making generalization to other institutions limited.
Conclusion
This study shows that the Spanish translation of the HCAHPS survey is valid for our institution, a private, for-profit hospital in Central America. It confirms that communication from nurses remains the element most correlated with the overall hospital rating. We also have demonstrated that, in our context, the second most important aspect associated with the overall rating of patient experience is the patients' evaluation of the hospital administrative processes at admission and discharge. This component of healthcare services seems to have a significant role in patient experience at our hospital. We suggest that improving the administrative processes throughout the patients' journey might be a valuable addition to enhancing the patient experience.
Supplemental Material
sj-docx-1-jpx-10.1177_23743735221138242 - Supplemental material for Inpatient Experience Surveys and Overall Hospital Rating: A Correlation Analysis for a Private Hospital in Central America
Supplemental material, sj-docx-1-jpx-10.1177_23743735221138242 for Inpatient Experience Surveys and Overall Hospital Rating: A Correlation Analysis for a Private Hospital in Central America by Eve Riopel, Idalina Cubilla-Batista, Emilio Romero-Romero and Fernando Diaz S in Journal of Patient Experience
Footnotes
Acknowledgments
We thank Ms. Sarai Ardila for her help in the management of the survey results database used in this research.
Author’s Note
Eve Riopel is currently a DrPH student at the Johns Hopkins Bloomberg School of Public Health, Baltimore, USA.
Fernando Diaz is currently Chef Medical Officer at The Panama Clinic, Ciudad de Panamá, Panamá.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Ethical Approval
Ethical approval to report this case was obtained from Comité de Bioética en Investigaciones Pacífica Salud (PS-20-Exp1).
Statement of Human and Animal Rights
All procedures in this study were conducted in accordance with the Comité de Bioética en Investigaciones Pacífica Salud approved protocols.
Statement of Informed Consent
Informed consent for patient information to be published in this article was not obtained because this study used retrospective data, already collected by the hospital. No information used within the course of this study was of personal or clinical nature.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
