Abstract
Our objective was to validate a 5-item compassion measurement tool as a reliable measure of patient assessments of clinician compassion in the pediatric outpatient setting. We completed a cross-sectional study in a U.S. academic healthcare system consisting of six pediatric clinics between February and September 2023. We adapted the original 5-item tool to elicit responses regarding the pediatric patient. The measure was disseminated with the Clinician and Group Consumer Assessment of Healthcare Providers and System (CG-CAHPS) survey. We included patients aged <18 years old who had an outpatient clinic visit and had a parent complete the CG-CAHPS survey. Validity testing of the 5-item tool was performed using confirmatory factor analysis. Internal reliability was tested using Cronbach’s α, and convergent validity with overall provider rating questions from the CG-CAHPS survey was evaluated using Spearman correlation. We analyzed 640 responses. The median (interquartile range) patient age was 8 (3–13) years, and 47% of patients were female. Sixty-one percent of patients were White/Caucasian and 25% were Black/African American. Confirmatory factor analyses found a good fit. The compassion measure demonstrated good internal consistency (α = 0.97) and convergent validity with overall provider rating (rs = 0.63 (95% confidence interval 0.56–0.70)) but reflected a patient experience domain distinctly different from what is currently captured in CG-CAHPS surveys. Our results suggest that the 5-item compassion measure is a valid tool that can reliably and distinctly measure patient assessments of clinician compassion in the pediatric outpatient setting.
Introduction
Compassion is an emotional response to the pain and suffering of another, involving an authentic desire to help. 1 It is largely considered a hallmark aspect of high-quality patient care, associated with reducing patient anxiety and improving patient-reported outcome measures.2,3 Given the positive impacts of clinician compassion, testing perceptions of clinician compassion across healthcare systems has a significant value in improving care.
Pediatric patients, due to their limited cognitive development, are more likely to express their anxieties and fears as behavioral distress, which can impede medical protocols and affect quality of care.4,5 Decreased cognitive development can also make pediatric patients more susceptible to trauma responses while receiving medical care, leading to more negative health outcomes. 4 Testing patient perceptions of clinician compassion can further efforts to promote compassionate care and minimize these harmful sequelae. Parents can also benefit, with evidence suggesting that emotional support can decrease parental distress and help with parents’ abilities to cope. 6
Previously, we developed and validated a 5-item compassion measure for use in conjunction with the Clinician and Group Consumer Assessment of Healthcare Providers and System (CG-CAHPS) survey among adult patients. 7 The CG-CAHPS is a provider rating survey for adult outpatient clinic visits used by the United States Centers for Medicare and Medicaid Services for all healthcare organizations that receive payments from Medicare. The 5-item compassion measure has been validated for use among adult patients in the inpatient and emergency department settings.7–9 However, there is currently no measurement tool to specifically measure the experience of compassion in the outpatient pediatric setting on a large scale.
The objective of this study is to assess the validity and reliability of a 5-item compassion measure in the pediatric outpatient setting. This information can help improve quality of care for pediatric patients and their families.
Significance statement
Compassion in the outpatient pediatric setting has been shown to improve patient outcomes and reduce parental distress, but there is no measurement tool to measure the experience of compassion in the outpatient pediatric setting on a large scale.
With the validation of this 5-item compassion measure as a reliable tool, pediatric outpatient providers and institutions will be better equipped to understand how they provide compassionate care.
The 5-item compassion measure can be used to create efficacious interventions to improve patient outcomes and experiences.
Materials and methods
Setting
This study was conducted across an academic healthcare system and Level 1 Trauma Center in the United States (Cooper University Health Care, Camden, NJ) with six pediatric clinics, including specialty clinics. The study took place from February 1 to September 6, 2023. Given this research involved only survey procedures and information was recorded in such a manner that subjects could not be identified, the Institution Review Board at our institution considered this study exempt from 45 Code of Federal Regulations (CFR) requirements as per regulation 45 CFR 46.104. This study is reported in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology statement. 10
Study population
The study population included patients aged <18 years old who had an outpatient clinic visit and had a parent complete the CG-CAHPS survey. As per our prior methodology of validating the 5-item compassion measure in adults, we matched our inclusion criteria to correspond with the CG-CAHPS survey given a primary objective of this measurement tool is to work in conjunction with the CG-CAHPS. 7 At our institution, the CG-CAHPS is sent to all patients who have an outpatient clinic visit and have a mailing address or email on file. Patients are excluded from receiving a CG-CAHPS if they already received the survey for a visit with the same specialty in the past 90 days. At our institution, we send patient visit data to the survey vendor (Press Ganey Associates, Inc., South Bend, IN, USA) on a daily basis for patients seen the day before. The vendor sends out surveys to patients within 48 h of receiving the data. The vendor collates responses and returns the de-identified survey data to our institution. 7
CG-CAHPS survey
The CG-CAHPS survey used within the pediatrics department at Cooper University Hospital is adapted from the 3.0 version of the CG-CAHPS survey, a tool that has been reliably shown to collect patient reports of their experiences with their healthcare providers. 11
5-Item compassion measure
For the purposes of this study, and for dissemination with the pediatric version of the CG-CAHPS survey, each of the five items of the original measurement tool was modified to specifically elicit responses regarding the pediatric patient and is displayed in Table 1.
Pediatric 5-item compassion measure.
Each item response scaled as 1 = very poor; 2 = poor; 3 = fair; 4 = good; 5 = very good.
Provider refers to physicians or advanced nurse practitioners.
Statistical analysis
Patient survey responses were described using median and interquartile range for continuous variables, and frequency and proportions for categorical variables. Survey responses for each of the five items were tabulated.
Confirmatory factor analysis (using structural equation modeling with maximum likelihood estimator) was used to test how well the 5-item compassion measure assessed a single construct of compassion, and to calculate standardized coefficients for each item. Structural equation modeling tests if the hypothesized model (i.e., single construct of patient experience of compassion) matches the observed data. 8 We examined fit indices (which take into account total sample size), including Comparative Fit Index (CFI), Tucker–Lewis Index (TLI), and standardized root mean squared residual (SRMR). We a priori chose our model to have good fit if CFI >0.95, TLI >0.95, and SRMR <0.08.7–9,12 We chose to examine fit indices because, when the sample size is large, chi-square difference tests for model fit are often significant (i.e., suggesting the model is a poor fit) even when the model is, in practice, a good fit.7,13,14 Similarly, in cases of simple models with a small degree of freedom and/or high factor loadings, the use of Root Mean Square Error of Approximation (RMSEA) is not recommended as it often results in the rejection of correctly specified or closely fitted models. 15
Reliability was tested using Cronbach’s α. Cronbach’s α is an estimate of internal consistency reliability based on the correlations among the items and the number of items in a scale. Given the majority of responses in patient experience data in the United States tend to be favorable (i.e., negatively skewed), 16 and our prior studies found the majority of responses were perfect scores,7,8 we repeated these analyses excluding perfect responses (i.e., response of 5—“very good” for all 5-item compassion items) to ensure the results remained consistent on the lower end of the scale.
We summed the scores of the 5-item compassion measure to obtain a composite score. Using Spearman correlation coefficients (rs), we tested convergent validity between the 5-item compassion measure total score and the CG-CAHPS provider communication questions (provider’s ability to listen, explain, give instructions, show courtesy and respect, and spend enough time with patient) and overall provider rating (i.e., CG-CAHPS question, using any number from 0 to 10, where 0 is the worst provider possible and 10 is the best provider possible, what number would you use to rate this provider?). We used bootstrap sampling with 1000 replications to estimate 95% confidence intervals (CIs) for the Spearman correlation coefficient. We hypothesized that while the 5-item compassion measure would have a positive correlation with the CG-CAHPS constructs (i.e., patients who report higher compassion would also report better communication and overall provider rating), these correlations would not be perfect (i.e., would be distinct and not redundant measures).7–9
To further determine whether the items in the 5-item compassion measure form a discrete construct or whether they simply reflect the clinician’s ability to communicate with the patient, confirmatory factor analysis was used to test a single construct model (i.e., both the 5-item compassion measure and the CG-CAHPS communication questions loading on a single latent variable). We further tested the null hypothesis that the covariance between the two latent structures is 1 (i.e., single construct model). We tested this hypothesis using a likelihood ratio test to compare two nested models: one model with covariance between the two latent models constrained at 1 (i.e., single construct) vs a second model with covariance between the two latent models allowed to be a free parameter (i.e., two-construct model). All the data was exported into Stata/SE 16.1 for Mac, StataCorp LP (College Station, TX, USA) for analysis.
Sample size calculation
There are many approaches to ensuring an adequate sample size for structural equation models and confirmatory factor analysis. 17 Prior Monte Carlo simulation studies have found for a single factor model with five items and factor loadings >0.65, 7 a minimum sample size of ∼100 subjects would be required. 18 Our two-construct model with five indicators per model would require ∼200 subjects. 18 To further ensure an adequate sample size, we used the conservative rule of thumb of 20:1 ratio of responses to free parameter. The primary model estimated in this study contains 10 free parameters: five factor loadings (5-item compassion measure) and five variances of error terms. The variance of the single latent variable was restrained to 1. Using a 20:1 ratio of responses to free parameter, we would require a minimum of 200 responses. 19 The most complex model estimated (two-construct model) contained 21 free parameters: 10 factor loadings (5-item compassion measure and five CG-CAHPS communication questions), 10 variances of error terms, and the covariance of the two latent variables. The variances of the two latent variables were both restrained to 1. For this model, a minimum of 420 responses would be required.
Results
During the study period, 18,444 surveys were delivered and we received 671 responses (4%). Thirty-one returned surveys had a partial response for the 5-item compassion measure and were excluded from analyses, resulting in 640 complete measurement responses. Table 2 displays the patient self-reported characteristics.
Patient and visit characteristics.
Confirmatory factor analysis found the five items loaded well on a single construct (all standardized coefficients >0.90; Table 3). We found our model had excellent fit based on our a priori definition: CFI = 0.99, TLI = 0.98, and SRMR = 0.009. As expected, the chi-square test for model fit was significant, p < 0.001. Internal reliability was also excellent (Cronbach’s α = 0.97). Repeating the analysis excluding perfect scores found good model fit and internal reliability on the lower end of the scale (CFI = 0.98, TLI = 0.95, SRMR = 0.027, and Cronbach’s α = 0.94).
Standardized coefficients from confirmatory factor analysis.
CI, confidence interval.
Table 4 displays the distribution of responses for the 5-item compassion measure, and the CG-CAHPS communication and overall provider rating questions. The composite 5-item compassion measure ranged the full scale (5–25). The composite 5-item compassion measure had a moderate correlation with clinician communication (rs = 0.56 (95% CI 0.47–0.66)) and overall provider rating (rs = 0.63 (95% CI 0.56–0.70)). Using confirmatory factor analysis, we found a single factor model (both the 5-item compassion measure and the CG-CAHPS communication questions loading on a single latent variable) to have poor fit (CFI = 0.87, TLI = 0.83, and SRMR = 0.07). Further, the likelihood ratio test comparing two nested models resulted in p < 0.001. Thus, we reject the null hypothesis that the covariance between the two latent structures is 1 (i.e., the two-factor model has better fit).
Response frequencies for the 5-item compassion measure, as well as the CG-CAHPS survey communication and overall provider rating questions among the validation cohort.
CG-CAHPS, Clinician and Group Consumer Assessment of Healthcare Providers and System.
Discussion
This study provides the initial validation of the 5-item compassion measurement tool as a way to assess the experience of clinician compassion in the pediatric outpatient setting. Using confirmatory factor analysis, we found the pediatric 5-item compassion measure had a good model fit and measured a single construct (i.e., compassion). Cronbach’s α demonstrated good internal consistency. Additionally, this study shows that the 5-item compassion measure distinctly measures the construct of clinician compassion and does not redundantly measure constructs already measured by CG-CAHPS. For example, the 5-item compassion measure only had a moderate correlation with clinician communication and overall provider rating. Further, confirmatory factor analyses found the 5-item compassion measure measured a construct distinct from that of clinician communication.
There is an abundance of literature to suggest that clinician compassion provides a wide range of benefits to patients.3,20,21 Overall, compassionate care is associated with enhanced well-being and a higher quality of care. 20 Psychologically, clinician compassion has been shown to reduce patient suffering, anxiety, and depressive symptoms. 3 It has also been shown to promote positive patient behaviors such as adherence to treatment, engagement, and self-efficacy. 3 Physiologically, clinician compassion has been associated with shorter recovery times, decreased amounts of experienced pain, and improved immune function. 3 These factors can lead to largely improved health outcomes for patients and create a strong precedent for studying compassionate care. With validated tools to measure the construct of clinician compassion, providers and institutions are better equipped to understand how they provide compassionate care and create efficacious interventions that can improve patient outcomes and experiences.
Research also suggests that clinician compassion can provide significant benefit to the providers themselves. It is commonly thought that practicing compassion as a healthcare provider can lead to compassion fatigue and exhaustion. However, more recent literature supports the notion that practicing more compassion does not necessarily correlate with increased compassion fatigue. 22 In fact, studies show that compassion promotes resilience and a positive affect that can actually decrease levels of stress. 23 Thus, compassionate care can be crucial not only for patients but for their providers as well.
Pediatrics is a specialty serving a particularly vulnerable population, which can add complexities to the provision of compassionate care. Children have more difficulties processing information given their cognitive development, which can lead to challenges in communication. 4 Further, social emotional skills are still being developed in childhood, making children especially vulnerable to stressful situations. 24 Many children report anxiety and fear in healthcare settings, which can make it more difficult for pediatric providers to care for their patients’ emotional well-being. 4 Additionally, due to their limited cognitive development, children often react to stress with behavioral responses (i.e., aggression, regression) which can impede patient care. 4 Children are also especially susceptible to traumatic experiences, known as adverse childhood experiences, which can slow the progression of these developmental processes and amplify the intensity of these issues. 25 Therefore, this study addresses a patient population that can uniquely benefit from further research in clinician compassion.
There is also evidence that compassionate care can provide benefits to parents and family. Literature strongly indicates that interventions to support parents’ emotional health can help to benefit children with physical illness, thereby reducing parental stress, anxiety, and depression. 6 Education, social support, and emotional self-care can help mitigate the disruption to family life caused by a child’s physical illness. 6
We recognize that there are limitations to this study that must be taken into account. First, the response rate for our 5-item compassion measure was low, consistent with prior studies utilizing after-care surveys.7,26,27 It is unknown whether other factors (e.g., poor experiences during the visit) impacted survey completion, resulting in non-response bias and potentially decreasing the generalizability of the results and impacting the external validity of the study. Non-responders may have experienced less compassion than responders thus limiting generalizability to patients who experience a low degree of compassion. However, we found our model to have good fit even when those with a perfect score were excluded. Non-response bias could also be introduced due to other systemic differences between survey responders and non-responders, such as varying literacy rates or levels of technology use, thus making the study less generalizable to the general population. However, the goal of this study was to evaluate the psychometric properties of the 5-item compassion measure, not to measure the degree of compassion specifically. Further, healthcare organizations can only estimate their patient experience metrics using data from responders, and the results of the psychometric analyses among those who did respond suggest the 5-item compassion measure is a valid and reliable assessment of compassion. It is also important to note that, while the response rate was low, the actual total sample size was robust (n = 640) for use of maximum likelihood estimator. The low response rate could affect generalizability but, based on our sample size calculation, the total number of responses allowed for good model estimation among those who did respond, as exemplified by the narrow 95% CIs for all five factor loadings. To improve response rates in future studies, surveys could be administered at the point of care so that responders can complete the survey before leaving the visit. Further, follow-up emails and calls can also be made after the visits to ensure survey completion. Second, the 5-item compassion measure was completed by caregivers of the patients, and we did not gather perceptions of clinician compassion from the patients themselves. There are inherent limitations due to young age, but further research can be done to survey adolescent patients and obtain a more direct perception of clinician compassion from the patient perspective. Third, while we aim to assess clinician compassion based on clinician behaviors, it is possible that there are other factors that might impact perceptions of clinician compassion, such as clinician characteristics (i.e., sex, age), visit details (i.e., appointment duration, waiting room duration), and patient characteristics (i.e., acuity of illness). Future studies need to be completed to evaluate these factors and how they might play a significant role in patient perception of clinician compassion. Fourth, while certain underrepresented minorities are well-represented in our study, there are others that are less represented. Survey respondents were 61% White, 25% Black/African American, 6% Asian, 0.3% Hawaiian/other Pacific Islander, and 1% Native American/Alaskan Native. It is also important to note that the 5-item compassion measure was only distributed in English and Spanish. Future studies including a wider range of languages and a greater representation of all underrepresented minorities can help to better identify trends in compassionate care among diverse groups, which is especially important in efforts to ensure equitable care. Fifth, this survey only includes patients and caregivers in the pediatric outpatient setting. Future studies are needed to validate the use of the 5-item compassion measure in the pediatric inpatient setting. Sixth, the original 5-item compassion measure items underwent face and construct validity. 7 We modified the original measure to specifically elicit responses regarding the pediatric patient. The modified version did not undergo additional face or construct validity through cognitive interviews prior to this study. It is possible that different items could perform better in this population. However, the results of this study found excellent model fit in the pediatric population. We chose to use similar modified versions of the original items to allow for future testing of invariance, with the hope that the measures can be used to compare experiences of compassion across populations. Seventh, there was a high correlation among the five items included in the compassion measure, which could suggest redundancy among the five items. However, the high correlation in this study can most likely be attributed to the majority of respondents recording the highest score possible for all five items. Eighth, we were unable to test the reliability of the compassion measure across various providers. There was not an adequate number of survey responses per provider to assess this.
Footnotes
Acknowledgements
The authors would like to acknowledge our patients at Cooper University Hospital.
Declaration of conflicting interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Drs. Trzeciak and Mazzarelli are authors of a book about compassion science, entitled Compassionomics. No other disclosures were reported.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Presentation of data
Data from this study was presented at the 2024 Pediatric Academic Societies Conference (abstract and poster) and at the 2024 Annual Eastern Society for Pediatric Research Meeting (abstract and oral presentation).
