Sage Journals: Discover world-class research

Abstract

Patient experience data are used to set performance targets and monitor effectiveness of quality improvement (QI) activities. However, assessing and determining areas that need improvement can be challenging, especially when there are many measures, making it harder to synthesize and identify clear priorities. We describe a QI priority metric and technique for assessing cross-sectional patient experience data that explicitly examine subgroup performance and priorities. The QI priority metric combines 2 patient experience metrics (case-mix-adjusted mean for a patient experience measure and partial correlation of that measure with an overall rating) into a single priority value, allowing leaders to quickly identify improvement areas across multiple measures and patient groups. We examined the priority metric overall and by patient groups (ie, by race, ethnicity, and language preference). We found the priority metric synthesized and identified 2 priority areas for improvement for the entire patient population and revealed several additional improvement priorities specific to patient groups. This metric has the potential to be an informative and self-educating technique for promoting uniformly high-quality care and enhancing performance.

Keywords

quality improvement high-quality care patient experience priority metric

Introduction

Patient experience is a crucial quality measure. Many healthcare organizations measure patient experience using the Consumer Assessment of Healthcare Providers and Systems (CAHPS^®) surveys. The adoption of CAHPS surveys and their use for quality improvement (QI), public reporting, value-based purchasing, and accreditation programs is widespread in the United States. For example, to assess adult inpatient care experiences, hospitals often use the Hospital CAHPS survey. HCAHPS was designed to identify gaps and variations in hospital performance.¹ The Centers for Medicare & Medicaid Services (CMS) initiated the national administration of HCAHPS in 2016. The Child HCAHPS survey was developed in 2015 to assess inpatient pediatric care experiences.^2–4 Although CMS does not require that hospitals administer Child HCAHPS, it has national benchmarks and is often used for QI.^5,6

Patient experience data are often used to set performance targets and to monitor the effectiveness of QI activities. To do this, healthcare leaders conduct regular assessments using CAHPS surveys to identify improvement opportunities and create dashboards to monitor trends.⁷ This process generally consists of measuring patient experience with a CAHPS survey, identifying an area of interest or need for improvement, selecting the relevant measures or metrics to track from a CAHPS survey, incorporating other measures that complement the CAHPS measures, developing a QI initiative with the buy-in and engagement from clinicians and staff, implementing the various phases of QI (such as the steps in the plan-do-study-act cycle), tracking changes over time in the key metrics, adjusting the improvement plan as results emerge, and monitoring whether improvements are sustained over time.⁸ This process aligns with the evidence,^9–13 is delineated in The CAHPS Improvement Guide: Practical Strategies for Improving the Patient Care Experience,⁷ and is recommended by many healthcare and improvement organizations as part of the QI process.

The CAHPS Improvement Guide lays out the specific steps for assessing performance and identifying areas for improvement. As recommended by many healthcare and improvement organizations, the guide suggests a 2-dimensional priority matrix to graphically display (1) relative performance on CAHPS composite measures and (2) the extent to which each measure relates to an overall care rating.¹⁴ Figure 1 shows an example of the priority matrix. Measures in the bottom-right quadrant are likely the highest priorities for improvement. The other quadrants convey similar information about performance in each aspect of care and the relative importance of this area to patients.

Figure 1.

Example priority matrix.

The priority matrix can be challenging to act upon when there are many measures, such as the 16 Child HCAHPS survey measures.^15,16 In addition, ensuring uniformly high-quality care for all can necessitate examining performance for subgroups as well as overall performance.^17,18 A single summary metric combining the 2 dimensions of the priority matrix can help leaders target their QI efforts.

This article describes the construction and use of a QI priority metric and technique for assessing multiple patient experience measures that explicitly examine both overall and patient subgroup performance and priorities to promote uniformly high-quality care, by identifying areas needing prioritized improvement. This metric combined 2 standard patient experience metrics to more easily compare their importance for QI across multiple measures and for patient groups. We illustrate this using Child HCAHPS data.

Method

Setting

We partnered with a medium-to-large children's hospital in the South with a predominantly Hispanic and often Spanish-preferring patient population that administers the Child HCAHPS survey (paper mailing in English and Spanish). The hospital leadership and their QI team were incorporating the 18 Child HCAHPS survey measures into its QI process and were keen to synthesize and identify clear priorities based on the measures in the hospital's cross-sectional data over time. Typically, the QI team reviewed the overall composite performance to determine areas needing improvement, but they were also interested in understanding the underlying patterns of care experiences in terms of quality and importance based on their main patient subgroups.

Data

We used 498 completed Child HCAHPS surveys based on hospital discharge dates from May 2021 to September 2023. These included 158 English-preferring Hispanic, 78 Spanish-preferring Hispanic, 92 non-Hispanic Black, and 170 non-Hispanic White patients. The Child HCAHPS survey includes 10 composite measures, 2 rating measures, and 6 single-item measures.¹⁹ We focused on the overall rating item, 10 composites, and 3 single-item measures (Responsiveness to call button, Keeping you informed about your child's care in the ER, and Privacy when talking with doctors, nurses, and other providers).

Analysis

We developed a QI priority metric and technique to promote uniformly high-quality care that extends the use of the priority matrix (see Figure 1).

The QI priority metric we propose combines these same 2 pieces of information—(1) the room for improvement in the overall mean case-mix adjusted top-box score for a particular measure and (2) the partial Pearson correlation between the measure and the overall rating of the hospital—into a single number where a higher value indicates higher priority. The priority metric = (partial correlation squared * 100) * (1- mean adjusted top-box proportion). The top box indicates the proportion of responses in the highest-response category (eg, “Always” for Never-to-Always scales).¹ The partial correlation is the product-moment Pearson correlation between the residuals of the measure and the residuals of the overall rating of the hospital, both regressed on the case-mix variables. The priority metric ranges from 0 (if the measure had a zero partial correlation with the overall rating or if performance was perfect) to almost 100 (if the measure entirely, uniquely predicted the overall rating or the performance on the measure approached no top-box responses). Multiplication ensures that only measures with both low performance and high importance are prioritized.

We calculated the priority metric for each measure using the overall patient sample to assess overall performance. To understand the priority areas for each patient group, we calculated and examined the metrics for 4 patient groups based on race, ethnicity, and language preference. We calculated priority scores overall and for each patent group. Lastly, we examined the priority metric for each item within the composites for additional insight and QI guidance.

Illustrative Example

In collaboration with the hospital's QI team and examining the sample sizes of completed surveys by patient groups, we determined it was possible to examine 4 main patient populations of interest. We combined the respondent's self-reported preferred language and the child's race and ethnicity, as done previously^20,21 and hereafter called race, ethnic, language groups, to create 4 patient groups: (1) English-preferring non-Hispanic White patients, (2) English-preferring non-Hispanic Black patients, (3) English-preferring Hispanic patients, and (4) Spanish-preferring Hispanic patients. All other racial, ethnic, and language groups represented 15% of the surveys (ie, total N = 85) including non-Hispanic Black patients with unknown or other language preference (n = 10), non-Hispanic White patients with unknown or other language preference (n = 21), Asian patients (n = 6), Alaskan or Native American patients (n = 2), Pacific Islander or Hawaiian Native (n = 1), Multiracial (n = 24), and unknown or refused to report race/ethnicity (n = 21) and were excluded from the analysis because sample sizes were too small or too broadly defined (ie, multiracial) to obtain reliable metrics per patient group for interpretation. We used Fisher exact test or χ² test, as appropriate, to compare respondent and child characteristics across these patient groups.

We computed mean case-mix adjusted top-box scores and partial correlations for each measure, overall and for 4 race, ethnic, and language groups. Case-mix adjustment variables included child's age (continuous, centered), child's general health (Good/Fair/Poor vs Very Good/Excellent), respondent's age (18-34, 35-44, 45-54, and 55+ years), education (no college, some college, 4-year college degree), and respondent's relationship to child (Mother vs other). These simplified categorizations were necessary due to the small sample sizes in some categories. Missing data (0.4%-2.4%) were imputed using the overall mean.

We calculated the priority metric for each measure overall and for each race, ethnic, and language group. When calculating the priority metric for the patient groups, we used race, ethnic, and language group-specific partial correlations with the overall adjusted top-box proportion rather than the race, ethnic, and language group-specific adjusted top-box proportions for better precision because we found no significant differences in adjusted top-box proportions for the measures across the race, ethnic, and language groups (data not shown); group-specific top-box proportions may be preferred when significant differences across groups exist. The mean adjusted top-box proportion is in Table 1, column 1, and the partial correlation is in column 2.

Table 1.

Adjusted Mean Top Box Score, Partial Correlation (PC) of Measure With Overall Rating of the Hospital, and QI Priority Metric (PM) for Child HCAHPS Measures, for Overall Sample and by Race-and-Ethnicity and Language Preference, Sorted by Overall Sample Priority Metric.^a

	Overall sample				English- preferring Hispanic patients		Spanish-preferring Hispanic patients		Non-Hispanic Black patients		Non-Hispanic White patients
Child HCAHPS Composite Measures (Sorted by Overall QI Priority Metric (PM))	Adjusted mean top box proportion	PC	PM	Diff.	PC	PM	PC	PM	PC	PM	PC	PM
Helping your child feel comfortable	0.73	0.42	4.7	-	0.51	6.9	0.30	2.3	0.54	7.7	0.40	4.3
Communication between you and your child's nurses	0.86	0.57	4.5	0.2	0.61	5.2	0.57	4.6	0.53	3.9	0.54	4.1
Keeping you informed about your child's care	0.82	0.44	3.6	0.9+	0.45	3.8	0.54	5.4	0.48	4.2	0.40	2.9
How well doctors communicate with your child ^b	0.79	0.39	3.2	0.4	0.34	2.6	0.51	5.7	0.12	0.3	0.41	3.6
Responsiveness to call button^b,c	0.76	0.36	3.2	0.0	0.38	3.6	0.55	7.3	0.39	3.8	0.28	1.8
How well nurses communicate with your child ^b	0.81	0.40	3.0	0.2	0.40	3.0	0.57	6.2	0.38	2.8	0.35	2.4
Preventing mistakes and helping you report concerns	0.58	0.27	3.0	0.0	0.33	4.6	0.29	3.6	0.25	2.6	0.23	2.2
Preparing you and your child to leave the hospital	0.87	0.46	2.8	0.2	0.54	3.8	0.57	4.2	0.40	2.0	0.42	2.3
Communication between you and your child's doctors	0.88	0.45	2.4	0.4	0.52	3.2	0.41	2.0	0.54	3.4	0.39	1.8
Hospital Environment	0.76	0.29	2.1	0.3	0.29	2.0	0.35	3.0	0.44	4.8	0.18	0.8
Keeping you informed about your child's care in the ER ^b,c	0.88	0.38	1.7	0.4	0.13	0.2	0.30	1.1	0.80	7.5	0.41	2.0
Communication about your child's medicines	0.80	0.29	1.7	0.0	0.31	1.8	0.29	1.6	0.20	0.8	0.32	1.9
Privacy when talking with doctors, nurses, and other providers	0.89	0.37	1.5	0.2	0.53	3.1	0.27	0.8	0.23	0.6	0.34	1.3

Abbreviations: HCAHPS, Hospital Consumer Assessment of Healthcare Providers and Systems; PC, partial correlation; PM, priority metric; QI, quality improvement.

Adjusted indicates case-mix adjusted including controls for child's age (continuous, centered), child's general health (dichotomized as Good/Fair/Poor vs Very Good/Excellent), respondent's age (categorized as 18-34, 35-44, 45-54, and 55 years of age or older), education (classified as HS graduate or fewer years of education, some college, 4-year college degree or more), and relationship to child (dichotomized as Mother vs Not Mother). Partial correlation with the overall hospital rating; the partial correlation is the product-moment Pearson correlation between the residuals of the measure and the residuals of the overall rating of the hospital, both regressed on the case-mix variables. Diff. indicates the difference in the ranked ordered priority metric. + indicates the largest difference in the rank-ordered priority metrics. Bold indicate the QI priority metric is over 7.0, and italics indicate the QI priority metric is over 4.0.

Single Item.

Has a screener item.

Users can select an overall importance cutoff on the metric for prioritizing items based on the distribution of priority scores (ie, looking for “gaps’ in the range of priority scores) and the resources available to focus on multiple measures. Here, we selected a threshold of greater than or equal to 4 as it distinguished the 2 areas most needing improvement in the overall patient population and had the largest difference between the rank-ordered priority metrics (ie, a difference of 0.9; see Table 1, column 4). We then used the same threshold (priority metric > 4) to identify important aspects of the care experience that needed improvement overall and for each patient group (ie, group-specific distribution). Finally, for all composites meeting this threshold, we also calculated the priority metric for the overall sample and for each patient group on each item within the identified composite, using the same approach. This facilitated examination of patterns within a composite that also pointed to areas needing improvement.

All analyses were conducted using R version 4.3.3. Study protocols were approved by our institution's Human Subjects Protection Committee (IRB Assurance No.: FWA00003425; IRB_No.: IRB00000051; Project ID: ID:2022-N0410; Approval Determination date: 2/21/2023). We used the Strengthening the Reporting of Observational Studies in Epidemiology cross-sectional reporting guidelines.²²

Results

Table 2 provides patient characteristics. We found no statistically significant differences between race, ethnic, and language groups, except for respondents’ education (P < .001), which is accounted for in the case-mix adjustment of the CAHPS scores (Table 2).

Table 2.

Child and Respondent Characteristics.^a

Characteristic	Overall N = 498	English-preferring Hispanic patients N = 158	Spanish-preferring Hispanic patients N = 78	Non-Hispanic Black patients N = 92	Non-Hispanic White patients N = 170	P value
Child's age						.50
Mean (SD)	8 (6)	8 (6)	9 (5)	8 (6)	8 (6)
Child's general health						>.90
Excellent or very good	390 (79%)	124 (79%)	61 (80%)	73 (79%)	132 (79%)
Good, fair, or poor	102 (21%)	32 (21%)	15 (20%)	19 (21%)	36 (21%)
Unknown	6	2	2	0	2
Respondent's age						.10
Under 18-34	105 (21%)	38 (24%)	13 (17%)	22 (24%)	32 (19%)
35-44	217 (44%)	74 (47%)	33 (43%)	36 (40%)	74 (44%)
45-54	134 (27%)	40 (25%)	26 (34%)	20 (22%)	48 (28%)
55 or older	37 (7.5%)	5 (3.2%)	4 (5.3%)	12 (13%)	16 (9.4%)
Unknown	5	1	2	2	0
Respondent's highest level of education						<.001
High school graduate or less	61 (13%)	15 (9.6%)	16 (22%)	21 (24%)	9 (5.4%)
Some college	118 (24%)	49 (31%)	19 (26%)	22 (25%)	28 (17%)
4-year college graduate	125 (26%)	38 (24%)	8 (11%)	21 (24%)	58 (35%)
More than 4 years of college	182 (37%)	55 (35%)	31 (42%)	23 (26%)	73 (43%)
Unknown	12	1	4	5	2
Respondent's relationship to child						.30
Mother	412 (83%)	135 (86%)	67 (87%)	72 (78%)	138 (81%)
Not the mother	84 (17%)	22 (14%)	10 (13%)	20 (22%)	32 (19%)
Unknown	2	1	1	0	0

Bold indicates statistically significant distribution across the 4 patient groups based on P < .05.

Table 1 above (in the Illustrative Example section) presents the priority metric and its components for each Child HCAHPS measure overall and for each race, ethnic, and language group. Table 1 is also sorted by the overall priority metric for ease of identifying those measures with the highest overall priority metric and those over the threshold (ie, priority metric ≥ 4), which indicates which areas are priority improvement areas. Notably, the 2 measures with the highest overall priority metric (Helping your child feel comfortable and Communication between you and your child's nurses) were also the same measures with priority metrics over the threshold for 3 of the 4 race, ethnic, and language groups. Importantly, by assessing this metric across race, ethnic, language groups, several additional priorities were revealed, for example, Responsiveness to call button for Spanish-preferring Hispanic patients and Keeping informed about child's care in the ER for Non-Hispanic Black patients.

From Table 1, it is clear that the identified aspects of patient experience prioritized for improvement (ie, measures with priority metric ≥ 4) differed across the patient groups. Spanish-preferring Hispanic patients had the most aspects of patient experience prioritized for improvement; these 6 aspects were: Responsiveness to call button (priority metric = 7.3), How well nurses communicate with child priority metric = 6.2), How well doctors communicate with child (priority metric = 5.7), Keeping you informed about child's care (priority metric = 5.4), Communication between you and child's nurses (priority metric = 4.6), and Preparing you and child to leave hospital (priority metric = 4.2). Four aspects of patient experience were prioritized for improvement for Non-Hispanic Black patients: Helping child feel comfortable (priority metric = 7.7), Keeping you informed about child's care in the ER (priority metric = 7.5), Hospital Environment (priority metric = 4.8), and Keeping you informed about child's care (priority metric = 4.2). English-preferring Hispanic patients had 3 prioritized aspects for improvement, and non-Hispanic White patients had 2, respectively. However, both groups had their highest priority metric for Helping child feel comfortable (priority metric = 6.9, English-preferring Hispanic patients; priority metric = 4.3, non-Hispanic White patients), followed by Communication between you and child's nurses (priority metric = 5.2, English-preferring Hispanic patients; priority metric = 4.1, non-Hispanic White patients). Also, Preventing mistakes and helping you report concerns had a priority metric of 4.6 for English- preferring Hispanic patients.

Table 3 presents the same information as Table 1 for all items within a composite measure for those measures with a priority metric ≥ 4 (ie, those aspects of patient experience that the metric suggests should be prioritized for improvement). Reviewing the items with the composite measures facilitates an investigation and understanding of differences across patient group priorities and nuances related to the specific components that comprise a given aspect of patient experience. For example, the high-priority metrics for Helping your child feel comfortable are from one of its 3 items: Providers asked about things the family knows best about your child. These data show that English-preferring Hispanic patients, non-Hispanic Black patients, and non-Hispanic White patients all prioritized this one aspect as an area of improvement out of the 3 items. On the other hand, when examining the items that comprise Communication between you and child's nurses, the high priority metric for Non-Hispanic White patients is from the item How often did child's nurses listen carefully to you; in contrast, for both English-preferring and Spanish-preferring Hispanic patients, the high priority metrics are from the item How often did child's nurses explain things to you in a way that was easy to understand.

Table 3.

Adjusted Mean Top Box Score, Partial Correlation (PC) of Measure with Overall Rating of the Hospital, and QI Priority Metric (PM) for Most Important Child HCAHPS Composite Measures and Their Items, Overall and by Race-and-Ethnicity and Language Preference, Sorted by Overall Priority Metric.^a

	Overall			English-preferring Hispanic patients		Spanish-preferring Hispanic patients		Non-Hispanic Black patients		Non-Hispanic White patients
Most Important* Child HCAHPS Composite Measures (in Bold) and their Items (in regular text) (Composites Sorted by Overall QI Priority Metric (PM))	Adjusted Top-Box Proportion	PC	PM	PC	PM	PC	PM	PC	PM	PC	PM
Helping your child feel comfortable ^b	0.73	0.42	4.7	0.51	6.9	0.30	2.3	0.54	7.7	0.40	4.3
Things that a family might know best about a child include how child usually acts, what makes child comfortable, and how to calm child's fears. During this hospital stay, did providers ask you about these types of things?	0.55	0.27	3.4	0.29	3.8	0.15	1.0	0.50	11.2	0.30	4.0
Hospitals can have things like toys, books, mobiles, and games for children from newborns to teenagers. During this hospital stay, did the hospital have things available for your child that were right for your child's age?	0.76	0.22	1.2	0.32	2.4	0.17	0.7	0.10	0.3	0.20	0.9
During this hospital stay, how often did providers talk with and act toward your child in a way that was right for your child's age?	0.90	0.47	2.3	0.58	3.4	0.41	1.7	0.59	3.6	0.40	1.7
Communication between you and your child's nurses ^b	0.86	0.57	4.5	0.61	5.2	0.57	4.6	0.53	3.9	0.54	4.1
How often did your child's nurses listen carefully to you?	0.83	0.48	3.9	0.48	3.8	0.46	3.6	0.47	3.6	0.50	4.1
How often did your child's nurses explain things to you in a way that was easy to understand?	0.85	0.49	3.6	0.55	4.4	0.50	3.7	0.46	3.1	0.45	3.1
How often did your child's nurses treat you with courtesy and respect?	0.89	0.49	2.5	0.57	3.4	0.56	3.4	0.32	1.1	0.44	2.0
Keeping you informed about your child's care ^b	0.82	0.44	3.6	0.45	3.8	0.54	5.4	0.48	4.2	0.40	2.9
How often did providers keep you informed about what was being done for your child?	0.85	0.45	2.9	0.48	3.4	0.54	4.3	0.43	2.7	0.41	2.5
How often did providers give you as much information as you wanted about the results of these tests?	0.76	0.40	3.8	0.41	4.1	0.48	5.6	0.53	6.9	0.32	2.5
How well doctors communicate with your child ^b,c	0.79	0.39	3.2	0.34	2.6	0.51	5.7	0.12	0.3	0.41	3.6
How often did your child's doctors listen carefully to your child?	0.86	0.43	2.5	0.38	2.0	0.60	5.0	0.26	0.9	0.41	2.3
How often did your child's doctors explain things in a way that was easy for your child to understand?	0.78	0.32	2.2	0.29	1.8	0.33	2.4	−0.10	0.2	0.39	3.3
How often did your child's doctors encourage your child to ask questions?	0.71	0.26	1.9	0.28	2.3	0.34	3.4	0.06	0.1	0.22	1.4
Responsiveness to call button ^b,c,d	0.76	0.36	3.2	0.38	3.6	0.55	7.3	0.39	3.8	0.28	1.8
How well nurses communicate with your child ^b,c	0.81	0.40	3.0	0.40	3.0	0.57	6.2	0.38	2.8	0.35	2.4
How often did your child's nurses listen carefully to your child?	0.89	0.36	1.4	0.46	2.4	0.39	1.6	0.26	0.7	0.36	1.4
How often did your child's nurses explain things in a way that was easy for your child to understand?	0.84	0.38	2.4	0.28	1.3	0.58	5.4	−0.06	0.1	0.44	3.1
How often did your child's nurses encourage your child to ask questions?	0.69	0.25	1.9	0.27	2.3	0.43	5.6	0.43	5.6	0.12	0.4
Preventing mistakes and helping you report concerns ^b	0.58	0.27	3.0	0.33	4.6	0.29	3.6	0.25	2.6	0.23	2.2
Before giving your child any medicine, how often did providers or other hospital staff check your child's wristband or confirm his or her identity in some other way?	0.90	0.24	0.6	0.29	0.8	0.40	1.7	0.08	0.1	0.22	0.5
During this hospital stay, did providers or other hospital staff tell you how to report if you had any concerns about mistakes in your child's health care?	0.27	0.21	3.2	0.24	4.3	0.15	1.6	0.30	6.5	0.16	1.9
Preparing you and your child to leave the hospital ^b	0.87	0.46	2.8	0.54	3.8	0.57	4.2	0.40	2.0	0.42	2.3
As a reminder, a provider in the hospital can be a doctor, nurse, nurse practitioner, or physician assistant. Before your child left the hospital, did a provider ask you if you had any concerns about whether your child was ready to leave?	0.85	0.34	1.8	0.41	2.6	0.51	4.0	0.19	0.6	0.31	1.4
Before your child left the hospital, did a provider talk with you as much as you wanted about how to care for your child's health after leaving the hospital?	0.92	0.39	1.3	0.42	1.4	0.59	2.8	0.45	1.7	0.30	0.7
Before your child left the hospital, did a provider explain in a way that was easy to understand when your child could return to his or her regular activities?	0.80	0.36	2.5	0.44	3.8	0.33	2.2	0.40	3.1	0.31	1.9
Before your child left the hospital, did a provider explain in a way that was easy to understand what symptoms or health problems to look out for after leaving the hospital?	0.89	0.38	1.6	0.44	2.1	0.50	2.7	0.37	1.4	0.33	1.1
Before your child left the hospital, did you get information in writing about what symptoms or health problems to look out for after your child left the hospital?	0.87	0.27	0.9	0.40	2.0	0.20	0.5	0.06	0.1	0.30	1.2
Hospital environment ^b	0.76	0.29	2.1	0.29	2.0	0.35	3.0	0.44	4.8	0.18	0.8
During this hospital stay, how often were your child's room and bathroom kept clean?	0.80	0.31	2.0	0.26	1.4	0.40	3.3	0.46	4.3	0.24	1.2
During this hospital stay, how often was the area around your child's room quiet at night?	0.71	0.16	0.7	0.19	1.1	0.17	0.8	0.26	2.0	0.05	0.1
Keeping you informed about your child's care in the ER ^b,c,d	0.88	0.38	1.7	0.13	0.2	0.30	1.1	0.80	7.5	0.41	2.0

Abbreviations: HCAHPS, Hospital Consumer Assessment of Healthcare Providers and Systems; PC, partial correlation; QI, quality improvement.

Adjusted indicates case mix adjusted including controls for child's age (continuous, centered), child's general health (dichotomized as Good/Fair/Poor vs Very Good/Excellent), respondent's age (categorized as 18-34, 35-44, 45-54, and 55 years of age or older), education (classified as HS graduate or fewer years of education, some college, 4-year college degree or more), and relationship to child (dichotomized as Mother vs Not Mother). Partial correlation with the overall hospital rating; the partial correlation is the product-moment Pearson correlation between the residuals of the measure and the residuals of the overall rating of the hospital, both regressed on the case-mix variables. Bold indicate the QI priority metric is over 7.0 and italics indicate the QI priority metric is over 4.0. Composite measures are bolded, and the items within each composite are in regular text and listed in rows under the composite.

The composite had a QI priority metric (overall or for a given patient group) ≥ 4.0 (see Table 2).

Has a screener item.

Single item.

Limitations

This work has limitations. A single metric may not capture all patterns in the data, including trends over time. However, by examining the QI metric for each measure and its individual items, both for the overall population and for each of the patient subgroups, the single metric reveals the underlying patterns of performance and importance. Additionally, we developed and utilized this QI priority metric with a single freestanding children's hospital that receives approximately 20 completed CAHPS surveys per month, resulting in an overall sample of 498 completed surveys. Replicating the metric in other settings, such as using ambulatory Clinician and Group CAHPS survey data, or an organization with larger completed patient survey samples, or with other surveys, are warranted. In practice, organizations also need to have resources and staff with expertise to conduct such analyses or be able to negotiate with their patient experience vendors to conduct such group-level and item-level analyses. This illustration documents its value and the specifics of what steps to follow to allow for its broader adoption.

Discussion

The QI priority metric combines 2 standard patient experience metrics, making the patient experience data across multiple measures more easily understandable and actionable. This priority metric synthesized how much a measure is associated with the overall rating and how much room there is for improvement overall and for specific groups of patients. We recommend setting a threshold based on the metric's identification of the top 2 priority improvement areas, depending on the resources available, not necessarily a value of 4, as both resources and the threshold may differ for a given healthcare organization. For example, these values will be higher for lower-performing organizations with more room for improvement.

In this illustrative example, we found that the overall priority metric identified 2 priority areas for improvement, which were also identified as priority improvement areas for 3 of the 4 race, ethnic, and language patient groups. Importantly, assessing this metric across race, ethnic, and language groups revealed several additional improvement priorities expanding the areas to intervene and improve from 2 to 8 measures, namely it added responsiveness to call button, doctor communication with the child/patient, nurse communication with the child/patient, providing information on preventing mistakes, and helping people report concerns, as well as keeping people informed about care (overall and in the ER).

Examining data for multiple subgroups allow for more specific QI efforts, which may better achieve the goal of a good patient experience across patients. Assessing the underlying patterns of items within a composite was also informative. It highlighted nuances and behaviors requiring improvement related to specific components that comprise a particular aspect of the patient experience, such as doctors listening carefully during communication with patients. Furthermore, this allowed for the identification of interventions specific to the areas in need of improvement.

Even with a small sample, the metric identified and distinguished areas of needed improvement across patient groups, allowing it to successfully be used to determine areas of focus for QI activities that incorporate differences across patient groups and thereby promote attention to improvements in care across patient groups.

The results from this illustrative example were communicated to and resonated with the partner hospital's QI team and executive leadership. The review of the metric information overall and by patient group was sorted to easily identify areas with a priority metric greater than or equal to 4 (ie, the results presented in Table 1 and Table 3). This expanded their strategic initiative from focusing narrowly on preventing mistakes and broad communication improvement to a more specific, nuanced and defined initiative that aimed to improve 8 targeted Child HCAHPS measures (ie, those identified by priority metric higher than 4 across the patient groups): 4 measures related to communication, 2 related to being kept informed, 1 related to mistake prevention, and 1 about helping the child feel comfortable. The use of the metric information coupled with the examination of performance overall and by patient group including item-level data resulted in a 2-month long hospital-wide training initiative for clinical and non-clinical staff to improve safety, communication (with emphasis and training on listening, explaining, and informing) with children and their families and to help children feel more comfortable and at ease with care team interactions. The intervention and it's evaluation is described elsewhere.²³ Thus, the QI priority metric refined and clarified the areas of needed improvement, creating a more robust QI initiative. This was achieved by identifying behaviors within the prioritized composites and the areas of prioritized improvement for specific patient groups.

Conclusions

This QI priority metric synthesized performance and importance into a single dimension, making it easier to identify priorities for overall patient improvement and pinpoint areas for improvement in specific patient groups. Synthesizing information across multiple measures and readily identifying areas of needed improvement for all patients and patient subgroups is critical to the planning step of QI. This QI priority metric has the potential to be an informative, self-educating technique for promoting uniformly high-quality care and improving performance. It also assists organizations in achieving their goals and priorities related to patient care. Further use of this metric and technique for performance improvement is warranted. Further research could replicate the use of the QI priority metric in other settings, utilizing other patient experience survey data during these essential steps to identify areas for improvement in the QI process.

Supplemental Material

sj-docx-1-jpx-10.1177_23743735261437482 - Supplemental material for Using a Quality Improvement Priority Metric to Promote Uniformly High-Quality Care: An Illustrative Example Using Child HCAHPS Survey Data

Supplemental material, sj-docx-1-jpx-10.1177_23743735261437482 for Using a Quality Improvement Priority Metric to Promote Uniformly High-Quality Care: An Illustrative Example Using Child HCAHPS Survey Data by Denise D. Quigley, Marc N. Elliott, Mary E. Slaughter and Ron D. Hays in Journal of Patient Experience

Footnotes

Acknowledgments

The authors acknowledge the time and support of the hospital leaders and vendor liaisons who assisted in obtaining the survey data for this study, particularly Maria S. Panayotou, Toni Pollifrone, Amy Jones, Paula Arias, and Pamela Mori for their support and efforts in reviewing and improving Child HCAHPS metrics. The authors also acknowledge the parents and guardians who completed the surveys and provided narrative comments about their pediatric inpatient stays that were analyzed for this study. In addition, the authors recognize Lynn Polite for her administrative support with the manuscript.

Author Contributions

Denise D. Quigley conceptualized and designed the study; gained funding acquisition; led partnership and acquisition of the data; analyzed and interpreted the data; drafted the article; revised the article critically for important intellectual content; and provided final approval. Marc N. Elliot conceptualized the study’s analysis, analyzed and interpreted the data, revised the article critically for important intellectual content, acquired funding, and provided final approval. Mary E. Slaughter led the data analysis, advised on the draft of the article, analyzed and interpreted the data, drafted the article, provided critical input and revisions, and was involved in the final approval. Ron D. Hays acquired funding, provided input and critical revisions to the article regarding important intellectual content, and provided final approval.

Consent to Participate

All participants provided passive informed consent by completing and returning their survey responses.

Consent for Publication

Consent for publication was provided by the participating hospital.

Ethical Considerations

Study protocols were approved by RAND’s Human Subjects Protection Committee (IRB Assurance No.: FWA00003425; IRB_No.: IRB00000051; Project ID:2022-N0410; Approval Determination date: 2/21/2023).

Data Availability Statement

Data sharing is not applicable to this article as the datasets generated and analyzed during the current study are protected under human subjects and not consented to share.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Agency for Healthcare Research and Quality [Grant number U18HS029321].

ORCID iDs

Denise D. Quigley

Ron D. Hays

Supplemental Material

Supplemental material for this article is available online.

References

Giordano

Elliott

Goldstein

Lehrman

Spencer

. Development, implementation, and public reporting of the HCAHPS survey. Med Care Res Rev. 2010;67(1):27-37. doi:10.1177/1077558709341065

Feng

Toomey

Elliott

Zaslavsky

Onorato

Schuster

. Factors associated with family experience in pediatric inpatient care. Pediatrics. 2020;145(3):e20191264. doi:10.1542/peds.2019-1264

National Quality Forum (NQF). About us. National Quality Forum. Accessed September 18, 2020. www.qualityforum.org/story/About_Us.aspx.

Toomey

Zaslavsky

Elliott

, et al. The development of a pediatric inpatient experience of care measure: child HCAHPS. Pediatrics. 2015;136(2):360-9. doi:10.1542/peds.2015-0966

Quigley

Predmore

. What parents have to say: content and actionability of narrative comments from child HCAHPS survey. Hosp Pediatr. 2022;12(2):205-19. doi:10.1542/hpeds.2021-006032

Quigley

Slaughter

Gidengil

Palimaru

Lerner

Hays

. Usefulness of child HCAHPS survey data for improving inpatient pediatric care experiences. Hosp Pediatr. 2021;11(10):e199-209. doi:10.1542/hpeds.2020-004283

Edgman-Levitan

Shaller

McInnes

Joyce

Coltin

Cleary

. The CAHPS Improvement Guide: Practical Strategies for Improving the Patient Care Experience. 2003.

Quigley

Qureshi

Rybowski

, et al. Summary of the 2020 AHRQ research meeting on ‘advancing methods of implementing and evaluating patient experience improvement using Consumer Assessment of Healthcare Providers and Systems (CAHPS(R)) surveys’. Expert Rev Pharmacoecon Outcomes Res. 2022;22(6):883-90. doi:10.1080/14737167.2022.2064848

Berwick

James

Coye

. Connections between quality measurement and improvement. Med Care. 2003;41(1 Suppl):I30-8. doi:10.1097/00005650-200301001-00004

10.

Hempel

Bolshakova

Turner

, et al. Evidence-based quality improvement: a scoping review of the literature. J Gen Intern Med. 2022;37(16):4257-67. doi:10.1007/s11606-022-07602-5

11.

McDonald

Schultz

Chang

. Evaluating the state of quality-improvement science through evidence synthesis: insights from the closing the quality gap series. Perm J. Fall 2013;17(4):52-61. doi:10.7812/TPP/13-010

12.

Portela

Pronovost

Woodcock

Carter

Dixon-Woods

. How to study improvement interventions: a brief overview of possible study types. Postgrad Med J. 2015;91(1076):343-54. doi:10.1136/postgradmedj-2014-003620rep

13.

Weiss

. Methods for studying programs and policies. 1998.

14.

Agency for Healthcare Research and Quality. Section 5: Determining Where To Focus Efforts To Improve Patient Experience (Page 1 of 2). Agency for Healthcare Research and Quality. Accessed January 31, 2025. https://www.ahrq.gov/cahps/quality-improvement/improvement-guide/5-determining-focus/index.html.

15.

Institute for Healthcare Improvement. QI Essentials Toolkit. 2017. https://www.ihi.org/sites/default/files/QIEssentialsToolkit.pdf.

16.

Jazieh

. Quality measures: types, selection, and application in health care quality improvement projects. Glob J Qual Saf Healthc. 2020;3(4):144-6. doi:10.36401/JQSH-20-X6

17.

Agniel

Martino

Burkhart

, et al. Incentivizing excellent care to at-risk groups with a health equity summary score. J Gen Intern Med. 2021;36(7):1847-57. doi:10.1007/s11606-019-05473-x

18.

Beckett

Hambarsoomian

Martino

, et al. Measuring equity in the hospital setting: an HCAHPS application of the health equity summary score. Med Care. 2023;61(1):3-9. doi:10.1097/MLR.0000000000001769

19.

Agency for Healthcare Research and Quality. Patient experience measures from the CAHPS® Child Hospital Survey. 2017. https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance/hospital/about/measures-child-hcahps-909.pdf.

20.

Quigley

Elliott

Hambarsoomian

, et al. Inpatient care experiences differ by preferred language within racial/ethnic groups. Health Serv Res. 2019;54(Suppl 1):263-74. doi:10.1111/1475-6773.13105

21.

Quigley

Elliott

Slaughter

Talamantes

Hays

. Shadow coaching improves patient experience for English-preferring patients but not for Spanish-preferring patients. J Gen Intern Med. 2023;38(11):2494-500. doi:10.1007/s11606-023-08045-2

22.

von Elm

Altman

Egger

, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann Intern Med. 2007;147(8):573-7. doi:10.7326/0003-4819-147-8-200710160-00010

23.

Quigley

Panayotou

Slaughter

. Back-to-basics hospital-wide training improves inpatient pediatric care experiences. Hosp Pediatr.. 2025;16(3):219-29. doi:10.1542/hpeds.2025-008831.