Abstract
We lack knowledge on how patient-reported experience relates to both quality of care services and visit attendance in the primary care setting. Therefore, in a cross-sectional analysis of 8355 primary care patients from 22 primary care practices, we examined the associations between visit-triggered patient-reported experience measures and both (1) quality of care measures and (2) number of missed primary care appointment (no shows). Our independent variables included both overall patient experience and its subdomains. Our outcomes included the following measures: smoking cessation discussion, diabetes eye examination referral, mammography, colonoscopy screening, current smoking status (nonsmoker vs smoker), diabetes control Hemoglobin A1c (HbA1c [<8]), blood pressure control, cholesterol control Low Density Lipoprotein (LDL) among patients with diabetes (LDL < 100), and visit no shows 2 and 5 years after the index visit that triggered the completed patient-experience survey. We found that patient experience, while an important stand-alone metric of care quality, may not relate to clinical outcomes or process measures in the outpatient setting. However, patient-reported experiences with their primary care provider appear to influence their future visit attendance.
Keywords
Introduction
Over the past decade, there has been increasing emphasis on the use of patient-reported experience data in the performance evaluations of primary care practices and providers (1). Despite this, we lack knowledge on how patient reported experience relates to both quality of care services and utilization, specifically visit attendance in the primary care setting. Prior work has examined the relationship between patient experience and inpatient clinical outcomes with mixed results (1 –7). To our knowledge, there lacks prior studies examining how patient experience relates to ambulatory quality of care metrics and visit attendance. The relationship between patient experiences and outcomes may be different in the outpatient as compared to the inpatient setting, given the vastly distinct environments and quality of care indicators. In addition, how patient experience relates to uptake of elective primary care services, specifically patient appointment attendance, is an understudied area. Therefore, the objective of our study was to evaluate the associations between patient-reported experiences with care, visit attendance, and quality of care process and outcome measures in the outpatient primary care setting.
Methods
Data Sources
In order to estimate the relationships between patient-reported experiences with care and measures of quality and uptake of ambulatory care services, we examined data from the 34-item visit-triggered patient experience survey sent via mail with a prepaid return envelope to all patients after an ambulatory care visit collected between January 2012 and July 2014 from primary care patients at the 22 eligible primary care sites. Sites that were deemed eligible were health system-affiliated sites that provided (patients 18 and older) primary care services to adults (age 18 and older) and had completed the process for National Care Quality Assurance recognition as a patient-centered medical home. (8, 9) Of 27 affiliated health system primary care clinics, 22 met those criteria. These sites were a diverse group of 22 urban and suburban primary care practices in New Jersey and Pennsylvania. For those patients who completed more than 1 visit survey during this process (n = 959), we randomly selected one, for a final analytic sample of n = 8355 unique patients. We also retrieved and analyzed data for all our study sites from Penn Medicine’s Clinical Data Warehouse, on ambulatory quality of care process and outcome measures from the same period, January 2012 to 2014. Lastly, we retrieved and analyzed the number of missed scheduled appointments up to 5 years after the index ambulatory visit that triggered the completed patient experience survey. The institutional review board of the University of Pennsylvania approved this study.
Patient Experience Measures
Our independent variable was a measure of overall patient experience from a 34-item visit-triggered survey, that assessed patient reported experiences with the following subdomains: access, moving through the medical encounter (visit), ancillary staff (nurse, medical assistant), the care provider, patient safety, and privacy (no reported issues with safety or privacy). The access subdomain captures patient perceptions of ability to reach practice personnel via phone, the ease of scheduling appointments, the convenience of the practice’s office hours, the approachability, and courtesy of registration/front desk staff. The visit subdomain evaluates patient experiences with moving through the medical encounter, such as wait times and whether or not patients receive notifications about any delays. The ancillary staff subdomain assesses patient perceptions of care by nurses and/or medical assistants. The care provider subdomain asks patients if their care provider is courteous, provides clear and concise explanations, demonstrates concern for patients’ worries or questions, includes them in treatment decisions, and ensures understanding of medications and treatment plans. In addition, it elicits the patient assessment of the amount of time the provider spent with them and their overall confidence in their care provider. The last subdomain (no reported issues with safety or privacy) evaluates patient perceptions of overall cleanliness, practice staff responsiveness, and adherence to hygiene and safety practices. The survey generates a subdomain score (0-100) from each 5-point Likert-scale question as follows: very poor (score = 0), poor (25), fair (50), good (75), and very good (100). The scores for all questions within each subdomain are averaged to generate a mean score for that subdomain. Then the overall experience with care is calculated from the mean scores from the five subdomains weighted equally (10). Prior psychometric analyses of these subdomains reveal reliability estimates that range from a Cronbach α of 0.81 to 0.97. Additional methodological details of the survey are described elsewhere (11).
Ambulatory Process and Outcome Quality of Care Measures
We selected all quality metrics that were routinely collected and tracked during our study period by our study ambulatory practices for external reporting and/or pay for performance incentives (12, 13). These Center of Medicaid and Medicare outpatient measures of quality were consistent with what other primary care practices nationwide measure and report on. We examined data that captured the following as binary process measures (Yes vs No): smoking cessation discussed among identified tobacco users, diabetes eye examination referral among patients with diabetes, and breast cancer and colorectal cancer screening among eligible patients. We also examined data that captured the following clinical outcomes as binary variables: current smoking status (nonsmoker vs smoker), diabetes control Hemoglobin A1c (HbA1c [<8]), blood pressure control, and cholesterol control Low Density Lipoprotein (LDL) among patients with diabetes (LDL < 100). Table 1 defines the specific metrics we assessed and their respective eligible populations.
Ambulatory Quality of Care Metrics.
No Show Outcome
We used scheduling data from practice electronic medical records to determine this outcome. Using the definition set by the primary care practices in our study for routine monitoring, we classified a no-show appointment as an unattended scheduled visit that occurs without the practice receiving any prior notification from the patient/family to reschedule or cancel. Given our patient experience survey data is visit-triggered, every patient in our study has an index visit between 2012 and 2014, which they attended and afterward completed a survey that captured their visit experience. We measured the number no show visits over 2 distinct time intervals—in the 2 years and 5 years after the index visit that triggered the completed patient-experience survey.
Other covariates
In addition to self-reported patient experience measures, we examined the following patient and practice characteristics: race/ethnicity, age, gender, insurance type, place of residence, clinical comorbidities (Charlson Comorbidity Index [CCI]), practice location, number of providers, and patient volume.
Statistical analysis
We first determined the characteristics of patient in our sample and thereafter computed descriptive statistics for our patient experience measures and outcomes of interest. We then employed generalized estimating equations with an exchangeable correlation structure to account for patient clustering by provider and practice. In multivariate logistic regression models, we estimated the relationship between our patient experience measures and binary ambulatory quality of care metrics, adjusting for the covariates listed above. Using multivariate negative binomial models, we estimated the relationship between our patient experience measures and visit no show counts, adjusting for the covariates above, while accounting for clustering of patients by provider and practice. To account for multiple hypothesis testing, we also generated adjusted
Results
Of 28 201 patients, 8355 (22.8%) completed a survey. Of those 8355 patients in our study, 64% were female, 24% were black, 5.3% had Medicaid, 21.2% Medicare, 5.3% had multiple chronic conditions (CCI > 2). The mean age (SD) was 57 (17.5; Table 2). The mean patient experience scores (SD) for the 5 domains were as follows: 85.8 (16.0) for experience with access, 81.6 (18.4) for visit experience, 88.5 (16.7) for nurse/staff experience, 88.3 (15.0) for patient safety and privacy, and 91.5 (14.8) for care provider experience. The mean (SD) score for overall assessment of patient experience was 87.4 (13.6).
Baseline Characteristics of Patients.
Performance on quality of care metrics varied depending on the measure. About 56% of patients with diabetes received a necessary eye examination referral and about 68% had achieved lipid control (LDL < 100). Table 1 details the process and outcome measures along with the number and percent of patients that achieved them. About 15% of the patients in our sample had not shown up to one or more of their appointments within 2 years of their index visit and close to 27% of the patients had 1 or more no shows within 5 years of their index visit. Two years post index visit, the median no show count, or number of scheduled visits they missed, for patients in our sample was zero, as well as the 25th and 75th percentiles. The maximum appointment no show count was 15. The average the no show count was 0.28 with a standard deviation of 0.88. Five years post index visit, the median no show count as well as the 25th percentile for patients was zero, the 75th percentile was one. The maximum appointment no show count was 29, with a mean (SD) of 0.63 (1.7)
We found no discernable patterns in associations between our patient experience measures and our clinical process and outcome measures (Table 3). However, in adjusted models, we did find a significant association between patient-reported experiences with their care provider and no-show rates 2 and 5 years post index visit. These associations retained significance even after accounting for multiple hypothesis testing. Two years post index visit, holding other variables at their reference level or mean, going from a care provider score at 87.5 (25th percentile) to 100 (median and 75th percentile), the no show count would decrease from 1.6 to 1.4 or by 7% (Figure 1). Similarly, 5 years post index visit, holding other variables at their reference level or mean, going from a care provider score at 87.5 (25th percentile) to 100 (median and 75th percentile), the no show count would decrease from 1.7 to 1.6, or by 6% (Figure 1). Supplemental Appendix A has the complete results of this analysis with adjusted
Adjusted Associations between care provider experience score and no show counts at 2 years and 5 years post index visit.
Patient-Reported Experiences With Care and Ambulatory Process and Clinical Outcome Measures.
**,*
Discussion
We analyzed patient experience data over a 2-year period across 22 primary care practice sites and did not find any significant relationships between patient-reported experiences with care and both patient-level quality of care process and clinical outcomes.
We did find a significant association between patient-reported experiences with their care provider and whether or not they show up to scheduled appointments (appointment no-shows). This association was significant at both time intervals we examined, thus showing a sustained association even 5 years after the visit where patients reported their care provider experiences. Prior evidence has shown appointment no-shows to be an independent predictor of suboptimal primary care outcomes and acute care utilization. (15) Prior estimates reveal that the cost of one no show to a primary care visit ranges from US$125 to US$274 depending on the study and that with a no-show rate akin to the national average of 18%, monthly losses may be upward of US$60 000 (16).
Our findings from outpatient settings did not mirror prior evidence from inpatient settings that demonstrate significant associations between patient-reported experiences with care and process and clinical outcomes (6, 7). Key differences between these settings may explain why inpatient associations between care processes and patient experience do not translate to similar outpatient associations. Patient-reported experience measures from an inpatient encounter are isolated to one discrete care episode that encompasses linear set of care processes within a controlled setting (17). In contrast, primary care outpatient settings are complex adaptive nonlinear systems, where patient-reported visit experience does not necessarily reflect all the inputs, including those out of a health care provider’s control, required to effectively provide preventive care and manage chronic disease (17 –20). Our findings also did not mirror prior evidence from a national study that found an association between higher patient satisfaction and greater inpatient use, higher overall health care and prescription drug expenditures, and increased mortality (5). While there may be a tenuous relationship between patient satisfaction and quality of care, the evidence that satisfied patients are more likely to adhere provider’s recommended treatments is further underscored by our findings of a significant association between patient satisfaction with their providers and visit attendance (21).
Previous studies have highlighted the critical role of providers as drivers of overall patient experiences with care as well as care quality (8, 9, 22, 23). Building on those findings, this study demonstrates that the experience patients have with their providers may also influence subsequent uptake of and adherence to ambulatory services. Miscommunication has been found to be 1 of the 2 key contributors to appointment no shows, the other being forgetfulness (24). Initiatives to mitigate no-show rates to date have centered on appointment reminders with variable success (16, 24). Limited studies have evaluated the role of patient experience in predicting no shows. Goldman et al evaluated this and found that a patient’s visit experience, if negative, strongly predicted future no shows (25). There is also prior evidence that patient overall experience with visit is dictated by their experience with their primary care provider (8, 9). Future work should consider developing and implementing initiatives that improve provider–patient relationships and communications and evaluate the effects on appointment no shows.
The authors acknowledge our study’s limitations. We cannot infer causality, given our cross-sectional design. These findings may lack generalizability, as we conducted this study in one network of practices. However, University of Pennsylvania Health System (UPHS) primary care practices are a diverse group of suburban and urban practices in New Jersey and Pennsylvania. While our response rate was slightly higher than national averages for patient experience surveys (26), this study is likely subject to nonresponse bias (27). Reported-patient experiences from responders to the survey who took the time to complete the survey, likely reflect a more engaged patient population. Given this, we hypothesize that significant relationship we found between patient experience measures and appointment no show counts may underestimate the influence of care experiences on no-show rates in the actual population. Nonresponse bias may also play a role in the lack of significant associations between patient experience scores and quality of care outcomes. Patients that respond to visit-triggered surveys may be more engaged in their health and have better outcomes.
Our study found that patient experience, while an important stand-alone metric of care quality, may not necessarily relate to clinical outcomes or process measures in the outpatient setting. The relationship between patient experience and quality of care in the inpatient setting has been utilized as a key rationale for patient experience measurement. Yet, patient experience measurement in the primary care setting as a stand-alone measure of quality may be the only rationale needed. Prior work supports the need for the patient voice and feedback on organizational processes in our primary care redesign efforts. (8, 9] Moreover, we found that patient experience with their primary care provider did appear to influence no show rates. Practices should make efforts to minimize appointment no shows, which have shown to result in significant financial costs for practices and decreased value of care for patients.
Conclusion
Our findings reveal that practices may benefit from a focus on improving the provider–patient relationship as a potential method to improve visit attendance and that ongoing measurement of patient experience in the ambulatory setting is critical to such efforts.
Supplemental Material
Supplemental Material, Appendix_JPX - The Relationships Between Patient Experience and Quality and Utilization of Primary Care Services
Supplemental Material, Appendix_JPX for The Relationships Between Patient Experience and Quality and Utilization of Primary Care Services by Jaya Aysola, Chang Xu, Hairong Huo and Rachel M Werner in Journal of Patient Experience
Footnotes
Authors’ Note
The Institutional Review Board of the University of Pennsylvania approved this study. An abstract of this study was an oral presentation at the Society of General Internal Medicine on May 14, 2016, in Hollywood, FL, USA.
Acknowledgments
The authors would like to thank Sarah Lamar for her assistance in acquiring the data on missed scheduled visits.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following no financial support for the research, authorship, and/or publication of this article: Jaya Aysola was funded by the grant AHRQ’s PCOR Institutional Award (K12 HS021706-01) for the implementation and analysis of the results of this study. Rachel Werner is funded in part by K24-AG047908 from the National Institute on Aging.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
