Abstract
There is a current and ongoing shortage of primary care providers that is projected to increase through 2034. Over the last decade, this shortage has resulted in increased reliance on nurse practitioners and physician assistants to fill the gap. The purpose of this study was to investigate Medicaid recipients’ experiences with their respective primary care providers using responses from the Consumer Assessment of Healthcare Providers and Systems database with a focus on nurse practitioners compared to other provider types identified. Responses from 68,761 Medicaid recipients were analyzed. The results showed that Medicaid recipients’ scores for nurse practitioners were comparable to their peer primary care provider types. Areas for improvement were noted. Primary care services provided by nurse practitioners can contribute to the quality goals set by national healthcare organizations for quality and cost-effectiveness of care. This study expands the knowledge about nurse practitioners and other providers in the primary care role.
Introduction
A current and ongoing shortage of primary care providers is projected to increase through 2034 with a possible 48,000 vacancies. 1 This scarcity has resulted in an expansion of the primary care providers with the practice authority to fill the role. Nurse practitioners (NP) are filling this gap in services and are the fastest-growing profession in the country. This growth and expansion in NP preparation have led to concerns over the quality of patient care. For example, the American Medical Association (AMA) 2 claimed NP do not have sufficient knowledge and educational training when compared to physicians. Furthermore, the AMA claims the NP education process lacks high standards and sufficient clinical hours comparable to medical doctor (MD) programs.
The Centers for Medicare and Medicaid Services (CMS), 3 the administrator of Medicare and Medicaid programs, established the goal of delivering optimal health and well-being to United States citizens in a high-quality, safe, equitable, and effective healthcare system. With a Medicare population numbering 68.0 million 4 and a Medicaid population exceeding 72 million, 5 Medicare and Medicaid insurance programs have the largest enrollment in the United States. To care for these patients, there are 1,484,780 total non-institutional Medicare providers. Of these, 221,170 are primary care specialists serving over 72 million CMS patients and families. 6 Given the vast number of patients, families, and workforce personnel, CMS developed a national quality strategy plan aimed to achieve the highest quality outcomes and person-centric approach 7 to meet the established goal of optimal health and well-being.
The Agency for Healthcare Research and Quality (AHRQ) 8 and the CMS work in conjunction toward the same goal by different approaches. The AHRQ emphasizes the assessment of the patients’ experiences along with other factors through data collection and research that measure and demonstrate healthcare quality. Examples of quality indicators include patients following medical recommendations, reaching clinical outcomes, preventing harm, and reducing unnecessary healthcare services.
Evaluating patients’ experiences is critical to improving healthcare outcomes and recognizing patients are the experts in knowing and rating their experiences. To accurately gauge the patient's experience, healthcare institutions must ask patients whether events in a healthcare setting occurred. Consequently, the CMS joined with AHRQ to create and implement a patient experiences survey tool. The survey, known as the Consumer Assessment of Healthcare Providers and Systems (CAHPS) was the first national method of collecting data to measure patient experiences about the healthcare received 9 by Medicare and Medicaid recipients. An important aspect of healthcare identified in the CAHPS survey is the primary care provider type. Recognizing similarities and differences in the patients’ experiences with these providers can further inform these professional roles and guide decisions to meet the goal of quality healthcare.
The AHRQ 9 stated that positive patient experiences are correlated with financial indicators as well as improved clinical outcomes. Evidence cited that positive financial implications of patient experiences were related to improved patient retention, preventive disease management, enhanced communications, and adherence to treatment plans. Higher patient experience ratings were associated with lower medical risk, greater employee satisfaction and turnover, and increased patient loyalty to the institution. However, the evidence cited on the patient experiences, provider type, and health care outcomes was extremely limited in the literature, especially comparing provider types such as NP.
Included in the evidence surrounding the subject is a systematic review by Doyle et al 10 that analyzed 55 studies finding positive associations between patient experience, patient safety, and clinical effectiveness across various diseases and outcome measures. These outcomes included preventive care interventions such as immunization and screening, length of hospital stay, and primary-care visits. The authors found more studies with positive associations (206) between patient experiences, patient safety, and clinical effectiveness compared to 72 studies with no associations with the above variables. A limitation of the systematic review was that the provider type was not identified.
The evidence about NP effectiveness as a provider is strong. Htay and Whitehead 11 conducted a systematic review of 13 randomized controlled trial studies focused on the NP clinical effectiveness compared to physician-led usual care. The findings were NP provided cost-effective care that was considered efficient with general patient satisfaction. Specifically, there was improvement in patient physical function, increased patient perception of their health status, blood pressure control, reduced waiting time, and associated cost with care provided by NP.
In another study, Liu et al 12 analyzed Veterans Affairs outpatient and inpatient clinical data by provider type. Measures such as utilization cost (outpatient, inpatient, and total), and clinical outcome (control of hemoglobin A1c, Low Density Lipoproteins, and blood pressure) differences were found to be negligible between patients receiving care from NP and MD.
Considering the limited evidence comparing outcomes associated with provider type this study seeks to add to the body of knowledge on the subject. The purpose of this study was to investigate Medicaid recipients’ experiences with their respective primary care providers using responses from the CAHPS database with a focus on NP compared to other provider types identified.
Methods
Design
The study was quantitative and descriptive using the secondary data source CAHPS survey results. The University Institutional Review Board approved the study with exempt status under protocol number 230547186. The CAHPS data were deidentified prior to receipt from the managing digital healthcare research division of AHRQ. 13
Instrument
Medicaid recipient responses from the CAHPS survey were used for the investigation. The primary purpose of the CAHPS database is to “boost our scientific understanding of patient experience with healthcare as part of a larger effort to advance the delivery of safe, patient-centered care.” 8 The CAHPS survey databases are funded by the US AHRQ and administered by Westat. The scientists requested the most recent responses available from the CAHPS survey. The dataset provided by Westat included responses from the last 6 months in 2019. Data were not consistently collected or analyzed during the pandemic years of 2020-2023. Data collection and reporting were due to resume sometime after 2023 or 2024.
The CAHPS survey contains an item that identifies the provider type. The survey also included several other items that would describe the patient's experience in more detail. Patients’ experience specific to provider interactions was found to be reflected in 5 survey items. These items were compared across the categories of providers identified by the survey. The variables of interest were overall provider rating and specific survey item responses that would measure important aspects of the patient's experiences with their respective provider.
The survey used categorical, ordinal (yes/no; never/sometimes/usually/always; not at all/a little/some/a lot) and a Likert-type scale (0 = lowest rating of provider to 10 = highest rating of provider) to measure the overall providers’ care. The items selected to reflect patient experiences with care included: (a) provider explained things clearly (n = 62,777), (b) provider listened carefully (n = 62,713), (c) provider knew important information about your medical history (n = 62,138), (d) provider showed respect (n = 62,470), (e) provider spent enough time (n = 62,183), and (f) provider talked about prescription medicines at each visit (n = 46,629). These items were measured using a 4-point Likert-type scale (1 = never, 2 = sometimes, 3 = usually, 4 = always). Provider type was assigned a numbered category (1 = medical doctor, 2 = doctor of osteopathic medicine, 4 = physician assistant, 5 = nurse practitioner) for group comparisons. Only those surveys with a response to the survey item for provider type were included in the analysis.
Data Analysis
AHRQ provided a file containing the requested data in an SPSS-supported format using a secure link. IBM© SPSS version 28.0.0.0 was used to conduct all statistical procedures. The dataset contained missing responses to provider type (n = 3269) from the total N = 68,761 survey responses received. Measures of central tendency were calculated to describe the sample.
Observation of the differences in sample sizes and consistently high overall patient ratings of their providers indicated violations of the assumptions of normality. Levene's statistics (P = .001) were calculated confirming the violation of assumption for homogeneity of variances within the overall patient ratings of providers by type. The violations of assumptions associated with the data concluded that the robust Welch's analysis of variance (ANOVA) procedures were appropriate for investigating differences between the variables. To compensate for the violation in assumptions the post hoc comparisons were completed using the Games-Howell and Dunnett C procedures. The results were consistent across statistical procedures and approaches. The results of the Games-Howell test were reported.
Results
The aggregate dataset contained responses from 68,761 Medicaid recipients from July 1 through December 31, 2019, from regions in the United States. The surveys with missing provider type information were removed from the study resulting in 65,492 remaining for the final analysis. Surveys were completed by mail (38%), web/internet (24.6%), interview (22.3%), or phone (15.2%). Ninety-six percent of surveys were presented in English with 3.4% presented in Spanish and 0.9% selecting the “other/NA” category.
Patients answered demographic items for age, race, gender, ethnicity, educational level, and region of the country. See Table 1 for the demographics of the sample. AHRQ provided no rationale for the absence of surveys or the disproportionately small number of surveys from the Southeast and Northeast. Considering the 6 months of responses included in this dataset, the date for distribution of surveys may have fallen earlier in the year. Another consideration is the voluntary nature of survey completion and possibly inconsistent instructions for survey completion.
Demographic Characteristics of the Survey Participants.
The independent variable of interest was provider type. The largest percentage of provider type in the sample was identified as MDs (90.4%; n = 62,126), followed by doctors of osteopathy (DO; 2.9%; n = 2022), NP (1.3%; n = 909) and physician assistants (PA: 0.6%; n = 435). Respondents reported the rated visit with a “family practice” specialist for 23% of the surveys. “Internal medicine” visits accounted for 17.9% of the surveys and the remainder of visits were across 15 specialties. Practice ownership and affiliation were reported as primarily “provider/physician” at 66% and 25.6% “university/academic med center.” Only 4.4% of the affiliations were “hospital/health system.” The remaining affiliations were “county or other” at 4%.
In the 60,004 responses to the survey item rating the provider, over 51% of all provider categories received a rating of 10 (0 = lowest rating, 10 = highest rating). Over 87% of all providers rated 8 or above. Spearman Rho correlation coefficients were calculated between the overall patient rating and the demographic variables. Significant positive linear relationships were found between age, gender, and practice ownership affiliation (P < .001, α = .01). These relationships could affect the overall results and should be considered when interpreting the findings.
The ANOVA statistics showed a significant difference between the categories of providers when comparing overall ratings (df = 3, F = 33.32, P < .001). However, Levene's test (df = 3, F = 67.57, P < .001) for homogeneity of variance revealed that equal variances cannot be assumed consistent with the observed sample sizes for each category of provider type. To correct for the unequal sample sizes and variances, the more robust Welch's ANOVA test was conducted with similar findings (df = 3, F = 95.31, P < .001). Significant differences were found between group mean scores for MD and DO, PA and NP at the set alpha level of .05 at a 95% confidence level. In the post hoc analysis, the Games-Howell procedure was selected to discover the differences between the variables. These results were confirmed using the Dunnett C procedure to correct for the unequal sample sizes and variances. Table 2 reports the Games-Howell statistics indicating significant differences in overall mean scores by provider type.
Significant Games-Howell Results Comparing Patient Rating by Provider Type 0-10 Rating.
Abbreviations: CI, confidence interval; MD, medical doctors; DO, doctors of osteopathy; NP, nurse practitioners; PA, physician assistants.
Significant at .05 level.
Further analysis was conducted comparing provider type on each of the 5 survey items selected for rating-specific interactions. Each of the survey items included referred to care provided by “This provider…” using a rating scale from 1 to 4 (1 = never, 2 = sometimes, 3 = usually, 4 = always). Again, Welch's ANOVA and Games-Howell statistics were calculated. Overall, DO provider ratings were consistently higher than their peers on the selected survey items for this study. Table 3 shows mean differences and levels of significance in the comparisons for specific survey items by provider type. These results should be carefully considered due to the violation of normality associated with the data which could affect the validity of the results. Also note that the linear relationships between age, gender, and practice affiliation could impact reliability and validity of the findings.
Games Howell Post Hoc Results Comparing Ratings by Provider Type for Survey Items 1-4 Rating.
Abbreviations: CI, confidence interval; MD, medical doctors; DO, doctors of osteopathy; NP, nurse practitioners; PA, physician assistants.
Significant at .05 level.
Discussion
This pioneering study investigated patient experiences with primary care providers using a large US national data source. The results highlight that Medicaid patients consistently rated their experiences with their respective primary care providers (MD, DO, PA, and NP) with similar high scores. DOs (µ = 9.4) rated highest among the 4 categories of providers in the aggregate. Notably, our findings indicated the overall mean rating scores across all 5 survey items used to investigate patient experiences with providers, found that NP ratings (µ = 9.1) compared favorably to MD (µ = 8.8) and PA (µ = 8.9) peers with no significant differences found.
NP scores were high and rated comparably in most survey items, but their DO colleague's ratings were higher for the provider-specific items. These findings signal an opportunity for NP practice to evolve and emphasize the importance of ongoing quality improvement initiatives.
Our research findings echo other authors’ results on patients’ experiences of NP. A 2019 study reported on a sizable sample of over 53,000 Medicaid patients surveying their satisfaction levels. The researchers found that NP received commendable overall provider ratings, akin to those of MDs, PAs, and DOs. Notably, patients receiving care from NP reported higher overall ratings and communication scores than patients with MD providers. 14 Patient experiences were investigated in primary health clinics located in one Canadian providence. Survey responses were collected from approximately 2300 patients. Findings showed that patients consistently rated their NP providers highly across all measures. 15 Compelling evidence exists in a systematic review of 309 articles. 16 The 11 articles selected for inclusion employed rigorous studies such as meta-analysis, randomized controlled trials, or systematic reviews. Their findings supported NP delivering patient care of equal or superior quality compared to physicians in primary care settings. The clinical implication of this study emphasized the scientific evidence supporting NP effectiveness when compared to other primary care provider types. 12 This robust evidence contradicts the assertions made by the AMA and highlights the vital role the NP fills in ensuring access to quality healthcare, particularly in contexts where physician shortages and other barriers exist.
The academic implications of this study are important for nursing programs to understand the needs of the practice role for attaining professional practice competencies. While the survey items do not directly target NP practice exclusively, the context does underscore the importance of evaluating the NP based on the patients’ experiences that include assessments of clear communication, attentive listening, knowledge of patient history, respect, and time spent during interactions. The findings demonstrate that the NP consistently achieved comparable outcomes to other healthcare providers in these crucial competencies and continue to positively impact the patient experiences with primary care providers. 11
Given these insights, it is timely for nursing schools to embrace competency-based learning and interdisciplinary education to inform and adapt their curricula accordingly to cement their vital role in providing primary care services to the ever-growing US population. Furthermore, the study suggests that similar evaluative data should be gathered from practicing NP to continuously assess and refine individual graduate program competencies for quality improvement.
Contrary to the findings, recent statements made about NP preparation from other medical professionals suggest that academic programs are lacking in developing graduates as competent primary care providers. 2 The evidence supports the need for interdisciplinary knowledge and practices to be shared to improve quality of patient care that can result in highly rated patient experiences. By integrating this recommendation into nursing education and practice, we can ensure that NP and their primary care colleagues are equipped to deliver high-quality care that meets patients’ expectations and contributes to positive health outcomes.
Limitations
The federal government's primary purpose for the CAHPS data did not specifically target an analysis of patients’ experiences with provider types. Rather, the broader aim of CAHPS is to improve the overall patient experience in healthcare settings, promoting safety and patient-centered care along with information to inform physician and clinic decisions. Another limitation is that the researchers were unable to insert questions specifically tailored to capture patients’ experiences with NP exclusively. The researchers opted to utilize CAHPS data due to its extensive sample size, national representation, clear delineation of provider types, and availability; however, the number of patient responses to the CAHPS survey was overwhelmingly focused on MDs as the provider type. The absence of surveys from the southeast region of the United States clearly limits generalizability of the findings. AHRQ suspended data collection during COVID. Moving to differing care environments such as telehealth could affect the patient's experience. In addition, changes in health delivery and provider availability could have been affected during COVID which could be a factor impacting the patient experience with providers. While acknowledging these limitations, the study was able to draw valuable conclusions regarding patients’ experiences with NP compared to other primary care providers, offering important insights despite the constraints of the data source. Future research comparing patient experiences during COVID-19 and before or after would be an interesting and informative inquiry.
Conclusion
Nurses continue to be critical to the well-being of society by providing primary care services in a time of challenges in access to care. This study expands the knowledge about NP and other providers in the primary care role. NP patient experience ratings are comparable to MD, DO, or PA based on the patient-specific responses received on CAHPS survey. However, room for improvement in NP preparation for the role is evident since DO earned higher ratings from many patient responses. With this evidence and acknowledgment, there is support for NP-prepared nurses to bridge the void in primary care providers to meet the CMS’s established goal of delivering optimal health and well-being to US citizens in a high-quality, safe, equitable, and effective healthcare system. It will take all of us!
Footnotes
Acknowledgments
The CAHPS data used in this analysis were provided by the CAHPS Database. The CAHPS Database is funded by the US Agency for Healthcare Research and Quality (AHRQ) and administered by Westat under Contract No. HHSP233201500026I/HHSP23337004T. Support was received from the Open Access Publishing Fund administered through the University of Arkansas Libraries.
Author Contributions
DeAnna Jan Emory and Thomas A. Kippenbrock have developed the research project study design, interpretation, and manuscript writing for final submission. DeAnna Jan Emory performed the data analysis using SPSS.
Data Availability
Data acquired for the investigation were received through an application process. All survey data received contained no identifying information and were retrospective in nature from survey responses. The researchers agreed to hold the dataset in confidence. An application process is required to obtain the information and responses to survey items.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethical Approval
This study was approved by the Internal Research Review Board. The study was given exempt status (approval number: 2305471862).
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Statement of Human and Animal Rights
This article does not contain any studies with human or animal subjects.
Statement of Informed Consent
Informed consent for patient information to be published in this article was not obtained because this dataset was provided to the researchers upon request. All data were retrospective in nature and informed consent is not applicable.
