Abstract
Team-based care has emerged as a promising strategy for primary care practices to provide high-quality care. We examine changes in patient experience of care and recommended cancer screening rates associated with a primary care transformation initiative that established team-based care. Our observational study included 13 academically affiliated primary care practices in the Boston, Massachusetts area that participated in 2 learning collaboratives: the first (2012-2014) aimed to establish team-based primary care, while the second (2014-2016) focused on improving patient safety and cancer screening. We identified 37 comparison practices of similar size and network affiliation. Using a difference-in-differences approach, we compared pre (2013) and post (2015) patient experience and recommended cancer screening rates between intervention and comparison practices. We estimated linear regression models, using inverse probability weighting to balance on observable differences. Massachusetts Health Quality Partners data on patient experience comes from surveys (with communication, integration, knowledge of patient, access, office staff, and willingness to recommend domains), and its data on screening rates for breast, colorectal, and cervical cancers is derived from chart abstraction. Relative to comparison practices, the communication score in intervention practices increased by 1.47 percentage points on a 100-point scale (
Transitioning to team-based approaches to delivering primary care has been associated with achieving modest improvements in some measures of clinical processes.
We examine changes in 6 domains of patient-reported experiences of care and 3 rates of screenings for cancers associated with a primary care transformation initiative that established team-based care.
Our results suggest that communication may be the first dimension of patient experience that improves following establishment of team-based primary care, and that changing care processes may require more time and attention in the transition to team-based care.
Introduction
Adoption of team-based approaches to delivering primary care has emerged as a promising strategy to improve clinical quality and patient experience of care. In team-based primary care models, individuals from different disciplines collaboratively manage the care for a panel of patients.1,2 Care teams range in size and structure, and can include both clinical (eg, physicians, medical assistants) and non-clinical (eg, social workers, community health workers) members, depending on patient needs. 3 Through distribution of responsibilities and improved care coordination, team-based care can be an important facilitator of improved clinical quality (eg, recommended cancer screenings) and patient experience 4 ; however, evidence suggests primary care practices remain slow to adopt these approaches. 5
Transitioning to team-based primary care has been associated with achieving modest improvements in some process measures of clinical quality.6,7 However, our current understanding of the effectiveness of team-based care is limited by studies conducted in a single academic medical center and a limited range of clinical services, such as improvements in diabetes bundle adherence and reductions in preventable adverse drug events. Since establishing team-based primary care represents a practice-wide change, it is important to evaluate many areas of clinical practice (eg, the patient experience via better communication or preventative services via greater attention to systems). Elements of team-based models may improve clinical quality of care and receipt of preventative care: patient empanelment or integration of new members to a care team, for example, could facilitate monitoring of healthy patients and ensure timely cancer screening.8,9 Some evidence suggests that adoption of the patient-centered medical home model—of which team-based care was one of several key attributes—was associated with improvements in recommended cancer screening rates; however, additional research is needed to measure the association between team-based primary care approaches and rates of recommended cancer screenings. 10
Though patient experience is increasingly recognized as an important metric of care quality, the association between team-based primary care and patient experience of care remains infrequently and inconclusively studied.11-15 Through the integration of personnel like medical assistants or medical scribes into care teams, team-based care models may allow primary care providers to dedicate more time on direct patient contact, which has been associated with improved patient experience in previous studies.16-18 Additionally, most prior studies have focused primarily on a single global metric of patient satisfaction, rather than examining multiple dimensions of patient experience of care, such as integration of care, knowledge of patient, and provider communication.11,12 A meta-analysis found some evidence of team-based care being better than traditional care delivery models at improving patient satisfaction; however, to our knowledge, fewer studies have measured multiple dimensions of patient experience, and these have found marginal or non-significant differences after redesigning primary care delivery.12,19
In 2012, Harvard Medical School began a multi-year, multi-site, phased transformation initiative to establish high-functioning interdisciplinary teams and improve patient safety. We examine the impact of this primary care transformation initiative on changes in patient experience and recommended cancer screening rates.
Methods
Description of Initiative
In 2012, 18 hospital- and community-based primary care practices across 6 academic medical centers (AMCs) affiliated with Harvard Medical School entered a 2-year learning collaborative, the Academic Innovations Collaborative (AIC). The initiative aimed to improve experience of primary care delivery for patients, providers and staff through establishing team-based care. 20 Practices reorganized clinical and non-clinical personnel into interdisciplinary care teams, to which patients were assigned. In 2014, a 2-year continuation of the AIC began, called the AIC Comprehensive, Accessible, Comprehensive, Accessible, Reliable, Efficient, and Safe (CARES) initiative, which focused on using team-based care strategies to improve patient safety by improving cancer-screening processes. 21
AIC and AIC CARES required practices to establish regular care team huddles, attend triannual 1.5-day learning sessions, and participate in monthly webinars to discuss quality improvement strategies through team-based care. At the beginning of each phase, practices received unrestricted lump sum payments equivalent to $3 per member per month for AIC and $0.50 to $1 for AIC CARES to support systems transformation.20,21
Past work studying the AIC has found that care team members at AIC practices reported neutral to positive ratings on team dynamics (eg, perceived team effectiveness, acting and feeling like a team). 22 The transition to team-based care in AIC practices was associated with greater career satisfaction and perceptions of patient safety for physicians, nurse practitioners, physician assistants and medical assistants.22-27 Following the establishment of team-based care, healthier patients in AIC practices had more annual outpatient visits, while chronically ill patients experienced significant reductions in hospitalizations and emergency department visits. 28 Studies have also examined the diagnostic evaluation processes of patients presenting to AIC practices with high-risk symptoms for breast and colorectal cancer.29,30 However, until now, there have been no studies of the patient reported experience of the transition to team-based care in AIC practices.
Identification of AIC and Comparison Practices
Massachusetts Health Quality Partners (MHQP), an independent non-profit organization, produces an annual database that captures the structure of ambulatory care in the state, maps primary care and specialty providers to the locations where they deliver care, and includes information about organizational hierarchy and an individual’s role at the practice. 31 Using the 2013 MHQP Massachusetts Provider Database (MPD) we identified 13 of the 18 AIC practices and a comparison group of 37 practices; the remaining practices either were too small or lacked sufficient number of commercial claims to be measured in MHQP’s sampling methods. To include more provider characteristics (eg, specialty, gender) in our dataset, we used National Provider Identifier data available from the Center for Medicare and Medicare Services’ National Plan and Provider Enumeration System. 32
To identify appropriate comparison practices, we included all academically affiliated primary care practices in the Boston metropolitan area, as determined by practice zip code. We then calculated practice-level counts of each clinician type: primary care physicians, specialty physicians, nurse practitioners, physician assistants, and other (eg, urgent care, obstetrics and gynecology). Because there were some observable differences in practice characteristics, we estimated propensity scores for assignment to an AIC practice using a logit model, where AIC assignment was the binary outcome. Prior studies suggest that provider characteristics (eg, gender, type), as well as practice size and network affiliation can influence the quality of primary care delivered.33-37 As such, to balance practice-level characteristics, covariates included count of each clinician type, proportion of clinicians that was female, and network affiliation. We then calculated inverse probability of treatment weights (IPTWs) using the estimated propensity scores. 28
Outcomes
MHQP also produces the Patient Experience Survey (PES) and Clinical Quality (CQ) datasets, which provide practice-level quality measures based on survey and claims data for patients enrolled in any 1 of 5 commercial health plans. 38 These 5 plans comprised the majority of the commercially insured population in Massachusetts (eg, approximately 70% of the state in 2016). 39 The PES, based on the Consumer Assessment of Healthcare Providers and Systems Patient Centered Medical Home Survey, is a 60-question collection of standardized measures of patient perceptions of care that are compiled and reported as 6 domains: communication, integration of care, knowledge of patient, organizational access, office staff, and willingness to recommend. While practice-level response rates for surveys were not available in the data, the overall response rate in recent years has been around 20%, consistent with similar regional health care surveys. 40 Patients who reported at least 1 visit to their primary care provider in the last 12 months were randomly sampled based on visit and membership data provided by their health plan. MHQP attributed patients to a primary care provider based on whether the patient received primary care services, the patient’s most recent encounter, and frequency of visits. 40 Starting samples were obtained by randomly drawing an equal number of patients from each provider’s panel, where practices with more primary care providers had larger starting samples. All questions and measure domains underwent extensive psychometric testing, and MHQP evaluated each measure for its site-level reliability. Examples of PES survey questions based on the 6 measure domains include: communication (eg, “how often did this provider spend enough time with you?”), integration of care (eg, “how often did this provider seem informed and up-to-date about the care you got from specialists?”), knowledge of patient (eg, “how often did this provider seem to know the important information about your medical history?”), organizational access (eg, “how often could you get an appointment as soon as you needed?”), office staff (eg, “how often did the front office staff at this provider’s office treat you with courtesy and respect?”), and willingness to recommend. Descriptions of our outcomes are presented (see Appendix Table 1). Most survey questions were on a four-point Likert scale (eg, response categories were “never,” “sometimes,” “usually,” or “always”), and the willingness to recommend survey question was binary. MHQP provided practice-level average scores for each PES measure domain, which were calculated using a proportional scoring method: the average proportion responding in each Likert category across the survey questions in each domain were calculated. 41 Likert categories were converted to a numerical scale, wherein the “Never to Always” scale translated “1 to 4.” The average practice-level domain score was then converted to a 0 to 100 scale. 42 PES measures were case mix-adjusted by MHQP for age, gender, education, chronic conditions, race, language, health plan, and region.
The CQ dataset includes Healthcare Effectiveness Data and Information Set (HEDIS) performance measures, which are calculated using either an “administrative data method,” which uses only claims or encounter data for the entire HEDIS-eligible population at a practice, or a “hybrid method” which supplements the administrative calculation with a random sample of medical record reviews. MHQP applies an adjustment methodology for practices that use the hybrid method, but does not case-mix adjust results. Breast cancer screening measures are calculated using the administrative data methodology, while the cervical and colorectal cancer screening measures allow both administrative and hybrid methodologies. Health plans shared HEDIS results with MHQP, which then aggregated the data to the practice level. Specifications for our 3 CQ measures are available (see Appendix Table 2). We used 2013 and 2015 PES and CQ data for in our analysis. We limited our sample to practices that had observations in both 2013 and 2015, which differed by outcome, so sample sizes likewise varied by outcome.
Statistical Analysis
To analyze the effects of AIC CARES on patient experience and recommended cancer screening rates, we employed a standard difference-in-differences framework defining 2013 as the pre period and 2015 as the post period. The difference-in-differences approach allowed us to calculate the average treatment effect of the AIC CARES initiative by comparing changes in the pre and post periods between AIC and similar comparison practices. We used a linear regression model, weighting the observations using the IPTWs. All observations were clustered at the practice level to account for correlation. We include indicators for AIC participation, the post period, and the interaction between the 2 (AIC*post period), which provides the difference-in-differences estimate, and adjust for counts of each clinician type, network affiliation, and provider characteristics. Each of our outcomes are analyzed in a separate regression model. As a sensitivity analysis, we also examine our outcomes adjusting for practice-level covariates without IPTWs.
The institutional review board approved this study.
Results
Table 1 describes both unweighted and propensity-weighted characteristics of AIC and comparison practices. Prior to weighting, there were no significant differences between AIC and comparison practices, though, on average, AIC practices were larger in size. Using the IPTWs, means were comparable. In addition to formal inspection of covariate balance between AIC and comparison practices, we visually inspected the common support condition of our propensity scores (see Appendix Figure 1). 43
Descriptive Characteristics of Intervention and Comparison Primary Care Practices, 2013.
Weighted by inverse probabilities from a logit propensity score model adjusting for covariates in this table.
Table 2 summarizes the propensity-weighted scores for our outcomes for AIC and comparison practices pre-initiative (2013) and post-initiative (2015). We observed a statistically significant improvement in the communication score of 1.47 percentage points (
Weighted Difference-in-Differences Regression Results*.
Adjusted analyses weighted for network affiliation, proportion of practice clinicians that were female, as well as number of primary care physicians, specialists, physician assistants, nurse practitioners, and other (eg, urgent care, obstetrics and gynecology).
Discussion
Our findings suggest that the AIC CARES initiative was associated with a modest, statistically significant improvement in patient perceptions of communication after establishing team-based primary care in 13 practices across 6 AMCs; however, we did not detect significant improvements in other measures of patient experience or recommended cancer screening rates.
This study builds upon previous work on the effects of team-based care in several key ways. First, we examined our outcomes in 13 primary care practices across 6 different AMCs, while most previous studies focus on changes within 1 AMC. Second, while many studies include a single measure of patient satisfaction, we described 6 different domains of patient experience. Third, we measure changes in recommended cancer screening rates associated with practice transformation. Fourth, we used IPTWs in our analyses to balance on observable practice-level covariates to estimate the effect of the initiative on our outcomes using a difference-in-differences approach.
A previous study suggested that changes in utilization in AIC practices was potentially driven by patient empanelment and improvements in care management. 28 Improvements in patient perceptions of communication may be related to the integration of new care team members, which can allow primary care providers more time for direct contact with patients.16-18 In contrast to our findings, some studies have found no changes in patient experience or improvements in other dimensions of patient experience (eg, willingness to recommend) after transitioning to team-based care delivery models, which could reflect the wide variation in the design of care teams.12,19,44 Another potential explanation is that some measures of patient experience or satisfaction observe a ceiling effect (eg, most patients scoring mid-to-high ratings on survey items) that would be less sensitive to transitioning to team-based care. 45
Team-based approaches to primary care have been suggested as an important facilitator in improving adherence to recommended cancer screening processes. 46 Moreover, the AIC CARES initiative focused on improving patient safety, and specifically targeted cancer screening; however, we found few immediate changes in screening rates. This could reflect that practices had less time to improve screening rates.
Some individual AIC practices reported improvements in colorectal cancer screening rates, but in the aggregate the changes in screening rates were statistically similar to comparison practices, a finding that is consistent with evaluations of similar learning collaborative initiatives.21,47 These findings suggest a need to better understand the heterogeneity of contextual and implementation factors that enable some practices to transition to team-based care, and to use teams to improve colorectal cancer screening. Some team-based primary care interventions have improved colorectal cancer screening rates using innovative strategies, including the use of community health worker outreach and the introduction of fecal immunochemical tests; however, these practices had lower baseline rates than AIC practices, which exceeded the national average among commercially insured patients in 2013 (approximately 60%).48-50 There is mixed evidence on the association between breast cancer screening rates and team-based care: 1 study reported significant improvements of 3.5 percentage points over 3 years in practices with care teams, while another found no significant changes.51,52 Fewer studies have examined changes in cervical cancer screening rates associated with adoption of team-based care, though 1 found no significant differences among army primary care patients following the formation of care teams. 53
Limitations
Our study has several limitations. Given the number of outcomes we analyzed, our single measurable effect may due entirely to chance. Our analysis did not include all 18 AIC practices, as 2 were pediatric and 3 were either too small or had an insufficient number of commercial claims, and may be underpowered. It is possible that there were similar redesign efforts at comparison practices of which we are not aware and are therefore unable to account for this in our analysis. This may be of concern, as our study period includes implementation of Affordable Care Act’s care delivery initiatives. We therefore use a difference-in-differences framework, which is robust to systemic changes, though it assumes that no factors change differentially over time between AIC and comparison practices, which is possible.
Practice-level PES response rates were not available in the data, and we were unable to measure or account for differences in response rates between AIC and comparison practices in our analysis. However, the overall PES response rate was comparable to other regional health care surveys. 40 Further, our data is limited in the observable characteristics available to estimate propensity scores. In addition to practice-level covariates in the MPD, we identified additional provider-level characteristics from the NPI registry to address this issue. We also lacked sufficient data before the AIC initiatives to confirm parallel trends. Moreover, we estimated our outcomes among commercially insured patients and, though most AIC patients report commercial insurance, results could differ among patients with other forms of coverage. 23
There is wide variation in the size, structure, and composition of multidisciplinary teams established following the transition from traditional practice organization to team-based care.23,44 The effectiveness of the care teams, and the quality of care delivered, is impacted by environmental and contextual factors related to implementation, culture of the primary care teams, and perceived team dynamics.10,54-56 As we are unable to control for these factors, there may be residual confounding. Lastly, our sample includes academically affiliated primary care practices in the Boston area, which, although a diverse group including hospital-based, community-based and federally qualified health centers, may not be generalizable to other practices.
Redesigning primary care practice through team-based delivery models is intended to improve the patient experience and clinical quality of care. The transition to these models, though, can be a complex, continuous, and long-term process that can vary immensely by practice capacity and leadership. 57 We found no immediate improvements in most measures of patient experience and recommended cancer screening rates relative to comparison practices following the establishment of team-based care in AIC practices. It is possible that improving quality of care—particularly cancer screening rates—may take more time or require more attention in the transition. 58 Subject to the limitations noted above, our findings may suggest that communication is the first dimension of quality that patients see improvements following transformation to team-based primary care.
Supplemental Material
Supplemental_File_4_07072020 – Supplemental material for Team-Based Primary Care Practice Transformation Initiative and Changes in Patient Experience and Recommended Cancer Screening Rates
Supplemental material, Supplemental_File_4_07072020 for Team-Based Primary Care Practice Transformation Initiative and Changes in Patient Experience and Recommended Cancer Screening Rates by Kevin H. Nguyen, Alyna T. Chien, David J. Meyers, Zhonghe Li, Sara J. Singer and Meredith B. Rosenthal in INQUIRY: The Journal of Health Care Organization, Provision, and Financing
Footnotes
Acknowledgements
The authors thank Tanya Rudakevych and Jack Huang for their research assistance on this project.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by funding from the Harvard Medical School Center for Primary Care and the Controlled Risk Insurance Company (CRICO) Risk Management Foundation of the Harvard Medical Institutions (36212).
IRB Approval
The institutional review board at the Harvard T.H. Chan School of Public Health approved this study.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
