Sage Journals: Discover world-class research

Abstract

Introduction:

Primary care clinics often undertake quality improvement (QI) projects to enhance care delivery. Disruptions to clinical practice, such as staffing changes, can hinder QI efforts. Limited research examines how primary care clinics experience and respond to disruptions. This study explores the types and impacts of disruptions and factors linked to resilience for rural clinics during a Human Papillomavirus (HPV) vaccination QI study.

Methods:

We used a mixed-methods, longitudinal design to assess disruption frequency and type alongside clinic-level vaccination outcomes. Qualitative case studies explored how disruptions influenced QI participation and resilience factors.

Results:

Among 41 rural clinics studied, all reported at least 1 disruption during QI implementation, including staffing loss (37 clinics, 90%), EHR implementation (10 clinics, 24%), and organizational changes (4 clinics, 9%). Disruption frequency and type did not differ across quartiles of HPV vaccination improvement. Eight clinics were selected for in-depth case study analysis, which revealed factors shaping resilience: (1) clinic characteristics, (2) practice facilitator support, and (3) disruption characteristics.

Conclusion:

Disruption type or number did not explain differences in vaccination improvement. Multiple factors were found to influence resilience and continued QI. Identifying and enhancing these facilitators is critical to supporting primary care clinics through inevitable challenges.

Keywords

primary care quality improvement mixed methods pediatrics practice management program evaluation rural health

Background

Primary care clinics take on quality improvement (QI) projects to enhance patient experience, improve population health outcomes, reduce cost, and ensure provider wellbeing.¹ Significant variation exists among clinics’ experience and structure surrounding QI efforts. This may vary from no formal experience or structure to dedicated improvement teams tasked with improving specific metrics.² Studies have shown that many barriers exist to making and sustaining QI changes,^3-5 and primary care clinics are often under-resourced to support this work. Rural clinics in particular experience more barriers to making and sustaining QI changes.⁶

Disruptions can pose a significant challenge for clinics participating in QI, including unexpected events like natural disasters, or chronic challenges like financial strain.⁷ Common primary care disruptions include staff turnover, implementing a new electronic heath record (EHR), and organizational changes like merging with another health system.^8-11 These disruptions can negatively affect clinical care as well as QI.⁹ Understanding resilience—how health systems adapt and transform to recover from disruptions—is critical to ensuring long-term participation in QI, but little research describes the impact and response to disruptions among primary care clinics.¹²

The Rural Adolescent Vaccine Enterprise (RAVE) study utilized a practice facilitator led QI intervention during a period of extreme healthcare disruption, the COVID-19 pandemic (2020-2022), making RAVE an ideal setting to explore the impact of a variety of disruptions on QI. This manuscript aims to describe clinic disruptions experienced during RAVE, demonstrate the impacts of disruptions on QI efforts, and describe factors associated with resilience.

Methods

We used a mixed-methods, longitudinal design to examine disruption frequency and type alongside clinic-level HPV vaccination improvement. Qualitative collective case studies explored how disruptions influenced QI and identified factors that mitigated their impact on RAVE QI efforts.¹³ Oregon Health & Science University’s Institutional Review Board (IRB# 18660) approved all study activities.

Study Setting

RAVE was conducted through the Oregon Health & Science University.¹⁴ Eligible clinics were located in rural areas (defined by Oregon Office of Rural Health as any geographic area in Oregon that is 10 or more miles from a population center of > 40,000 people), participated in the Vaccines for Children program, and had at least 20 adolescents aged 13 to 17 and 10 aged 11 to 12. From April 2019 to March 2021, participating clinics received 18 months of practice facilitation to improve HPV vaccination. Support included monthly meetings with a facilitator and QI-based strategies to enhance HPV vaccination rates.¹⁵

Data Collection and Variables

This analysis drew from 3 primary data sources: (1) clinic-reported data, (2) the ALERT Immunization Information System (IIS), and (3) practice facilitator fieldnotes.

Clinic-reported data were collected via structured surveys at enrollment, capturing clinic specialty, staffing, and roles.

ALERT IIS was used to assess baseline patient characteristics and vaccination outcomes.¹⁶ Patients were linked to clinics using a validated weighting algorithm previously used to measure clinic-level vaccination outcomes.¹⁷ Baseline data were collected in early 2019. HPV vaccination completion rates were measured quarterly, and change was calculated by subtracting the baseline rate (quarter before intervention) from the final rate (quarter after the 18-month intervention). Rates were divided into quartiles, with the fourth quartile representing the most improvement.

Practice facilitator fieldnotes were collected monthly in REDCap during meetings with clinic teams.¹⁸ Practice facilitators were trained in collecting rigorous fieldnotes prior to implementation. Notes included structured prompts on disruptions, categorized as new EHR implementation, staff loss, new location, organizational change (e.g., mergers or new billing systems), and “other” (mostly COVID-related). Due to low frequency, location changes, mergers, and billing system changes were combined into a single category: “new organizational structures.” Fieldnotes also documented clinic context, disruption details, response strategies, and adjustments to implementation plans, serving as the primary data source for case study analysis. Clinic context, disruption details, and response strategies were free-text fields with rich thick descriptive text. Fieldnote data was reviewed for completeness, quality, and accuracy by an analyst prior to selecting and finalizing case studies to ensure robust data.

Descriptive Analysis

Descriptive statistics were used to summarize clinic characteristics, changes in HPV vaccine completion rates, and types of disruptions experienced. Disruptions are reported by quartile of HPV vaccine improvement. Due to small sample sizes per quartile and frequent overlap in disruption types, no between-group statistical analyses were conducted. Stratified data are presented to illustrate observed trends only.

Case Study Analysis

Case study clinics were selected using a maximum-variation approach to capture heterogeneity in clinic context, experience, and outcomes.¹⁹ This approach allowed for analytic comparison across contrasting cases to examine how similar factors did or did not manifest across different conditions. Clinics were chosen to reflect diverse disruption experiences: those with unique or infrequent disruptions, high or low disruption frequency, and those facing common disruptions to ensure representativeness. Selection also considered disruption impact on project participation, categorized inductively from facilitator fieldnotes as: continued participation (minimal change), reduced participation (delayed or scaled-back efforts), or halted QI (ended implementation). Clinics with poor data quality were excluded. Case study data were analyzed using latent projective content analysis to explore patterns and nuance in disruption impact.²⁰ Two qualitative analysts reviewed fieldnotes to identify meaningful units, then abstracted case study data from REDCap into Atlas.ti 23 for auto-coding using an a priori codebook (disruption description, impact, response plans, follow-up). Coded data were then organized into a matrix (by disruption type, response, impact, actor, timing, and outcome) by 1 analyst and reviewed by the second for accuracy. Both analysts reviewed the matrix to group similar cases and identify factors related to clinic resilience via analytical memos. Memos were discussed with the analytic team, project manager, and a RAVE facilitator to develop and refine themes through group consensus over 3 meetings. Final themes were validated with the full study team.

Results

Of the 46 clinics randomized in the RAVE project, 1 was deemed ineligible, leaving 45 participating clinics. Four clinics did not launch RAVE activities and lacked required disruption data, resulting in 41 clinics included in the HPV completion rate quartile analysis. Of the 41 included clinics, 8 were selected as case studies for further exploration. Clinic characteristics are shown in Table 1.

Table 1.

Characteristics of RAVE Clinics.

Baseline clinic characteristics at study initiation, 2019	N = 41 clinics
Clinic specialty, n (%)
Family Medicine	29 (70.7)
Pediatric or both	12 (29.3)
Mean proportion of patients with non-white race, % (SD)	6.8% (3.3)
Mean proportion of patients insured by Medicaid, % (SD)	50.5% (12.3)
Practice type, n (%)
Community health center	7 (17.1)
Health system	20 (48.8)
Private/independent	14 (34.1)
Size, n (%)
<200 adolescent patients	10 (24.4)
200-1000 adolescent patients	22 (53.7)
>1000 adolescent patients	9 (22.0)
HPV vaccination:
Mean baseline clinic rate of HPV vaccination^a for 11-17 year old patients, (SD)	39.8 (9.7)
Mean post-intervention clinic rate of HPV vaccination^a for 11-17 year old patients, (SD)	42.8 (9.8)
Improvement in clinic-level HPV vaccination^a for 11-17 year old patients by quartile, range of change in rates
Quartile 1	-9.0, -0.1
Quartile 2	0.0, 2.9
Quartile 3	2.9, 4.8
Quartile 4	5.3, 17.5

Vaccination up to date defined as 2 vaccine doses administered 6-12 months apart for 11-14 year olds and 3 vaccine doses administered 6 months apart for 15-17 year olds.

Quantitative Findings

HPV vaccination rate changes ranged from a 9 percentage point decrease to a 17.5 point increase. Clinics in the lowest improvement quartile saw changes from −9.0 to −0.1 points, while those in the highest quartile improved by 5.3 to 17.5 points. Disruption characteristics were summarized by HPV rate change quartile (Table 2). All 41 clinics reported at least 1 disruption during RAVE implementation. Staffing loss was most common (37 clinics, 90%), followed by new EHR implementation (10 clinics, 24%) and new organizational structures (4 clinics, 9%). “Other” disruptions, reported by 37 clinics (90%), were mostly related to COVID-19 but also included natural disasters. Disruption frequency and type were similar across quartiles, with no clear pattern linking disruption type or number to HPV rate change.

Table 2.

Summary of Disruptions Experienced, Overall and by HPV Vaccination improvement Quartiles.

	All clinics	Q1	Q2	Q3	Q4
	N = 41 (100%)	N = 11 (27%)	N = 10 (24%)	N = 10 (24%)	N = 10 (24%)
Any disruption	41 (100%)	11 (100%)	10 (100%)	10 (100%)	10 (100%)
Disruption types
Loss of staff	37 (90%)	10 (91%)	9 (90%)	9 (90%)	9 (90%)
New electronic health record	10 (24%)	3 (27%)	3 (30%)	2 (20%)	2 (20%)
New organizational structures	5 (12%)	2 (18%)	1 (10%)	1 (10%)	1 (10%)
Other^a	37 (90%)	10 (91%)	9 (90%)	9 (90%)	9 (90%)

Disruptions in the other category mostly included COVID-19 modified operations.

Qualitative Findings

The 8 clinic case studies analyzed qualitatively are summarized in Table 3. Of the 8 case studies, 3 were able to continue with their QI efforts despite disruptions (Case study 25, 1, 3). All 3 of these clinics were also able to improve their HPV completion rates. Three of the case clinics continued QI through disruptions, but at a reduced capacity, either delaying efforts or scaling down to accommodate changes caused by disruptions (Case study 33, 13, 42). Of the 3 clinics that reduced their QI efforts, only 1 was able to improve their HPV completion rates (Case study 33). Finally, 2 clinics had to stop their RAVE QI efforts due to disruptions (Case study 19, 5). Of these 2 clinics, 1 was able to improve their HPV rates (Case study 5), and 1 was not (Case study 19).

Table 3.

Case Study Clinic Summaries and Outcomes.

Case study ID and primary theme(s)	Case study summary	QI impact	Improved HPV completion rates
25 Clinic staffing	This medium size pediatric clinic that is part of a large health system experienced 16 disruptions during RAVE implementation. These disruptions included the loss of multiple staff members across multiple months. The clinic’s lost staff also spanned multiple roles including providers, medical assistants, nurses, and front desk staff. This left the clinic understaffed for almost the entire implementation period. Because this clinic was part of a health system, they were able to reduce the impact of staffing losses by relying on staff from other departments in their system. This included utilizing float medical assistants, triage nurses from another department, and having providers cover for vacant roles. By using these staff resources, the clinic was able to continue working on RAVE implementation despite chronic understaffing. Further, this clinic had QI experience and was motivated to improve HPV rates. This allowed them to continue to prioritize working on HPV QI despite reduced staff capacity. The clinic was able to find creative ways to reduce the disruption impact and continue RAVE implementation.	Continued RAVE QI	Yes
33 Practice facilitator accountability and Concurrent disruptions	This small physician-owned family medicine clinic reported 8 disruptions during RAVE implementation. These disruptions included the loss of clinic staff and COVID-19 related modified operations. The clinic lost staff because of the COVID-19 vaccine mandate for healthcare workers. In addition, the clinic had to re-organize roles and responsibilities for clinic staff who had vaccine waivers and needed to work from home. The clinic also had their nurse care manager retire. The clinic operated with reduced and a modified staffing model for the entire second half of RAVE . Towards the end of implementation, they also had a surge of COVID-19 in their community, leading to an increase in sick visits for their patients. All these disruptions reduced the clinics’ time for QI and hindered their implementation efforts. However, the clinic remained engaged by normalizing the changes they were experiencing and staying dedicated to working on QI. They viewed their disruptions as temporary and not things that would have long-term negative impacts. This clinic also used a team approach, including the involvement of leadership, for QI. Similarly, they remained flexible through staffing changes and adjusted roles to fill gaps as needed. Finally, the practice facilitator leveraged asynchronous check-ins between meetings to help the clinic stay on track with their implementation plan.	Reduced RAVE QI	Yes
19 QI experience and structure and disruption acuity	This medium size family medicine clinic that is part of a large health system reported 5 disruptions during RAVE implementation. These disruptions included the loss of one or more staff members and a natural disaster (wildfire). This clinic started RAVE having recently lost key staff. Then at month 3 of the project, there were wildfires in the area that caused the clinic staff to evacuate and temporarily practice in another location. This disruption, though severe, was brief. The clinic did have to pause implementation until the clinic could return to normal operations. The practice facilitator adjusted their implementation plan at this time to help the clinic accommodate the disruption delay. Then in month 5, the only provider working on the RAVE project left the clinic. The only other provider in the clinic was not interested in participating, so the clinic dropped out of RAVE and stopped all implementation.	Halted RAVE QI	No
1 Clinic staff mindset	This medium size rural health clinic, part of a group of practices, reported 12 disruptions during RAVE implementation. These disruptions included the loss of one or more staff members, moving to a new location, new leadership, and COVID-19 modified operations. This clinic experienced multiple disruptions almost constantly throughout the QI project. They started off with the loss of key staff including their CEO who was also a provider and the RAVE QI champion. They lost additional key staff including COO, lead MA, and others shortly after. The clinic then opened an additional clinic location. All of this caused re-organization of staff and roles. Shortly after, they entered modified operations due to the onset of the COVID-19 pandemic. During the remaining 9 months of the project, the clinic struggled with changes to operations from COVID-19 and being understaffed. Despite these challenges, the clinic was able to continue implementation and make small progressive changes. Even through changes in leadership, the clinic had leadership support and engagement in the RAVE project, which helped them continue to have protected time for QI through disruptions. Similarly, the clinic was large enough with enough staff to buffer losses by having staff cover or flex their roles. The clinic prioritized and valued QI and made it a priority despite several challenges.	Continued RAVE QI	Yes
3 Anticipated vs. unexpected disruptions	This medium size physician-owned pediatric clinic reported 7 disruptions during RAVE implementation. These disruptions included the loss of one or more staff members and the implementation of a new EHR. At month 4 of the project, the clinic had a provider go out on leave. Then 4 months later, they began the process of getting a new EHR. Two months after that, COVID-19 started. They reduced clinic and staff hours and reduced their patient load. A month after that, they lost their RAVE champion. A couple months after, their EHR roll out began. The clinic was proactive in dealing with staff changes and often had coverage for the roles before the person left, as was the case when the RAVE champion left. They also had a long lead up time that allowed for planning for the EHR change to reduce the impact of that disruption. For COVID-19 they modified their operations significantly (reduced clinic hours) for only about a month and then resumed “normal” operations, again reducing the impact of that disruption.	Continued RAVE QI	Yes
13 Clinic staff mindset	This small physician-owned pediatric clinic reported 12 disruptions during RAVE implementation. These disruptions included changes to clinic operations due to the onset of the COVID-19 pandemic. At month 4, the clinic began modified operations due to COVID-19. They had a major reduction in clinic visits at first. Then as visits increased, they implemented new staffing models with staff working from home on reduced hours. They operated under this model for the remainder of the study. Toward the end, they also lost some staff from the front desk, who were the ones working the most reduced hours. The clinic made plans to modify their clinic appointments, activities, and staffing. They used downtime from reduced patient visits to work on QI. Once they were operating with more appointments, they continued to work on making changes, such as less front desk staff hours. Though this clinic had to make significant changes in response to COVD-19, they saw it as an opportunity to prioritize QI instead of stopping it. This aligned with the clinic’s culture of QI. They also had a clinic culture that normalized change and flexibility. This clinic was motivated to find solutions to disruptions that allowed them to continue successfully implementing.	Reduced RAVE QI	No
42 QI experience and structure	This medium size family medicine clinic, part of a critical access hospital, reported 12 disruptions during RAVE implementation. These disruptions included the loss of one or more staff members, the implementation of a new EHR, and COVID-19 modified operations. In month 3 of implementation, this clinic lost staff due to the COVID-19 vaccine mandate for health care workers. The clinic also had a surge in COVID-19 cases in patients at this time. They continued to be understaffed and to prioritized their COVID-19 response for several months. Shortly after these COVID-19 related disruptions, the clinic began the process of preparing for an EHR change. This caused the clinic to struggle to prioritize RAVE QI, particularly at the clinic leadership level. The local clinic staff wanted to do RAVE QI but they did not have support from providers or leadership. The clinic reported they were having to “justify” spending time on RAVE QI to clinic leaders. The clinic also had a mindset of waiting for a good time to do QI, when they had extra time aside from core operations, viewing QI as a bonus activity. These factors influenced the clinic’s ability to continue QI through disruptions.	Reduced RAVE QI	No
5 Clinic staffing	This medium size family medicine group practice reported 2 disruptions during RAVE implementation. These disruptions included being purchased by or affiliated with a larger organization and the loss of one or more staff members. When this clinic started RAVE, they had just been purchased by a new health system. Shortly after, the central QI coordinator quit suddenly, and the health system decided to restructure staff roles and responsibilities. When this happened, the clinic wanted to stop QI efforts because they did not have the time or staff resources. The practice facilitator kept engaging with the clinic and offered delayed and adjusted implementation plans. However, the clinic did not engage in any QI efforts for several months and then the team and leadership decided they would no longer participate in RAVE.	Halted RAVE QI	Yes

Cross-case comparison produced with-in case understanding and cross-case pattern identification that led to the identification of 3 categories of factors that impacted resilience from disruptions. These categories included: (1) clinic characteristics; (2) supports from practice facilitators; and (3) disruption characteristics. For most clinics, a combination of factors were present, which had both positive and negative influences on resilience. No one factor stood out as exclusively protective. However, clinic factors that influenced engagement in QI during disruption, namely clinic staff mindset, did appear to often buffer other factors that had a prohibitive influence on continued QI.

Clinic characteristics that influenced engagement in QI during disruptions

Clinic Staffing: Staffing significantly influenced clinic engagement in QI during disruptions. Clinics with sufficient or flexible staffing (i.e., float staff) were better able to adapt and maintain QI when disruptions affected team structure, roles, or staffing levels. For instance, clinic case study 25 experienced major staffing losses across multiple roles during implementation. As a health system-based clinic, they accessed float staff like medical assistants and triage nurses from other departments. This flexibility helped them sustain RAVE implementation and improve HPV completion rates. Alternatively, clinic case study 5 lost staff roles to restructuring, which included QI specific roles. Without QI staff, the clinic halted participation in RAVE QI and ultimately decided to no longer participate.

QI Experience & Structure: Prior QI experience and structure influenced clinic engagement during disruptions. Clinics with limited QI experience or lacking formal structures often struggled to continue implementation. In contrast, those using a team-based approach had greater capacity to share workloads or absorb staff loss. Clinics relying on a single person, like clinic 19 with only 1 engaged provider, often stalled when that person left. Clinics without formal QI teams lost momentum, especially when implementing QI ad hoc and responding to competing demands like COVID-19. Lack of leadership support further hindered efforts; for example, clinic 42 had to justify RAVE participation to unsupportive system leadership. Finally, clinics with an established QI culture—where QI was routine and valued—were better able to sustain efforts during disruption.

Clinic Staff Mindset: Staff mindset played a critical role in QI efforts during disruption. Clinics that embraced change, viewed disruptions as opportunities, and demonstrated determination were more likely to sustain QI. For example, clinic 1, a small clinic with limited resources, implemented RAVE and improved QI rates despite staff loss and COVID-19 by normalizing change and persisting through challenges. Clinic 13, facing reduced visits and staff hours, used the extra time to focus on HPV vaccine initiatives, turning disruption into progress. Similarly, clinics that used creative problem-solving, assessed capacity, and communicated needs were better able to adapt implementation plans. Strong motivation to improve vaccine rates or engage in QI also helped clinics persevere through adversity.

Practice Facilitator Strategies that Moderated Disruptions Effects on QI

Accountability: Practice facilitation supported QI by providing accountability, especially during disruptions. Facilitators helped clinics stay on track with RAVE by adjusting plans to match capacity, reminding teams of action items, and checking on progress. This was especially valuable when disruptions diverted attention from QI. For example, clinic 33 credited their facilitator’s accountability as the key reason they continued implementation during COVID-19.

Adapting Implementation Plans: Practice facilitators helped clinics revise implementation plans during disruptions by assessing needs, revisiting goals, and identifying new, flexible paths forward. This adaptability allowed clinics to continue QI, even at a smaller scale or with a different focus. In some cases, it was key to sustaining RAVE efforts despite challenges. It also gave clinics space to recover without losing momentum, making it easier to resume progress as capacity returned.

Disruption Characteristics that Influenced QI Impact

Anticipated vs. Unexpected Disruptions: The ability to continue QI often depended on whether disruptions were expected. Planned changes (i.e., EHR transitions or staff retirements) allowed clinics to prepare and minimize impact. For example, clinic 3 faced staff loss and an EHR change but maintained RAVE efforts due to proactive planning. In contrast, unanticipated disruptions, such as natural disasters or sudden COVID-19 illnesses, made it harder to sustain QI, as clinics had to balance immediate needs with long-term solutions like hiring, often leading to greater struggle.

Disruption Acuity: The duration and severity of disruptions shaped clinic engagement in QI. Short but severe disruptions often paused implementation but had minimal long-term impact, as clinics resumed QI quickly afterward. In contrast, persistent but less severe disruptions (lasting months) created greater barriers. Some clinics continued QI by leveraging other supports, but others experienced significant delays or cessation. For example, clinic 19 briefly halted RAVE due to a wildfire evacuation but resumed operations within a month. However, ongoing staffing losses afterward ultimately led them to stop implementation and not complete RAVE.

Concurrent Disruptions: Experiencing multiple disruptions simultaneously made it harder for clinics to sustain QI. This was especially challenging when staffing shortages coincided with other issues, like a COVID-19 surge. For example, clinic 33 faced 9 months of reduced staffing while also managing increased sick visits due to COVID-19. The compounding disruptions led the clinic to scale back their RAVE QI efforts.

Discussion

In this study, we found no clear patterns between HPV completion rate quartiles and reported disruptions, suggesting no specific disruption type consistently influenced QI performance. However, qualitative analysis identified several key factors supporting QI continuity during disruptions. The level of anticipation and acuity of disruptions affected clinics’ capacity for change. Case studies highlighted the importance of staffing, QI experience and structure, staff mindset, and the role of practice facilitators in providing accountability and adapting implementation plans. These findings align with prior research showing that disruption type and number did not significantly affect clinics’ capacity for facilitated QI, while internal QI structures, mindset, and facilitator adaptations supported resilience and sustained engagement.⁹

Historical research on organizational disruption focused on absorbing change to maintain a steady state.¹² However, disruptions observed in this study—such as the COVID-19 pandemic—support emerging views that emphasize organizational transformation to establish a new operational norm and reduce disruption impact.⁷ Our qualitative findings reinforce this, showing that clinic mindset and perception of disruption influenced continued QI. As the healthcare landscape evolves, the nature and impact of disruptions are also shifting, highlighting the need for an adaptive, evolutionary approach to supporting clinic QI.^21-24

Disruptions—both anticipated and unanticipated—are inevitable in primary care and are expected to increase due to rising pandemics and natural disasters.⁸ Projected healthcare worker shortages may worsen this challenge, especially as this study highlights the importance of sufficient and flexible staffing for sustaining QI during disruptions.²⁵ These findings align with Thomas et al., who identified key health system functions for clinics facing disruption: (1) governance—strong, participatory leadership; (2) financing—adequate resources; (3) resources—sufficient and well-distributed staff and infrastructure; and (4) service delivery—flexible care models.²⁶ Future research should examine strategies to help clinics strengthen modifiable factors that support QI resilience.

Limitations

This study has several limitations. First, the small sample of clinics in each quartile limited analyses to observational trends only. In addition, the number of clinics included in the case study favored in-depth analysis of cases over representativeness or enumeration of all available cases. Further, research with a larger sample could provide more nuance into how disruption influences clinic performance. Second, this study was a part of a randomized trial using QI to improve 1 clinical performance outcome. Though our findings are likely relevant to QI for other areas of clinical care, further research is needed to determine if results of this study are the same for a variety of QI projects. Further, the clinics in the study were all located in rural areas of 1 state and may not be representative of the experiences of urban clinics or rural clinics in other regions. Finally, qualitative data collected for this study were collected and recorded systematically, but reflect the point of view of practice facilitators, which is subject to bias based on their role and individual perceptions.

Conclusion

Disruptions are inevitable in primary care, and how clinics respond significantly affects their engagement in improvement activities. Understanding disruptions and identifying intervention opportunities is essential for developing strategies that support consistent, sustainable QI in changing contexts. Since resilience is modifiable, further research is needed to optimize it amid ongoing uncertainty.

Footnotes

Acknowledgements

The authors would like to acknowledge the 45 rural primary care clinics and their staff for their work in this research study and beyond to improve the health of their patients and communities. The authors would also like to acknowledge the ORPRN Practice Facilitators who supported the participating clinics through RAVE implementation.

ORCID iDs

Chrystal Barnes

NithyaPriya Ramalingam

Caitlin Dickinson

Kylie Lanman

Patricia A. Carney

Brigit A. Hatch

Ethical Considerations

Oregon Health & Science University’s Institutional Review Board (IRB# 18660) approved all study activities.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research project was funded by the American Cancer Society, RSG-18-022-010.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data Availability Statement

Data used in this study may be made available upon reasonable request pending institutional approvals.

References

Bodenheimer

Sinsky

From triple to quadruple aim: care of the patient requires care of the provider. Ann Fam Med. 2014;12(6):573-576. doi:10.1370/afm.1713

Balasubramanian

Marino

Cohen

, et al. Use of quality improvement strategies among small to medium-size US primary care practices. Ann Fam Med. 2018;16(1):S35-S43. doi:10.1370/afm.2172

Cohen

McDaniel

Jr Crabtree

, et al. A practice change model for quality improvement in primary care practice. J Healthc Manag. 2004;49(3):155-168; discussion 169-170.

Crabtree

Nutting

Miller

, et al. Primary care practice transformation is hard work: insights from a 15-year developmental program of research. Med Care. 2011;49:S28-S35. doi:10.1097/MLR.0b013e3181cad65c

Gupta

Boland

Jr. Aron

DC.

The physician’s experience of changing clinical practice: a struggle to unlearn. Implement Sci. 2017;12(1):28. doi:10.1186/s13012-017-0555-2

Fagnan

Ramsey

Dickinson

Kline

Parchman

ML.

Place matters: closing the gap on rural primary care quality improvement capacity-the healthy hearts northwest study. J Am Board Fam Med. 2021;34(4):753-761. doi:10.3122/jabfm.2021.04.210011

Fridell

Edwin

von Schreeb

Saulnier

DD.

Health system resilience: what are we talking about? A scoping review mapping characteristics and keywords. Int J Health Policy Manag. 2020;9(1):6-16. doi:10.15171/ijhpm.2019.71

Mold

Walsh

Chou

Homco

JB.

The alarming rate of major disruptive events in primary care practices in Oklahoma. Ann Fam Med. 2018;16(1):S52-S57. doi:10.1370/afm.2201

Hemler

Edwards

Valenzuela

, et al. The effects of major disruptions on practice participation in facilitation during a primary care quality improvement initiative. J Am Board Fam Med. 2022;35(1):124-139. doi:10.3122/jabfm.2022.01.210205

10.

Zhang

Bannon

, et al. Identifying practice facilitation delays and barriers in primary care quality improvement. J Am Board Fam Med. 2020;33(5):655-664. doi:10.3122/jabfm.2020.05.200058

11.

Ruhe

Gotler

Goodwin

Stange

KC.

Physician and staff turnover in community primary care practice. J Ambul Care Manage. 2004;27(3):242-248. doi:10.1097/00004479-200407000-00008

12.

Blanchet

Thinking shift on health systems: from blueprint health programmes towards resilience of health systems Comment on “Constraints to applying systems thinking concepts in health systems: a regional perspective from surveying stakeholders in Eastern Mediterranean countries”. Int J Health Policy Manag. 2015;4(5):307-309. doi:10.15171/ijhpm.2015.49

13.

Crowe

Cresswell

Robertson

Huby

Avery

Sheikh

The case study approach. BMC Med Res Methodol. 2011;11:100. doi:10.1186/1471-2288-11-100

14.

Fagnan

Morris

Shipman

Holub

King

Angier

Characterizing a practice-based research network: Oregon Rural Practice-Based Research Network (ORPRN) survey tools. J Am Board Fam Med. 2007;20(2):204-219. doi:10.3122/jabfm.2007.02.060140

15.

Carney

Hatch

Stock

, et al. A stepped-wedge cluster randomized trial designed to improve completion of HPV vaccine series and reduce missed opportunities to vaccinate in rural primary care practices. Implement Sci. 2019;14(1):30. doi:10.1186/s13012-019-0871-9

16.

Robison

SG.

Addressing immunization registry population inflation in adolescent immunization rates. Public Health Rep. 2015;130(2):161-166. doi:10.1177/003335491513000209

17.

Hatch

Valenzuela

Darden

, et al. Clinic-level differences in human papillomavirus vaccination rates among rural and urban Oregon primary care clinics. J Rural Health. 2023;39(2):499-507. doi:10.1111/jrh.12724

18.

Harris

Taylor

Minor

, et al. The REDCap consortium: building an international community of software platform partners. J Biomed Inform. 2019;95:103208. doi:10.1016/j.jbi.2019.103208

19.

Palinkas

Horwitz

Green

Wisdom

Duan

Hoagwood

Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Adm Policy Ment Health. 2015;42(5):533-544. doi:10.1007/s10488-013-0528-y

20.

Kleinheksel

Rockich-Winston

Tawfik

Wyatt

TR.

Demystifying content analysis. Am J Pharm Educ. 2020;84(1):7113. doi:10.5688/ajpe7113

21.

Goetz Goldberg

Kuzel

Feng

DeShazo

Love

LE.

EHRs in primary care practices: benefits, challenges, and successful strategies. Am J Manag Care. 2012;18(2):e48-e54.

22.

Ryan

Shih

Winther

Wang

JJ.

Does it get easier to use an EHR? Report from an urban regional extension center. J Gen Intern Med. 2014;29(10):1341-1348. doi:10.1007/s11606-014-2891-0

23.

Welch

Cuellar

Stearns

Bindman

AB.

Proportion of physicians in large group practices continued to grow in 2009-11. Health Aff. 2013;32(9):1659-1666. doi:10.1377/hlthaff.2012.1256

24.

Muhlestein

Smith

NJ.

Physician consolidation: rapid movement from small to large group practices, 2013-15. Health Aff. 2016;35(9):1638-1642. doi:10.1377/hlthaff.2016.0130

25.

Lopes

Almeida

Almada-Lobo

Handling healthcare workforce planning with care: where do we stand?

Hum Resour Health. 2015;13:38. doi:10.1186/s12960-015-0028-0

26.

Thomas

Sagan

Larkin

Cylus

Figueras

Karanikolos

Strengthening health systems resilience: Key concepts and strategies. European Observatory Policy Briefs; 2020.