Harmonizing Qualitative Data Across Multiple Health Systems to Identify Quality Improvement Interventions: A Methodological Framework Using PROSPR II Cervical Research Center Data as Exemplar

Abstract

Background: Heterogeneity in healthcare systems’ organizational structures, policies and decisions influences practice implementation and care delivery. While quantitative data harmonization has been used to compare outcomes, few have conducted cross-site qualitative inquiry of healthcare delivery; thus, little is known about how to harmonize qualitative data across multiple settings. Objective: We illustrate a methodological approach for a theory-driven qualitative data harmonization process for the PROSPR II Cervical Research Center, a large multi-site, mixedmethods study evaluating cervical cancer screening across three diverse healthcare settings. Methods: We compared three geographically, socio-demographically, and structurally diverse healthcare systems using a multi-modal qualitative data collection strategy. We grounded our sampling strategy in a cervical cancer screening process model, then tailored it for system-specific differences (e.g., clinic staffing structure and individual roles). Data collection tools included domains corresponding to shared research objectives (e.g., abnormal follow-up) while accommodating local context. Analysis drew on operational domains from the screening process model and constructs from the Consolidated Framework for Implementation Research and Normalization Process Theory. Results: Exemplars demonstrate how data harmonization revealed insights suggesting opportunities to improve clinical processes across healthcare systems. Discussion: This analysis advances the application of qualitative methods in implementation science, where assessing context is key to responding to organizational challenges and shaping implementation strategies across multiple health systems. We demonstrate how systematically collecting, analyzing and harmonizing qualitative data elucidates the impact of process factors and accelerates efforts to identify opportunities for quality improvement interventions.

Introduction

Variation in healthcare delivery processes have been widely studied at the patient-level (e.g. knowledge, preferences) and provider-level (e.g., attitudes and beliefs), (MacLaughlin et al., 2019; Perkins et al., 2013; Plourde et al., 2016); however, system-level factors are less well-documented (Kim et al., 2019). Heterogeneity in healthcare systems’ organizational structures, policies, and approach to decision-making influences practice implementation and care delivery. Research consortia, like the Population-based Research to Optimize the Cancer Screening Process (PROSPR), (Barlow et al., 2020; Kamineni et al., 2019; National Cancer Institute Division of Cancer Control and Population Sciences, 2021; UT Southwestern Medical Center, 2021) that aim to understand multi-level variation in the delivery of cancer screening care across health care settings, have helped to establish associations between care delivery stepsand outcomes (Barnes et al., 2018; Martin et al., 2017; Pruitt et al., 2018). Adding qualitative inquiry to the examination of cancer screening at diverse healthcare systems has the potential to produce a more complete understanding of the impact of care delivery and context on cancer outcomes. This helps inform future design of modifications to system-level processes and practical interventions that improve quality of care and outcomes (Lee et al., 2018; Persaud & Nestman, 2006).

The importance of harmonizing quantitative data in population health studies has been extensively discussed (Rolland et al., 2015; Sholle et al., 2017). Benefits include the ability to ask questions for which individual studies are underpowered and to focus on rare outcomes. Similarly, harmonizing qualitative data across heterogeneous settings would enable analyses to be conducted across the combined data set as a whole; this would be valuable for understanding the role of local context and variations in organization and management across multiple health systems and processes (Jenkins et al., 2018). However, less is known about the methodology for harmonizing qualitative data because most studies to date have been conducted in similar healthcare settings (e.g., small community organizations, clinics within a large managed care organization, the Veterans Administration) (Burau et al., 2018; Coronado et al., 2018; Damschroder et al., 2009, 2022; Hamilton et al., 2017). Given uniformity in organizational structure, such studies emphasized adherence to policies and procedures, adoption and integration of interventions to existing practices, and provider decision-making. Few have qualitatively studied clinical care delivery processes across diverse organizational settings, (Burau et al., 2018; Damschroder et al., 2009, 2022; Freeman et al., 2019) or across the cancer care continuum (Anhang Price et al., 2010; Scott, 2012).

Harmonization refers to processes of data integration – from different participants (e.g., individuals in various roles encompassing different responsibilities, decision-making authority, and activities in each setting), clustered in different organizational units (e.g., clinic, hospital, multi-hospital system), across sites. Thus, while preserving the context-relevant details at each site, harmonization requires identification of common data elements across heterogeneous care delivery processes to conduct meaningful analyses across diverse healthcare systems. Harmonization can be used to identify, for example, how organizational leaders decide and enact clinical guidelines, which in turn affect clinical practices. These analyses are then synthesized to identify targets for quality improvement that are potentially transferrable across sites to enhance care delivery in diverse systems.

Cervical cancer screening has become an increasingly complex process (Beaber et al., 2015); organizations must integrate evolving screening and management guidelines (Fontham et al., 2020; Perkins et al., 2020; US Preventive Services Task Force, 2018) that require data on an individual’s age, prior screening history, current screening modality and test results (Kinney et al., 2018). This integration involves multiple steps and interfaces (communication and transfer of responsibilities) across primary and gynecology care to provide care for women across the life course (Anhang Price et al., 2010; Beaber et al., 2015) and management commensurate with individual cancer risk (Kinney et al., 2018; Loomans-Kropp & Umar, 2019; Marcus et al., 2016). Because the effectiveness of screening relies on identifying and managing abnormal findings, (Anhang Price et al., 2010; Taplin et al., 2012; Zapka et al., 2010) differences in health insurer payer mix (e.g., use of public payer programs to cover the uninsured) may further complicate a healthcare system’s policy decisions on the delivery of screening services and management of abnormal results (Breen et al., 2019).

In this article, we illustrate a methodologic approach guiding qualitative data harmonization for the PROSPR Cervical Research Center, a large multi-site, mixed methods study evaluating cervical cancer screening care delivery across three diverse healthcare systems. We attend to both the context and the broader applicability of findings, and offer data sampling, collection, and analytic tools and techniques for strengthening qualitative claims. By sharing our framework, we advance use of qualitative methodology in implementation science, (National Cancer Institute, 2018) where assessing the role of context is key to responding to organizational challenges and shaping implementation strategies across multiple healthcare systems (Harrison & Grantham, 2018; Nilsen & Bernhardsson, 2019). We illustrate how harmonized data enables the production of aggregate findings and site-specific insights to inform system-level quality improvement across diverse healthcare systems.

Methods

Healthcare Settings

The NCI-funded PROSPR II Cervical Research Center – Multilevel Optimization of the Cervical Cancer Screening Process in Diverse Settings & Populations (METRICS) – is designed to elucidate multi-level factors that hamper or facilitate the cervical cancer screening process across multiple healthcare systems. We collected and analyzed data from three geographically, sociodemographically, and structurally diverse healthcare systems (see Table 1): A mixed-model healthcare system providing coverage and care in Washington State; a vertically-integrated county safety-net healthcare system in Dallas, Texas; and an integrated healthcare system with primary care provider networks and their affiliated tertiary care hospitals accepting primarily commercial insurance in Massachusetts. Based on previous work, (Tosteson et al., 2016) we hypothesized heterogeneity in organizational structures, decision-making processes, and clinical practices across the three healthcare systems.

Table 1.

Healthcare System Characteristics of METRICS PROSPR II Cervical Research Center, 2010–2019.

Characteristics	Parkland-UTSW (Parkland)	Kaiser Permanente Washington (KPWA)	Mass General Brigham (MGB)
Geographic region	Dallas County, Texas	Washington State	Eastern Massachusetts
System type	Integrated county safety-net healthcare system owns and operates hospital and outpatient clinics (primary care, women’s health, and specialty)	Mixed-model healthcare system with diverse insurance plans, a large integrated group practice and contracted network of hospitals and providers	Integrated healthcare system of general and specialty hospitals, physicians organizations and primary care provider networks
Hospitals	1	1	6
Outpatient Care Centers a	26	49	8
Providers (Physician/APP) b	2965/430	2151/722	10,392/1185
Electronic health record	Epic®
Pathology system	Cerner Millennium® transition to Epic Beaker in Sep 2019	Progeny®	PowerPath® transition to Co-Path® (select sites)
HPV Assays used
High-risk (hr) HPV DNA (*genotyping data)	Cervista HPV HR (2012-19)	*Aptima HPV + reflex 16/18/45 (2018–19)	*Aptima HPV + reflex 16/18/45 (2017-19 select sites) *BD Onclarity + reflex 16/18/45 (2018–19 select sites)
Screening Strategies c
Pap with reflex HPV testing	2004-19 (Primary Care)2011-19 (Wome’s health)	2006–19	2004–19
Co-testing*	Option for ages 30-65 (2004-present)	Option for ages 30-65 (2013-19)	Option (2004-present)
ages order	*Policies encourage for symptomatic or under-screened (no Pap history) women in primary care clinics		*Policies suggest use for 30-65 but providers can for any woman regardless of age
Screening programs	Eligible women are offered ing clinic visits. EHR has automated alert reminding providers to ascertain screening status and offer screening if due/overdue.	“Birthday letter” reminders to eligible women sent by population health management team. Telephone outreach by primary care if overdue. EHR has automated alert reminding providers to offer screening if overdue.	Eligible women are offered screlinic visits. EHR has automated alert reminding providers to ascertain screening status and offer screening if due/overdue.
Management of abnormal screening results	Primary care or OB/GYN providers refer patients to a centralized dysplasia clinic, which also reviews system-wide abnormal results as a fail-safe	Primary care or OB/GYN provider teams, who performed screening, manage referrals and scheduling	Primary care or OB/GYN provider teams, who performed screening, manage referrals and scheduling

^aAs of 2019. Outpatient care centers may have multiple clinics or medical offices onsite as well as a laboratory department and a radiology facility.

^bIncluding MD/DO fellows and residents, nurse practitioners and physician’s assistants providing primary care, specialty care (i.e., cervical procedure), or HPV vaccination services to at least one cohort member from 2010-19.

^cSites differed in policies regarding screening strategies, particularly whether policies endorsed a particular screening modality. For example, MGB supported clinician discretion regarding Pap with reflex HPV versus co-testing; KPWA offered Pap only with reflex HPV until 2012, then added a co-testing option in 2013; PHHS has preferred Pap only with reflex HPV for average-risk women during the entire period and supports clinician selection of co-testing for under-screened women.

Sampling

We executed a multimodal qualitative data collection strategy consisting of participant observation, semi-structured interviews, and document analysis. The first step in the data harmonization process was to develop a data dictionary to define stakeholders and identify a common sampling strategy of target interview and observation participants. To promote consistency across healthcare systems, we grounded our sampling strategy in Barlow et al.’s (2020) cancer screening process model (Figure 1) (Barlow et al., 2020). Accordingly, we recruited participants from roles across the multiple phases of the clinical care continuum (e.g., detection, diagnosis, treatment), locations delivering each care step (e.g., primary care, dysplasia or gynecology clinics, pathology), specific duties and roles (see Table 2). Data harmonization requires a balance between uniformity and flexibility to accommodate heterogeneity in organizational structures and staffing across sites and must recognize variation of staff roles and responsibilities. For example, selection of interview and observation targets was tailored, in part, on whether particular care delivery policies and steps were managed by a centralized team (e.g., population health) versus local clinics, or physicians versus advanced practice providers (APPs). Thus, during a 3-day in-person conference and subsequent biweekly videoconferencing meetings, members of each site collaboratively developed a strategy to ensure that participants were sampled according to comparable duties/roles in the screening process model, regardless of their unique physical location at each site.

Figure 1.

Cancer screening process model, adapted from Barlow, et al. (2020). Footnotes: * Organ-specific screening modalities—Breast: mammography; Cervical: Pap or Pap/HPV (co-test); Colorectal: gFOBT/FIT, sigmoidoscopy, colonoscopy. † For cervical and colorectal detected abnormalities, excisional treatment may precede surveillance. ‡ Depends on cancer type and screening modality.

Table 2.

Sampling Strategy of Exemplar Roles by Screening Process Phase, Location, and Duties.

Screening Process Model Phase	Potential Locations	Description of Duties	Exemplar Roles Involved at Sites
Risk assessment, detection to diagnosis	Primary Care Clinics	Screening and triage to specialty care	Director of Primary Care (administrator), Primary care provider, Charge nurse, Case Manager
Detection to diagnosis	Specialty Clinic: Dysplasia or OB/GYN	Follow-up to abnormal results	Clinic Director or other administrator, Specialty care provider, Nurse coordinator/navigator
Diagnosis	Pathology	Evaluation of cervical cytology and histopathology specimens, generates reports, coordinates with physicians	Pathologist, Cytologist, Pathology Technician
Treatment	Specialty Clinics: Dysplasia or OB/GYN	Treatment procedures, surveillance	Specialty care provider, Case Manager, Nurse coordinator/navigator
Across phases	Administrative offices	Designing and revising EMR/IT communication and reporting functions	Chief information Officer, Epic Technician

Based on the sampling strategy described in Table 2, and with the understanding that each system could further diversify data collection according to their unique organizational structures, we collected observation and interview data from a range of roles (see Table 3). For example, at the Massachusetts healthcare system, unique targets included leaders involved in the screening process workflow design, providers with potential insight into workflow challenges, and providers representing hospital-based clinics and community health centers. At the Texas system, researchers prioritized sampling from a dysplasia specialty clinic within the OB/GYN department, which centralizes abnormal screening referrals from primary care and women’s health clinics, and where providers include attendings, trainees, and advanced practice providers (e.g., nurse practitioners). At the Washington healthcare system, researchers purposefully sampled members of an organizational team responsible for oversight of cervical cancer screening guidelines. They also purposefully sampled providers from different service areas within western Washington to gain broader perspective of varying clinic cultures and patient populations.

Table 3.

Multi-Modal Data Collected by Role and Healthcare System.

Observation and Interview Data Collected						Documents Collected
System	Providers	Ancillary Staff	Administrative Leaders	EHR/IT Professionals	Pathologists/Cytologists
A	13	1	3	0	2	42
B	14	5	3	1	5	32
C	19	7	4	4	3	44
Totals	45	14	10	5	10	118

Within our multimodal data collection strategy, observations generally preceded interviews in order to first obtain an initial understanding of organizational processes, key players, and roles. Individuals in leadership positions were purposefully selected for interviews to provide an overview of clinic functions and “big picture” concerns. Other individuals were selected for interviews based on their role as identified through observation or on the recommendation of the administrative leader. Documents were requested when they were identified during observation and interviews as being relevant to communication and documentation processes, e.g., letters from providers/clinics to patients; screen shots of EMR functions; and clinic policies and procedures. Data collectors at each healthcare system were responsible for identifying the most appropriate data collection modality and target participants based on the unique qualities of their organization.

Data Collection

We co-developed tools to structure data collection. Semi-Structured interview guides included questions informed by the overall research objectives: to better understand how clinicians understood and managed abnormal cervical pathology results; how they managed populations with elevated risk (e.g., women who are immunocompromised) or altered risk (e.g., women who are pregnant); and how they used their EHR systems, created workarounds, or identified gaps in care delivery. Observation guides directed the observer to assess potential areas of difference across healthcare systems, such as workflow patterns, communication among stakeholders, and use of the EHR. Document collection guides targeted key domains across systems, such as tailored guidelines for screening and management of abnormal results, scheduling forms and procedures, lab order forms and procedures, EHR processes, patient communication tools, and provider training documents.

As with the sampling strategy, master guides were then adjusted based on system-specific characteristics and individual informants’ responsibilities. For example, within the Texas system, the centralized dysplasia clinic within the OB/GYN department performed most of the follow-up procedures and surveillance tests for patients with abnormal screening results while at the Massachusetts system, some individual primary care practices and community health centers managed abnormal results locally. These local adjustments to the interview guides provided flexibility to also identify influential individuals who functioned as champions in their local settings and who may not be in leadership roles. Data were collected by local teams of qualitatively trained researchers from diverse disciplinary backgrounds (e.g., anthropology, psychology, medicine). Data collection strategically encompassed the entire cervical cancer screening continuum in each health system while also capturing variation at individual (e.g., roles and responsibilities) and system (e.g., centralized vs. de-centralized) levels. We obtained verbal informed consent from participants in observation and interviews in accordance with human subjects protocols approved by the Institutional Review Boards at each institution [UT Southwestern Medical Center #STU 2019-0979, Kaiser Permanente Washington Research Institute #1480422, and Mass General Brigham IRB #2019P002451].

Data Analysis

The research teams across the three healthcare systems shared data as it was collected through an online portal. Members from all three teams met in biweekly videoconferencing calls to assess appropriateness of data collection guides, identify gaps in sampling to adjust recruitment, and review findings to note commonalities and differences across systems. As such, data collection and analysis proceeded iteratively in a grounded theory approach, with intra-system development and inter-system comparisons continually informing data collection strategies, suggesting new theories of interpretation.

Systematic thematic coding commenced approximately half-way through the data collection process, with one healthcare system serving as the analysis hub based on capacity and expertise. Team members at the hub first reviewed approximately 60% of existing interview transcripts, interview field notes, and observation field notes to develop a codebook consisting of nodes and definitions corresponding to the cervical cancer screening process model and operational domains addressed in the interview guides. Coding also drew from the Consolidated Framework for Implementation Research (CFIR) (Damschroder et al., 2009) constructs to reflect internal and external organizational motivators, and from the Normalization Process Theory (May et al., 2018; McNaughton et al., 2020) (NPT) constructs to reflect actors’collective awareness and actions related to the screening delivery process. Integrating CFIR and NPT provides a more nuanced understanding of interactions among actors, processes, and contexts that can support the development of quality improvement strategies within unique settings (Schroeder et al., 2022). The analysis hub circulated the codebook draft to solicit feedback from data collection team members at the other health systems.

Next, three members of the analysis hub jointly reviewed and coded 15% of the transcripts, refining code definitions to finalize the codebook. Finally, using an alternating matrix for coder assignments, members of the analysis hub double-coded remaining transcripts and collected documents, meeting weekly to resolve discrepancies by consensus with a senior team member included when necessary to achieve clarity. Table 4 demonstrates how the thematic codes, screening process model, and CFIR and NPT constructs were applied to each type of data source.

Table 4.

Coding Exemplar By Data Source, Theme, and Analytic Constructs.

Data Type and Source	Excerpt	Screening Process Model Phase	Thematic Code	CFIR a Construct	NPT b Construct
Document: Cervical Cancer screening guidelines [System A]	“For the management of Pap and HPV test results and follow-up colposcopy results, the recommendations of the 2012 ASCCP Consensus guidelines Conference (Massad et al., 2013) have been adopted.”	Detection	Guidelines	Outer setting	Collective action
Interview transcript: EHR/IT (quality and safety specialist) [System B]	Interviewer: Where do you realistically think cervical cancer prevention ranks as an organizational priority?Participant: It is kind of left up to the discretion of the institutions to figure out what they want to focus on. I mean, at a [health system] level I think it’s certainly a priority in the sense that it's in the [list of internal performance metric targets] and they are looking at this every year to see what the performance is.”	Detection	Quality improve-ment	Inner setting	Reflexive monitoring
Observation: Ancillary staff (Case Manager) [System C]	The report…was a monthly compilation of all cytology results. This report acts as a double checking process to make sure no patient was accidently overlooked”	Detection	Pathology	Inner setting	Collective action

^aConsolidated Framework for Implementation Research (CFIR).

^bNormalization Process Theory (NPT).

A non-coder member of the analysis hub conducted quality review checks on 10% of the sample to ensure consistency and enhance rigor. Members of the data collection team met monthly with the analysis hub to review and clarify any data that were unclear or could benefit from additional contextual understanding. Once all documents had been coded, members of the analysis hub created summary reports for each code to synthesize findings and extract representative quotes for major themes. Team members from all healthcare systems reviewed and discussed summary reports. Data management and analyses were facilitated by NVivo 12.0 (QSR Australia).

Finally, we harmonized data by comparing findings that were coded under the same thematic code (e.g., abnormal follow-up) that also shared at least one CFIR or NPT construct (e.g., intervention characteristics). Harmonized data could originate from any data source type (i.e., observation, interview, document) and involve any staff role (i.e., primary care provider, pathologist, specialist).

Results

As mentioned in Table 3, across the three healthcare systems we collected a total of 118 documents, and observed and conducted 84 total interviews with providers, ancillary staff, administrative leaders, EHR/IT professionals, pathologists, and cytologists. Findings are based on our analysis of these 202 total data sources, including interview transcripts, fieldnotes, and documents.

As a result of the harmonization process, we identified similarities and differences across health systems at various points in the screening process model. These areas indicate where process inefficiencies or opportunities for potential adaptation or improvement at one system might be tested in other health systems to enhance patient outcomes. For example, at both Systems B and C, processing of abnormal results was centralized in the pathology department; however, communication of abnormal results differed. At System B, results were communicated individually from pathology to ordering providers, whereas at System C, pathology sent weekly aggregrated reports to centralized dysplasia clerks who then communicated abnormal diagnoses to patients and scheduled follow-up appointments; ordering providers were notified by EHR inbox. The process exemplified at System C presented a potential process improvement opportunity for the other healthcare systems.

We note other findings in Table 5, where we have highlighted two coded exemplars of health system differences and the harmonization insights that were produced. The first example describes various quality improvement techniques to enhance follow-up of abnormal screening outcomes, including: use of an EHR dashboard, education via patient navigators, and a standardized ordering process between providers and pathology. In the second example, harmonization yielded insights regarding methods for disseminating guidelines through virtual consults, direct communications between pathologists and providers, and via continuing medical education curricula.

Table 5.

Harmonization Insights Across Health Systems by Thematic Codes and Constructs.

Example 1: Hamonization insight:Systems used different quality improvement techniques to improve follow-up of abnormal screening results. Techniques included use of an EHR dashboard for provider- and clinic-level monitoring of screening outcomes (System A), enhanced education via a patient navigator program during abnormal follow-up (System B), and creation of systematic EHR-based abnormal follow-up process between providers and pathology (System C).			Theory-Based constructs:Screening process model phase: DetectionTopical theme: abnormal follow-up CFIR: Intervention characteristicsNPTb: Reflexive monitoring Site A) Collective action (Sites B & C)
System A interview: Administrator (Quality improvement specialist)		“We have a dashboard on [name] that’s broken down from the big division level all the way down to the individual provider level, providing that data is showing how well we’re doing in certain measures, one of those measures being cervical cancer screening. So we really look at that data to determine whether our processes are reliable, to identify gaps of why aren’t we screening at least – I forget the target of it now, 70 to 80% of our women for cervical cancer screening -- what’s occurring? Is it that we’re not doing it? Or there’s a data issue? Or is it that this care is being done outside and we’re not getting to the system? So that’s kind of what our data reports look at.”
System B interview: Provider (OB-GYN)		“We’ve also started a patient navigator program, which the Pap smear evaluation center has had. And so we’re implementing that patient navigator program at my clinic. […]We met with the patient navigator at [Clinic], and so we’re starting to incorporate some education into our colpo appointments just to try to address the patients' concerns and use it as an education point.”
System C interview: Pathology (Cytology Supervisor)		“I think it’s really helped patient care when it comes to having the providers placing an actual order in EPIC which then translated into EPIC Beaker to where it’s all one system now, when before with Cerner and EPIC there could be something lost in translation or there wasn’t that electronic component by the providers. So now instead of the providers putting it on a piece of paper, which as we know piece of papers can be lost, dropped, whatever or could be mismatched by providers as well by their clinic. You could have a requisition of one patient be put into a Pap sample of another patient with the specimen. And so that has been eliminated with that taking away that paper.”
Example 2: Hamonization insight: Processes for disseminating guideline and policy changes occurred through communication channels: Virtual consults for abnormal follow-up care (System A), via pathologists’ reports and communications with clinicians for abnormal follow-up (System B), and through continuing medical education (System C).			Theory-based constructs:Screening process model phase: Detection to diagnosisTopical theme: GuidelinesCFIR: Inner settingNPT b : Coherence
System A interview: Provider (OB/Gyn)	Family practice also does virtual consults with us when there is some sort of abnormal Pap or follow-up testing they’ve done that’s abnormal or not. They’re able to send us virtual consults to ask what the next steps are for a patient because sometimes either it’s unclear to them, it’s not your typical black-and-white situation that they plug into the ASCCP algorithm or they are not aware of the ASCCP algorithm. It’s kind of a variance of all three.
System B interview: Pathologist	The other thing that pathologists do… I would talk to the clinicians on a periodic basis about making sure they were up to date on the latest screening guidelines and follow-up guidelines, so that they had algorithms to follow in their practices. And this was particularly important for all the nurse practitioners who might be more interested in sort of an algorithmic approach to management. So that if, for instance, they got a diagnosis of low-grade squamous intraepithelial lesion, what should they do about it?...So the pathologist’s base role is primarily in interpreting a Pap test and correlating that with an HPV test, sending out a combined report that goes to the clinician that should put them in a bucket. And by bucket, I mean, one of the columns for what the clinician can do in terms of further screening or further interventions or anything else that they need to do based on that report.
System C interview: Provider (Community Clinic Primary Care Provider)	Interviewer: “How do you learn about what are the appropriate guidelines to follow? So who communicates that do you, what the policies or guidelines are?Participant: CME…We used to have a GYN presentation in quality leadership meeting, but that was many years ago. We haven’t had any lately. But just by doing CME of regular screening, you know, we get updates.

^aConsolidated Framework for Implementation Research (CFIR).

^bNormalization Process Theory (NPT).

Discussion

In this article, we provided selected excerpts to illustrate how we harmonized qualitative data sampling, collection, and analysis across three diverse healthcare systems using cervical cancer screening care as an exemplar care delivery process. To enable systematic comparison, a consistent sampling strategy aligned participant roles with process model phases from Barlow et al.’s cancer screening model. The shared conceptual model facilitated discussion among team members and allowed us to identify commonalities and differences in the care continuum across systems while offering flexibility to accommodate local contextual variation in actual decision-making authority and responsibilities of individuals across the continuum. Interviewing and observing healthcare providers in diverse roles provided insight into variations in processes within healthcare systems that identify opportunities for potential intervention (Weiner et al., 2012).

Using the cervical cancer screening process as an exemplar, we demonstrate how qualitative data harmonization is an underutilized tool that can greatly enhance multi-site system comparisons to highlight opportunities for cross-system learning and quality improvement. Identified commonalities based on aggregate findings from multiple healthcare systems increases relevance and potential feasibility that may motivate stakeholders for planned initiatives. As opportunities for future interventions are identified, implementation strategies can be developed to leverage local variation (Bracher & May, 2019; Hawe et al., 2009). Knowledge of local context across multiple system settings, then, should lead to greater feasibility for dissemination to diverse systems (Gold et al., 2016).

Our methodology posed several challenges in its operationalization. The COVID-19 pandemic disrupted our data collection efforts, specifically hindering observational research as the healthcare systems reduced services and limited access. As a result, we adjusted our strategy and emphasized document collection to reduce response burden on clinical partners who necessarily prioritized COVID-related clinical responsibilities. Additionally, interview data reflect processes before and during COVID-19 as systems responded to the pandemic by modifying clinic workflows (e.g., prioritizing or delaying procedures, increasing telehealth appointments). Analytically, multi-sited primary data collection is resource intensive, requiring local experts at each study site and strong communication to manage and coordinate qualitative data collection and analysis. Social distancing prevented the study team from meeting in-person and the impact of relying solely on virtual communications to harmonize our data across sites is not clear.

Understanding the impact of multi-level factors on care delivery processes and outcomes is a growing area of research. By examining cervical cancer screening and follow-up processes and the different forms in which care can be delivered, findings from harmonized qualitative data can produce greater insight into harmonized quantitative data by identifying potential factors affecting cervical cancer screening utilization and outcomes across settings (Hawe et al., 2004). Our approach also has implications for screening and management of abnormal results in other preventive services. Systematically collecting and harmonizing qualitative data from stakeholders at each step in the screening continuum elucidates the impact of process factors and may accelerate efforts to identify potential interventions with greater likelihood of adoption.

By illustrating how we have harmonized qualitative data from three different healthcare systems, we hope this description enables others to apply our theory-driven methodologic framework to further advance implementation science in other disease contexts. In the United States, healthcare settings are quite diverse in their organizational structures, resources, and challenges; better understanding of their heterogeneity can shape selection of implementation strategies and better predict what adaptations to strategies would be needed across different health systems. As a whole, more rigorous qualitative investigation should better elucidate the impact of process factors and accelerate efforts to identify intervention opportunities to improve healthcare delivery.