Effort Required and Lessons Learned From Recruiting Health Plans and Rural Primary Care Practices for a Cancer Screening Outreach Study

Abstract

Introduction:

Recruiting organizations (i.e., health plans, health systems, or clinical practices) is important for implementation science, yet limited research explores effective strategies for engaging organizations in pragmatic studies. We explore the effort required to meet recruitment targets for a pragmatic implementation trial, characteristics of engaged and non-engaged clinical practices, and reasons health plans and rural clinical practices chose to participate.

Methods:

We explored recruitment activities and factors associated with organizational enrollment in SMARTER CRC, a randomized pragmatic trial to increase rates of CRC screening in rural populations. We sought to recruit 30 rural primary care practices within participating Medicaid health plans. We tracked recruitment outreach contacts, meeting content, and outcomes using tracking logs. Informed by the Consolidated Framework for Implementation Research, we analyzed interviews, surveys, and publicly available clinical practice data to identify facilitators of participation.

Results:

Overall recruitment activities spanned January 2020 to April 2021. Five of the 9 health plans approached agreed to participate (55%). Three of the health plans chose to operate centrally as 1 site based on network structure, resulting in 3 recruited health plan sites. Of the 101 identified practices, 76 met study eligibility criteria; 51% (n = 39) enrolled. Between recruitment and randomization, 1 practice was excluded, 5 withdrew, and 7 practices were collapsed into 3 sites for randomization purposes based on clinical practice structure, leaving 29 randomized sites. Successful recruitment required iterative outreach across time, with a range of 2 to 17 encounters per clinical practice. Facilitators to recruitment included multi-modal outreach, prior relationships, effective messaging, flexibility, and good timing.

Conclusion:

Recruiting health plans and rural clinical practices was complex and iterative. Leveraging existing relationships and allocating time and resources to engage clinical practices in pragmatic implementation research may facilitate more diverse representation in future trials and generalizability of research findings.

Keywords

primary care recruitment pragmatic trial clinical practice-health plan partnership cancer screening cancer screening outreach

Introduction

Pragmatic trials are an important component of implementation science, yet not much is known about the effort required to recruit a representative sample of organizations into studies.¹ A recent report identified the need to increase representation of diverse populations in research as “urgent” with wide ranging health implications.² In particular, rural populations are historically underrepresented in cancer research. Only about 3% of National Institutes of Health (NIH)-funded cancer research is conducted in rural communities, yet 14% to 19% of the US population lives in rural areas and 1 in 4 Medicaid patients aged 65 years and older is served by a rural clinical practice.^3
-6

NIH investment in rural-focused cancer control research may bridge these funding disparities,⁶ but only if research teams have effective strategies to recruit rural clinical practices and community organizations. Research has shown some facilitators and barriers to engaging clinical practices in pragmatic trials,^7
-12 but less about how much effort it takes to effectively recruit organizations (i.e., health plans or health systems) into this research.^1,13,14 Given the growing focus on engagement of community-based primary care practices in research by the NIH, additional insight into these processes is critical.¹⁵

To address this gap, this manuscript reports on recruitment into SMARTER CRC, a pragmatic clinical trial to increase colorectal cancer (CRC) screening for Medicaid patients served by rural clinical practices (i.e., clinical practices).¹⁶ Building on prior evidence that recruitment is often more challenging than expected,^12,17 our study team sought to quantify the effort required to recruit a representative sample of health plans and clinical practices. In addition, to empirically understand the key factors for successful trial recruitment, we analyzed why health plans and diverse clinical practices chose to participate in a complex interventional study. We hypothesized that to meet recruitment targets for successful randomization into the pragmatic trial, we would need to learn and address the motivations for research participation at more than 1 type of organization (i.e., clinical practices nested within health systems nested within health plans). Further, because of these complex relationships and other constraints (i.e., organizational resources, the pandemic), we hypothesized that recruitment might require more time than prior studies. We include the timeline and activities required for recruitment, characteristics of clinical practices, and reasons why rural health plans and clinical practices participated. We anticipate these findings can inform future efforts to engage rural health care organizations in cancer prevention trials and facilitate more diverse representation in research.

Methods

Study Setting

SMARTER CRC is a pragmatic trial to implement mailed fecal immunochemical testing (FIT) and patient navigation to follow-up colonoscopy.¹⁸ It is a collaboration between the Oregon Rural Practice-based Research Network (ORPRN),¹⁹ Kaiser Permanente Center for Health Research (KPCHR), health plans serving a Medicaid population (i.e., Medicaid health plans), and rural and frontier clinical practices. Building on existing community-academic partnerships,^20
-28 the study used a collaborative health plan-clinical practice implementation model to overcome challenges in implementing mailed FIT programs in rural primary care settings.^21,29 Between 2013 and 2019, CRC screening was an incentivized quality metric for Oregon Medicaid health plans, who received annual performance-based payments for reaching CRC screening targets.³⁰ Study activities were approved by the Oregon Health & Science University Institutional Review Board (STUDY00020681), with a ceding agreement with KPCHR.

Eligibility Criteria: Health Plans and Rural Clinical Practices

At the time of study initiation, 16 regionally based Medicaid health plans were operating in Oregon. Eleven of the 16 health plans met eligibility criteria defined as serving primary care practices located in rural areas defined by Rural-Urban Commuting Area (RUCA)³¹ codes or the Oregon Office of Rural Health.³² Rurality classifications were operationalized as RUCA codes 1 to 3 as metropolitan, RUCA codes 4 to 6 as micropolitan, RUCA codes 7 to 10 as rural areas, or rural area defined by the Oregon Office of Rural Health as “any geographic areas in Oregon ten or more miles from the centroid of a population center of 40,000 people or more.”^31,32 The intervention relied on health plan insurance claims to identify the patient population eligible for CRC screening; this approach was intended to overcome data challenges in implementing population outreach interventions with rural clinical practices.^33,34

We sought to recruit 30 clinical practices within participating health plans meeting the following criteria: (1) rurality as defined above using RUCA³¹ or the Oregon Office of Rural Health definitions,³² (2) more than 30 eligible Medicaid or dual-eligible Medicaid/Medicare patients, and (3) a claims-reported CRC screening rate less than 60% in 2019.

Recruitment and Outreach Protocol

Per the study design, we recruited Medicaid health plans and clinical practices clustered within health plans. Medicaid health plans were approached between January 2020 and April 2021. Although in person recruitment was anticipated, strategies were adapted to virtual engagement modalities to comply with COVID-19 mandates in 2020. Generally, the recruitment team made an initial contact via e-mail to a known contact with a quality improvement focused role (e.g., quality performance coach, quality improvement supervisor, director of quality) at various levels of the health plan to schedule a half-hour virtual or phone meeting with the research team (i.e., project manager and practice facilitator). Following initial contact the research team confirmed interest and obtained a Letter of Support (LOS) or Partnership Agreement Letter (PAL) to complete health plan recruitment.

Clinical practice recruitment occurred between January 2020 through April 2021. The study team developed an initial list of eligible practices in collaboration with each participating health plan. Recruitment outreach varied by lead recruiter, modality (e.g., email, phone, text), intensity, and prior engagement (e.g., “cold call” vs. leveraging existing relationships). In some cases, health plans assisted with practice-level outreach. Study team members conducted regular outreach with eligible clinical practices to answer questions, provide information, and collect LOS or PAL forms. Outreach targeted roles with decision-making authority (i.e., medical directors, quality managers and directors, and clinic managers), or population health champions who could reach decision-makers.

Data Collection and Qualitative Analysis

Here we describe recruited clinical practice and health plan characteristics, and qualitative findings on clinical practice participation. Data sources included research staff recruitment logs (including contact counts and encounter fieldnotes), semi-structured interviews and clinic intake surveys for recruited clinical practices, and publicly available data to identify federal designation and RUCA code for recruited and non-recruited clinics.

Recruitment contact logs

The study team tracked contacts in a recruitment log, including organization, recruitment status, CRC screening rates if known, organization affiliation, contact information, next steps, and contact attempt details. Fieldnotes were taken during recruitment meetings. For the purposes of analysis, 1 recruitment touch was counted as 1 email or text thread regarding the same topic, 1 phone call, or 1 virtual meeting. Descriptive analyses were conducted to summarize, describe, and explore activity patterns.

Two members of the study team (BB, EM) reviewed data from the recruitment contact logs and classified clinical practices into categories. Eligible clinical practices were classified as “Did Not Approach” if the research team did not reach out, “Unable to Engage” if they were non-responsive after several multi-modal outreach attempts, or “Halted” if the research team did not have direct leads for outreach (i.e., a known organizational contact). Clinical practices were categorized as “Declined” when they notified the study team they would not participate, and “Disengaged” when non-responsive after initial engagement. Clinical practices were “Recruited” if they signed a LOS or PAL.

Semi-structured interviews

Semi-structured qualitative interviews were conducted with 5 staff members from the 3 participating health plans and 58 staff members across 27 participating clinical practices at study baseline between January 2021 and June 2021. Purposive sampling was used to identify interviewees, and interviews occurred after recruitment and before intervention implementation. Interviews were recorded, professionally transcribed, and validated by a member of the research team. Interview transcripts were uploaded into ATLAS.ti (Version 9.0) for data management and analysis.

Qualitative data analysis was led by the study qualitative lead (EK) with support from 2 qualitative research assistants (EM, CL), and findings were reviewed with the broader research team. The analysis team used thematic analysis to classify emergent themes by reading and coding transcripts independently, meeting to discuss codes and identify emergent themes, and sharing back query report summaries.³⁵ Once all interviews were coded, analyzed, and summarized, the Consolidated Framework for Implementation Research (CFIR)³⁶ was used to categorize the reasons why health plans and clinical practices chose to participate. After quantitative data were summarized, we re-reviewed qualitative themes to inform final results on recruitment and pull representative case examples.

Results

Effort Required to Meet Study Recruitment Targets

Overall recruitment outreach was extended by 6 months due to external disruptions (e.g., COVID-19, wildfires, dissolved CRC incentive metric). Figure 1 displays the consort diagram related to study recruitment. Nine health plans were approached to participate in study activities. Ultimately, 5 health plans chose to participate and 3 regional divisions chose to operate centrally with 1 point-of contact (noted as collapsed), resulting in 3 recruited health plan sites. Four health plans declined to participate, each with distinct reasons. One was unable to obtain organizational buy-in, 1 was unable to commit due to the demands of COVID-19, 1 lacked the bandwidth, saying “we just don’t think we have the bandwidth nor the interest at the clinics to take on the project,” and 1 did not operate in a manner conducive to the health plan and clinic collaborative model and was therefore a mutual decision between the health plan and study team. Two health plans were not approached because they did not have the capacity to support the intervention at the time.

Figure 1.

Health plan and clinical practice recruitment consort diagram.

A total of 101 clinical practices were identified, however 25 did not meet eligibility criteria (see Figure 1). Pilot clinical practices were excluded because they had already implemented the intervention. Seventy-six clinical practices were deemed eligible for the study and 58 were approached between January 2020 and April 2021. Ultimately, 51% (n = 39) clinical practices enrolled in the study and 49% (n = 37) did not enroll. However, upon obtaining additional practice eligibility data, practices needed to be collapsed or excluded altogether if they had too few patients or could only provide research data as a single site. Another 5 clinical practices withdrew from the study prior to randomization, resulting in the randomization of 29 sites.

Of recruited clinical practices (Table 1), most either did not have any federal designation (38%, n = 15) or were Rural Health Clinics (RHCs) (36%, n = 14). The majority of both recruited clinical practices (67%, n = 26) and non-recruited clinical practices (59%, n = 22) were micropolitan (i.e., had a RUCA code between 4 and 6). Of clinical practices that did not enroll, most (42%, n = 16) either did not have a federal designation or were RHCs (32%, n = 12). Notably, of both recruited and non-recruited practices, only 1 (3%) was designated as a Tribal Health Center; this practice enrolled in the study.

Table 1.

Characteristics for Recruited and Not Recruited Eligible Clinical Practices.

	Health Plan 1 (n = 27 clinics)		Health Plan 2 (n = 6 clinics)		Health Plan 3 (n = 43 clinics)		All (n = 76 clinics)
	Recruited	Not recruited	Recruited	Not recruited	Recruited	Not recruited	Recruited	Not recruited
N clinical practices	16	11	4	2	19	24	39	37
Rurality, n (%)
RUCA 1-3	0 (0)	2 (18)	0 (0)	0 (0)	1 (5)	4 (17)	1 (3)^a	6 (16)
RUCA 4-6	12 (75)	5 (45)	3 (75)	1 (50)	11 (58)	16 (67)	26 (67)	22 (59)
RUCA 7-10	4 (25)	4 (36)	1 (25)	1 (50)	7 (37)	4 (17)	12 (31)	9 (24)
Federal designation, n (%)
Federally Qualified Health Center (FQHC)	3 (19)	3 (27)	3 (75)	0 (0)	3 (16)	6 (25)	9 (23)	9 (24)
Rural Health Clinic (RHC)	9 (56)	4 (36)	0 (0)	0 (0)	5 (26)	8 (33)	14 (36)	12 (32)
Tribal Health Center	1 (6)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	1 (3)	0 (0)
No federal designation	3 (19)	4 (36)	1 (25)	2 (100)	11 (58)	10 (42)	15 (38)	16 (42)
Health Plan Support, Population Data	• Health plan shared study information with networks and provided warm hand-off emails • Population data provided to study team prior to outreach		• Health plan shared study information with networks • Population data • provided to study team prior to outreach		• Health plan shared study information with networks and occasionally provided warm hand-off emails • Population data not provided until after recruitment		—

Percentages may not total 100% due to rounding. Rurality: Metropolitan, RUCA code 1 to 3; Micropolitan, RUCA code 4 to 6; Rural Area, RUCA code 7 to 10.

Clinical practice recruited within the 1 to 3 RUCA code met the Oregon Office of Rural Health definition of rural area.

Touches to clinical practice recruitment

The recruitment period for the 39 clinical practices extended 467 days (from January 17, 2020 through April 28, 2021) and included 177 total recruitment touches. Figure 2 provides a graphic representation of recruitment efforts (e.g., number of recruitment touches, modality, length of recruitment) for the 39 recruited clinical practices by health plan. Seventeen clinical practices were recruited individually (i.e., study staff communicated directly with the clinical practice) while the remaining 22 clinical practices were recruited in 9 groups (i.e., study staff communicated with a single point of contact about recruitment). The median number of recruitment touches for clinical practices recruited individually was 5 (range: 2-17). Clinical practices in groups often required more recruitment touches per point-of-contact (median: 7 touches, range: 3-10).

Figure 2.

Recruitment Effort for Recruited Clinical Practices by Health Plan (N = 39).

Time to clinical practice recruitment

The time to recruitment differed for clinical practices recruited individually versus in groups. We also saw different lengths of time for “active recruitment,” defined as the initiation of outreach through a verbal agreement to participate, versus “full recruitment,” defined as the initiation of outreach through signed participation agreements (e.g., LOS or PAL). The active recruitment window among individual sites was a 69-day median, whereas group recruitment had a 134-day median—nearly twice as long. However, the full recruitment periods showed a smaller difference with a 190-day median for individual practices (range: 5-296 days) and a 209-day median for groups (range: 35-329 days). It was not uncommon to wait months to receive the PAL from a clinical practice after the last recruitment touch: for 7 individual sites and for 3 groups the PAL was submitted more than 3 months after the last recruitment touch. Additionally, groups sometimes submitted a single PAL for all participating sites and sometimes submitted individual PALs. We highlight 2 cases in Table 2 representing the range and complexity of the recruitment effort.

Table 2.

Illustrative Cases of Practice Recruitment Efforts.

Case 1: Simple to recruit. For a clinical practice that was relatively simple to recruit, a clinical practice representative reached out to the research team expressing interest prior to the onset of COVID-19 after becoming aware of the study through KPCHR partners. In June 2020, the research team asked that they sign a letter of support to join the study and within 3 days, the team held an informational meeting and received the signed letter of support. Notably, this partner had worked with ORPRN and KPCHR in the past and we were working directly with the Quality Director. Additionally, this practice had high quality improvement capacity including dedicated staff (clinical informatics specialist and a CRC screening champion) to do this type of work. In this case, the research team knew the decision maker.

Case 2: Challenging to recruit. Other clinical practices required many outreach attempts and follow-up to get agreement and obtain a signed PAL. Multiple modes of outreach were needed over several months and the research team was unsure of who the decision maker was. In attempting to obtain the signed PAL, the practice facilitator was told by 1 practice representative, “[Clinic Quality Assurance Lead] thinks that I’m heading it and I think that (Clinical Project Manager) is heading it. Just because I’m QA (quality assurance) doesn’t mean I can effect clinical change.” This practice has a quality committee, but shared that priorities are mainly set by the Chief Medical Officer; however, it was their executive director that officially agreed to participate.

Reasons for Participation

Table 3 summarizes health plan and clinical practice participation in relation to factors in the outer setting, inner setting, and intervention characteristics. Health plans and clinical practices often differed in their reported reasons for participating. However, for the outer setting, both health plans and clinical practices reported patient needs and resources as a reason for participation. Health plans and clinical practices did not report any of the same inner setting reasons for participation. Regarding intervention characteristics, both reported cost as a reason for participation.

Table 3.

Medicaid Health Plan and Rural Primary Care Clinical Practice Reasons for Participation Utilizing the Consolidated Framework for Implementation Research (CFIR).

CFIR constructs	Health plan reported	Clinical practice reported
Outer setting
Patient Needs and Resources ^a	Recognized the common barriers Medicaid patients experience with CRC and prioritized elimination of those barriers. “We did work really hard quite a few years back when we started our first mail FIT program, to make sure that providers felt comfortable with mail FIT or just FIT kits in general, . . . because they would default to colonoscopies. And I think that was definitely a deterrent for some patients especially in the rural service area also potentially Medicaid patients who don’t have as much time and energy to try and make it to an appointment. They might have a lot of barriers to the process in general of a colonoscopy. And so we wanted to make sure that we were able to really increase access to options.” “The educational piece is really important and one that’s a little bit harder to get across to Medicaid patients. And so that’s something that I really appreciate with the SMARTER CRC project is they’re very focused on the educational component and making sure that the patients are aware of the importance of the screening.”	Expressed a patient-centered mindset and had a strong desire to reach their patients where they are at through alternative CRC screening options. “I think a lot of people are forgetting to do preventative stuff in the midst of a pandemic. And so, we don’t want to lose sight of, not only that quality piece, but also just patient care.” “I think with us being a rural clinic, it really gives our clinic a second option. We didn’t really have another option, and I think just to say, “Hey, here’s another option we have for you” and give my providers another option.”
Networks and Communications	Actively engaging with their social networks and expressed interest in more CRC research and communications. “We have been in conversation through all of our regions for a number of years on how to improve colorectal cancer screening rates. That has been through programs like this, it’s also been in direct collaboration with the American Cancer Society and sitting on some of their work groups and just having conversations on a regional basis of identifying what are others seeing, what are the other plans doing? What are some of the other PCP groups doing? What is the messaging and campaigning from other organizations? It’s one of those things where the more hands on deck, the better, and the more research and communication that we can have, we just want to be a part of that.”	—
Implementation climate	Implemented mailed FIT programs in the past, making them receptive and supportive. “I think that’s been helpful that we’ve had a mail FIT program in the past so a lot of clinics are aware of it and kind of aware of the time commitment on their end too which isn’t super significant. So we had a lot of clinics that were interested and even a handful more that are interested in participating for the next year, which is exciting.”	—
Peer pressure	—	Felt pressure as a rural practice being compared to urban practices, which influenced their participation. “It’s really important to us to not be the best clinic, but to be one that is really out there trying to work with the community, and/or the state to say, hey, we’re going to take our part just as much as this doctor that lives in Portland that sees three times the amount of patients has even bigger staff. But hey, look at us, at least we’re doing our part to make patients better and to help patients.” “We’re in a rural area, we don’t have the resources of a large metropolitan area. I will take any additional help that I can get to help work with the clinic to teach us better ways, better options, just to make us better. And like I said, especially being in a rural area where we don’t have the ability to have mass staff to calculate the data and pull the data and work the data, we’ll take any extra help we can get.”
External policies and incentives	—	Mentioned the need for data support and initiatives to help improve metrics. “I think it’s an important metric for the demographic. We do need to participate with and medically to ensure that their health is monitored in terms of the colorectal screening, and it’s just been something that’s low on the list in terms of being able to get it done because there’s really no incentive from anyone asking for it. So, it’s dealing with a lot of the metrics, things that are incentivized, things that aren’t and those based on health and finding balance between those two.” “It aligned with some of the quality metrics that we already put some effort into for our net and what will soon be our direct contracting initiative.”
Inner setting
Compatibility	Understood the value in screening patients (even without a state incentive metric) and believed that the intervention was an effective way to screen their patients. “We chose to participate primarily because CRC screening still a major focus for us even though it’s not an incentive measure. And we really wanted to be able to carry out and continue a mail FIT program into 2020 and 2021, 2022 since we had to discontinue ours specifically.” “I’m hoping that we can get more people in to be screened from those outlying areas. And I’m hoping that if this is successful and it’s a workflow that we can have in place that they’ll continue to do it. And this is just part of day-to-day business, this is how to get people screened.” “Even though this isn’t a [quality improvement metric] anymore, this is still a population that we feel needs to be looked at and essentially monitored.”	—
Culture	—	Had a prior connection to ORPRN, which influenced their decision to participate in the program, or felt their community needed this program. “I really enjoyed working and learning with ORPRN. ORPRNs management of quality improvement is definitely a learning experience and that’s personally why I was excited about this project.” “I think it’s something that we need in this community. We’re always thinking about different ways of preventative health and the things that we need to do and trying to get patients to be thinking more forward instead of just thinking in the moment right now.”
Learning climate	—	Appreciated and valued the training and education opportunities. “Our medical director liked how this project was set up, especially with being able to provide some education, greetings, and follow-up letters. He was very intrigued on how it could be implemented within our own community.”
Leadership engagement	—	Commitment, involvement, and accountability at the leadership level. “The medical director had identified cancer screenings as being a priority this year, especially with having patients hesitant about coming in during COVID.”
Tension for change	—	Had a need for a change at the practice level, such as needing to relieve colonoscopy waitlists, streamline workflows, and improve staffing capacity. “SMARTER CRC could reduce the burden for colonoscopy.” “We’ve done a lot of improvement, and we’ve tried different things, but nothing seems to stick. Unless we have one team handle it, and they have to do a lot of work on it, and then it takes away from other things.”
Intervention characteristics
Design quality and packaging	Wanted the program’s intervention structure and support to establish a mailed FIT program. “Again, it was really helpful having assistance with that structure because I feel like now that it’s in place, once it gets going I hope it’ll just be second nature to how the [Medicaid health plan] does business.”	—
Cost	Felt the intervention was balanced when considering investment, supply, and opportunity cost. “There was buy-in as long as it was budget neutral.” Continued, “What was really the business reply mail the big thing, because once we got everything, the bids out and all this other kind of stuff. Having the cost to mail out 1900 kits and then have to pay for postage back was going to be. . . We were going to have bow out. The business reply mail is going to make it so that we can continue to participate.”	Emphasized the importance behind cost and benefit and believed that our program struck the right balance between the 2 factors. “There’s things that we need to do for our contract, things that we need to do to make sure that we can do to balance budget, and then there’s things we need to do for patients’ health. All three of those things in a small clinic, cost a lot of money, take all that time and you’re trying to find a balance between all those things.”
Relative advantage	—	Ability to screen patients for CRC even in the midst of the COVID-19 pandemic. “Before the COVID-19 hit, we were slowly working on ideas to reach the public in this area. With COVID-19 now, everything is shut down and we can’t do anything. Screening is very important and if it’s going to be free for the patient, it’s even better.” “Our medical director read information that suggested that we will see an increase in certain types of cancer over the next five years simply because of the delays that have occurred with COVID and primary care. When we were approached about doing the SMARTER CRC, our medical director kind of jumped on it and said, “This is the type of stuff I’m talking about. This is why we need to try to find other means of reaching out to our patients in the event that they don’t come to the clinic, but we still need to make sure we’re screening.””

CFIR Constructs in bold were mentioned by both health plans and clinical practices.

Health plan motivations to participate also included the opportunity to network with similar organizations to improve screening rates (outer setting), the importance of organizational commitment to CRC screening (inner setting), and perceived value of the program (intervention characteristics). Each participating health plan had prior experience with mailed FIT programs and prior research or technical assistance experience with the study team. Clinical practice motivations also included the pressures of meeting preventive care needs as under-resourced rural clinical practices and quality metric reporting requirements (outer setting). Inner setting reasons included a prior relationship with the research team, improving access for patients, seeing value in the offered training, leadership enthusiasm, and the “COVID-19 proof” nature of the intervention (i.e., mailing FITs to patients’ homes could be accomplished during COVID-19 care interruptions).

Discussion

Our research suggests that recruiting rural clinical practices for pragmatic clinical trials requires flexibility to enable success. Due to the multi-level nature of the recruitment process (i.e., recruiting health plans prior to clinical practices), research team time was needed to identify which clinical practices identified by the health plans met recruitment eligibility criteria. Ultimately, 76 clinics were deemed eligible. Consistent with prior research, we found that it is important to build ample time for recruitment as clinical practices have competing priorities.³⁷ Many of the competing priorities were related to COVID-19, but not exclusively, and other factors routinely strain clinician and staff time. For example, prior research found that an additional 13 months was required to meet recruitment targets for an implementation study, even after a generously planned 27-month recruitment period.¹² This was consistent with our results, where a 9-month recruitment period was planned and ultimately 15 months was needed.

The process of recruiting health plans and clinical practices was complex and iterative. We incorporated known strategies to enhance recruitment, including building on prior relationships,^7,8,12 multi-modal outreach methods (i.e., email, phone, and text),⁷ remaining highly flexible,^11,37 and using messaging that resonated with clinical practice needs.¹² Our findings support prior research emphasizing the perceived importance of the research topic,¹¹ an opportunity to improve patient care¹³ and offering a streamlined intervention with limited burden on clinical practices.³⁷

While health plans and clinical practices chose to participate for different reasons, most had prior relationships and a commitment to CRC screening. Despite external disruptions (e.g., COVID-19 and wildfires), clinical practices appreciated the health plan-clinical practice partnered approach to reduce the administrative burden. Notably, even with CRC screening eliminated as a statewide incentivized quality metric, health plans and clinical practices described the importance of screening. Similar to prior research,¹³ participating sites valued the support to improve screening.

CRC research in vulnerable populations is often focused on FQHCs,^13,38
-40 and with only 23% of our enrolled clinics being FQHCs, our findings suggest the importance of including other practice types. This may be especially important for RHCs, as they’re historically underrepresented in research.⁶ Additionally, only 3% (n = 1) of practices were designated Tribal Health Centers.

Our descriptive analysis provides a robust foundation for future research. Future studies could more systematically explore recruitment across heterogeneous rural environments, similarities and differences between rural and urban health plan and clinical practice recruitment, study team relationships with recruited and non-recruited organizations, characteristics of recruited and non-recruited organizations, the level of contact required (i.e., level of study team member conducting outreach to reach organizational decision makers), the relationship between a provider champion and recruitment outcomes, and cost analysis of recruitment. Notably, future studies could actively test comparative outreach strategies by controlling the modality of recruitment or recruitment lead.

Limitations

This study uses observational data to shed light into the complexities of clinical practice recruitment. However, there are a number of important limitations to inform future research. First, while we tracked recruitment activities for all clinical practices approached, we only have primary qualitative and quantitative data from health plans and clinical practices that chose to participate; we did not conduct interviews or surveys with sites that declined. Second, it is possible that not every recruitment interaction or point of contact was documented. While we anticipate any missed contacts did not occur systematically, this may have impacted our ability to quantify recruitment efforts and contribute to limitations due to wide variation and confounding variables not captured in our data. Third, we used publicly available data to identify federal designation and RUCA codes for non-recruited clinical practices after recruitment ended, thus it is possible there were changes in status from the time sites were approached during the recruitment period. Lastly, because this study focused on a rural health intervention, there is not an urban comparison.

Conclusion

Iterative and adaptable strategies were key to recruiting health plans and clinical practices into this pragmatic trial. The need to engage diverse sites into pragmatic research warrants additional research in this area and substantial investment. Our research identifies clinical practice and health plan motivators to participation and demonstrates the facilitators to multi-level, multi-tiered recruitment and the time and effort required to recruit a representative sample into the study.

Footnotes

Acknowledgements

The authors express their gratitude to Mary Patzel, MBA for leading qualitative data collection efforts; Erin Kenzie, PhD for leading qualitative data analysis; Caroline Lawrence for assisting in the analysis of qualitative data; Tiff Weekley, MA for assisting in the compilation of publicly available data for non-recruited clinical practices; and to health plan research partners for sharing information with their networks and providing data for research purposes. This study was conducted as part of the NCI-funded consortium The Accelerating Colorectal Cancer Screening and Follow-up through Implementation Science (ACCSIS) Program. The overall aim of ACCSIS is to conduct multi-site, coordinated, transdisciplinary research to evaluate, and improve colorectal cancer screening processes using implementation science strategies.

Abbreviations

Colorectal Cancer (CRC); Fecal Immunochemical Testing (FIT); Oregon Rural Practice-based Research (ORPRN); Kaiser Permanente Center for Health Research (KPCHR); Screening More Patients for CRC through Adapting and Refining Targeted Evidence-based Interventions in Rural settings (SMARTER CRC); Accelerating Colorectal Cancer Screening and Follow-up through Implementation Science (ACCSIS); National Institutes of Health (NIH).

Availability of Data and Materials

The datasets generated and/or analyzed for this study are available by request. Information to request data can be found at under SMARTER CRC, 2019-2024.

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: From 2021 to 2023, Dr. Coronado served as PI on a contract through the Center for Health Research funded by Guardant Health that assessed adherence to a commercially available blood test for colorectal cancer.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Research reported in this publication was supported by the National Cancer Institute of the National Institutes of Health under Award Numbers UG3CA244298 and UH3CA244298. Dr. Davis’ time was supported in part by a career development award from the National Cancer Institute (1K07CA211971-01A1). Dr. Petrik’s time during manuscript preparation was supported by grant number K12HS026370 from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or Agency for Healthcare Research and Quality. The funders were not involved in the study design, collection, analysis, interpretation of data, the writing of this article or the decision to submit it for publication.

Ethical Approval and Consent to Participate

Study activities were approved by the Oregon Health & Science University Institutional Review Board (STUDY00020681) and was granted a waiver of informed consent, given minimal risks to patients.

Consent for Publication

All authors have received a final version of this manuscript and provide consent for its publication.

ORCID iD

Brittany Badicke

Trial Registration

Registered at clinicaltrial.gov (NCT04890054) and at the NCI’s Clinical Trials Reporting Program (CTRP #: NCI-2021-01032) on May 11, 2021.

References

Solberg

LI.

Recruiting medical groups for research: relationships, reputation, requirements, rewards, reciprocity, resolution, and respect. Implement Sci. 2006;1:25.

National Academies of Sciences E, Medicine, Policy, et al. The National Academies Collection: reports funded by National Institutes of Health. In: Bibbins-Domingo

Helman

, eds. Improving Representation in Clinical Trials and Research: Building Research Equity for Women and Underrepresented Groups. National Academies Press; 2022.

Foutz

Samantha

Garfield

The Role of Medicaid in Rural America. 2017. Accessed October 4, 2022. https://files.kff.org/attachment/Issue-Brief-The-Role-of-Medicaid-in-Rural-America

Cromartie

Dobis

Krumel

McGranahan

Pender

Rural America at a Glance. 2020. Accessed 2022. https://www.ers.usda.gov/webdocs/publications/100089/eib-221.pdf?v=2348.2

Blake

Moss

Gaysynsky

Srinivasan

Croyle

RT.

Making the case for investment in rural cancer control: an analysis of rural cancer incidence, mortality, and funding trends. Cancer Epidemiol Biomarkers Prev. 2017;26(7):992-997.

Kennedy

Vanderpool

Croyle

Srinivasan

An overview of the national cancer institute’s initiatives to accelerate rural cancer control research. Cancer Epidemiol Biomarkers Prev. 2018;27(11):1240-1244.

Ellis

Bertoni

Bonds

, et al. Value of recruitment strategies used in a primary care practice-based trial. Contemp Clin Trials. 2007;28(3):258-267.

Cuthel

Rogers

Daniel

Carroll

Pham-Singer

Shelley

Barriers and facilitators in the recruitment and retention of more than 250 small independent primary care practices for EvidenceNOW. Am J Med Qual. 2020;35(5):388-396.

Messner

Moloney

Warriner

Wright

Foster

Saag

. Understanding practice-based research participation: the differing motivations of engaged vs. Non-engaged clinicians in pragmatic clinical trials. Contemp Clin Trials Commun. 2016;4:136-140.

10.

Loskutova

Smail

Ajayi

Pace

Fox

CH.

Recruiting primary care practices for practice-based research: a case study of a group-randomized study (TRANSLATE CKD) recruitment process. Fam Pract. 2018;35(1):111-116.

11.

Johnston

Liddy

Hogg

Donskov

Russell

Gyorfi-Dyke

Barriers and facilitators to recruitment of physicians and practices for primary care health services research at one centre. BMC Med Res Methodol. 2010;10:109.

12.

Fernald

Jortberg

Hessler

, et al. Recruiting primary care practices for research: reflections and reminders. J Am Board Fam Med. 2018;31(6):947-951.

13.

Coronado

Retecki

Schneider

Taplin

Burdick

Green

BB.

Recruiting community health centers into pragmatic research: findings from STOP CRC. Clin Trials. 2016;13(2):214-222.

14.

Balio

Riley

Grammer

Weathington

Barclay

Jonas

DE.

Barriers to recruiting primary care practices for implementation research during COVID-19: a qualitative study of practice coaches from the Stop Unhealthy (STUN) Alcohol Use Now trial. Implement Res Pract. 2022;3:26334895221094297.

15.

National Institutes of Health OoSC. Program Snapshot. National Institutes of Health, Office of Strategic Coordination Clinical Research in Primary Care Web site. 2024. Accessed May 20, 2024. https://commonfund.nih.gov/clinical-research-primary-care

16.

Coury

Coronado

Currier

, et al. Methods for scaling up an outreach intervention to increase colorectal cancer screening rates in rural areas. Implement Sci Commun. 2024;5(1):6.

17.

Fagnan

Walunas

Parchman

, et al. Engaging primary care practices in studies of improvement: did you budget enough for practice recruitment? Ann Fam Med. 2018;16(1):S72-S79.

18.

Coronado

Leo

Ramsey

, et al. Mailed fecal testing and patient navigation versus usual care to improve rates of colorectal cancer screening and follow-up colonoscopy in rural Medicaid enrollees: a cluster-randomized controlled trial. Implement Sci Commun. 2022;3(1):42.

19.

Davis

Gunn

Kenzie

, et al. Integration of improvement and implementation science in practice-based research networks: a longitudinal, comparative case study. J Gen Intern Med. 2021;36(6):1503-1513.

20.

Coronado

Petrik

Vollmer

, et al. Effectiveness of a mailed colorectal cancer screening outreach program in community health clinics: the STOP CRC cluster randomized clinical trial. JAMA Intern Med. 2018;178(9):1174-1181.

21.

Coronado

Green

West

, et al. Direct-to-member mailed colorectal cancer screening outreach for medicaid and medicare enrollees: implementation and effectiveness outcomes from the BeneFIT study. Cancer. 2020;126(3):540-548.

22.

Coronado

Thompson

Petrik

, et al. Patient-refined messaging for a mailed colorectal cancer screening program: findings from the PROMPT study. J Am Board Fam Med. 2019;32(3):318-328.

23.

Coronado

Johnson

Leo

, et al. Patient randomized trial of a targeted navigation program to improve rates of follow-up colonoscopy in community health centers. Contemp Clin Trials. 2020;89:105920.

24.

Davis

Schneider

Gunn

Rivelli

Vaughn

Coronado

GD.

A qualitative study of patient preferences for prompts and reminders for a direct-mail fecal testing program. Transl Behav Med. 2021;11(2):540-548.

25.

Davis

Freeman

Shannon

, et al. A systematic review of clinic and community intervention to increase fecal testing for colorectal cancer in rural and low-income populations in the United States: how, what and when? BMC Cancer. 2018;18(1):40.

26.

Davis

Nambiar

Mayorga

, et al. Mailed FIT (fecal immunochemical test), navigation or patient reminders? Using microsimulation to inform selection of interventions to increase colorectal cancer screening in Medicaid enrollees. Prev Med. 2019;129s:105836.

27.

Pham

Cross

Fernandez

, et al. “Finding the right FIT”: rural patient preferences for fecal immunochemical test (FIT) characteristics. J Am Board Fam Med. 2017;30(5):632-644.

28.

Coury

Miech

Styer

, et al. What’s the “secret sauce”? How implementation variation affects the success of colorectal cancer screening outreach. Implement Sci Commun. 2021;2(1):5.

29.

Davis

Coury

Larson

, et al. Improving colorectal cancer screening in rural primary care: preliminary effectiveness and implementation of a collaborative mailed fecal immunochemical test pilot. J Rural Health. 2023;39(1):279-290.

30.

Authority

OH.

Oregon Health System Transformation CCO Metrics 2019 Final Report. Oregon Health Authority Office of Health Analytics; 2020.

31.

Agriculture USDo. Rural Urban Commuting Area Codes. Economic Research Service; 2020. Accessed January 5, 2021. https://view.officeapps.live.com/op/view.aspx?src=https%3A%2F%2Fwww.ers.usda.gov%2Fwebdocs%2FDataFiles%2F53241%2FRUCA2010zipcode.xlsx%3Fv%3D8648&wdOrigin=BROWSELINK

32.

Health OOoR. Oregon Office of Rural Health geographic definitions. 2019. Accessed May 10, 2024. https://www.ohsu.edu/oregon-office-of-rural-health/about-rural-and-frontier-data

33.

Cohen

Dorr

Knierim

, et al. Primary care practices’ abilities and challenges in using electronic health record data for quality improvement. Health Aff (Millwood). 2018;37(4):635-643.

34.

Petrik

AFCJ

Hiebert

Badicke

Coronado

Davis

. Data challenges in identifying patients due for colorectal cancer screening in rural clinics J Am Board Fam Med. 2023;36:118-129.

35.

Braun

Clarke

Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77-101.

36.

Damschroder

Aron

Keith

Kirsh

Alexander

Lowery

JC.

Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.

37.

Goodyear-Smith

York

Petousis-Harris

, et al. Recruitment of practices in primary care research: the long and the short of it. Fam Pract. 2009;26(2):128-136.

38.

Thompson

Davis

Leo

, et al. Participatory research to advance colon cancer prevention (PROMPT): study protocol for a pragmatic trial. Contemp Clin Trials. 2018;67:11-15.

39.

Ratnapradipa

Chen

Watanabe-Galloway

Farazi

PA.

Cross-sectional study of colorectal cancer screening barriers in a Latino-serving federally qualified health center. J Cancer Educ. 2023;38(1):193-200.

40.

Leeman

Askelson

, et al. Understanding the processes that Federally Qualified Health Centers use to select and implement colorectal cancer screening interventions: a qualitative study. Transl Behav Med. 2020;10(2):394-403.