Sage Journals: Discover world-class research

Abstract

Introduction:

Using a digital process that leverages electronic health records (EHRs) can ease many of the challenges presented by the traditional enrollment process for clinical trials. We tested if automated batch enrollment using a technology-enabled subject recruitment system (TESRS) enhances recruitment while preserving representation of research subjects for the study population in our study setting.

Methods:

An ongoing community-based prospective adult cohort study was used to randomize 600 subjects who were eligible by age and residential address to TESRS (n = 300) and standard mailing method (n = 300), respectively, for 3 months. Then, TESRS was initiated and included automatic identification of patients’ preference for being contacted (online patient portal vs postal mail) from EHRs and automatic sending out of invitation letters followed by completion of a short online survey for checking eligibility and the digital consent process if eligible. We compared (1) median time to consent from invitation sent out per subject and total subjects recruited after a 3-month recruitment period, (2) the estimated study staff’s time, and (3) representation of sociodemographic characteristics (e.g., age, sex, race, SES measured by HOUSES index, and rural residence) between subjects recruited via TESRS and those via traditional mailing methods.

Results:

Median age of randomized subjects (n = 600) was 63 years with 52.0% female and 89.2% non-Hispanic White. Over a 3-month period, results showed consent rate via TESRS was 13% (39/297) similar to 11% (31/295) via standard mailing. However, recruitment was significantly faster with the TESRS approach (median 7 vs 26 days) given the study staff’s effort. Study staff’s time saved by using TESRS compared to standard mailing approach was estimated at 40 min per subject (equivalent to 200 h for 300 subjects). No significant differences in characteristics of research subjects from the study population were found.

Conclusion:

Our study demonstrated the utility of TESRS as a subject recruitment digital technology which significantly enhanced the recruitment effort while reducing the study staff burden of recruitment while maintaining the consistency of characteristics of recruited subjects. The strategy and support for implementing and testing TESRS in other study settings should be considered.

Keywords

technology decentralized trial recruitment batch

Introduction

In the era of electronic health records (EHRs), enrollment of a large number of research subjects in clinical trials or prospective studies can be challenging but can be facilitated by leveraging EHRs. For example, a qualitative study showed most researchers, institutional review board chairs, and primary care physicians considered it acceptable for researchers to contact patients directly.¹ Patient’s preferred communication methods are often documented in EHRs (e.g., online patient portal vs postal mail) along with most of the information that helps determine eligibility for different studies (e.g., demographics, comorbidities). A batch enrollment for a recent digital clinical trial utilized EHRs for the identification of eligible subjects, sorting out their preferred communication methods, and sending out automatic invitation letters to target recruitment.^2,3

Despite the widespread utilization of EHR screening for recruitment purposes, there are ongoing challenges in terms of technical aspects, governance, and regulatory requirements.⁴ The ethical and legal imperatives of the digital consent process have also been raised due to the potential of not fully understanding the information, especially potential risks, harms, or side effects of a treatment.⁵ However, studies with no greater than minimal risk (e.g., observational studies) may reap great benefits for timely subject recruitment by utilizing automated batch enrollment, especially for time-sensitive studies such as COVID-19-related observational studies.^2,3 A recent review on digital tools in the informed consent process showed digital technologies for informed consent were not found to negatively affect any of the outcomes studied, and overall, multimedia tools seem desirable.⁶ However, none of the studies compared and reported the utility of using digital technology tools in informed consent for subject recruitment as primary outcomes by comparing with the traditional approach (e.g., standard mailing). Batch enrollment using digital tools may also introduce selection bias due to difficulty in accessing technology by some patients (e.g., digital screening and consenting).^7,8 This bias may further exclude vulnerable populations (e.g., older people, people from lower socioeconomic status, or rural residents). Herein, we aimed to test the utility of automated batch enrollment (named technology-enabled subject recruitment system [TESRS]) by comparing study staff’s time saved for subject recruitment and representation of sociodemographic characteristics of study subjects who were recruited via TESRS versus the traditional standard mailing approach.

Materials and Methods

Study Setting and Population

According to 2010 U.S. census data, age, sex, and ethnic characteristics of Olmsted County, Minnesota, residents (where 89% of the study subjects lived) were similar to those of the state of Minnesota and the Upper Midwest (19) except for a large proportion of residents working in the health care industry. We enrolled patients paneled in the Mayo Clinic primary care practice, Rochester, Minnesota, who resided in Olmsted and 6 surrounding counties using electronic health records (EHRs), and the details of the study design were previously reported.⁹

Study Design and Cohort

We tested if the TESRS approach helped recruit new subjects of an expanded, ongoing community-based prospective cohort study assessing burden of respiratory infections.⁹ As a proof-of-concept study, for comparison purposes, we randomized 600 subjects who were eligible by age and residential address to TESRS (n = 300) and standard mailing method (n = 300), respectively, during the subject recruitment period (July 27, 2021-October 25, 2021). Randomization was done by SAS (statistical software) and stratified based on a history of confirmed COVID-19 to account for willingness to participate in a research study during the COVID-19 pandemic, not necessarily due to biologic impact of COVID-19.

Technology Enabled Subject Recruitment System (TESRS): A batch enrollment system has been used and reported in a previous study from our institution.² Briefly, after a list of potential subjects was generated by the study team and imported into the EHRs, TESRS was initiated and included automatic identification of patients’ preference for being contacted (online patient portal vs postal mail) from EHRs and automatic sending out of invitation letters followed by a reminder after 2 weeks (see the flow diagram depicted in Figure 1 for the involved systems and their processes). Briefly, interested subjects completed a short online survey through Qualtrics (Provo, UT), and if eligibility criteria were met, their information was interfaced to a Participant Tracking System (PTrax), an in-house clinical trials management system, to carry out the digital consent process with coordinator-assisted methods as an option for patients who encountered difficulty using the electronic enrollment method. To ensure that subjects who digitally consented to this study were properly informed about the study, study staff reviewed the key contents of the informed consent (e.g., study procedures) and verified understanding with the teach-back method while baseline visit was scheduled over the phone. An educational video was sent to all subjects demonstrating this procedure. Characterization of the recruited subjects was available via a Customer Relationship Management (CRM) system which constantly interfaced with the survey and clinical trials management systems to provide a dashboard using Microsoft Power BI to gauge subject’s enrollment status.

Figure 1.

Involved digital systems for TESRS and work and information flow diagram.

Other variables: For characterizing study subjects, demographic variables (age, sex, race/ethnicity) were extracted from EHRs at the time of randomization. Socioeconomic status (SES) was defined by the HOUSES index, a validated, individual-level housing-based SES measure which is based on 4 real property variables of an individual housing unit.^10
-12 Rurality was defined by the Census Bureau’s urban-rural classification for each Census block.¹³ History of COVID-19 prior to invitation was extracted from EHRs as history of COVID-19 may affect participation rate due to health concern or fatigue from numerous invitations to studies related to COVID-19.

Data Analysis: We compared the estimated study staff’s time and representation of sociodemographic characteristics (e.g., age, sex, race, SES measured by HOUSES index, and rural residence) of a sample of recruited subjects for the study population in our study setting between subjects recruited via TESRS and those via traditional mailing methods. We also compared 2 recruitment approaches (TESRS vs standard mailing) regarding median time to consent from invitation sent out per subject and total subjects recruited after a 3-month recruitment period. We measured time to consent “Enrolled” versus “Accrued” because time to accrue (i.e., completion of initial visit for the first procedure) was dependent on the limited slots for the initial study visit due to institutional restrictions for site visits during the COVID-19 pandemic. Categorical variables are presented as count (percentage) and tested with chi-square and when the categorical variables were ordinal in nature, they were tested with Cochran-Armitage trend test. Continuous variables are described with median (interquartile range) and compared with the Wilcoxon rank sum test. All statistical analyses were completed in SAS statistical software (version 9.4M6 Cary, North Carolina), and a P value less than .05 was considered statistically significant.

Results

Summary of characteristics of study subjects

Median age of randomized subjects (n = 600) was 63 years with 52.0% female and 89.2% non-Hispanic White. Supplemental Table 1 summarizes characteristics of study subjects who were invited by TESRS versus a traditional standard mailing approach after randomization. Overall, there was no significant difference in age, race/ethnicity, socioeconomic status defined by HOUSES index, rural status, and history of COVID-19 positivity, except sex (male 52.7% in TESRS vs 44.1% in standard mailing arm; P = 0.036). Three subjects for TESRS and 5 for the standard mailing arm were excluded, respectively, in the subsequent analysis as they were found to have died or moved out of the study setting before or after the letters were sent out.

Study Staff Time Spent for Recruitment

Table 1 shows estimated time for study staff’s effort for subject recruitment process (e.g., organizing and sending out the list of eligible subjects as a batch). Study staff’s time saved by using TESRS compared to standard mailing approach was estimated at 40 min per subject (equivalent to 200 h for 300 subjects). The median time for invited subjects to consent via TESRS was significantly shorter than the traditional approach (median 7 days [interquartile range: 0-14] vs 26 days [interquartile range: 20-36] with standard mailing (P < .001) (see Supplemental Table 2 for details).

Table 1.

Comparisons of Recruitment Effort and Process by Study Staff and Estimated Time Between the Approach Using TESRS and the Standard Mailing Approach.

Recruitment effort under the traditional mailing approach	Estimated time spent per patient	Recruitment effort under TESRS	Estimated time spent per patient
Organize and send recruitment list to Media Service Team^a who verifies the list and sends out invitation letter via traditional postal mail	Done once per batch	Saves recruitment list to a network share for dissemination to study support systems to send invitations (most invites go out electronically, some mailed by Media Service Team)	Done once per batch
When mail arrives, open and sort	5 min	When mail arrives, open and sort (only for those who prefer mail contact)	2-3 min
Coordinator calls patient to pre-screen, inclusion/exclusion	15 min	Automatic (self-prescreening)	N/A
Data entry	10 min	Data entry (only for those who prefer mail contact)^b	2-3 min
Onsite consent process	15 min	Automatic (self-consenting)	N/A
Once consent is received, everything will be the same after this point
Total	45 min		5 min

Some institutions may not have this service, so individual investigative team may need to put together all materials to be sent out to eligible subjects, which will add another layer of coordinators’ time.

For the individuals that signed consent electronically (TESRS), the data were electronically imported to REDCap by our statistics team. This was set up as an automatic process and ran every night. The manual data entry would be needed for those that did not prefer electronic consent.

Representation of a Sample of Recruited Subjects for the Study Population

Subjects who had the preference to be contacted via Mayo Clinic’s online patient portal (TESRS) were more likely to respond to research invitation (i.e., pre-screening rate 28.6%) compared to those who did not (13.6%; P < .001) (Supplemental Table 2), in addition to significantly faster response time with the TESRS approach (median 7 vs 26 days) as described above. The final number of subjects accrued (i.e., the subject completed the initial study visit) after 3 months were similar (32 vs 31) (see Table 2), and characteristics of subjects accrued via TESRS did not differ significantly from those via standard mailing in age, sex, race/ethnicity, SES, and history of COVID-19. There was no significant difference in proportion of rural residents enrolled to our study between the 2 approaches.

Table 2.

Comparison of Sociodemographics of Subjects Accrued* Between TESRS and Standard Mailing.

	Standard mailing (N = 31)	TESRS (N = 32)	P value
Age category, n (%)			.67^a
50-59 years	8 (25.8)	10 (31.3)
60-69 years	13 (41.9)	12 (37.5)
70-79 years	7 (22.6)	8 (25.0)
80 years or above	3 (9.7)	2 (6.3)
Male, n (%)	15 (48.4)	14 (43.8)	.71^b
Race/ethnicity, n (%)			.20^b
African American	0 (0)	0 (0)
Asian	0 (0)	0 (0)
Hispanic or Latino	1 (3.2)	0 (0)
Non-Hispanic White	28 (90.3)	32 (10.0)
Unknown	2 (6.5)	0 (0)
HOUSES quartile^c, n (%)			.90^a
Q1 (lowest SES)	5 (16.1)	4 (12.9)
Q2	3 (9.7)	4 (12.9)
Q3	14 (45.2)	14 (45.2)
Q4	9 (29.0)	9 (29.0)
Living in rural area, n (%)	9 (29.0)	11 (34.4)	.65^b
COVID 19-positive, n (%)	11 (35.5)	10 (31.3)	.72^b

Cochran-Armitage trend test.

Chi-square P value.

HOUSES: Individual-level socioeconomic status (SES) measure (the higher HOUSES, the higher SES).

Subjects that had been accrued after 3 months.

Discussion

To our knowledge, this is the first study that assessed the utility of TESRS for online versus traditional mailing invitation followed by a digital screening and consenting process. Timely subject recruitment is challenging for large clinical trials or observational studies, but it is often crucial for studies addressing time-sensitive outcomes such as COVID-19-related studies. Our study results showed that TESRS enabled us to more efficiently recruit a large number of subjects while keeping representativeness of recruited subjects for the study population compared to the traditional recruitment approach (e.g., standard mailing). While this system may be ideal for digital or fully decentralized clinical trials or observational studies,^2,3 traditional studies requiring onsite visits such as ours also found benefits of TESRS for timely and efficient subject recruitment. Indeed, after reviewing preliminary data 1 month after starting this pilot trial, our team switched our primary subject recruitment process to that using TESRS for recruiting the rest of the subjects (~800) to reach our target sample size.

Study staff’s recruitment effort requires a variety of study-specific, subject-dependent, and labor-intensive tasks. Sometimes, it requires significant additional time for addressing unanticipated challenges for subject recruitment. Table 1 summarized a study staff’s activities during the subject recruitment process, comparing between TESRS and standard mailing approach. If TESRS is applied to a large study, for example, recruiting 4,000 subjects, about 1.0 Full Time Equivalent (FTE) of a study coordinator’s time could be saved with TESRS. TESRS enabled our study team to complete subject recruitment of ~800 within 3 months without hiring additional study coordinators.

Our study has some limitations. First, immediate application of TESRS to other study settings may be uncertain or limited as institutions may not have all digital systems and the interface needed for operating TESRS (see the section of Technology enabled subject recruitment system (TESRS) in the Materials and Methods). Although it may take longer and be costly to set up this system initially, even part of the process (e.g., digital screening) may help reduce effort and time of traditional recruitment processes (e.g., phone screening). Second, digital consenting process may not address individual’s questions before they determine participation, especially for people from different cultural backgrounds. REDCap eConsent framework which is a validated tool to provide a personalized consent experience through a consent document that utilizes several methods including avatars, contextual glossary information supplements, and videos will enhance TESRS if adopted to TESRS, especially for multi-site studies.¹⁴ Third, our study population is predominantly non-Hispanic White raising uncertainty of whether our study findings can be generalizable in other study settings such as those with more ethnic minorities or people with limited English proficiency. However, despite the reported potential disparities in access to digital technology,^8,15,16 our study showed that characteristics of the accrued subjects through TESRS did not differ significantly from those via standard mailing in age, sex, race/ethnicity, SES, rural status, and history of COVID-19. Given this feature of TESRS, it is promising to continue to develop and refine this digital recruitment technology for various studies, and it may be worthy of testing this system in other study settings. Lastly, this exploratory study is based on a relatively small sample size.

Conclusion

Our study demonstrated the utility of TESRS as a subject recruitment digital technology which significantly enhanced the recruitment effort while reducing the study staff burden of recruitment and maintaining the consistency of characteristics of recruited subjects. The strategy and support for implementing and testing the TESRS in other study settings should be considered.

Supplemental Material

sj-docx-1-jpc-10.1177_21501319231194967 – Supplemental material for Application of Innovative Subject Recruitment System for Batch Enrollment: A Pilot Study

Supplemental material, sj-docx-1-jpc-10.1177_21501319231194967 for Application of Innovative Subject Recruitment System for Batch Enrollment: A Pilot Study by Chung-Il Wi, Katherine S. King, Euijung Ryu, Traci L. Natoli, Ryan P. Miller, Matthew J. Spiten, Bijan J. Borah, Paul Y. Takahashi, Xiaoxi Yao, Peter A. Noseworthy, Robert J. Pignolo and Young J. Juhn in Journal of Primary Care & Community Health

Footnotes

Acknowledgements

We greatly thank the staff of the Precision Population Science Lab, Reporting Systems & Solutions, Ambulatory Documentation Systems, Research CTMS, Survey Research Center, Data Delivery Applications, Innovate & Diffuse, and Enterprise Solution Activation & Services of Mayo Clinic. We thank Ms. Kelly Okeson for her administrative assistance for the manuscript preparation.

Credit Author Statement

All authors meet the criteria for authorship based on the following 4 requirements: (1) substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; (2) drafting the work or revising it critically for important intellectual content; (3) final approval of the version to be published; and (4) agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. Specifically, YJ had full access to all the data in the study, takes responsibility for the integrity of the data and the accuracy of the data analysis, and had authority over manuscript preparation and the decision to submit the manuscript for publication. Study concept and design: CW, KK, ER, NT, RP, and YJ. Acquisition, analysis, or interpretation of data: CW, KK, ER, NT, RM, MS, BB, PT, XY, PN, RP, and YJ. Drafting of the manuscript: CW. Critical revision of the manuscript for important intellectual content: all authors.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This collaborative observational study was supported by a research grant from GlaxoSmithKline Biologicals SA.

ORCID iDs

Chung-Il Wi

Young J. Juhn

Supplemental Material

Supplemental material for this article is available online.

References

Beskow

Brelsford

Hammack-Aviran

CM.

EHR phenotyping for research recruitment: researcher, IRB, and physician perspectives on approaches to contacting patients. J Clin Transl Sci. 2021;5:e32. doi:10.1017/cts.2020.524

Yao

Attia

Behnken

, et al. Batch enrollment for an artificial intelligence-guided intervention to lower neurologic events in patients with undiagnosed atrial fibrillation: rationale and design of a digital clinical trial. Am Heart J. 2021;239:73-79. doi:10.1016/j.ahj.2021.05.006

Noseworthy

Attia

Behnken

, et al. Artificial intelligence-guided screening for atrial fibrillation using electrocardiogram during sinus rhythm: a prospective non-randomised interventional trial. Lancet. 2022;400:1206-1212. doi:10.1016/S0140-6736(22)01637-3

O’Brien

Raman

Ellis

, et al. The use of electronic health records for recruitment in clinical trials: a mixed methods analysis of the Harmony Outcomes Electronic Health Record Ancillary Study. Trials. 2021;22:465. doi:ARTN 465 10.1186/s13063-021-05397-0

Tait

Voepel-Lewis

Digital multimedia: a new approach for informed consent?

JAMA. 2015;313:463-464. doi:10.1001/jama.2014.17122

Gesualdo

Daverio

Palazzani

, et al. Digital tools in the informed consent process: a systematic review. BMC Med Ethics. 2021;22:18. doi:10.1186/s12910-021-00585-8

Drew

Nguyen

Steves

, et al. Rapid implementation of mobile technology for real-time epidemiology of COVID-19. Science. 2020;368:1362-1367. doi:10.1126/science.abc0473

Anderson

Racial and ethnic differences in how people use mobile technology. Pew Research Center. 2015. Accessed November 7, 2022. https://www.pewresearch.org/fact-tank/2015/04/30/racial-and-ethnic-differences-in-how-people-use-mobile-technology/

Juhn

Ryu

, et al. Adherence to public health measures mitigates the risk of COVID-19 infection in older adults: a community-based study. Mayo Clin Proc. 2021;96:912-920. doi:10.1016/j.mayocp.2020.12.016

10.

Juhn

Beebe

Finnie

, et al. Development and initial testing of a new socioeconomic status measure based on housing data. J Urban Health. 2011;88:933-944. doi:10.1007/s11524-011-9572-7

11.

St Sauver

Jacobson

, et al. Ethnicity, socioeconomic status, and health disparities in a mixed rural-urban US community-olmsted county, Minnesota. Mayo Clin Proc. 2016;91:612-622. doi:10.1016/j.mayocp.2016.02.011

12.

Bjur

Ryu

, et al. Socioeconomic status, race/ethnicity, and health disparities in children and adolescents in a mixed rural-urban community-olmsted county, Minnesota. Mayo Clin Proc. 2019;94:44-53.doi:10.1016/j.mayocp.2018.06.030

13.

United States Census Bureau. 2010 Urban and Rural Classification, Geography. 2023. Accessed April 23. https://www.census.gov/programs-surveys/geography/guidance/geo-areas/urban-rural.html

14.

Lawrence

Dunkel

McEver

, et al. A REDCap-based model for electronic consent (eConsent): moving toward a more personalized consent. J Clin Transl Sci. 2020;4:345-353. doi:10.1017/cts.2020.30

15.

Vangeepuram

Mayer

Fei

, et al. Smartphone ownership and perspectives on health apps among a vulnerable population in East Harlem, New York. Mhealth. 2018;4:31-31. doi:10.21037/mhealth.2018.07.02

16.

Turner

APaE

. Smartphones Help Blacks, Hispanics Bridge Some – But Not All – Digital Gaps With Whites. Benton Institute for Broadband & Society; 2019.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB