Abstract
Background
The consequences of health disparities in underrepresented populations persist with increased disease burden and reduced access to care. Even with inclusion mandates, underserved populations are poorly represented across trials. This article describes recruitment and retention efforts of an underrepresented population in a large NIH-funded trial.
Methods
The BackInAction (BIA) study is a pragmatic, multi-site, three-arm, parallel-groups randomized controlled trial testing the effectiveness of acupuncture needling for reducing back pain-related disability among 800 older adults (≥65 years) with chronic low back pain. The Institute for Family Health (IFH), an FQHC in New York City, one of four BIA sites, provides primary care to largely underrepresented patients. The IFH recruitment goal was 123 participants. PCPs were oriented to trial referral, clinical research coordinators worked as navigators, and electronic health records (EHR) mechanisms were adapted to allow seamless communication between trial acupuncturists and the research team.
Results
IFH met its goal of 123 trial participants with sociodemographic (22.8% ≥ 75 yrs of age, 72.4% female, 59.4% reported having at least some college education, 62.6% reported an annual household income of less than $25 000) and ethnic/racial diversity (39.0% Hispanic, 35.6% Black, 22.0% White non-Hispanic, 26.8% Spanish-speaking). IFH study withdrawal rate was 12.2% with 18.7% missingness in follow-up data rates at the trial’s 6-month primary endpoint.
Conclusion
The IFH site team successfully recruited and retained diverse participants through trusted connections with the study population, building on experience with acupuncture research, engaging PCPs, study team members, primary care clinical sites and EHR communication options.
Background
The National Institutes for Health (NIH) has designated populations that experience health disparities to include racial and ethnic minority groups, people with lower socioeconomic status (SES), underserved rural communities, sexual and gender minority groups and people with disabilities. 1 The consequences of health disparities in these underserved populations persist: more serious chronic health problems, less access to quality healthcare, health effects of structural racism and discrimination, and differences in the incidence, prevalence, and burden of disease. 2 Involvement of ‘women and minority groups’ in clinical research was required by the NIH Revitalization Act of 1993, 3 and remains a federal policy priority.4,5 Yet, even with inclusion mandates and increased funding, underserved populations lack representation in mainstream clinical research.4,6 The inclusion of ethnically diverse participants is also poorly reported across trials. “When reported, the relative proportions of diverse groups (e.g. Black, Asian, Hispanic/Latino participants) are under-represented…” with White participants making up between 70% and 95% of most samples.7,8 “Ethnic disparities are grounded in the assumption that health-care tailored to the needs of White ethnic groups will be sufficient for all in society… results from studies on White ethnic groups are used as a proxy for all, which leads directly to poorer health outcomes in ethnic minorities.” 9 This limits our understanding of factors that contribute to disparities within and across groups and remains a critical challenge to the nation’s health.6,10
Poor recruitment and retention contributes to persistent underrepresentation of historically marginalized groups.3,11 Despite considerable research on recruitment and retention strategies,12-14 innovative and tailored methods are essential, especially to the success of large, multi-site collaborations involving minoritized populations.15,16 Inclusion of a diverse and representative sample in trials, including integrative medicine trials, means study findings are more likely to be generalizable to all. 17 This article describes the insights gained through the processes, achievements, and efforts required to optimize recruitment and retention for the BackInAction (BIA) trial in this underrepresented population.
Context
Acupuncture therapy is an evidence-based nonpharmacologic intervention effective for chronic low back pain (cLBP)18-20 and recommended as one first-line treatment option for acute, subacute and cLBP by the American College of Physicians (ACP) 21 and the American Academy of Family Physicians. 22 However, access has been limited in low-income and ethnically diverse communities. Common barriers include financial obstacles such as lack of insurance coverage, logistical barriers including limited neighborhood accessibility, limited awareness on the benefits of acupuncture therapy, and general mistrust.23-25 Moreover, there is an added barrier in the federally qualified healthcare center (FQHC) setting, where services like acupuncture, physical therapy, or massage can only be billed on a day a particular patient also has a primary care visit. Essential follow-up visits cannot be provided as it would burden the primary care schedule.
While research on acupuncture for cLBP has not excluded older adults, no trial has focused specifically on older adults with cLBP. On January 21, 2020, the Centers for Medicare & Medicaid Services (CMS) announced a decision to cover acupuncture ‘needling’ for cLBP only in Medicare recipients, with limits on number of visits per year and stipulations on who may provide and directly bill for acupuncture care. 26 Namely, acupuncture is covered if provided by a physician or by a physician assistant, nurse practitioner/clinical nurse specialist or auxiliary personnel if they have a master or doctoral degree from an accredited acupuncture school and are licensed to practice acupuncture in their state. Acupuncture can only be billed by a qualified Medicare billing provider which precludes almost all licensed acupuncturists, the primary acupuncture provider group in the U.S. 27 CMS also partnered with the National Institutes of Health (NIH) to issue a ‘Funding Opportunity Announcement’ (RFA-AT-19-005) to conduct a pragmatic randomized controlled trial (PRCT) of acupuncture needling for cLBP in older adults.
The BIA trial was designed to address the evidence gap on the effectiveness and safety of acupuncture in older adults with cLBP. 28 The aim of this three-arm pragmatic clinical trial was to test the effectiveness of acupuncture needling for improving back pain-related disability among 800 adults aged 65 and older with cLBP across four healthcare systems. These included: two integrated insurance and care delivery systems on the West Coast (Kaiser Permanente Northern California and Kaiser Permanente Washington); a fee-for-service system in California (Sutter Health) and the Institute for Family Health (IFH), a network of FQHCs in New York City and the Mid-Hudson Valley region of New York.
IFH provides primary care and behavioral health services to predominantly underserved communities. The Institute engages in clinical and health services research in support of its core mission to provide access to superior primary care, especially to those who are medically underserved. 29 The IFH practice sites, providers, and organizational leaders comprise a primary care-based research network (PBRN) registered with the Agency for Health Care Quality and Research (AHRQ). While engaging in a wide variety of research projects, focus areas include racial and ethnic health disparities, patient-centered care, health information technology, integration of mental health care and primary care, and women’s health. 29 IFH’s experience in research, education and the use of evidence-based integrative approaches in primary care supported both the BIA recruitment and retention plan and the way it was implemented.
Methods
All study procedures described here were approved by the institutional review board at IFH (IRB # 2371) and the single IRB for the overall clinical trial housed at the Kaiser Permanente Northern California Region (KPNC sIRB #1474280; IRB FWA #00002344). Study participants were consented for all study activities described in this paper.
IFH maintained a keen awareness of the population they serve and the needs of its administrative, clinical and research staff, ensuring that all those involved were recognized as essential to a successful research trial. This was particularly important as the acupuncture intervention was slated to take place at participants’ ‘home’ clinic. The IFH target sample size for the BIA trial was 123 participants receiving their primary care within the IFH system. Prior acupuncture research conducted at IFH
30
demonstrated the effectiveness of recruiting participants through their primary care provider’s (PCP’s) referral. As a result, two referral pathways were created: (1) a direct mailing of recruitment postcards to eligible patients; and (2) a means of referral through the EHR from IFH PCPs. To optimize and leverage the second pathway, via referral, several steps were taken to alert and orient the IFH PCPs and utilize recruitment through the EHR, in this case Epic Systems Corporation’s EpicCare®: • The site Principal Investigator (PI) (RT) met with IFH PCPs, providing an orientation to acupuncture research and the BIA trial opportunity • A list of eligible participants was created that were, in turn, identified when scheduled to be seen in the clinic • The scheduled PCP then received notifications in the EHR about these potential participants before the day’s clinic session started • PCPs made their referrals to the research trial within the EHR, employing the EHR ‘in-basket’ to message the Clinical Research Coordinators (CRCs). This referral process intentionally mirrored how a PCP would routinely communicate with staff regarding clinical matters. Notice to the PCP that the potential participant would first need to meet eligibility requirements and might be randomized to the usual care arm was included in all referral prompts.
As the first phase recruitment period progressed, the rate of referrals did not keep pace with the recruitment target, coinciding with the emergence of the COVID-19 pandemic and lack of wide availability of a COVID-19 vaccine. This disrupted operations in all clinical trial activities in the U.S. and globally, reducing trial participation due to fear of exposure to the virus. IFH clinics and medical staff followed, and in turn required that patients follow the then-current masking, distancing, protective barrier and handwashing guidelines to mitigate exposure to COVID-19. The COVID-19 pandemic incentivized broad and swift adoption of innovative approaches to recruitment, described below (Results). 31 BIA recruitment at IFH was also affected by East coast weather and dense urban public transit, both making access to IFH appointments more difficult for differently-abled older adults. In addition, familial responsibilities, increasingly during COVID-19, affected patient recruitment due to the role of older patients in providing home care for family members. 32 An essential IFH innovation leveraged the EHR to review and record potential barriers in order to overcome these hurdles and support recruitment. The IFH Information Technology (IT) department created a ‘Best Practice Alert’ (BPA) that was triggered for the PCP when a patient was identified as having cLBP and within the inclusion age for the study. A response button in the BPA created an automated referral to the CRC, reducing a PCP’s referral input to a series of mouse clicks. This automated referral more closely mirrored how a provider would generate a clinical action in the EHR, such as a referral for a colonoscopy for colon cancer screening. Recruitment remained flexible, and providers could still communicate research referral messages through the EHR in-basket.
Spanish-Speaking Participant Recruitment
The IFH site was exclusive among BIA study sites in recruiting Spanish-speaking patients. As such, IFH was responsible for having all documents and scripts translated from English to Spanish to build a research database that was appropriate for Spanish-speaking participants. Inherent in recruitment of a second language population was additional time required for IRB evaluation of translated study documents. This additional IRB review slowed early recruitment of Spanish-speaking participants.
Participant Retention
Clinical Research Coordinators (CRCs) were employed for specific skills in research, data collection, and bilingual English and Spanish fluency. Beyond meeting recruitment objectives, the CRCs had an expanded role as navigators of all facets of the research experience from recruitment to intervention. Participant navigators have been shown to facilitate retention in clinical trials, particularly in underrepresented communities. 33 CRCs built and sustained relationships with trial participants, acted as the point-of-contact for PCPs and IFH administration, and were an integral part of the research team.
The team of Research Acupuncturists (RAs) participated in IFH cross-clinical site trial orientations. 34 Most had provided acupuncture interventions in previous trials, 35 including at IFH, 30 and were likewise acculturated to both the IFH and the trial aims in an expanded role as part of the patient participants’ care team. A Research Clinic Director (RCD) was also identified, which was one of the Research Acupuncturists (DM). This lead served to both manage the implementation of the acupuncture intervention in the participants’ home clinic and coordinate ongoing acupuncture delivery for the research trial. Acupuncture was provided at three IFH clinic sites from which patient recruitment also occurred. Thus, delivery in the home clinic reduced a significant barrier to access. Acupuncture session forms that were identical to the other trial site session forms 34 were created in the EHR through collaboration between the RCD and IFH (IT team). This plan enabled a seamless capture of research data as the session was recorded directly into the EHR, normalizing and building transparency of both research and acupuncture intervention to the participants and their PCPs.
RAs also regularly completed surveys in REDCap® (Research Electronic Data Capture) to track and manage critical indicators of each clinic session ‘flow’ (see Supplement 1). REDCap is a secure, web-based software platform designed to support data capture for research studies.36,37 Any urgent matters were relayed by text or phone to the IFH project manager (PM) (MB) to ensure that concerns were addressed immediately by the RCD and CRC.
Results
IFH successfully recruited members of underrepresented groups to the BIA trial to meet its goal of 123 randomized participants with acceptable levels of retention.
The following data were collected as recruitment and retention efforts were tracked: • Demographics of IFH participants, including preferred language • Rate of recruitment for provider referrals vs postcard mailers • Rate of recruitment from providers before and after creation of the EHR BPA • Rate of retention (provided by the coordinating center)
Baseline Demographic and Clinical Characteristics.
aData are presented as number (percentage) of participants observed. Data were missing for the following: income 32, race/ethnicity 5, married/domestic partnered 1, BMI 3, frailty 5.
bPatient reported.
cDerived from the electronic health record (EHR).
dCategories are not mutually exclusive.
In addition, the hypothesis of provider referrals facilitating the majority of the recruitment proved true: 85% of the sample were recruited through referrals vs only 15% from postcard outreach. The BPA with ‘referral button’ also showed an advantage. Prior to BPA the rate of referral per week was 4.4. After the BPA, that rate of referral increased to 13.4 per week. IFH study withdrawal rate was 12.2% with 18.7% missingness in follow-up data rates at 6 months, the trial’s primary outcome timepoint.
Discussion
Up to 20% of clinical trials are unable to meet recruitment targets and are either terminated early or are completed shy of their goal.41-43 IFH experienced a successful rate of recruitment and retention of members of underrepresented groups in the BIA trial through understanding and anticipating the challenges experienced by IFH participants. The IFH team understood that there is often mistrust of institutions among underrepresented populations, a potential obstacle to successful trial recruitment and retention. 44 Therefore, the research team was comprised of individuals who had extensive experience working with underserved populations, facilitating trust-building with the participants. 29
The IFH research team consisted of the site PI (RT) who is an experienced researcher and PCP at IFH; a pivotal Program Manager (PM) (MB) working at IFH for many years; a RCD (DM) who had experience at IFH delivering acupuncture in both clinical and research milieu; and the CRCs who were recruited for specific skills in research, data collection, and bilingual English and Spanish fluency.
We hypothesized that providing care at IFH participants’ home clinic would facilitate trust and access, and that referrals from PCPs would help participants feel at ease with the trial, encouraging them to participate. Referral to the trial also matched how PCPs might refer to specialty services, helping to normalize the idea of acupuncture as a service and as research for both PCPs and patients. IFH has had acupuncture services available since 2015, providing some familiarity for referring providers and patients as well. Care was taken to bring nuance to the recruitment process that (a) the trial participant referral was not interpreted as a consent process for the study; and (b) that referral was for the study and not for acupuncture service per se, in that the participant could be randomized to usual care. Overall, study referrals needed to be as organic and simple as possible for busy providers. Embedding the process largely in the EHR facilitated our success, as evidenced by the increase in rate of referral after creation of the BPA.
While trial-wide incentives supported ongoing participation in the BIA study, 7 the incentive amounts had potentially more impact for lower socioeconomic status participants. IFH participants were compensated $5 for each of the 9 short surveys, $15 for completing baseline, $20 for the 3-month survey, $25 for the 6-month survey, and $30 for the 12-month survey, for a potential total of $135 provided by way of a reloadable debit card used to pay research participants (ClinCard). 45 Even so, retention of IFH participants remained a challenge, consistent with significantly lower trial retention rates for minorities and those who speak a language other than English at home. 46 Almost 60% (8 of 14) participant withdrawals were in the usual care group. Reasons for intervention group withdrawals varied from mobility issues, transportation, scheduling, level of pain or other health issues.
Efforts were made to prevent missing measures data. CRC contacts were used to remind participants of upcoming survey calls and the IFH trial coordination team used reminder calls to participants to expect timed survey calls. IFH CRCs and KPWA personnel also made attempts to troubleshoot any incentive ClinCard issues ahead of the 6-month survey call to maintain engagement and to alert participants to expect an upcoming survey call from a Washington state phone number. While IFH retention rate was lower than the other BIA clinical trial sites, the rate reflects the rate of retention for this population in other NYC area acupuncture trials. For ‘Acupuncture Approaches to Decrease Disparities in Outcomes of Pain Treatment Two Arm Comparative Effectiveness Trial’ (AADDOPT-2), a large RCT comparing group acupuncture to individual session acupuncture for chronic musculoskeletal pain in an underrepresented population, the loss to follow-up (those with no follow-up data) at 12 weeks was 12.8% (group sessions) and 12.3% (individual sessions), with between 25-27% of participants' missing pain outcome data at 12 and 24 weeks. 35 GAPYOGA, the IFH feasibility study of group acupuncture followed by therapeutic Yoga sessions experienced a 16% loss to follow-up, with 78 (84%) of the 93 participants completing the intervention and 10 weeks interview. 30 IFH retention is well within the retention rates for these trials with underrepresented participants.
Personal and continued communication by the CRCs and IFH PM (MB) with the IFH study participants provided ‘high touch’ navigation support. This navigation support was also enhanced by the RCD (DM), who would closely communicate with the CRC around any obstacles to those participating in the intervention. Lastly, close collaboration between the study coordinating center and IFH was critical in operationalizing the research structure while allowing for flexibility to optimize recruitment and retention. The PM role (MB) at IFH was an important nexus of communication in this regard.
IFH recruitment and retention of historically underrepresented participants in the BIA trial provides an exemplary strategy that was responsive to both the trial needs, the primary care sites, the providers, and the participants. While the NIH has developed a strategic plan (2021-2025) for involvement and inclusion to address minority health and health disparities in drug trials, 5 it is important to support ongoing strategies to involve and include historically marginalized populations in trials of nonpharmacologic pain care, such as acupuncture therapy. 47
Limitations
Like many trials conducted during the COVID-19 pandemic, the IFH team needed to respond to unique challenges at the New York City IFH site to stay on track with trial milestones. Additionally, there might have been a larger percentage of Spanish-speaking patients earlier in the trial if IRB approval of the translated forms had come sooner. Furthermore, IFH missing follow-up data rate was larger than the three West coast sites, potentially due in part to trial design wherein the follow-up interviews were conducted by phone from the non-local Kaiser Permanente Washington site, introducing an unfamiliar component as well as a time difference of 3 hours to interview East coast participants. Lower completion rates at IFH were driven by the withdrawal rate in the usual care arm which was over twice as high as any other site. Potentially the PCP referral process that facilitated recruitment may have contributed to an expectation that participants were being referred for acupuncture, increasing ‘disappointment withdrawals’. Moreover, the receipt of active treatment may have been a more important motivator for trial participation at IFH than among participants at the West coast sites who may have had greater resources to access such services.
Conclusion
The inclusion of underrepresented participants in acupuncture research is essential to the generalizability of any findings. The IFH site team successfully recruited and retained diverse participants through trusted connections with the study population, building on experience with acupuncture research recruitment and retention by engaging PCPs, study team members, primary care clinical sites and EHR communication options.
Supplemental Material
Supplemental Material - Recruitment and Retention for an Acupuncture Trial in an Underrepresented 65 and Older Population With Chronic Low Back Pain
Supplemental Material for Recruitment and Retention for an Acupuncture Trial in an Underrepresented 65 and Older Population With Chronic Low Back Pain by Ray Y. Teets, Arya Nielsen, Donna Mah, Matthew Beyrouty, Marsha J. Handel, Morgan Justice, Hyowoun Jyung, Carolyn M. Eng, and Lynn L. DeBar in Global Advances in Integrative Medicine and Health
Footnotes
Acknowledgments
We gratefully acknowledge the members of our research coordination team: Phoebe Rosenheim, Ariel Jacobs, Luz E. Soto Cossio, Mirta Milanes, Saskia Shuman and Eve Walter. We also acknowledge the caring and expert treatment provided by our research acupuncturists consenting to be named here: Patricia Botet, Valentina Duque, Selina Greene, Maria Macchia, Dana Moore, and Dorothy Wong. Finally, we gratefully acknowledge the IFH IT team, including Rebecca McGinley, for EHR support.
Author Contributions
R.Y.T., A.N., D.M., M.B., M.J., H.J., C.M.E., L.L.D. made a significant contribution to the concept, design, acquisition, analysis or interpretation of data. R.Y.T., A.N., D.M., M.B., M.J.H., M.J., H.J., C.M.E., L.L.D. drafted the article or revised it critically for important intellectual content. All authors approved the final version of the article for publication and agree to be accountable for all aspects of the work and resolved any issues related to its accuracy or integrity.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by a contractional agreement from the National Institutes of Health, National Center for Complementary and Integrative Health [NCCIH UG3/UH3AT010739 and PRISM Resource Coordination Center PU24AT010961]. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Trial registration
ClinicalTrials.gov Identifier: NCT04982315. Clinical trial registration date: July 29, 2021.
Ethical Statement
Data Availability Statement
The data that support the findings of this study are available from the National Institute of Mental Health (NIMH) Data Archive (NDA). NDA is a collaborative informatics system created by the National Institutes of Health to provide a national resource to support and accelerate research in mental health. Dataset identifier: Not yet available but will be available in Spring 2025. To receive only IFH data one would need to request with the study MPI Andrea Cook
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
