Abstract
Background
Difficulties recruiting to clinical trials are well-documented. Strategies to engage staff from the clinical site where recruitment takes place may be helpful in increasing recruitment rates.
Aim
To systematically review the literature to evaluate the effectiveness of interventions that aim to increase recruitment to clinical trials, focused on clinical site staff who support recruitment.
Methods
A systematic search for randomised studies within a trial (SWATs) that aimed to improve recruitment to a randomised host trial in the field of health or social care aimed at clinical site staff was conducted. Studies were excluded if they aimed to increase retention, were targeted at participants, or the SWAT or host trial were non-randomised. Database and hand searches were conducted up to 25th July 2024. The primary outcome was the rate of recruitment. The Cochrane RoB2 tool was used to assess the risk of bias of included studies.
Results
A total of seven studies were retrieved; all had a high risk or some concerns of bias. Studies evaluated heterogenous interventions and were synthesised narratively. A digital training package for trainee principal investigators was the only intervention to demonstrate a statistically significant effect.
Conclusion
Due to the small number of studies retrieved and the heterogeneity between them, it was not possible to make any conclusions of effectiveness of any strategy at helping clinical site staff to recruit optimally. To prevent research waste, future SWATs need to focus on replications of recruitment interventions in populations and settings of need, rather than further single-study replications.
Registration
PROSPERO CRD42022346585.
Keywords
Introduction
Randomised controlled trials (RCTs) are considered the gold standard in assessing the effectiveness of interventions. 1 However, difficulties recruiting to RCTs are well documented.2,3 Suboptimal recruitment to clinical trials impacts on statistical power, increases costs and delays the production of evidence for patient benefit, leading to effective treatments not being implemented or ineffective treatments continuing to be used, thus contributing to research waste.4–7
Analysis of trials funded by UK healthcare funder, the National Institute of Health and Care Research (NIHR) between 1997 and 2020 showed that only 63% (n = 245) reached their final recruitment target, illustrating the challenge of recruiting to time and target. 8
Although evidence-based healthcare decision-making originates from information produced by RCTs, this is not always the case for trial conduct decisions such as recruitment strategies. 9 Studies within a trial (SWATs) are a method to evaluate the effect of recruitment interventions. The availability of evidence regarding effective recruitment strategies, generated through SWATs, is growing, but recruiting to time and target continues to be hindered by the lack of robust evidence-based strategies. 9
Previous research has identified the difficulty ‘picking apart’ factors influencing recruitment due to several strategies being used concurrently and not being evaluated using randomised methods, making it difficult to delineate the effects of each. 10 One of these factors is the role of clinical staff from recruiting sites. Bower et al. 11 identified a clear knowledge gap with regard to effective strategies aimed specifically at recruiters to RCTs. There are many different job roles within clinical teams, such as ward clerks, record keepers and nurses, who can act as gatekeepers to potential participants, but their role in recruitment is not always as explicit as this. They can be key to successful recruitment to host trials by providing unofficial support, such as reminders about the trial to staff who can refer/recruit. It has been recognised that there is a need to credit and support this range of staff members whose involvement in recruitment is traditionally overlooked but is no less crucial to trial success. 12 In addition, gatekeeping by healthcare professionals has been identified as one of the most difficult recruitment barriers to overcome. 13
Another factor suggested to contribute to low trial enrolment is the inaccurate assessment of challenges and risks associated with taking part in clinical trials, by both patients and staff. 14 Furthermore, a systematic review investigating the factors which affect recruitment to clinical trials found that both patients’ and gatekeepers’ decisions to participate are affected by their attitude towards the research and intervention under investigation, and a subsequent judgement between risk and reward of participation. 15 More recently, a qualitative evidence synthesis illuminated the influence of staff recruiters on participant screening and the complexities they navigate when recruiting to clinical trials. 16
Despite this evidence, there has been relatively little investigation into interventions to aid this population to recruit participants effectively.9,17 Much of the investigation related to recruitment to clinical trials has focused on interventions targeted at prospective participants, with few focusing on effective interventions that could be used by staff recruiters. Of the 68 studies included in a Cochrane review by Treweek et al., 9 only 7% (n = 5) were aimed at recruiters, which further highlights the dearth of high-certainty evidence in this area, and translates into a lack of evidence upon which trialists can use in the planning of clinical trials.
Given the growing body of evidence acknowledging the effect staff recruiters can have on recruitment, there have been attempts to develop interventions which target this population. For example, use of a designated person to recruit participants or the Qualitative Research Integrated in Trials (QuinteT) recruiter training intervention, which was developed to aid recruitment to surgical trials.10,18 Despite these interventions, there remains limited evidence of strategies aimed at clinical site staff who are involved in recruitment to clinical trials and, with the increasing prevalence of SWATs, it is likely that previous systematic review evidence on this topic is now outdated.9,17
The aim of this review was to systematically review the literature to evaluate the effectiveness of interventions focused on clinical site staff (defined as: staff who are part of a clinical team at the recruitment site but separate to the core research team) who support recruitment to clinical trials.
The following objectives were set to achieve the aim: 1. Quantify the effect of identified interventions on recruitment rates to the host trial. 2. Explore the availability of cost and acceptability data of identified interventions 3. Evaluate the effect of interventions on participant screening rates.
Methods
The protocol for this review was prospectively registered on the International Prospective Register of Systematic Reviews (PROSPERO CRD42022346585).
This paper follows the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA). 19
Eligibility
Participants
Any member of staff from the clinical trial site (≥18 years), separate from the core research team, who was involved in any aspect of participant recruitment, either formally, such as taking informed consent, or informally, such as administrative staff reminding clinical staff to present a trial to eligible participants.
Interventions
Any randomised SWAT aimed at improving recruitment to a randomised host trial in the fields of health or social care.
Outcome measures
Recruitment rate to the host trial at the end of the SWAT period.
Types of studies
Included studies were randomised studies within a trial (SWATs) embedded in a host RCT. Trials were excluded if they aimed to improve participant retention or were targeted at host trial participants. Interventions evaluated using non-randomised methods, or evaluating recruitment to non-randomised host trials were also excluded.
Information sources and search strategy
The electronic databases EMBASE, MEDLINE and PsycINFO were searched via the OVID platform from date of inception to 25th July 2024. Due to time and resources constraints, searches were limited to the English language, using a combination of terms related to SWATs and recruitment. Relevant terms were identified using the online resource PubMed PubReminer. 20 The search strategy was then piloted to ensure it retrieved a relevant, known paper. The full search strategy is given in Supplemental Appendix 1.
The following resources were also hand-searched: the reference lists of included studies, F1000 Research SWAT collection, 21 the Northern Ireland Hub for Trials Methodology Research: SWAT Repository Store, 22 the Trial Forge SWAT Network Group website 23 and the Online Resource for Research in Clinical triAls (ORRCA). 24
To ensure ongoing or recently ended and unpublished research was included, the PROMoting the USE of SWATs (PROMETHEUS) Team at the University of York 25 were contacted to ask if they were aware of any relevant research.
Study selection and data collection
Retrieved records were managed using Endnote 26 for deduplication and exported into Covidence 27 for further deduplication and screening. Titles and abstracts were screened independently and in duplicate by two reviewers (L.H. and A.T.). Full texts of articles meeting the inclusion criteria were screened independently and in duplicate by L.H. and A.T. Disagreement between reviewers was resolved by discussion and by the inclusion of a third reviewer (L.C.). A data extraction template was developed and piloted for use in Microsoft Excel 28 (Supplemental Appendix 2). Independent, double data extraction was performed by L.H., C.A. and L.C. The following data items were extracted:
SWAT
• Authorship • Year of publication • Title of publication • Type of strategy / intervention used in the SWAT • Length of intervention • Randomisation type (e.g. - parallel / cluster) • Allocation concealment • Sample size (overall and by trial arm) • Proportion of participants recruited to each of the SWAT arms • Participant screening rate for each of the arms of the SWAT • Intervention cost-effectiveness data • Data on acceptability of the intervention • Timing of SWAT in relation to host trial (at the start, or part way through) • Participant characteristics (age, gender, ethnicity, role in host trial, job role, experience)
Host study
• Country • Sample size (target / actual) • Health condition area • Randomisation type • Setting (e.g. – primary / secondary care) • Time / funding extension needed to recruitment period (Y/N)
Outcome measures
Primary
Proportion of participants recruited to the intervention arm and the control arm (as a percentage for each arm). The final report of recruitment was accepted. Data were extracted per publication and could not be converted due to heterogeneity in outcome reporting.
Secondary
Costs and acceptability of interventions.
Participant screening rates
The number of participants screened over the screening period (in months).
Data synthesis
The heterogeneity of interventions and outcome measures meant that statistical pooling of study results was inappropriate and pooled recruitment rates could not be calculated. Therefore, studies were grouped by intervention type and synthesised narratively.
To capture all studies meeting the inclusion criteria and allowing for replication recommendations, those at high-risk of bias were included in the narrative synthesis.
Planned subgroup analysis to explore the impact of interventions in different clinical settings, such as surgical and mental health settings, could not be performed due to the level of clinical heterogeneity and small number of records retrieved.
There were no planned sensitivity analyses.
Risk of bias
Two reviewers independently and in duplicate assessed risk of bias for each of the included records (L.H. and M.T.) using the Cochrane Risk of Bias 2 for cluster-randomised trials tool, 29 resolving discrepancies by discussion between them.
Missing data was not sought from authors due to time constraints.
Results
Study selection
As detailed in the PRISMA flow diagram
19
(Figure 1), after deduplication, 1611 records were screened. Twenty-eight studies were assessed for eligibility, of which 21 were excluded due to ineligible study design (n = 14) and ineligible study population (n = 7). PRISMA flow diagram of identified records.
A total of seven studies published between 2006 and 2022 were eligible for inclusion in this systematic review.
Characteristics of included studies
All seven studies were randomised using cluster randomisation. Three out of seven retrieved records were SWATs embedded in a host trial in a surgical setting30–32 two in oncology,33,34 one in stroke 35 and one in diabetes. 36
Study settings varied, with most studies based in UK countries (n = 4).30–32,35 The remaining studies were based in Australia (n = 1), 33 France (n = 1) 34 and multiple continents incorporating Asia, Australasia, Europe and North America (n = 1). 36
Information on the individual job roles comprising the participant population were not well-described in any included study. Four studies reported some information on the roles of people targeted by the interventions: healthcare professionals, research nurses, principal investigators, radiology contacts, research physiotherapists and surgeons. 30–32,35
Study characteristics are presented in tables one, two and three.
Risk of bias
Risk of bias was assessed using the Cochrane Risk of Bias 2 tool for cluster-randomised trials.
29
Results of the assessment are shown in Figure 2. Risk of bias of included studies.
Overall, three studies were assessed as having a high risk of bias 33,34,36 and four as having some concerns.30–32 The main domains driving high risk of bias were missing outcome data (n = 2)33,36 and bias due to deviations from intended intervention (n = 1). 34 The bias in the studies assessed as having some concerns was largely driven by bias due to deviations from intended intervention and bias in the selection of the reported result.
Analysis
Due to the lack of consistency in the interventions evaluated in the included studies, it was inappropriate to undertake a meta-analysis and hence a narrative synthesis is presented below.
Interventions and outcomes
Site initiation visits (SIV)
Two studies investigated the impact of onsite visits.31,34
Characteristics of studies investigating site initiation visits.
Liénard et al. 34 found site initiation and ongoing monitoring visits resulted in 11% higher recruitment in sites that had onsite initiation visits and ongoing monitoring visits versus sites that did not. The authors state this difference was not significant but did not report statistical values (302 participants vs 271 respectively).
Training
Two studies examined the effectiveness of interventions with a staff training element at increasing recruitment rates.30,32
Characteristics of studies investigating staff training interventions.
An enhanced digital training for trainee principal investigators (TPI) was the only intervention to demonstrate a statistically significant effect. 30 Over a six-month recruitment period, there was statistically significant benefit to recruitment in TPI arm (Incidence risk ratio (IRR) 1.23, 95% confidence interval (CI, 1.09 to 1.40, p = 0.001). When a digital nudge, involving a personal email expressing appreciation and encouragement for recruitment was coupled with TPI, a significant interaction was observed, enhancing recruitment (IRR 2.09, 95% CI, 1.64 to 2.68. p < 0.001). However, in the nudge only arm, recruitment was reduced at a significant level (IRR 0.62, 95% CI, 0.51 to 0.75, p < 0.001) indicating the digital nudge only had a beneficial effect on recruitment when combined with TPI and was harmful to recruitment when used in isolation.
Additional communication, funding, and recruitment plus software to aid patient identification
Three studies reported a range of heterogenous strategies. One study investigated the effect of additional communication, including frequent email contact and report of recruitment performance 36 ; one examined the provision of additional funding to the sites 33 ; and one investigated whether the provision of software package aimed at helping recruiters identify eligible patients was effective. 35
Characteristics of studies investigating additional communication, funding, and recruitment plus software to aid patient identification.
In one study, additional funding was provided to some recruitment sites. This could be used as sites wished, with most using the funds to increase staff hours or staffing levels, followed by being added to general funds to cover budget deficits. The additional funds provided to these sites did not result in a significant difference to recruitment when compared to site with no additional funding (ratio, 0.99; 95% CI, 0.69 to 1.43, p = 0.96). 33
Recruitment advice focusing on the use of software aimed at identifying eligible patients and a 6-month review of recruitment progress did not significantly improve recruitment when compared to sites that did not receive the intervention (rate ratio: 1.06; 95% CI 0.55 to 2.03, p = 0.87). 35
Cost-effectiveness of interventions
An exploration of cost data found that costs of interventions were not reported well in any included records.
Three studies provided a descriptive commentary of costs but only Jefferson et al. 31 reported costs of the intervention; the average cost to set up a site in the intervention arm was £1,016.93 versus £727.10 in the control arm.
Acceptability of interventions
Four studies provided data on the acceptability of interventions, which were assessed using survey methods.30–32,35
Jefferson et al. 31 found a preference for remote SIVs (16/28 respondents) and an on-site meeting for the final site visit (17/28).
Support using software to identify eligible participants from audit data was found to be useful by clinical site staff (93%, n = 28), but time and resource pressures constrained its use. 35
All of the participants who received an enhanced trainee PI training package were extremely satisfied with the induction and ongoing support. 90% of participants engaged with the induction activity and 86.7% with follow-up communication during the 6-month SWAT. 30
Parker et al. 32 found that participants felt positive about the training intervention, they learned a lot from it, and that it would influence their future recruitment practices. Average ratings from 0-10 (10 = highest rating): how positive participants felt about the training intervention 9.3 (SD 1.0), how much they learned during the intervention 9.2 (SD 0.9), and how much difference the intervention made to future recruitment practices 8.7 (SD 0.9).
Screening rate
None of the included studies provided information on the effect of the intervention on screening rates.
Discussion
Main findings
This review identified seven studies investigating the effect of randomised interventions aimed toward clinical site staff at improving recruitment rates to trials in the field of healthcare. The level of clinical and statistical heterogeneity between populations and interventions prevented the calculation of a pooled effect estimate for any single intervention. Only one intervention showed a statistically significant improvement in recruitment to the host trial, an enhanced trainee principal investigator training package. 30
Similar to previous reviews on clinical trial recruitment,9,17 these results highlight the lack of high-quality evidence regarding strategies targeted at staff at the clinical recruitment site upon which trialists can make recruitment decisions. Little appears to have changed since previous reviews on this topic with studies still characterised by incomplete reporting and evaluations of single-study, heterogeneous interventions.9,17
Strengths and limitations
Previous research identified a wide range of clinical site staff roles as having the potential to affect participant recruitment 12 This review built on this by including a wide population, including any member of staff from the clinical recruitment site, unrestricted in terms of profession. This meant that any studies aimed at a broad range of job roles, such as ward clerks and administration staff could be included and summarised, although none of the included studies described inclusion of this often-overlooked population. The nature of cluster randomisation means it is difficult to understand the nuanced effect the intervention may have on individuals and how this could differ across job roles. Including this information in detail would highlight areas of need and prevent research waste.
Due to time and resource constraints, included studies were restricted to the English Language only, introducing the possibility of selection bias, although it is likely this would have a limited impact on the number of studies retrieved, as the majority of SWAT results are published in English Language journals.
This review is only concerned with interventions evaluated by randomised methods and therefore does not take account of non-randomised interventions, identified as potentially providing value to recruitment. 37
Recommendations for future research
• SWATs aimed at clinical site staff are needed in all fields of health and social care. • Future studies should provide information regarding the job roles and level of experience in research of staff targeted by interventions, which will aid understanding for who and how the interventions are effective or ineffective and so inform future replications. • To prevent research waste, it is recommended that cost and acceptability data is included in all future planning and reporting of SWATs. • The authors advocate for much wider promotion of the need for evidence-based trial methodology. • Future SWATs should report standardised outcome data, such as recruitment rates, to allow for comparison between studies and meta-analysis of data to produce pooled effect estimates.
Conclusion
This paper highlights the dearth of good-quality evidence of effective interventions aimed at clinical site staff recruiters to enable them to recruit participants at optimal levels. The few records that were identified were too dissimilar to draw any useful conclusions regarding their effectiveness. To prevent research waste, future SWATs need to focus on replications of recruitment interventions in populations and settings of need, rather than further single-study replications.
Supplemental Material
Supplemental Material - Interventions to improve clinical trial recruitment with a focus on clinical staff from the recruiting site; a systematic review
Supplemental Material for Interventions to improve clinical trial recruitment with a focus on clinical staff from the recruiting site; a systematic review by Laura Hermann, Catherine Arundel, Abin Thomas, Maureen Twiddy and Laura Clark in Research Methods in Medicine & Health Sciences.
Supplemental Material
Supplemental Material - Interventions to improve clinical trial recruitment with a focus on clinical staff from the recruiting site; a systematic review
Supplemental Material for Interventions to improve clinical trial recruitment with a focus on clinical staff from the recruiting site; a systematic review by Laura Hermann, Catherine Arundel, Abin Thomas, Maureen Twiddy and Laura Clark in Research Methods in Medicine & Health Sciences.
Footnotes
Acknowledgments
The authors would like to thank Sarah Greenley, Information Specialist at Hull York Medical School, for her valuable assistance in developing the search strategy and conducting the search.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Laura Hermann, NIHR Pre-doctoral fellowship [grant number NIHR302008] was funded by the NIHR for this research project. The views expressed in this publication are those of the authors and not necessarily those of the NIHR, NHS or the UK Department of Health and Social Care.
Supplemental Material
Supplemental material for this article is available online.
Appendix
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
