Studies within a trial priorities to improve the evidence to inform recruitment and retention practice in clinical trials

Abstract

Background

Trial execution commonly relies on experience and judgement because there is a lack of evidence to inform how best to design and deliver clinical trials. Recruitment and retention are critical determinants to trial success have been persistent challenges that impact various stakeholders including funders, researchers, and the public. Studies within a trial (SWATs) are a way to discover best practices for recruitment and retention strategies, however, the current SWAT landscape has not been formally explored to date. This study aimed to (i) identify where current activity is taking place (ii) understand if SWATs are addressing PRioRiTY questions (iii) highlight gaps in the literature for future research.

Methods

In November 2020, registered SWATs in the SWAT repository store were extracted and categorised into ‘recruitment’, ‘retention’ or ‘other’ based on the primary outcome. Recruitment and retention SWATs were subsequently mapped against PRioRiTY 1 and 2 questions and descriptive statistics were used to present the findings.

Results

125 registered SWATs were extracted from the repository. 50 and 36 SWATs reported recruitment and retention as their primary outcome, respectively. A majority of recruitment SWATs investigated what and how information should be designed and delivered to potential trial participants (n = 23, 46%) and the advantages and disadvantages of using technology during the recruitment process (n = 9, 18%). Three of the Top 10 PRioRiTY 1 questions had no SWATs mapped against them. A majority of retention SWATs focused on the best ways to encourage participants to complete trial tasks (n = 24, 67%), how incentives should be implemented (n = 10, 28%) and strategies to make participants feel valued (n = 9, 25%). Five of the Top 10 PRioRiTY 2 questions had no SWATs mapped against them.

Conclusions

This study identified a mismatch between registered SWAT activity and the priority questions in recruitment and retention. Trial teams should consider the PRioRiTy 1 and 2 questions for recruitment and retention, respectively, when designing a SWAT. In addition, there is a great breadth of research taking place, but replication of existing research is needed to produce confident evidence-based guidance for trialists and researchers to implement into their work.

Keywords

planning the research recruitment and retention retention and compliance SWAT trial conduct

Background

Randomised trials are essential to provide evidence that advances clinical care and informs clinical practice guidelines. How the trials themselves are run, however, can be inefficient and sub-optimal.

‘There is a peculiar paradox that exists in trial execution - we perform clinical trials to generate evidence to improve patient outcomes; however, we conduct clinical trials like anecdotal medicine: (1) we do what we think works; (2) we rely on experience and judgement and (3) limited data to support best practices.’

Monica Shah, quoted in Gheorghiade et al. 2014¹

The conduct of clinical trials commonly relies on experience and judgement. It does so because there is no alternative: the evidence available to inform decisions is either limited or non-existent. This might not matter if we knew trials to already be highly efficient but as anyone who works in trials knows, this is not the case.² This disconnect between evidence (not much) and practice (decisions that need to be made every day) highlights the need to research how best to conduct clinical trials. This lack of evidence on how best to approach trials spans the entire clinical trial lifecycle from site selection to the dissemination of results.

Studies within a trial (SWAT) are used as a way to investigate and test alternative ways of delivering a trial process to generate evidence to support decisions about that process. A SWAT is defined as “a self-contained research study that has been embedded within a host trial with the aim of evaluating or exploring alternative ways of delivering or organising a particular trial process”.³ SWATs can be randomised evaluations, non-randomised evaluations, or more exploratory studies using qualitative methods to provide an answer to a given question. The process they test may be at any point of the trial life cycle and target anyone involved in the trial endeavour, such as participants, trial staff, clinicians, regulators, etc.

Recruitment and retention are two of the thorniest trial process challenges.^4–7 Participation in trials is dependent on the willingness of people, be it patients, the public or professionals, to volunteer their time and energy to the trial. It is not just the initial decision to participate; the commitment to the trial often needs to be sustained over long periods and may involve multiple questionnaires and/or follow up visits. As well as diminishing certainty, inadequate trial recruitment and retention can also raise ethical concerns for those already recruited into a trial in which the objectives cannot now be achieved.^8,9 Understandably, recruitment and retention have been identified as the top methodological priorities for trial methodology research within the UK.¹⁰

Which aspects of trial recruitment and retention were most in need of research attention has been unclear. However, two recent prioritisation processes, Prioritising Recruitment in Randomised Trials Study (PRioRiTy I, 2018¹¹) and Prioritising Retention in Randomised Trials Study (PRioRiTy II, 2019¹²), have changed this. The PRioRiTY 1 and 2 projects were completed as collaborative projects with the public, carers, and professionals to identify and prioritise unanswered questions for trial recruitment and retention respectively, which help direct research teams to create evidence that is important and can be applied in everyday trial conduct. The research agendas produced by the prioritisation exercises now require SWATs to help fill the evidence gaps.

To understand the current landscape of recruitment and retention SWATs registered on the SWAT repository (http://www.qub.ac.uk/sites/TheNorthernIrelandNetworkforTrialsMethodologyResearch/SWATSWARInformation/Repositories/SWATStore/), we conducted a gap analysis to identify where efforts are currently focussed and, critically, where there are gaps. We had a particular focus on where effort is required to address the PRioRiTy projects Top 20 questions.

Methods

In November 2020 we extracted data on the title, lead contact, intervention and the status (i.e. active, inactive, completed) of the SWATs registered in the SWAT Repository Store.¹³ The registered SWATs were organised into one of three groups, ‘recruitment’, ‘retention’ or ‘other’ based on the primary outcome stated in the SWAT protocol. Any studies listed on the SWAT repository that were not focused on recruitment, retention or not explicitly set within clinical trials were excluded.

The included SWATs were then mapped against the research questions that came from the recruitment and retention PRioRiTy projects (https://priorityresearch.ie). Mapping was completed by comparing the recruitment SWATs with PRioRiTy 1 questions, and retention SWATs with PRioRiTy 2 questions. Where a recruitment and retention SWAT could not be mapped onto a PRioRiTy question, the research team developed ‘codes’ for new categories. SWATs were mapped to all PRioRiTy questions considered relevant.

One researcher conducted the initial mapping independently (CB for recruitment and KG for retention) with subsequent verification of findings through discussion between three researchers (CB, KG, ST). The final complete mapping document is available in Supplementary File 1. Results are presented descriptively using frequencies.

Results

At the time of mapping (Novembber 2020), there were 125 SWATs listed on the database, with 86 of these reporting a primary outcome that focussed on recruitment or retention and was set within a trial. Of the 86, 50 SWATs considered recruitment and 36 SWATs retention, with 38 of the recruitment SWATs also proposing to evaluate retention as a secondary outcome. A total of 13 SWATs were inactive (i.e. had not started), 24 were ongoing, 41 had reached completion and the status of the remaining 8 SWATs was unclear. Reasons for inactive registrations is likely to be for one of two reasons. Firstly, the repository is a place where protocols can be posted for others to use, and it may not be the intention of the team to conduct the SWAT. Secondly, some SWATs were delayed because of delays faced by the host trial.

Of the 50 recruitment SWATs, 5 did not map onto a PRioRiTy I Question and 22 SWATs mapped onto more than one PRioRiTy I Question (e.g. both Question 4 and Question 6).The remaining 45 mapped SWATs aiming to improve recruitment primarily focused on two broad areas:

1. What and how information should be designed and delivered to potential trial participants (n = 23, 46%), which mapped to PRioRiTy I Question 4 (n = 14, 28%), and Question 2 (n = 9, 18%);

2. Advantages and disadvantages of using technology during the recruitment process (n = 9, 18%), which mapped to PRioRiTy I Question 10) (Table 1).

Table 1.

Recruitment SWATs mapped to PRioRiTy I questions. The shaded rows highlight the top three areas of activity.

PRioRiTy I question	Number of SWATs mapped to question n (%) (total number of recruitment SWATs = 50)
1. How can randomised trials become part of routine care and best utilise current clinical care pathways?	0 (0%)
2. What information should trialists communicate to members of the public who are being invited to take part in randomised trials in order to improve recruitment to the trial?	9 (18%)
3. Does patient/public involvement in planning a randomised trial improve recruitment?	0 (0%)
4. What are the best approaches for designing and delivering information to members of the public who are invited to take part in a randomised trial?	14 (28%)
5. What are the barriers and the enablers for clinicians/healthcare professionals in helping conduct randomised trials?	3 (6%)
6. What are the key motivators influencing members of the public’s decision to take part in a randomised trial?	4 (8%)
7. What are the best approaches to ensure inclusion and participation of under-represented or vulnerable groups in randomised trials?	0 (0%)
8. What are the best ways to predict recruitment rates to a randomised trial and what impact do such predictions have on recruitment?	2 (4%)
9. What are the best approaches to optimise the informed consent process to 1improve recruitment of members of the public to randomised trials?	1 (2%)
10. What are the advantages and disadvantages to using technology during the recruitment process?	9 (18%)
11. How best can clinicians and other health professionals be educated to optimise recruitment to randomised trials?	5 (10%)
12. Does feasibility testing of recruitment procedure lead to improvements in recruitment?	0 (0%)
13. How can eligibility criteria for trial participants be optimised so that they ad recruitment to randomised trials?	0 (0%)
14. What is the value of making trials participants feel appreciated in being recruited to a trial and how can this be best achieved?	1 (2%)
15. What are the best approaches to ensuring manageable workloads for clinical and research staff responsible for recruiting members of the public to randomised trials?	1 (2%)
16. Does involvement of (i) members of the public and/or (ii) members of the public participating in a randomised trial, in designing trial information improve recruitment?	3 (6%)
17. What are the incentives that motivate members of the public to agree to participate in randomised trials?	5 (10%)
18. Do trial recruiters who have received focused, specialised recruitment training achieve better levels of recruitment than non-trained recruiters?	2 (4%)
19. Do randomised trials which have a low participant burden/requirements achieve better recruitment than those that have a greater patient burden?	0 (0%)
20. Does a central registry for members of the public (i.e. a list of people with contact details) who are interested in taking part in a randomised trial improve recruitment?	0 (0%)
The recruitment SWAT did not map onto a PRioRiTy 1 top 20 question	5 (10%)

Some SWATs mapped onto more than one PRioRiTy question leading the sum of percentages to be more than 100%.

Five SWATs with recruitment as a primary outcome could not be mapped to any of the PRioRiTy I questions and alternatively were categorised as ‘site engagement’ which involved interaction with a recruiting site (e.g. PI visiting site, use of site recruitment plan). Of the five unmapped SWATs, 1 was a protocol, 2 were inactive due to unopened host trials, 1 is unknown due to no response from the registered SWAT contact, and 1 is published.

Three of the Top 10 PRioRiTy I questions (Question 1, 4 and 7) had no recruitment SWATs mapped against them.

Of the 36 retention SWATs, all were mapped onto PRioRiTy II questions (Table 2), with 23 SWATs mapping onto more than one question (e.g. Both Question 4 and Question 19). A majority of SWATs mapped onto Question 4 (n = 24, 67%) which considered the best ways to encourage participants to complete trial tasks. The other top areas of retention activity cover PRIORITY II questions 18 and 19, on strategies to make participants feel valued and how incentives should be implemented.

Table 2.

Retention SWATs mapped to PRioRiTy II questions. The shaded rows highlight the top three areas of activity.

PRioRiTy II question	Number of SWATs mapped to question n (%) (total number of retention SWATs = 36)
1. What motivates a participant’s decision to complete a clinical trial?	0 (0%)
2. How can trials make better use of routine clinical care and/or existing data collection to improve retention?	0 (0%)
3. How can trials be designed to minimise burden on staff and participants and how does this affect retention?	0 (0%)
4. What are the best ways to encourage trial participants to complete the tasks (e.g. attend follow-up visits, complete questionnaires) required by the trial?	24 (67%)
5. How does involvement of patients/the public in planning and running trials improve retention?	0 (0%)
6. How could technology be best used in trial follow-up processes?	7 (19%)
7. What are the most effective ways of collecting information from participants during a trial to improve retention?	5 (14%)
8. How does a participant’s ongoing experience of the trial affect retention?	1 (3%)
9. What information should trial teams communicate to potential trial participants to improve trial retention?	1 (3%)
10. How should people who run trials plan for retention during their funding application and creation of the trial (protocol development)?	0 (0%)
11. What aspects of trial recruitment processes could be changed to improve retention?	0 (0%)
12. What aspects of trial retention do participants perceive as burdensome and how can these be addressed?	0 (0%)
13. What influence does the relationship between trial staff and participants have on retention?	0 (0%)
14. How does a sense of belonging or being part of something amongst trial participants affect retention?	0 (0%)
15. What are the best approaches for designing and communicating information about trial retention for trial participants?	2 (6%)
16. To what extent (if any) do studies that explore retention procedures before the main trial (i.e. feasibility study) lead to improvements in retention in the main trial?	0 (0%)
17. How can trials make better use of routine clinical care and/or existing data collection to improve retention?	0 (0%)
18. What strategies (e.g. sending Christmas cards or saying ‘thank you’) make participants feel valued and how do they affect retention?	9 (25%)
19. What are the best strategies for using participant incentives (e.g. monetary or non-monetary) and how should they be implemented (e.g. when should they be provided) when collecting information from participants in clinical trials?	10 (28%)
20. How does continuity (e.g. seeing/speaking to the same staff) and consistency (e.g. of trial information) affect retention?	0 (0%)

Some SWATs mapped onto more than one PRioRiTy question leading the sum of percentages to be more than 100%.

Five of the retention PRioRiTY II Top 10 questions had no SWATs mapped against them and included areas such as what motivates a participants decision to participate, use of routine data, and patient and public involvement (See Table 2).

Discussion

Most SWAT activity in both recruitment and retention is focused on a small number of PRioRiTY 1 and 2 questions. This might not be a problem (some focus is a good thing) but there remains much variability within SWATs that appear to be answering the same PRioRiTy question. For example, there is considerable intervention heterogeneity in studies exploring monetary incentives in terms of value (e.g. £5–25) and mode (e.g. voucher, money). There is also a large focus on the way information is provided to participants for recruitment, and also the completion of postal questionnaires to encourage retention, but large variability exists within the repository meaning that conclusive guidance for research conduct cannot be drawn from the evidence.

This variability makes it hard to provide a definitive answer as to whether these interventions are effective at improving recruitment and/or retention, and efforts should be made to work collaboratively and create a strong evidence base by replicating what has been registered or coordinating efforts with trial conduct methodologists. Moreover, there is little evidence on specific populations such as younger or older adults and diverse ethnicities to help trialists select strategies most applicable to their own trial. A concerted effort of multiple evaluations of identical interventions at scale in a range of populations is needed.

The mapping exercise highlights gaps where further research is needed (Tables 1 and 2). Unexplored gaps include how to maximise the potential of routine clinical care for trial recruitment and retention (PRioRiTy I Question 1 and PRioRiTy Question 2). Additionally, there are several gaps in the Top 10 questions exclusive to recruitment or retention that highlight areas for innovation and may be best suited to an exploratory design using qualitative methods, for example, what motivates a participant’s decision to complete a clinical trial and what strategies make participants feel valued? Qualitative enquiry is a means to explore the in-depth views and experiences of participants which may not otherwise be fully captured with quantitative methods alone.

Future consideration should be given to exploring how challenges or motivators to trial participation (both initial and continued) reported by participants can be used to drive the design of SWAT interventions for randomised evaluation. Approaching SWATs in this manner would strengthen existing evidence through the use of sequential research design methods where, for example, an initial qualitative study informs the need for further quantitative investigation and vice versa. An example of where this can be applied is conditional altruism, which describes the willingness to help others but with some perceived benefit to self, and has been identified as an influential reason for trial participation.14 Yet, there are no SWATs registered that target this influence to improve recruitment. Conversely, there are many studies on the use of monetary incentives, but no qualitative evidence to prove if, how and why it is a powerful motivator for the return of outcome data. Only one registered SWAT planned to use qualitative methods to investigate one of the PRioRiTy questions (PRioRiTy 1, Question 5). The absence of qualitative studies from the repository highlights that more discussion of SWATs with qualitative researchers would be helpful, as would the broader conceptualisation of SWATs as including qualitative work.

It should be recalled that the mapping exercise focused on SWATs registered on the SWAT Repository and did not encompass research publicised outside of this, meaning there may be existing research on the PRioRiTy gaps (e.g. Crocker et al., 2018 systematic review and meta-analysis of PPI involvement). This means that this SWAT mapping exercise is not an exhaustive list of all research on recruitment and retention and that some of our gaps may actually have some existing evidence to replicate/build upon. Those wishing to design and run SWATs should consult relevant, systematic reviews, or conduct their own, in addition to looking at the repository to ensure they are fully informed of the current evidence.

There any now many opportunities and resources available to support collaborative SWAT evaluations through the PROMETHEUS team (https://www.york.ac.uk/healthsciences/research/trials/research/swats/prometheus/) and Trial Forge SWAT Network (https://www.trialforge.org/2021/06/swat_network/), which provide guidance and advice on all aspects of SWAT design, implementation, analysis and reporting. A list of priority replications has been provided by Treweek et al. (2018¹⁵) and Gillies et al. (2021¹⁶) Cochrane systematic reviews in addition to this SWAT mapping analysis.

Conclusion

There is a mismatch between SWAT activity and prioritised unanswered research questions in recruitment and retention coming from PRioRiTY 1 and 2. Many questions considered research priorities by trial stakeholders have no or very little SWAT activity, while others have many, slightly different intervention evaluations from which it is, or will be, hard to make confident judgements about effects.

Trial teams should consider the PRioRiTy 1 and 2 priorities for recruitment and retention, respectively, when designing their SWATs. Moreover, in many cases replication is likely to make a greater contribution to the evidence base than further innovation.

Supplemental Material

Supplemental Material - Studies within a trial priorities to improve the evidence to inform recruitment and retention practice in clinical trials

Supplemental Material for Studies within a trial priorities to improve the evidence to inform recruitment and retention practice in clinical trials by Cherish Boxall, Shaun Treweek, Katie Gillies; on behalf of the Trials Methodology Research Partnership Trial Conduct Working Group in Research Methods in Medicine & Health Sciences

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the Trials Methodology Research Partnership (MR/S014357/1). The Health Services Research Unit, Institute of Applied Health Sciences (University of Aberdeen), is core-funded by the Chief Scientist Office of the Scottish Government Health and Social Care Directorates. The funders had no involvement in study design, collection, analysis and interpretation of data, reporting or the decision to publish.

ORCID iD

Cherish Boxall

Supplemental Material

Supplement material for this article is available in online.

References

Gheorghiade

Vaduganathan

Greene

, et al. Site selection in global clinical trials in patients hospitalized for heart failure: perceived problems and potential solutions. Heart Failure Reviews 2014; 19(2): 135–152.

Gaba

Bhatt

. The COVID-19 pandemic: a catalyst to improve clinical trials. Nat Rev Cardiol 2020; 17(11): 673–675.

Treweek

Bevan

Bower

, et al.

Trial Forge Guidance 1: what is a Study Within A Trial (SWAT)?

Trials 2018; 19(1): 139.

Bower

Brueton

Gamble

, et al. Interventions to improve recruitment and retention in clinical trials: a survey and workshop to assess current practice and future priorities. Trials 2014; 15(1): 399.

Duley

Gillman

Duggan

, et al. What are the main inefficiencies in trial conduct: a survey of UKCRC registered clinical trials units in the UK. Trials 2018; 19(1): 15.

Gul

Ali

. Clinical trials: the challenge of recruitment and retention of participants. J Clin Nurs 2010; 19(1–2): 227–233.

Chaudhari

Ravi

, et al. Recruitment and retention of the participants in clinical trials: challenges and solutions. Perspect Clinical Research 2020; 11(2): 64–69.

Fogel

. Factors associated with clinical trials that fail and opportunities for improving the likelihood of success: a review. Contemp Clinical Trials Communications 2018; 11: 156–164.

Sully

BGO

Julious

Nicholl

. A reinvestigation of recruitment to randomised, controlled, multicenter trials: a review of trials funded by two UK funding agencies. Trials 2013; 14(1): 166.

10.

Tudur Smith

Hickey

Clarke

, et al. The trials methodological research agenda: results from a priority setting exercise. Trials 2014; 15(1): 32.

11.

Healy

Galvin

Williamson

, et al. Identifying trial recruitment uncertainties using a James Lind Alliance Priority Setting Partnership – the PRioRiTy (Prioritising Recruitment in Randomised Trials) study. Trials 2018; 19(1): 147.

12.

Brunsdon

Biesty

Brocklehurst

, et al. What are the most important unanswered research questions in trial retention? A James Lind Alliance Priority Setting Partnership: the PRioRiTy II (Prioritising Retention in Randomised Trials) study. Trials 2019; 20(1): 593.

13.

The Northern Ireland Network for Trials Methodology Research . SWAT Store. https://www.qub.ac.uk/sites/TheNorthernIrelandNetworkforTrialsMethodologyResearch/SWATSWARInformation/Repositories/SWATStore/

14.

McCann

Campbell

Entwistle

. Reasons for participating in randomised controlled trials: conditional altruism and considerations for self. Trials 2010; 11(1): 31.

15.

Treweek

Pitkethly

Cook

, et al. Strategies to improve recruitment to randomised trials. Cochrane Database Syst Rev 2018; 2(2): MR000013.

16.

Gillies

Kearney

Keenan

, et al. Strategies to improve retention in randomised trials. Cochrane Database Syst Rev 2021; 3(3): MR000032.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB