Are retention strategies used in National Institute for Health and Care Research,Health Technology Assessment trials supported by evidence for their effectiveness? A systematic mapping review

Abstract

Background and Aims

Poor retention of trial participants is common and can result in significant methodological, statistical, ethical, and financial challenges. To improve trial efficiency, we aimed to assess the extent to which commonly used strategies to retain participants within trials are supported by evidence for their effectiveness.

Method

A systematic methodological review was carried out to identify commonly used retention strategies in National Institute for Health and Care Research (NIHR) Health Technology Assessment (HTA) trials (January 2020–June 2022). Strategies were then mapped to evidence for their effectiveness from the most recent Cochrane retention review (published 2021), and a future Study Within A Trial (SWAT) priority list was created.

Results

Amongst 80 trials, the most frequently reported retention strategies were: flexibility with data collection method/location (53%); participant diaries (38%); use of routine data (29%); PPI input (26%); telephone reminders for participants (26%); postal reminders for participants (25%); monitoring approaches (21%); offering flexibility with timing of data collection (20%); pre-paid return postage (18%); prioritising collection of key outcomes (15%); and participant newsletters (15%). Out of the 56 identified strategies, mostly no, very low or low evidence for their effectiveness was identified (64%; 14%; 13% respectively).

Discussion and Conclusions

Commonly used retention strategies are lacking good quality evidence for their effectiveness. The findings support the need for more SWATs and help identify priority areas for future SWAT research. These priorities could be used with other priority lists to inform future SWAT conduct.

Keywords

Study within a trial retention randomised controlled trial methodological research recruitment and retention recruitment,retention and compliance sampling randomised trials research designs & methods

Introduction

Background and rationale

Randomised Controlled Trials (RCTs) are considered the gold standard for clinical research due to their ability to reduce bias and confounding variables.¹ The random allocation of participants to study arms is essential to eliminate selection bias by balancing known and unknown participant characteristics between groups. Attrition bias, a form of selection bias, occurs when there is a difference in the characteristics of patients who are lost to follow-up by initial group allocation. Additionally, if participants withdraw before data collection time points, are lost to follow up, or provide incomplete patient-reported outcomes this results in missing data, which can question the trial’s external validity.² Poor retention will also reduce the trial’s statistical power, reducing the chance of a true effect being identified.

A review of National Institute for Health and Care Research (NIHR), Health Technology Assessment (HTA) funded trials found the median attrition rate to be 12%.³ This is higher than the 5% participant loss to follow-up which has been suggested to result in low risk of bias, and anything over 20% may seriously threaten the trial’s internal validity.⁴ A survey by Kearney et al.⁵ found that 78% of Chief Investigators (CIs) of NIHR HTA funded trials, recognised the challenges of participant retention at the beginning of the trial and implemented strategies within the trial design to overcome this. Adjusting the sample size in anticipation of missing data was also highlighted as common practice. Whilst this method maintains study power, it does not address any bias on outcomes caused by attrition. In addition, attrition is responsible for research waste. A recent study estimated the average cost per participant in trials funded by the NIHR HTA programme to be £2987.⁶ If retention rates were routinely higher than they are now, then the average trial could be smaller, and its costs would be reduced.

The strongest evidence for the effectiveness of retention strategies mainly comes from randomised trials of different retention methods which are often referred to as a Study Within A Trial (SWAT). These are self-contained studies embedded within a host trial, which evaluate an aspect of trial methodology.⁷ Evidence suggests that more randomised retention SWATs are needed. A recent Cochrane review evaluated 52 retention strategies and found none to be supported by Grading of Recommendations, Assessment, Development and Evaluation (GRADE) high-certainty evidence, and most retention interventions had only been evaluated in just one study.⁸

This review reports retention strategies that are used within recently published NIHR HTA trials, examines what justification exists for choice of retention strategy, and determines the extent that these strategies are supported by robust evidence for their effectiveness. Previous research has reported commonly used retention strategies by surveying UK CTUs⁵ but no published reviews have established current retention practice through reporting the strategies used in recently published NIHR HTA trials. Trials funded by the HTA programme have been chosen to examine current retention practice in this review for several reasons. Regardless of the trial’s outcome, virtually all trials funded by this programme are required to publish a report, and as these reports are extensive, more retention information is expected to be ascertained compared with trials published in traditional peer reviewed journals. No reviews have been identified that map retention strategies used in these trials, to evidence for their effectiveness. This review aims to address the current gap in the literature by assessing what strategies are actually used in practice to reduce attrition and if these are grounded in evidence.

Review objectives

1. To identify retention strategies that have been used in NIHR HTA trials published from January 2020 to June 2022.

2. To map strategies that are currently being used against evidence for their effectiveness from the Cochrane retention review.⁸

3. To create a priority list for future retention SWATS, using insight from PRIORITY 2’s important unanswered retention questions.⁹

4. To understand trial teams’ justification for using certain strategies.

Methods

A protocol for this review was registered prospectively on the Open Science Framework,¹⁰ The review was reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement where appropriate.¹¹

Eligibility criteria

Due to logistics, and the desire to include recently conducted RCTs, only trials published within the last 30 months were considered for inclusion. This review was interested in retention strategies used across all clinical areas and patient types, so there were no exclusion criteria on these factors. The inclusion and exclusion criteria are summarised below. Cluster trial designs were excluded as randomisation is at a site-level, and not patient-level so retention strategies may be different compared with non-clustered designs.

Inclusion criteria

• Trials published in the NIHR HTA Journals Library from January 2020 to June 2022

• Patient-level, parallel randomised trials in any clinical area

Exclusion criteria

• Cluster trial designs

• Feasibility studies

• External pilot studies

• Cross-over trials

• Phase I and Phase II trials

• N-of-1 trials

Search strategy

Due to the broad inclusion criteria, this review did not require an extensive or highly sensitive search strategy. ‘RCT’ was entered into the search bar within the NIHR HTA Journals Library, and filters were applied to identify trials published in 2020, 2021, and up to June of 2022.

Data management

Systematic review software Covidence,¹² was used to manage search results, independently screen studies and resolve any disagreements.¹³ This enabled efficient use of time and reduced the risk of human error. EndNote,¹⁴ was used to manage bibliographic references.¹⁵

Study selection

All search results were screened by two reviewers independently (RW and AP), in line with the recommended gold standard method.¹⁶ Titles and abstracts were screened first in accordance with the eligibility criteria. A third researcher (DT) was on hand to resolve disagreements if needed. Agreement was reached for inclusion/exclusion of all studies after reviewing titles and abstracts, and so full text assessment of studies took place simultaneously with data extraction.

Data extraction

A data extraction form was created in Microsoft Excel,¹⁷ and initially piloted with 13 eligible trials from 2022. Following the pilot, any necessary changes were made to the form. If data were missing, the published trial protocol was searched on Google Scholar,^18,19 to identify if the missing data item was recorded in the protocol. If the required information was still not located, the full trial protocol was searched via the NIHR webpage, which typically publishes the protocol of all NIHR funded trials. The second reviewer (AP) checked extraction accuracy for the first 16 (20%) studies by cross-checking each extracted data item. For the remaining studies, the second reviewer carried out spot checks for accuracy, and helped resolve any data items that had been queried by the lead reviewer (RW). Any data items that remained missing for eligible trials were marked as “NR” (not reported).

Data items

The data items that were extracted are included in Supplemental_material_1 along with details of any additional resources that were required to extract the data item, and where necessary, precise definitions of the data items.

Retention strategies were extracted when they were specifically reported as retention strategies by the trial, and in cases where a method was reported that was assumed by the lead reviewer that it would affect retention. Therefore, judgements had to be made, based on the available evidence reported in the HTA reports and trial protocols.

Data analysis

Characteristics of the included studies were narratively synthesised, and key comparative details tabulated. Retention characteristics of the included trials were also narratively synthesised, including detail on trials that included SWATs and if this related to retention; trials that included internal pilot studies with retention criteria; commonly reported reasons for missing data; and attrition summaries.

Retention strategies were identified for each trial and categorised, using guidance from the ORRCA retention research domains.²⁰ Strategies were then ranked according to their frequency of use, and all strategies were mapped to the results of the Cochrane retention review by Gillies et al.⁸ for evidence of their effectiveness (which is the most up to date systematic review of SWAT retention evidence). From this point onwards, this review is referred to as the Cochrane retention review. If for example a trial reported using multiple reminders of the same method (e.g., two postal reminders) this counted as one ‘occurrence’ of the strategy. This logic was consistently applied. A short narrative overview of retention strategy characteristics within the included trials was produced.

For each strategy, the following details from the Cochrane retention review were reported:

• If any relevant evidence existed

• GRADE certainty of evidence (very low, low, moderate, high)

• The intervention and comparator for the evidence

• The Risk Difference (RD)

• 95% Confidence Intervals

• Number of studies included in the meta-analysis

• The “conclusion” regarding the strategy’s effectiveness

For each strategy, evidence was sought that compared the intervention to usual follow-up. If such evidence did not exist this was indicated by “no evidence.” If additional evidence was available (for example evidence for telephone reminder vs usual follow up, and evidence for telephone reminders vs postal reminders) these other comparators were reported. If evidence was only available as a comparator against another strategy (e.g., the only evidence for telephone reminders was telephone reminders vs postal reminders, “no evidence” was reported, but this further evidence was discussed.

Strategies were then prioritised for future SWAT research, based on their frequency of use, the available evidence for their effectiveness, and the degree that it was felt they aligned with the PRIORITY 2 important unanswered retention research questions.⁹ With insight from these prioritisation factors, a list of priority SWAT retention research topics was created.

To map the identified retention strategies to the PRIORITY 2 questions, the reviewers compared each identified strategy to the list of top 10 PRIORITY 2 questions. If the reviewers decided the PRIORTY 2 question aligned with the identified strategy in the review, the PRIORTY 2 question number(s) was recorded next to the identified retention strategy. For example, any strategies that were aimed at encouraging participants to complete follow up (e.g. reminders, pre-paid postal strategies, newsletters) would be deemed aligned with the PRIORTY 2 question: “What are the best ways to encourage trial participants to complete the tasks required by the trial?.”

Results

Included studies

Between January 2020 and June 2022, 104 records were identified when ‘RCT’ was applied as a search term. No duplicate records were identified. Following independent screening with the eligibility criteria, two discrepancies between reviewers arose, which were discussed and resolved. A total of 26 records were excluded at the title and abstract screening stage, and no further studies were excluded during full text assessment. Some studies had multiple reasons for exclusion, but the main reasons were cluster (n = 10) and feasibility trials (n = 13). Three studies were excluded as they were not RCTs. A total of 78 records were included in the review, encompassing 80 trials. Two records^21,22 reported results from two RCTs in their reports. Supplemental_material_2 reports the included trials. The study selection process highlighting the numbers of records at each stage, and detailed reasons for exclusion is reported in Figure 1.

Figure 1.

PRISMA flow diagram (adapted from Page et al.,¹¹).

Characteristics of included studies

The characteristics of the 80 trials included in this review, are shown in Table 1. Most trials were of standard parallel design (95%, n = 76) and had two trial arms (81%, n = 65). All trials were multicentre, and the average number of sites was 37. The most common intervention type was drug (28%, n = 22), followed by procedure/surgery (21%, n = 17). Thirty-three percent of trials had an ‘Other’ intervention type (categorised according to the ISRCTN trial registration, no further breakdown of this category is available). Most trials were conducted solely in the UK (91%, n = 73), and 9% were multi-national (n = 7). Active controls were most common (75%, n = 60), followed by placebo controls (15%, n = 12). Few trials used other types of inactive controls (10%, n = 8). Patient blinding to intervention allocation was reported in one quarter of the trials (n = 20), but the majority did not implement patient blinding (75%, n = 60). 39% of trials included an internal pilot trial (n = 31), while 61% did not (n = 49). The most common trial setting was hospitals (68%, n = 54), and Urology and Oncology were the most common clinical areas under study (13% n = 10, 10% n = 8 respectively). Amongst the included trials, the median number of randomised participants was 518 (mean: 3,151; Interquartile Range (IQR) 362:893), the minimum was 9 (this trial was halted early), and the maximum was 160,921.

Table 1.

Characteristics of included trials.

Characteristics		n (%)
Trial design (n = 80)	Standard parallel trial	76 (95)
	Factorial 7^a	3 (4)
	Cohort RCT^a	1 (1)
Trial arms (n = 80)	2	65 (81)
	3	11 (14)
	4	4 (5)
Sites (n = 80)	Multicentre	80 (100)
Sites (n = 80)	Single	0 (0)
Intervention type (n = 80)	Drug	22 (28)
	Procedure/surgery	17 (21)
	Behavioural	5 (6)
	Mixed	5 (6)
	Device	5 (6)
	Other^b	26 (33)
Trial coordinating country (n = 80)	UK	73 (91)
Trial coordinating country (n = 80)	Multi-national	7 (9)
Control type (n = 80)	Active	60 (75)
	Placebo	12 (15)
	Other inactive control	8 (10)
Patient blinded (n = 80)	No	60 (75)
Patient blinded (n = 80)	Yes	20 (25)
Internal pilot trial (n = 80)	No	49 (61)
Internal pilot trial (n = 80)	Yes	31 (39)
Setting (n = 80)	Hospital	54 (68)
	Mixed	7 (9)
	Community	6 (8)
	General practice	4 (5)
	Other^c	9 (11)
Clinical area (n = 80)	Urology	10 (13)
	Oncology	8 (10)
	Gynaecology and obstetrics	7 (9)
	Musculoskeletal	7 (9)
	Neurology	6 (8)
	Stroke	4 (5)
	Mental health	3 (4)
	Ophthalmology	3 (4)
	Paediatrics	3 (4)
	Respiratory	3 (4)
	Other^d	26 (4)

^aThese trials also followed a parallel design, but their design was ‘non-standard’ so their other design details were deemed useful to report separately.

^bTrial intervention type was categorised as ‘Other’ when the ISRCTN trial registration reported the intervention type as ‘Other’. No further breakdown of this category is available.

^cDental practices (n = 2), online (n = 2), sexual health centres (n = 2), ambulance services (n = 1), health visiting services (n = 1), occupational therapy services (n = 1).

^dDementia (n = 2), dentistry (n = 2), orthopaedic (n = 2), otorhinolaryngology (n = 2), surgery (n = 2), trauma (n = 2), cardiology (n = 1), chronic disease (n = 1), colorectal (n = 1), critical care (n = 1), cystic fibrosis (n = 1), dermatology (n = 1), diabetes (n = 1), falls prevention (n = 1), gastrointestinal (n = 1), multiple sclerosis (n = 1), nephrology (n = 1), nephrology/geriatric (n = 1), sexual health (n = 1), smoking prevention (n = 1).

Summary of retention-associated activity in the included trials

Attrition and reasons for missing data

The majority of included trials adjusted their sample size calculation to account for expected participant attrition (89%, n = 70). Three of these studies mentioned they had adjusted for missing data/participant attrition but did not clearly state the percentage adjustment. Of those that reported an adjusted sample size, the mean adjustment was 16% (mode: 20%, median: 17%) with adjustments ranging from 3% to 30% (IQR 10%:20%). The mean attrition rate at the primary outcome analysis point was 12% (median: 11%), ranging from 0% to 33% (IQR 2%:19%). Five trials were not included in this summary as one did not clearly report a final attrition figure, and four did not reach target recruitment levels and were stopped early. These four trials were excluded as their attrition figure would not be comparable to the other included trials and did not represent a final figure based on the trial’s original expected sample size.

Of the included trials, the most common reasons for missing outcome data, were due to participant withdrawal, participants being lost to follow-up, and participant death. Reasons for participant withdrawal were varied, and often were not reported. Of those that reported participant withdrawal reasons, common reasons were due to the treatment allocation or perceived effectiveness of treatment; the burden of data collection; and personal reasons such as the participant moving away or becoming ill. It was not possible to report this data quantitively due to inconsistency of reporting.

SWATs and internal pilot studies

Just under one third of the trials included a SWAT (30%, n = 24), with a total of 26 SWATs identified. Mostly one SWAT was included per trial (n = 23), but one trial²³ included three. Out of the 26 identified SWATs, the most common type was a non-randomised evaluation of trial processes using qualitative methods (54%, n = 14). These studies were commonly reported as nested, embedded, or sub-studies, but as they explored aspects related to trial processes e.g., recruitment, they were classed as SWATs in this review. Randomised SWATs occurred slightly less frequently than non-randomised evaluations (42%, n = 11), and the majority of these evaluated the effectiveness of different retention interventions (n = 7). The other four randomised SWATs evaluated strategies relating to recruitment (n = 3) and site initiation (n = 1). One observational SWAT was identified,²⁴ where the study reported the addition of a telephone call to improve collection of key outcomes, and reported the effect that this had on outcome collection. All the non-randomised qualitative evaluations of trial processes had aims related to recruitment, but one also had aims related to understanding retention.²⁵ Supplemental_material_3 shows the trials that included SWATs, and the details of their included SWATs.

Of the 31 trials that included pilot studies, roughly one third (n = 11) had criteria relating to retention to inform continuation to the main study.

Identified retention strategies with mapping

Amongst the included trials, 56 retention strategies were identified, with 409 total occurrences. Most trials used several retention strategies, with the mean number of retention strategies identified per trial as 5 (IQR 3:7). Two trials had no retention strategies identified^26,27 and the maximum strategies identified per trial was 18.²⁸

Table 2 shows the retention strategies identified in the included trials, ranked by the frequency of each strategy’s use. Included in the table also, is the relevant ORCCA retention domain for each strategy, (strategies that did not clearly fit into ORRCA domains were attributed ‘Other’), the definition used in this review to identify each strategy, and mapping of each strategy to the Cochrane retention review for evidence for its effectiveness. Out of the 56 identified strategies, 36 strategies (64%) had no evidence for their effectiveness, 8 (14%) had very low evidence, 7 (13%) had low evidence, 2 (4%) had very low and low evidence, 1 (2%) had very low and moderate evidence, and 1 (2%) had moderate evidence that the strategy may reduce retention (diaries), with a final 1 (2%) having moderate evidence that it probably increases retention (giving a pen at recruitment).

Table 2.

Identified retention strategies with mapping, ranked by frequency of use.

Ranking (by frequency of use)	ORCCA domain	Strategy	Definition	No. of trials that used strategy (n=80) N (%)	Evidence grade	Risk difference (95% CI), No. of studies in the meta-analysis	Conclusion
1	Data collection	Flexibility with location/method of data collection	Offering multiple different locations or methods of data collection to the participant for a particular outcome measure. Flexibility doesn’t have to be offered to all participants, but could be a strategic decision, for example offering telephone completion to non-responders of a postal questionnaire.	42 (53)	No evidence	N/A	N/A
2	Participants	Diaries	Any use of diaries that were either mandatory or optional for participants or participants’ carers/parents/guardians to fill out as part of the trial process. Purpose does not have to be directly related to retention. Not counted if they formed part of the actual intervention delivery.	30 (38)	Moderate (diary with usual follow-up vs no diary)	-3% (-4% to -2%), 2	“Probably reduces retention”
3	Data collection	Use of routine data	Using ONS/HES/GP/hospital records etc. to obtain data for outcome measures.	23 (29)	No evidence	N/A	N/A
5	Central study management	PPI approaches	Using PPI to advise on strategies/methods that clearly relate to retention or participant engagement.	21 (26)	Low (peer led follow up strategy vs usual follow up)	22% (14% to 30%), 1	“May result in large increase in retention”
5	Participants	Telephone reminders	Telephone contact with participants after the data collection time point has been reached, but outcome measure not received.	21 (26)	Low (telephone reminders vs usual follow-up); Low (telephone reminders vs postal reminders)	-1% (-18% to 15%), 1; -19% (-33% to -5%), 1	“May result in little or no difference to retention”; “May result in a large increase in retention”
6	Participants	Postal reminders	Postal contact with participants after the data collection time point has been reached, but outcome measure not received.	20 (25)	No evidence	N/A	N/A
7	Central study management	Monitoring approach	Monitoring of missing data or upcoming appointments- not relating to data validation monitoring.	17 (21)	No evidence	N/A	N/A
8	Data collection	Flexibility with timing of data collection	Accepting outcome measures with a window of flexibility around the outcome measure time point (e.g., stating +/- 6 days for the 4-week questionnaire).	16 (20)	No evidence	N/A	N/A
9	Data collection	Pre-paid return postage	Providing participants with free post, pre stamped envelopes etc.	14 (18)	No evidence	N/A	N/A
11	Participants	Newsletter	Sending out a participant newsletter during the trial duration.	12 (15)	Very low (newsletter vs usual follow up)	-0% (-4% to 3%), 4	“Very uncertain”
11	Data collection	Prioritising collection of key outcomes	E.g., collecting minimum data items by an alternative method, shortening data collection measure to just collect key outcomes, putting outcomes in a priority order.	12 (15)	Very low (short vs usual questionnaire)	0% (-8% to 8%), 3	“very uncertain”
14	Participants	Electronic/email reminders	Email/web contact with participants after the data collection time point has been reached and outcome measure not received.	10 (13)	Low (electronic reminder vs usual follow-up); Low (electronic prompt vs electronic reminder)	1% (-4% to 6%), 3; 2% (-6% to 9%), 1	“May result in little or no difference to retention”; Electronic reminders “may increase retention slightly”
14	Participants	Extra contact	Additional contact with participants during the trial (e.g., updating or checking in with participants by telephone/post etc.) Excludes celebration cards (see “extra contact-cards”).	10 (13)	Low (Frequency of telephone contact comparing annual contact vs contact only at baseline)	8% (1% to 15%), 1	“May increase retention”
14	Participants	Monetary incentive (voucher/cash)	Monetary gift, unconditional of an action.	10 (13)	Very low (monetary incentive vs monetary reward); Very low (monetary incentive vs lottery inclusion); Low (monetary incentive vs no incentive)	-0% (-7% to 6%), 4; 2% (-9% to 12%), 1; 7% (4% to 11%), 3	“Very uncertain”; “Very uncertain”; “May increase retention”;
17	Participants	Monetary reward (voucher/cash)	Monetary gift, conditional of an action.	9 (11)	Very low (monetary reward vs usual follow up); Moderate (monetary rewards delivered with pre notification/reminder letter),	2% (-3% to 6%), 3; 9% (3% to 15%), 1	“Very uncertain”; “Probably increases retention”;
17	Data collection	Data collection during routine care	Collecting outcomes during routine care appointments, or using similar methods to routine care.	9 (11)	No evidence	N/A	N/A
17	Participants	Text reminders	Text contact with participants after the data collection time point has been reached and outcome measure not received.	9 (11)	No evidence	N/A	N/A
20	Other	Contacting participants after missed assessment visits	After in-person data collection has been missed- contacting participants to rearrange/follow up.	8 (10)	No evidence	N/A	N/A
20	Participants	Relationship with clinical staff	Any strategy that appears to help maintain good relationships between participants and clinical staff, e.g., supportive phone calls, helplines, continuity of data collection with the same staff member.	8 (10)	No evidence	N/A	N/A
20	Participants	Emphasising participants' value	Emphasising to participants the importance of attending follow up, and the participants’ value to the trial. E.g., through thank you messages.	8 (10)	Low (societal benefit message vs usual follow-up); very low (certificate of appreciation vs no certificate)	-0% (-4% to 4%), 1	“may result in little or no difference to retention”
22	Data collection	Home visits	Offering participants the option for clinical staff to visit them in their home to collect an outcome measure.	6 (8)	No evidence	N/A	N/A
22	Participants	Supporting participation	Offering to pay participants’ expenses e.g., travel, prescription costs.	6 (8)	No evidence	N/A	N/A
25	Data collection	Trial document appearance strategies	E.g., use of a cover letter with questionnaire, trial tagline, coloured documents and/or appealing trial documents.	5 (6)	Low evidence for personal form-amongst return postage strategies evidence (return postage strategies vs standard return postage)	4% (-0% to 9%), 3	“may increase retention slightly”
25	Participants	Trial webpage	Use of a trial webpage that participants can access.	5 (6)	No evidence	N/A	N/A
25	Data collection	Increased/decreased frequency of data collection	Increasing or decreasing number of data collection time points to increase retention of participants and/or their data.	5 (6)	No evidence	N/A	N/A
28	Participants	Contact information	Providing participants with clear contact information for trial team/trial clinicians.	4 (5)	No evidence	N/A	N/A
28	Data collection	Pre-addressed envelopes	If participant needs to return information, providing a pre-addressed envelope.	4 (5)	No evidence	N/A	N/A
28	Participants	Pre-notification	Contact made regarding the data collection measure before participants have received it.	4 (5)	Low (pre-notification cards vs no card)	3% (-3% to 10%), 2	“May increase retention slightly”
36	Participants	Extra contact- cards	Sending participants seasonal/celebration cards e.g., Christmas or birthday cards.	3 (4)	No evidence	N/A	N/A
36	Site and staff	Data collection training	Staff undergo data collection training concerning data completeness.	3 (4)	No evidence	N/A	N/A
36	Site and staff	Monitoring visits	Site monitoring visits to check factors related to retention/data completeness.	3 (4)	Very low (site monitoring compared vs no visits)	-5% (-20% to 10%), 1	“Evidence very uncertain”
36	Site and staff	Newsletter- maintaining staff engagement	Use of staff newsletter.	3 (4)	No evidence	N/A	N/A
36	Participants	Postal prompts	Postal contact made with the participant regarding the data collection measure, before the data collection time point has been reached.	3 (4)	No evidence	N/A	N/A
36	Data collection	Questionnaire design adaptions	Changes made to the questionnaire design with the aim to increase retention.	3 (4)	Very low (short vs usual questionnaire)	0% (-8% to 8%), 3	“very uncertain”
36	Site and staff	Regular contact with staff/sites	Regularly contacting sites/site staff (non-newsletter methods).	3 (4)	No evidence	N/A	N/A
36	Data collection	Visit schedule	Strategies related to the visit schedule to increase retention (e.g., using visit trackers for appointments, spacing out outcome measures, scheduling follow up appointments at prior appointment).	3 (4)	No evidence	N/A	N/A
43	Site and staff	Electronic reminders	Email/web-based contact concerning outcome measures with sites/staff after data collection time point has been reached.	2 (3)	No evidence	N/A	N/A
43	Site and staff	Monetary incentives	Monetary gift, unconditional of an action.	2 (3)	No evidence	N/A	N/A
43	Participants	Other non-monetary incentive	Non-monetary gift, unconditional of an action (e.g., trial membership card or trial welcome pack).	2 (3)	No evidence	N/A	N/A
43	Participants	Pen non-monetary incentive	Gifting a pen, unconditional of an action.	2 (3)	Low (pen vs no pen)	2% (0% to 4%), 5	“May increase retention slightly”
43	Site and staff	Prompt	Email/web-based contact concerning outcome measures with sites/staff before data collection time point has been reached.	2 (3)	Very low (site prompt for upcoming assessment vs no prompt)	-3% (-13% to 7%), 1	“Evidence very uncertain”
43	Data collection	Same frequency of data collection as routine follow-up	Data collection time points/outcome measures at same frequency of routine follow up.	2 (3)	No evidence	N/A	N/A
43	Participants	Telephone prompts	Telephone contact concerning outcome measure before data collection time point has been reached.	2 (3)	Very low (telephone prompt vs usual follow- up)	1% (-10% to 12%),2	“Very uncertain”
56	Participants	Non-monetary reward	Non-monetary gift, conditional of an action (e.g., toy or certificate upon attending a clinic visit).	1 (1)	No evidence	N/A	N/A
56	Participants	Behavioural interventions	Any intervention aiming to increase retention through behavioural methods.	1 (1)	Very low (theory informed vs usual cover letter)	3% (-2% to 8%), 4	“Very uncertain”
56	Participants	Conditional reward (prize draw)	Entry into a prize draw conditional of an action.	1 (1)	Very low (lottery vs usual follow up)	-1% (-3% to 2%), 1	“Very uncertain”
56	Other	Updating contact details	Creation of an easy pathway for participants to update the study team with changes to their contact details.	1 (1)	No evidence	N/A	N/A
56	Central study management	CRF design	Interventions aiming to increase retention through changes to the design of the case report form (note related to whole CRF, not to individual questionnaires).	1 (1)	No evidence	N/A	N/A
56	Participants	Cultural considerations	E.g., carer asked to help non-English speaking participants with data collection).	1 (1)	No evidence	N/A	N/A
56	Study design	Impact of recruitment- giving a pen	An intervention at the time of recruitment that may affect retention, e.g., giving a pen at recruitment.	1 (1)	Moderate (giving a pen at recruitment compared to no- pen)	20% (7% to 32%), 1	“Probably increases retention”
56	Other	Obtaining back up contact details for the participant	The participant is asked to provide additional contact details that differ from main details used by the trial team.	1 (1)	No evidence	N/A	N/A
56	Site and staff	Other relevant training	Staff training (not related to data completeness) that may increase retention.	1 (1)	No evidence	N/A	N/A
56	Study design	Other trial design (cohort RCT)	If the design of the trial is likely to increase retention.	1 (1)	No Evidence	N/A	N/A
56	Study design	Run in period	Use of a run-in period in the trial design.	1 (1)	No Evidence	N/A	N/A
56	Central study management	Study identity/branding	Distributing trial documents to participants with trial branding, creating a study identity.	1 (1)	No evidence	N/A	N/A
56	Other	Telephoning participants to clarify missing items from postal questionnaires	As stated.	1 (1)	No evidence	N/A	N/A
N/A	Participants	Incentive/reward (type not stated)	Conditional or unconditional gift.	2 (3)	N/A	N/A	N/A
N/A	Participants	Prompts (type not stated)	Contact made with the participant regarding the data collection measure, before the data collection time point has been reached.	3 (4)	N/A	N/A	N/A
N/A	Participants	Reminders (type not stated)	Any contact with the participant after the data collection time point has been reached regarding missed outcome measure.	6 (8)	N/A	N/A	N/A

To note, within the ORRCA ‘participant domain’ category, there were 11 occurrences where not enough detail on the strategy was provided to adequately classify the strategy. Three categories were created and are included at the bottom of the table for reference, but these categories were not included in the ranking, mapping, or prioritisation exercise. These were: reminders-type not stated (n = 6); prompt - type not stated (n = 3); and incentive/reward-type not stated (n = 2).

Commonly used strategies amongst the included trials, generally were participant or data-collection orientated. The most frequently occurring strategy was offering flexibility with data collection method/location, which was used by over half the included trials (n = 42, 53%). This was followed by participant diaries (n = 30, 38%) and use of routine data (n = 23, 29%). There was a tie at 5^th place (PPI approaches and telephone reminders for participants, both n = 21, 26%).

The following results are reported for strategies that were used by greater than 10% of trials, refer to Table 2 for the remaining strategies. Rankings refer to the commonality of the strategy amongst the included trials.

Postal reminders for participants ranked 6th (n = 20, 25%); monitoring approaches ranked 7th (n = 17, 21%); flexibility with timing of data collection ranked 8th (n = 16, 20%); pre-paid return postage ranked 9th (n = 14, 18%); participant newsletter and prioritising collection of key outcomes ranked joint 11th (both n = 12, 15%); participant electronic/email reminders, extra contact with participants, and monetary incentives for participants ranked joint 14th (all n = 10, 13%); monetary reward for participants, data collection during routine care, and text reminders for participants ranked joint 17th (all n = 9, 11%); and contacting participants after missed assessment visits, relationship with clinical staff, and emphasising participants’ value ranked joint 20th (all n = 8, 10%).

SWAT prioritisation

Table 3 shows the top-10 most frequently used strategies (those ranked 1st – 11th), alongside a summary of the available evidence for their effectiveness, and further mapping against PRIORITY 2⁹ questions (the PRIORITY 2 questions are shown for reference in Table 4). Out of the top-10 most frequently used strategies, no supporting evidence was available for just under half the strategies (n = 5). There was very low evidence (n = 2), and low evidence (n = 3) available in support of the remaining strategies, apart from for participant diaries, which was the only strategy that had evidence that when compared with usual follow-up, it had a negative effect on retention, which was at moderate certainty.

Table 3.

Top-10 most frequently occurring retention strategies, with mapping to evidence, PRIORITY 2 questions, and example SWAT research questions.

Top-10 strategies	No. of trials that used strategy (n=80) n (%)	Cochrane Evidence	Relevant Top 10 PRIORITY 2 questions	Example SWAT research questions
Flexibility with data collection method/location	42 (53)	No evidence	3, 4, 7	Home or clinic follow-up offered vs clinic follow-up only, postal or telephone follow-up offered vs telephone only
Participant diary	30^(a)(38)	Moderate (negative effect on retention)	3, 4, 7	Diaries to record compliance information vs no diaries, diaries to aid data collection vs no diaries
Use of routine data	23 (29)	No evidence	2, 3, 6, 7	Use of routine data vs participant reported data
PPI approaches	21 (26)	Low evidence	5	PPI led retention strategies vs no PPI input into retention strategies, PPI advice on data collection frequency vs no advice sought on data collection frequency
Telephone reminders for participants	21 (26)	Low evidence	4	Telephone reminders vs usual follow-up (more evidence needed), telephone reminders vs postal reminders (more evidence needed)
Postal reminders for participants	20 (25)	No evidence^(b)	4	Postal reminders vs no reminders, postal reminders vs postal prompts
Monitoring approach	17 (21)	No evidence	6	Automated missing data checks vs manual checks, weekly data monitoring vs monthly
Flexibility with timing of data collection	16 (20)	No evidence	3, 4, 7	1-week flexibility with data collection time points vs no flexibility, 1-week flexibility vs 2-week flexibility
Pre-paid return postage	14 (18)	No evidence^(c)	3, 4, 7	Pre-paid return postage vs no pre-paid return postage, pre-paid 1st class return postage vs second class return postage
Prioritising collection of key outcomes	12 (15)	Very low evidence for shortening questionnaire length	3, 4, 7	Including outcomes in priority order vs outcomes in any order, offering minimum data collection of key outcomes by an alternative method (e.g., by telephone) vs offering minimum data collection of key outcomes via original method (e.g., by post)
Participant newsletter	12 (15)	Very low evidence	1,4, 6, 8	Newsletter vs usual follow-up (more evidence needed), monthly newsletter vs quarterly newsletter

^aDue to evidence that the inclusion of a diary possibly reduces retention, this figure includes all occurrences of diaries used in the included trials, regardless of if they had a purpose related to retention, as all occurrences were deemed relevant.

^bThere was no evidence for postal reminders versus usual follow-up. There was however evidence for the effectiveness of telephone reminders versus postal reminders, where telephone reminders may be more effective.

^cThere was no evidence for the effectiveness of pre-paid return postage versus not using pre-paid return postage, but there was relevant evidence for standard return postage versus other return postage strategies, where it was concluded that return postage strategies “may increase retention slightly” compared to standard return postage.

Table 4.

PRIORITY 2 important unanswered questions ranking.⁹

PRIORITY 2 important unanswered questions ranking (adapted from Brunsdon et al.⁹)
1	What motivates a participant’s decision to complete a clinical trial?
2	How can trials make better use of routine clinical care and/or existing data collection to improve retention?
3	How can trials be designed to minimise burden on staff and participants and how does this affect retention?
4	What are the best ways to encourage trial participants to complete the tasks (e.g., attend follow-up visits, complete questionnaires) required by the trial?
5	How does involvement of patients/the public in planning and running trials improve retention?
6	How could technology be best used in trial follow-up processes?
7	What are the most effective ways of collecting information from participants during a trial to improve retention?
8	How does a participant’s ongoing experience of the trial affect retention?
9	What information should trial teams communicate to potential trial participants to improve trial retention?
10	How should people who run trials plan for retention during their funding application and creation of the trial (protocol development)?

Given that there was poor quality evidence to support all the top-10 most frequently used strategies, and that all these strategies could be mapped to PRIORITY 2 questions (indicating some alignment in research priorities), these strategies were identified as a priority for further SWAT research. The exception to this logic concerns participant diaries, which had moderate certainty evidence that they reduced retention. This strategy is included as a SWAT priority topic as future SWAT research into diaries is considered a priority. Given their high frequency of use, increased evidence is needed on the certainty of their effect on retention (e.g., moving from moderate, to high evidence certainty), and research is needed to determine in what situations diaries are most likely to reduce retention. PRIORITY 2 question nine and 10 were unable to be matched to the top-10 frequently used strategies in this review. Example SWAT research questions are displayed next to each priority retention research topic.

Retention strategy justification

Approximately one third of the included trials (30%, n = 24) reported some form of justification for their choice of retention strategies. Input from PPI groups to justify choice of retention strategy was most frequently reported (n = 10). Other justifications were based on literature evidence (n = 3), evidence from the trial’s pilot or previous study (n = 3), trial teams’ experiences (n = 2), and common practice (n = 1). Several trials justified their strategies with a combination of these approaches (n = 5). Justification of strategy may not be relevant to all trials, for example Maheshwari et al.²⁹ who acknowledge that due to their primary outcome being mandatorily reported in health records, they expect minimal attrition. Due to this, they did not report any retention strategies (however use of routine data was recorded as a strategy by the reviewers), so there was no justification expected.

Discussion

This review found a continuing problem with attrition in HTA funded trials (January 2020–June 2022). A total of 80 trials were identified for inclusion, with a median size of 518 randomised participants. There was a mean attrition rate of 12%, which was slightly lower than the mean anticipated attrition rate of 16%. For the average trial of 518 participants, this difference of 4% may on average result in the recruitment of 21 more participants than is necessary (4% of 518 = 20.7). As cost per participant in NIHR trials was previously estimated to be £2987,⁶ this may increase the costs by £62,727 (£2987 × 21) per trial. If this figure was applied across the 80 trials included in this review, the NIHR could potentially save over half a million pounds annually. In the future, if more effective interventions were implemented to reduce retention, then an even lower anticipated attrition rate could be built into sample size calculations, which would further drive down trial costs. In this report, the key attrition reduction strategies used by HTA trialists were identified and mapped against the best evidence from a recent Cochrane review of SWATs. The following section summarises and discusses the key findings.

Principal findings

This review found that the most used retention strategies (Table 4) have mostly low, very low, or no evidence in support of their effectiveness. Participant diaries were the second most common retention strategy (used by 38% of trials) but had moderate certainty evidence that they may reduce retention. As this was the only commonly used strategy that may be harming retention, this is a priority strategy for future SWAT research. Future SWAT research is also recommended for the other commonly used strategies which all need considerably more evidence to support their effectiveness. Example SWAT research questions are displayed next to each priority retention research topic in Table 4. Further transparency is needed surrounding why certain retention strategies are chosen by trial teams, as only approximately one third of the included trials reported some form of justification for their choice, with very few citing literature evidence. Justification for future use of participant diaries is particularly needed, due to the evidence that this strategy may reduce retention.

This review highlighted that only 9% of the included trials embedded a randomised retention SWAT into their design, implying this is not currently common practice. With the clear need to increase the evidence base for the effectiveness of retention strategies, and with randomised SWATs providing the best quality evidence for the effectiveness of such strategies, it is important that future trialists understand the benefits of embedding randomised retention SWATs into their trial design.

Out of the trials in this review that included an internal pilot, only approximately one third had retention criterion, to inform continuation to the main study. Based on this finding, it appears that retention may be commonly being deprioritised, compared with recruitment at this initial study stage. Considering the significant methodological, statistical, ethical, and financial challenges that poor retention can bring, early identification of poor retention is a must to enable appropriate counter strategies to be implemented. We hope that more triallists will consider the value of including retention criterion in the internal pilot.

Strengths

This is the first study that has examined current retention practice of trials published to the NIHR HTA journal library and has mapped strategies used in these trials to evidence for their effectiveness. Trials registered to the NIHR HTA library are required to publish a trial report so risk of publication bias in this review is low, and the detailed level of reporting required by the library ensures all relevant data is likely to have been captured. Retention strategy use has been established directly from the trial reports, which offers a more precise estimate compared to strategies trial teams may say are commonly used. Cochrane reviews are high quality, regularly updated, peer reviewed, systematic in nature, and aim to synthesise all relevant empirical evidence, so the mapping exercise in this review is highly likely to have encompassed most, if not all, available randomised SWAT evidence for each strategy.

Limitations

Trial authors were not contacted for further information, so this review was unable to include data items that were not reported in the HTA report or published trial protocol. If retention strategies were used by trial teams but not clearly reported, then they may not have been captured in this review. The results of this review may not accurately reflect trials published outside of January 2020–June 2022. The HTA programme is publicly funded and funds high quality research that is mostly conducted by CTUs, and so findings may not represent commercially funded trials, international trials, or other non-CTU delivered trials. Using the ORRCA retention research domains to classify retention strategies could be seen as a limitation. Using this structured method to initially identify strategies may have resulted in different classifications and/or grouping of strategies, had the ORRCA domains not been used, which consequently may have affected the mapping process. This review has examined the retention strategies for a sample of large individually randomised controlled trials. We excluded pilot, feasibility and cluster designs as these are likely to use different retention strategies from the ‘standard’ trial design and their findings may not be generalisable to the majority of large individually randomised trials. However, it would be useful to review retention strategies for these designs, particularly for cluster trials. It is possible that relevant evidence may have been identified from the Cochrane review if strategies had been identified and grouped differently. For example, there was evidence in the Cochrane review for telephone versus postal follow-up, but neither telephone follow-up nor postal follow-up were identified as individual retention strategies in this review. Instead, offering flexibility with data collection location/method was identified as the strategy and consequently, there was no evidence in the Cochrane review to directly support this strategy. This review did not consider non-randomised SWAT evidence or cost-effectiveness evidence. The Cochrane retention review included SWATs up to January 2020, and so there may be new relevant evidence for strategies that was unable to be accessed in the mapping exercise in this review.

Comparisons to other studies

Some consensus has been reached on current retention practice and priority SWAT research topics between the results of this review and research by Kearney et al.⁵ Both identify telephone reminders, flexibility of appointment times, pre-paid envelopes, and newsletter strategies as within the top-10 most commonly used retention strategies. Further, offering multiple methods of data collection was identified as the most frequently used strategy in this review, and was reported as the fourth most recommended strategy by CIs to mitigate missing data in Kearney’s research. In addition, just under half of this review’s recommended SWAT priority topics (n = 4) appear in the top 21 research priorities generated by Kearney et al.⁵ Research into participant diaries was identified as a high priority in this review, which is a stark contrast to the ranking of the priority of participant diaries from Kearney’s research.

Bower et al.³⁰ identified six priority recruitment and retention interventions, with none matching the priorities discussed in this review. Differences in results may be explained as some of their identified intervention priorities had a definite focus on recruitment over retention (e.g., observing recruitment), research was conducted prior to the Cochrane retention review⁸ used to inform this review, and different methods were used to establish current retention practice. Apart from the PRIORITY 2 findings,⁹ which were used to inform this research, no further studies were identified that advised priorities for retention research.

Conclusion

This study systematically identified retention strategies from a sample of 80 trials published to the NIHR HTA library and mapped these strategies to evidence for their effectiveness from the Cochrane retention review. Of the 56 retention strategies identified, there were none that had high quality supporting evidence for their effectiveness, and amongst the Top-10 most frequently used strategies, mostly no evidence, low evidence, or very low evidence was identified. Diaries were the second most frequently used strategy, despite moderate certainty evidence of their negative impact on retention. Little justification for the choice of retention strategy used by trial teams was identified in the final report. This study supports the need for more SWATs and helps identify priority areas for future SWAT research. These priorities could be used with other priority lists to improve the efficiency in the conduct of further SWATs.

Supplemental Material

Supplemental Material - Are retention strategies used in NIHR HTA trials supported by evidence for their effectiveness? A systematic mapping review

Supplemental Material for Are retention strategies used in NIHR HTA trials supported by evidence for their effectiveness? A systematic mapping review by Rosalind Way, Adwoa Parker and David J Torgerson in Journal of Research Methods in Medicine & Health Sciences.

Supplemental Material

Supplemental Material - Are retention strategies used in NIHR HTA trials supported by evidence for their effectiveness? A systematic mapping review

Supplemental Material

Supplemental Material - Are retention strategies used in NIHR HTA trials supported by evidence for their effectiveness? A systematic mapping review

Footnotes

Acknowledgements

The Authors acknowledge the contributions of Laura Clark and Izzy Coleman who were involved in initial project conceptualisation discussions.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors. Author AP is funded by the National Institute for Health Research (NIHR) Advanced Fellowship (award reference: NIHR302256). The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.

ORCID iDs

Rosalind Way

Adwoa Parker

David J Torgerson

Supplemental Material

Supplemental material for this article is available online.

References

Akobeng

. Understanding randomised controlled trials. Arch Dis Child 2005; 90: 840–844. DOI: 10.1136/adc.2004.058222.

Bell

Fairclough

. Practical and statistical issues in missing data for longitudinal patient-reported outcomes. Stat Methods Med Res 2014; 23: 440–459. DOI: 10.1177/0962280213476378.

Jacques

Ahmed

Harper

, et al. Recruitment, consent and retention of participants in randomised controlled trials: a review of trials published in the national institute for health research (NIHR) journals library (1997–2020). BMJ Open 2022; 12: e059230. DOI: 10.1136/bmjopen-2021-059230.

Dettori

. Loss to follow-up. Evid Base Spine Care J 2011; 2: 7–10. DOI: 10.1055/s-0030-1267080.

Kearney

Daykin

Shaw

ARG

, et al. Identifying research priorities for effective retention strategies in clinical trials. Trials 2017; 18: 406. DOI: 10.1186/s13063-017-2132-z.

Pirosca

Shiely

Clarke

, et al. Tolerating bad health research: the continuing scandal. Trials 2022; 23: 458. DOI: 10.1186/s13063-022-06415-5.

Treweek

Altman

Bower

, et al. Making randomised trials more efficient: report of the first meeting to discuss the trial forge platform. Trials 2015; 16: 261. DOI: 10.1186/s13063-015-0776-0.

Gillies

Kearney

Keenan

, et al. Strategies to improve retention in randomised trials. Cochrane Database Syst Rev 2021; 3: Mr000032. DOI: 10.1002/14651858.MR000032.pub3.

Brunsdon

Biesty

Brocklehurst

, et al. What are the most important unanswered research questions in trial retention? A James Lind alliance priority setting partnership: the priority II (prioritising retention in randomised trials) study. Trials 2019; 20: 593. DOI: 10.1186/s13063-019-3687-7.

10.

Way

. Are retention strategies used in NIHR HTA trials supported by evidence? A protocol for a systematic mapping review, https://osf.io/8y7z2 (2023).

11.

Page

McKenzie

Bossuyt

, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021; 372: n71. DOI: 10.1136/bmj.n71.

12.

Covidence - better systematic review management, https://www.covidence.org/ (2022).

13.

Kellermeyer

Harnke

. Covidence and rayyan. J Med Libr Assoc. 2018; 106. DOI: 10.5195/jmla.2018.513.

14.

EndNote , https://endnote.com/ (2022).

15.

Hupe

. EndNote X9. J Electron Resour Med Libr 2019; 16: 117–119. DOI: 10.1080/15424065.2019.1691963.

16.

Edwards

Clarke

DiGuiseppi

, et al. Identification of randomized controlled trials in systematic reviews: accuracy and reliability of screening records. Stat Med 2002; 21: 1635–1640. DOI: 10.1002/sim.1190.

17.

Microsoft Excel , https://www.microsoft.com/en-gb/microsoft-365/excel (2022).

18.

Google Scholar , https://scholar.google.co.uk/ (2022).

19.

Vine

. Google scholar. J Med Libr Assoc 2006; 94: 97–99.

20.

ORRCA , https://www.orrca.org.uk/ (2022).

21.

Hemming

Constable

Goulao

, et al. Surgical interventions for uterine prolapse and for vault prolapse: the two VUE RCTs. Health Technol Assess 2020; 24: 1–220. DOI: 10.3310/hta24130.

22.

Marson

Burnside

Appleton

, et al. Lamotrigine versus levetiracetam or zonisamide for focal epilepsy and valproate versus levetiracetam for generalised and unclassified epilepsy: two SANAD II non-inferiority RCTs. Health Technol Assess 2021; 25: 1–134. DOI: 10.3310/hta25750.

23.

Cockayne

Pighills

Adamson

, et al. Home environmental assessments and modification delivered by occupational therapists to reduce falls in people aged 65 years and over: the OTIS RCT. Health Technol Assess 2021; 25: 1–118. DOI: 10.3310/hta25460.

24.

Armstrong-Buisseret

Brittain

Kai

, et al. Lactic acid gel versus metronidazole for recurrent bacterial vaginosis in women aged 16 years and over: the VITA RCT. Health Technol Assess 2022; 26: 1–170. DOI: 10.3310/ZZKH4176.

25.

Harding

Chadwick

Homer

, et al. Methenamine hippurate compared with antibiotic prophylaxis to prevent recurrent urinary tract infections in women: the ALTAR non-inferiority RCT. Health Technol Assess 2022; 26: 1–172. DOI: 10.3310/QOIZ6538.

26.

Appleton

Rainford

NEA

Gamble

, et al. Levetiracetam as an alternative to phenytoin for second-line emergency treatment of children with convulsive status epilepticus: the EcLiPSE RCT. Health Technol Assess 2020; 24: 1–96. DOI: 10.3310/hta24580.

27.

King

Fernie

Hudson

, et al. Primary trabeculectomy versus primary glaucoma eye drops for newly diagnosed advanced glaucoma. TAGS RCT 2021; 25: 72. DOI: 10.3310/hta25720.

28.

Dias

Brealey

Cook

, et al. Surgical fixation compared with cast immobilisation for adults with a bicortical fracture of the scaphoid waist. The SWIFFT RCT 2020; 24: 52. DOI: 10.3310/hta24520.

29.

Maheshwari

Bari

Bell

, et al. Transfer of thawed frozen embryo versus fresh embryo to improve the healthy baby rate in women undergoing IVF: the e-freeze RCT. Health Technol Assess 2022; 26: 1–142. DOI: 10.3310/AEFU1104.

30.

Bower

Brueton

Gamble

, et al. Interventions to improve recruitment and retention in clinical trials: a survey and workshop to assess current practice and future priorities. Trials 2014; 15: 399. DOI: 10.1186/1745-6215-15-399.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.13 MB

0.26 MB

0.16 MB