Conducting an RCT of a sexual health intervention during the COVID-19 pandemic: Lessons learned

Sexually transmitted infections (STIs) pose a significant public health challenge and correct and consistent condom use is the most effective means of reducing STI transmission. We adapted the Kinsey Institute Home-Based Intervention Strategy (Kl-HIS) ¹ to promote condom use among young men aged 16-25 years in England. We developed and feasibility tested two intervention delivery models: face-to-face (ProHIS ² ) and digital-only (eHIS ³ ). Funded by the Public Health Research Programme of the National Institute for Health and Care Research (NIHR), we conducted a randomised controlled trial (RCT) to test whether the intervention, delivered by either ProHIS or eHIS, would be effective compared to usual condom information and distribution care offered by the National Health Service (NHS). ⁴ The trial was a three-arm parallel-group RCT (1:1:1 allocation, two intervention arms vs. usual care) with baseline measurement, monthly follow-up questionnaires, and up to three STI screening points (baseline, 6 months, and 12 months). Participants had to be aged between 16–25 years, have a penis, be a UK resident, and be at self-identified risk of STIs (e.g. reported condom use errors or condomless sex (anal/vaginal) with a casual or new sexual partner in the past 3 months). The primary outcome was Chlamydia trachomatis test (CT) positivity; secondary outcomes were frequency of condomless sex, reported condom use errors and problems, and condom use attitudes and behaviours. The trial was powered at 90% to detect a reduction in positivity rates from 11% (observed among men aged 15–24 years in U.K. national screening ⁴ to 6% at 6 months post-randomisation. For details of the intervention, see Stone et al.^3,5

The trial required recruitment of individuals from NHS sexual health clinics ⁴ ; our original target sample was 2231. Healthcare staff were trained to identify and enrol participants, deliver the intervention/usual care arms, and collect and process samples (urine and swabs) for on-site CT screening. The recruitment strategy and eligibility criteria were carefully tailored to target individuals at the highest risk of CT. ¹ Eight recruitment sites were established with the assistance of a Clinical Research Network (CRN): sexual health services in seven NHS trusts and one university health centre.

The HIS-UK trial opened to recruitment in one site on March 13, 2020. The first UK lockdown began on March 23, 2020 (see timeline-lockdown-social (instituteforgovernment.org.uk). On March 30, in line with NIHR guidance, recruitment was halted. Most sexual health clinics changed their working patterns; some staff were considered to be at high risk of contracting COVID and required to work from home, others were redeployed to other clinical areas. A UK outbreak of Mpox in May 2022 ⁶ further took up clinic staff time, particularly in London. Open clinics operated with a smaller number of staff. Clinics changed to telephone triaging all patients, offering telemedicine consultations, only seeing patients face-to-face if essential, and advising patients with no symptoms to order self-sampling STI kits via web-based services.

In June 2021, recruitment for our trial resumed with a revised protocol reflecting post-COVID NHS working directives, specifically a reduction in direct clinical contact time with trial participants, and increased provision of remote research participation and care delivery. NHS Trusts prioritized studies on COVID and Mpox. Later in 2021, restarting other research was dependent on local situations and capacity.

Amendments to trial

The following protocol amendments were made in response to changes in service operation following discussions with sites, our Trial Steering Committee, Ethics committee, and funders ⁴ (for details, see ¹ Online Supplemental Table 1).

(1) We used social media adverts/videos and direct text messages from participant identification centres (PICs). PICs are NHS/Non-NHS sites that identify potential research participants by searching patient record databases and referring details on to the researchers.

(2) We allowed potential participants to self-refer to the study by completing an online expression of interest form rather than having to visit a recruiting clinical site. Site staff completed the final recruitment and baseline tasks for eligible clinical participants residing within the catchment area of a recruiting NHS Trust site, following completion of online consent, registration and baseline questionnaires. HIS-UK research staff completed recruitment for community participants residing outside site catchment areas. Participants randomised to the control or eHIS arm completed all recruitment tasks by phone. Participants randomised to ProHIS completed recruitment tasks face-to-face in clinic or via online video consultation software to enable the required visual condom demonstration.

(3) Condom kits were posted to eHIS and proHIS participants instead of being collected in person.

(4) Postal self-sampling STI kits were used to collect and return samples, avoiding the need for a clinic visit.

Study recruitment after amendments

Unfortunately, the trial funder did not extend the study timeline to cover time lost due to the COVID-19 imposed halt to recruitment. We were consequently unable to reach the target sample size (percentage recruited/target: 725/2231 = 32.5%), necessitating amendment to the statistical analysis plan. The ProHIS and eHIS participants were combined to form a single HIS-UK trial arm for comparative purposes.

Despite the above challenges, we were able to recruit and randomise 725 participants: 575 (79%) completed all baseline activities; 189 (32%) of those completing baseline activities reached the final 6-months assessment. The HIS-UK intervention had a positive impact on attitudes towards condoms and lubricants, confidence in correct condom use, condom errors and problems, and recent condom and lubricant use. ⁵

Below we discuss the lessons learned from our research trial to support those designing future sexual health intervention trials, including during times of health service disruption.

Lesson #1: Be creative and flexible about recruitment strategies

To mitigate the risk of under-recruitment, we extended recruitment beyond NHS sexual health services, opened the trial up to all eligible young men in England and expanded our modes of advertising. Our first strategy was a targeted social media campaign. We created Instagram and Twitter (now X) accounts for the study, Facebook advertisements and posts with links to the study website; we produced a bank of social media advertisements that staff in the clinical sites could use (see https://www.his-uk.net/). Initially we only used these advertisements to increase recruitment to the clinical sites, but we expanded these to a national campaign. Unlike our successful experience using social media for recruitment in previous sexual health studies, we did not recruit many participants to the study with these adverts. The click-through rate (CTR) from our posts on social media was below 1% (any advertisement that generates over 2% CTR is performing well; https://www.smartinsights.com/internet-advertising/internet-advertising-analytics/display-advertising-clickthrough-rates/#:∼:text=Unfortunately%2C_there_isn't_a,far_as_I'm_aware.

A strategy to boost recruitment was through outreach visits to university freshers fairs (events held during the first week of term to welcome new students) and army and marine barracks within catchment areas of the study sites ⁷ to distribute leaflets and business cards with QR codes and information about the study.

The final strategy we adopted was to enlist GP practices and sexual health services to act as PICs, facilitated via the NIHR Clinical Research Network of local CRNs. Contracted PICs identified potential participants by conducting clinic record database searches for individuals who met the eligibility criteria and who had previously agreed to be contacted for research purposes. These individuals were then informed about the study via text messages. No client interaction with the GP practices/clinics occurred and the usual HIS-UK recruitment sites (i.e., clinics or the research team) took on the responsibility for seeking consent and undertaking research procedures. Payments were made to PICS for the text messages sent and for their administrative time. The introduction of these automated text messages by PICS significantly improved recruitment. Over 70,000 text message advertisements were sent by PICs located within 13 of the 15 local CRNs, achieving a CTR of 10%, meaning that 10% of those receiving these text messages clicked the expression of interest link provided. Of the 580 participants recruited, 210 (36.2%) were linked with a clinic, and 370 (63.8%) were community participants. Of the 370 community participants, 270 (72.9%) were recruited from PICs.

Sexual health researchers need to consider multiple recruitment strategies. Low resource and inexpensive PICs may be a promising recruitment strategy for future research.

Lesson #2: CT self-sampling via postal kits has both advantages and disadvantages

One of the changes with the expansion of community recruitment and the reduction in contact time in clinical settings was that CT self-sampling kits could be provided and returned by the participants via post, rather than in clinical settings. Participants were provided with postal, self-collected sampling kits for the detection of CT. Testing for Neisseria gonorrhoeae was not included, as gonorrhoea was not an outcome of the study. All samples were analysed by The Doctors Laboratory Pathology (TDL), an accredited UK diagnostic laboratory, using nucleic acid amplification tests (NAATs), who then shared the test results with the participants and the research team.

Participants who reported penetrative vaginal or anal sex exclusively with women in the previous 6 months were provided with a single first-catch random urine sample for CT testing. Participants who reported penetrative anal sex with men in the previous 6 months were offered a multi-site CT sampling kit consisting of a first-catch random urine sample, a self-collected rectal swab, and a self-collected pharyngeal swab, all tested for CT using Polymerase Chain Reaction (PCR)-based assays.

This change had both advantages and disadvantages. In qualitative interviews with a subset of men who completed the study, some expressed positive views, mainly about the convenience of self-sampling. ⁸ However, others believed seeing a health professional would reduce the risk of misdiagnosis and identify other potential health problems.

The main disadvantage of the provision of self-sampling kits was the associated reduction in screening data. Although 94% of our participants agreed to CT screening at baseline, only 58% of those provided samples for testing. All community participants who agreed to screening were sent self-sampling kits at baseline, whereas clinical recruiting staff had the option to either obtain samples in clinic or to order self-sampling kits for participants. A significant proportion of participants provided with kits failed to return samples, despite follow-up reminders. We were able to obtain a higher percentage of test results from participants linked to clinical sites than from those recruited via the community route; at baseline screening results were available for 75% of clinic participants and 40% of community participants. Had all samples been collected in clinics as was initially planned, our screening completion rates would have been higher.

One likely explanation for the higher follow-up rate among participants recruited in clinic was that at least a proportion of these were attending because they wanted STI testing, whereas community participants were typically recruited via PIC texts, would likely have been at lower risk of STIs than clinic attendees, and perhaps have been more motivated by the financial compensation for participation than the need for an STI screen.

Thus, there were likely two issues that affected our obtained CT positivity rates: a lower proportion of our community samples provided a sample for CT testing (and so less contributed to the calculation of positivity rates) but the nature of the community sample (i.e. less at risk) further undermined the assessment of the primary outcome of CT positivity.

Our experience suggests researchers should maximise use of opportunities afforded by care pathways where potential participants are already situated and where they are motivated to provide the necessary data in a way that requires little effort on their part. The CT test positivity rates among young men visiting sexual health services in the UK is high ⁹ and hence this population was our target for this study. When amending a study protocol, it is also important to assess risk and motivation levels of individuals recruited through different pathways to ensure participant numbers and retention are sufficient to achieve the desired statistical power. Since some participants preferred the home-based sampling kits over testing in clinic, allowing a choice of how individuals provide data may optimize recruitment, and offering more than one method for providing data may also increase the size and diversity of samples.

Lesson #3: Some face-to-face contact with researchers may enhance retention

The protocol amendments required reduced face-to-face clinical contact post-COVID-19, and the option for intervention delivery video consultation rather than face-to-face (ProHIS) enabled the trial to continue. These changes resulted in a substantial reduction in completion of baseline tasks (i.e. questionnaires, self-sampling), despite the dedicated efforts of research staff to follow up with participants. Completion of baseline tasks for the three arms was as follows: control 94.2%; eHIS 93.8%; ProHIS 51.9%.

In the original protocol, all recruitment and baseline activities took place during a single consultation and attrition was low. Unlike with the home-based self-sampling kits, it appeared that the more study tasks could be completed at one time and on-site, the less effort was required by the participants, and the more likely they were to complete the study. The requirement for participants in the ProHIS arm to schedule a follow-up online video consultation for intervention delivery led to significant attrition, despite efforts of the research staff to follow up with participants with text/email reminders. In total 241 participants were allocated to ProHIS but only 125 (52%) received the intervention (either in clinic or via video consultation); 116 did not schedule or attend a session.

The qualitative data in the interview sub-study ⁸ also indicated that while online recruitment was seen as convenient (particularly for those who would not normally attend a clinic because of discomfort or access issues), there may be some disadvantages as well. Some ProHIS arm participants, for example, suggested that they might be more likely to retain information provided if a health professional relayed the information in person, rather than accessing this at home via video consultation where there could be distractions. Some participants also commented that the sense of “distance” from the research team might increase disengagement with the study. This raises the question of whether there was drop-out related to aspects of the trial measures/procedure being digital or drop-out because of the intervention itself being digital. ¹⁰

Conclusions

We faced significant challenges and disruption in carrying out this RCT due to the COVID-19 pandemic and its legacy, ultimately leading to the trial’s premature end. To align with the trial’s original objectives, many protocol amendments were made. Some of these amendments enhanced the trial, yielding valuable insights for sexual health researchers in the future. For example, most participants found the home-based self-sampling kits convenient compared with in-clinic sampling. Some changes ultimately undermined the study by reducing the proportion of participants recruited that contributed to the primary endpoint, leading to reduced statistical power. For example, while the inclusion of community recruitment via PIC direct text messaging greatly boosted recruitment, the CT test positivity rate among the broader community population of young men was lower than that of those attending sexual health services. This meant that the assumptions made in the original sample size calculations were no longer relevant and this hampered our ability to make conclusions about our primary outcome ⁵

In summary, our lessons learned mainly relate to the recruitment and retention of participants. In response to the prolonged halt in recruitment due to the pandemic, our major focus was on increasing recruitment. After many unsuccessful strategies, recruitment using PICs led to a large improvement in the recruitment rate. We believe that the PICs strategy for recruitment was likely successful because of the low effort required and the fact that text messaging was used, which might be particularly appealing to young people. Some other aspects of the study related to the move away from in-person contact with participants e.g., return of STI sampling kit, receipt of the ProHIS intervention via video consultation, adversely affected the study. Our recommendation is that researchers conducting evaluations of sexual health interventions carefully consider potential “tradeoffs” when making changes to the design of a study. Our experience with video consultations suggests the need to examine influences on engagement with video consultation before prioritising as an intervention strategy with young people.

The severe disruption of clinical services caused by the COVID pandemic was unpredictable and a global pandemic is unlikely to occur again within the near future, but we believe that many of our lessons learned are relevant for future sexual health research in the context of unanticipated changes to health services. With pressures on sexual health services and research within the UK and globally, we continue to be in a very resource-constrained environment where conducting research is extremely challenging. Using low-cost recruitment strategies such as PICs is one recommendation we would make as well as designing future RCTs of interventions with possible disruptions to the study procedures and alternative recruitment strategies in mind. Different recruitment strategies may also lead to populations of different risk and collecting some data on this in advance of the trial start would help better inform sample size calculations. Capturing information to understand the impact of changes in research protocols, as we have done here, is also essential.

Supplemental material

Supplemental material - Conducting an RCT of a sexual health intervention during the COVID-19 pandemic: Lessons learned

Supplemental material for Conducting an RCT of a sexual health intervention during the COVID-19 pandemic: Lessons learned by Cynthia Ann Graham, Nicole Stone, Stephen Bremner, Nuala McGrath, Rowena Bedford, Katherine Brown, Katie Newby, Amanda Clarke, Louise Jackson, Leanne Morrison, Tom Nadarzynski, Ye To in International Journal of STD & AIDS.