Abstract
Objectives
We aimed to determine the feasibility of an online randomised controlled trial (RCT) of the Men’s Safer Sex website, measuring condom use and sexually transmitted infection (STI).
Methods
For this study 159 men aged ≥16 with female sexual partners and recent condomless sex or suspected STI were recruited from three UK sexual health clinics. Participants were randomised to the intervention website plus usual clinic care (n = 84), or usual clinic care only (n = 75). Online outcome data were solicited at 3, 6, and 12 months.
Results
Men were enrolled via tablet computers in clinic waiting rooms. Software errors and clinic Wi-Fi access presented significant challenges, and online questionnaire response rates were poor (36% at 3 months with a £10 voucher; 50% at 12 months with £30). Clinical records (for STI diagnoses) were located for 94% of participants. Some 37% of the intervention group did not see the intervention website (n = 31/84), and (as expected) there was no detectable difference in condomless sex with female partners (IRR = 1.01, 95% CI 0.52 to 1.96). New acute STI diagnoses were recorded for 8.8% (7/80) of the intervention group, and 13.0% (9/69) of the control group over 12 months (IRR = 0.75, 95% CI 0.29 to 1.90).
Conclusion
It is likely to be feasible to conduct a future large-scale RCT to assess the impact of an online intervention using clinic STI diagnoses as a primary outcome. However, practical and technical challenges need to be addressed before the potential of digital media interventions can be realised in sexual health settings.
Trial registration number: ISRCTN18649610
Keywords
Introduction
Men’s sexual health
Sexually transmitted infections (STI) are a major public health problem, with high social and economic costs. The majority of genital warts, herpes and chlamydia occurs in opposite sex partnerships, with other infections (syphilis, gonorrhoea and HIV) particularly affecting men who have sex with men. 1 Condoms are effective for prevention of STI; however, there are many barriers to successful use, for example decrease in sensation, interruption of sex, incorrect size or fit, use of alcohol/recreational drugs, anxiety affecting sexual performance and stigma associated with carrying condoms.2,3 Condoms can be a barrier to intimacy and trust, 3 and use is often lower in established relationships. 4
Prevention efforts are needed to target the obstacles to condom use that men face. 2 While there are a variety of health promotion interventions aimed at improving sexual health for men who have sex with men (MSM), there are fewer interventions specifically for adult men who have sex with women (MSW),5,6 despite the fact that MSW report more inconsistent condom use than MSM. 7 Men are less likely than women to visit health professionals and generally have shorter clinic appointments,8,9 so may be less likely to be offered health promotional advice or risk reduction counselling in routine appointments. Men may also be reluctant to discuss their sexual health with health professionals, partners or friends. 10 An online intervention may therefore offer an alternative avenue to reach men. 11
Digital interventions for sexual health
There has been an explosion of interest in digital health worldwide, since mobile phone and internet interventions can facilitate access to self-care as well as relieving pressure on over-stretched or scarce health services. 12 However, the pace of innovation is outstripping the capacity to evaluate digital interventions rigorously. 13
Interactive digital interventions (IDIs) are computer-based programs that provide information and one or more of decision support, behaviour-change support or emotional support for health issues, through provision of personally relevant tailored material. 11 IDIs are suitable for sexual health promotion because access can be private, anonymous and self-paced. 14 Interventions can be targeted for specific groups (e.g. by age, gender or sexuality), and content can be tailored for individuals. 15 IDIs can be costly to develop, but offer the advantages of intervention content fidelity 16 and the potential to reach large audiences at relatively low dissemination costs. 17 IDIs can impact upon sexual behaviour (such as condom use)11,18 as well as increasing knowledge, self-efficacy and safer sex intention.11,19 More evidence is needed to establish effects on biological outcomes (STI) and cost-effectiveness.
The Men’s Safer Sex website is an IDI designed to increase condom use and reduce STI. The website addresses men’s barriers to condom use, providing tailored advice for individuals.20,21
National Health Service sexual health clinics
Sexual health clinic provision varies across the UK. 17 National Health Service (NHS) provision is free and confidential, and services can be accessed using a pseudonym if wanted, since proof of identity is not required. Waiting times can be long, especially for drop-in services in areas of high demand. Clinics often have electronic medical records that are not integrated with other hospital or family doctor records, and facilities such as patient access to Wi-Fi are not routinely available. Since it is usual to wait some period of time before seeing a sexual health clinician, we chose to offer the Men's Safer Sex website in sexual health clinic waiting rooms, to take advantage of a potential ‘teachable moment’. 13
Digital interventions can be particularly appropriate for sexual health, 17 and online trials have the potential to be an efficient and cost-effective way to conduct research. 16 However, it can be difficult to recruit people to trials in clinical settings, and it can be difficult to retain participants in online trials. 16 This feasibility trial was designed to evaluate recruitment and retention rates and methods of sexual health outcome measurement, to add to evidence about how to conduct rigorous eHealth research in healthcare settings.
Aim
The aim of this study was to establish the feasibility and optimal design of a full-scale randomised controlled trial (RCT) to evaluate the effect of the Men's Safer Sex website on condom use and STI acquisition among men attending sexual health clinics.
Research questions
Is it feasible to recruit MSW to an online trial (in sexual health clinic settings)? What proportion of men can be retained in an online trial at 3, 6 and 12 months? What proportion of clinic records can be located (for STI diagnoses over 12 months)? Is it feasible to collect data on the main outcomes of interest at 3 and 12 months (condom use and STI acquisition)? What proportion of the intervention group will access the intervention website?
Methods
Design
This study was a feasibility RCT, 22 with ethical committee approval from City and East NHS Research Ethics Committee (reference number 13 LO 1801).
Participants and setting
Participants were recruited from three busy English sexual health clinics, two in central London and one in the West Midlands. 23 These clinics served a diverse range of patients in terms of age, socio-economic status, ethnicity and sexuality.
Inclusion criteria
Inclusion criteria were: men aged 16 years and over (with no upper age limit); able to read English; with access to the internet; and at high risk of future STI (i.e. two or more sexual partners in the past year and some non-condom use in the last 3 months; or symptoms of acute STI; or seeking treatment for an STI); and for whom at least half of their sexual partners are female. HIV-positive men and men with hepatitis B or C were excluded, since patients with these diagnoses are likely to receive health promotion in the course of routine clinical care. Men who have had sexual experience only ever with males, predominately with males, or no sexual experience at all were also excluded.
Intervention
The Men's Safer Sex website content and design was based on evidence from the sexual health literature, theories of behaviour change, qualitative interviews with men in sexual health clinics and discussions with clinical and academic experts in sexual health and digital technologies. 20 The development process was iterative, with a high level of user involvement. 20 The website addresses men’s barriers to condom use, particularly condom use skills, impact upon pleasure and judgement about potential risk of STI. The website content incorporates behaviour-change techniques, and provides tailored feedback for individual users to address their barriers to condom use (See Appendix 1, Men's Safer Sex website screenshots). The website is designed to be delivered initially in clinic, to make use of the time when patients are waiting to be seen, but also provides online access after patients have left the clinic. The intervention website content and design are reported in detail elsewhere.20,21 The website is available at www.menss.co.uk.
Procedures
Participants were screened for eligibility, gave informed consent and completed baseline questionnaires using a tablet computer positioned in the waiting room or a side room of the sexual health clinics. The study was advertised in leaflets and on posters, and research and clinic staff also directed participants to the tablet computer. Enrolment was self-directed, with staff available to help with technical problems only. Eligibility was checked with a short set of screening questions online, and eligible participants provided informed consent online. Baseline data were collected online, and then participants received a message informing them of whether they had been randomly allocated to the control group or the intervention group. Those allocated to the intervention group were asked to engage with the MenSS website for as long as they wished (usually until they were called in to see health professionals). They also had access to the website after leaving the clinic, by logging in.
At 3, 6, and 12 months after their initial clinic visit, participants were invited via email to complete an online follow-up questionnaire. If they did not initially complete questionnaires, they received three further email prompts at 1-week intervals, as well as two text messages to their mobile phone. If participants still did not respond, a researcher telephoned them 1 week after the final email and text message. Self-reported STI diagnoses were collected at all time points, and clinical diagnoses were collected at 12 months by recording diagnoses or suspected diagnoses noted in clinic records at the sexual health clinics that participants were recruited from.
Outcomes
We measured response rates for the online questionnaire at 3, 6 and 12 months, and proportion of clinic records which could be located using details given by participants. For sexual health outcomes, we adapted the Sexunzipped online sexual health questionnaire, 24 selecting items for inclusion based on a literature search for established measures and consultation with experts. Interviews were conducted with men in sexual health clinics (N = 11) to gain feedback on successive versions of the outcome questionnaire, and we modified the structure and content of the outcome questionnaire based on men’s feedback. 22
Variables assessed at each time point.
Sexual health outcome measures
Condom use
The objective of the study was to promote condom use for penetrative sex, 22 and so the main sexual health outcome of interest was the number of episodes of condomless vaginal or anal sex with female partner/s over the previous 3 months, assessed at the 3-month follow-up. We anticipated that the main opportunity for engagement with intervention website would be at the initial clinic visit, and that any change in condom use behaviour would occur over the subsequent 3 months.
Sexual partners
Participants were asked to report the number of sexual partners over the last 3 months (both female and male). The number of partners with whom participants had condomless sex over the previous 3 months was also assessed: female (vaginal and anal sex) and male (anal sex).
Contraception use and pregnancy
Participants were asked which types of contraception (if any) female partners were using. Participants were asked whether a female partner had been pregnant in the last 3 months, and the outcome of that pregnancy if known.
STI diagnoses
Participants were asked to report STI diagnoses over the past 3 months at every follow-up point. Participants were also asked whether they had received treatment due to a partner being diagnosed with an STI. In order to assess laboratory diagnoses, all STI diagnoses recorded in sexual health clinic records over the 12-month study period (at the participating sites) were collected at the end of the study.
Engagement with the intervention (patterns of website use)
Customised Drupal™ web analytics software was used to record website usage (times the website was visited, pages visited), to assess engagement with the Men's Safer Sex intervention.
Adverse effects
Adverse impacts on sexual health outcomes were recorded at 3, 6 and 12 months. Beyond noting any deterioration in outcome measures, participants were also asked to report whether they had experienced any adverse impacts as a result of the study, recording this in a free text box on each of the follow-up questionnaires.
Hypothesised mediators of condom use
We measured hypothesised mediators of condom use behaviour change which were specifically targeted within the intervention, to explore the mechanisms by which behaviour might have changed (see Appendix 2, outcome questionnaire). Determinants of condom use were identified using the theoretical frameworks of the COM-B model 27 and the PRIME theory of motivation. 28 Mediators included condom use errors and problems; 29 knowledge of risk of STIs and condom sizes; communication with partners; 30 perceptions of identity and condom use; 28 beliefs about pleasure and condom use; 31 self-efficacy; 32 motivation; and condomless sex due to intoxication. Full details of all trial outcome variables are described in the Men's Safer Sex RCT protocol. 22 Online questions were compulsory, so completeness of data for participants who submitted outcome questionnaires was excellent.
Randomisation
Once participants had been checked for eligibility, given informed consent and submitted baseline data, they were randomly allocated 1:1 using an automated computer algorithm to either the intervention or control group. This allocation was unalterable.
Blinding
It was not possible to blind participants to their allocation, as those in the control condition received ‘usual care’, with no access to an intervention website. Research staff were blind to allocation condition, as allocation occurred automatically on the tablet computer; however, allocation group could sometimes be inferred by the length of time that participants spent on the tablet computer. Data were analysed by statisticians who were blinded to group identification during analysis.
Statistical methods
The treatment effect at follow-up was estimated using generalised linear models with a loge link and Poisson error. The loge of the baseline values were included in each model as explanatory variables to account for baseline differences. Standard errors were estimated using variance components to account for over dispersion in the models. Incidence rate ratios (IRRs) comparing the control versus intervention groups were estimated from these models along with their 95% confidence intervals and p-values.
Results
Recruitment
Recruitment into the trial was successful: the target sample size was achieved in just over 2 months (28 April 2014 to 7 July 2014), with the greatest rates of recruitment on days when a researcher was present to remind staff and patients about the study. Follow-up data were collected between 28 July 2014 and 2 July 2015. Participant flow is outlined in Figure 1. Due to a technical error in the online recruitment software, reasons for exclusion were only available for 20 participants out of 159. In total, 176 participants were randomised. Of these, seven withdrew from the study, one was a duplicate account, and nine were withdrawn due to being later found to be ineligible (they had been erroneously classed as eligible due to a technical error in the eligibility screening questionnaire).
CONSORT diagram showing flow of participants.
Of 159 participants, 57 (36%) responded to the 3-month online questionnaire, which was the main outcome point (see Figure 1). An increase in the incentive from £10 (at 3 and 6 months), to £30 (at 12 months) appeared to have a positive effect on response rates: 79 out of 159 (50%) filled in the online questionnaire at 12 months. Data from clinic records on STIs and on sexual health service use were available for 149/159 (93.7%) participants.
Participants
Demographic characteristics of participants.
Website usage
Some 37% of participants in the intervention group visited the Men's Safer Sex website once (n = 31/84), 26% more than once (n = 22/84), and 37% did not see the website at all (n = 31/84). Twenty-two (26%) participants returned to the website after leaving the clinic, with 59% of these participants logging into the website twice in total (n = 13/22). The maximum number of return visits was eight. Participants visited a median number of 15 pages out of a total of 34 main topic or activity pages. Several factors hampered men’s access to the intervention website: technical problems with clinic Wi-Fi stability, time taken with online research procedures (registration, consent, baseline outcome measurement), and being called in to appointments before accessing the intervention website.
Descriptive statistics
Condomless sex and STI diagnoses at baseline and follow-up.
Potential mediators of behaviour change at baseline and follow-up.
It can be seen that there was an apparent increase in episodes of condomless sex reported by men in both the intervention and control groups over the 3 months of follow-up from baseline (Table 3). The median numbers of episodes of condomless sex reported at 3 months were small (12 and 10 for the control and intervention groups, respectively) and attrition was large, so these results are subject to bias. The median numbers of (condomless sex with) male and female partners, and STI over 3 months were very small (median 0 to 1). We present these results descriptively without significance testing since the small numbers preclude meaningful analyses.
Table 4 shows the distribution of outcomes which may be mediators of change (i.e. factors potentially influencing condom use and STI acquisition). We present these results descriptively without significance testing since the small numbers preclude meaningful analyses.
Group comparisons for condomless sex and STI
Group comparisons for condomless sex and STI diagnoses.
Discussion
In this study we explored the feasibility of a full-scale online RCT (in sexual health clinic settings) of the Men's Safer Sex website for MSW. Recruitment into the trial was successful, and retrieval of clinic records for clinical STI diagnoses was 94%. A larger incentive (£30 voucher) offered at 12 months and resolution of technical problems appeared to have a positive influence on the response rate for the online questionnaire (an increase from 36% to 50%). However, response rates remained poor despite repeated email prompts, text messages and phone calls. A third of the intervention group did not see the Men's Safer Sex website, and there were no differences in main outcomes of interest (condomless sex at 3 months, STI acquisition over 12 months), which was as expected in this small feasibility trial. There was no reported harm from the intervention website or online trial. We discuss the implications of this feasibility trial for the conduct of online trials of digital interventions in sexual health clinic settings.
Technical and practical problems
We encountered technical issues which hampered recruitment, access to the intervention website and online data collection at all time points. It was difficult to set up and maintain access to Wi-Fi for patients in clinic waiting rooms, since the process for permission and set-up was complicated, internet connections were often poor and staff were not always confident about remedying internet access problems. Our protocols for data security demanded passwords with a high level of security, but these requirements are likely to have made it harder to remember the passwords.
There were substantial software development errors that impacted on recruitment of eligible participants, and upon data collection and access to the intervention (e.g. errors in links to the outcome questionnaire, login and password reset functions). Despite careful specification of requirements, and manual testing by the research staff, the extent of these problems were realised only after patients had been recruited.
The reach and scalability of digital interventions is potentially excellent, and interest in digital media technology for health has exploded over the past decade or so, but the NHS lags behind many other institutions in terms of patient access to digital services.13,17 Reliable software frameworks for online RCTs are also needed, which are customisable, user-friendly, and which meet requirements for secure data protection and data storage.
Engagement with the MenSS website
Engagement with digital interventions for health promotion can be a major challenge. 35 We placed the intervention in clinic waiting rooms to take advantage of the waiting time that is common in drop-in sexual health clinics, and this depended upon clinic flow. A third of the intervention group did not see the MenSS website, which may have been due to a lack of available time: we shortened the length of the outcome questionnaire by capturing only key outcomes at baseline, but research procedures were still time-consuming (informed consent, online registration, outcome data collection), especially if participants were affected by the technical problems described above.
We sent emails at 1-month intervals to prompt men to access the intervention, and were pleased to find that a third of participants visited the website after their initial clinic visit, since we had anticipated that the main window of opportunity for an intervention would be at the time of the clinic visit. We offered financial incentives for follow-up data collection, but not for engagement with the intervention itself. Effective methods for engagement need to be established, for example integrating health promotion websites as a step in a patient pathway through the clinic. 17
STI diagnoses in clinic records
The retrieval rate for data on STI diagnoses collected from clinic records was 94%, searching by names given, date attended, email address or telephone number. The location of data from clinic records could be improved by requesting additional information from participants (e.g. date of birth, address, clinic number); however, soliciting additional identifiable data may discourage some from participating in research. Half of the sample (72/159) returned to the same clinic within a year; however, STI diagnoses and service use are likely to have been underestimated, since participants are free to attend other sexual health clinics and/or general practice, and records are not shared between all clinics. Differential service use would undermine the validity of clinically recorded STI diagnosis as an outcome if patients in different experimental arms used other services at different rates. It might be possible to capture STI diagnoses made at other sites by using Hospital Episode Statistics and GP practice data. However, patients in sexual health clinics can choose pseudonyms, which would make it difficult to link clinical records. Regardless of which services were used, clinic records are likely to underestimate the cumulative incidence of STI, particularly if STI are asymptomatic (and do not trigger a visit to a clinic).
Despite these drawbacks, recording STI diagnoses from clinical notes has several advantages over self-reported online outcome measurement: clinic record data are not subject to recall bias or social desirability bias, and data collection from records is cheaper in terms of researcher time and costs of participant incentives. In addition, filling in a sexual health questionnaire may prompt participants to reflect on their behaviour (for both control and intervention participants), 25 potentially reducing the apparent effect of an intervention. The cumulative incidence of acute STIs over 1 year was 9.4%, which compares with an annual recorded rate of 0.85% for new STI diagnoses for men attending sexual health services in England. 36 We were therefore successful in recruiting men at substantial risk of STI.
Online outcome measurement
Online trials have a number of potential advantages including automated enrolment, randomisation, data entry and reminders. 17 Online questionnaires can capture data on a range of outcomes, including sexual wellbeing outcomes, which reflect participants’ priorities.25,37 However, while online recruitment can be good, there can be high drop-out rates which substantially undermine internal validity.17,38 The response rates for online questionnaires sent by email were poor (maximally 50%), despite an incentive of £30 for the final questionnaire. We used evidence-based methods to enhance retention, such as contacting participants multiple times via different routes (by personalised email, text message and with telephone calls).24,38,39 However, we do not know whether emails ended up in ‘Junk Mail’, nor how many participants were put off by the initial technical problems with password access and malfunctioning links to the outcome questionnaire. The larger incentive (£30) offered at 12 months appeared to have a positive influence on the response rate (an increase from 36% to 50%), which is in line with studies which tested the effect of larger incentives.24,40 However, it is hard to evaluate the effect of the larger incentive, since the technical problems were resolved by 12 months, which may have also contributed to the improved response rate.
Conclusion
Our findings regarding the conduct of an online RCT of a website for sexual health for men in a clinic setting may be applicable more widely. We have shown that using STI diagnoses from clinical records as a biological outcome is feasible, but we did not achieve adequate response rates for self-reported online outcomes, which limits the measurement of sexual wellbeing and exploration of the mechanism of action of the intervention. Response rates improved with a larger-value voucher (and once technical problems had been remedied).
There are opportunities for exploring the potential of digital media interventions, and considerable policy support in the UK, but there are problems to be overcome before realising their potential in NHS settings, which include ensuring the reliability of software, patient access to IT, data security protocols which are not over-burdensome for patients, minimally time-consuming research procedures (e.g. collecting outcome data at follow-up only) and facilitating patient engagement with digital interventions, for example by integrating digital interventions into routine clinical pathways. 13
Footnotes
Acknowledgements
We would like to thank all of the men who participated in intervention development and in the trial, and to the clinical staff for supporting recruitment of participants. The Men’s Safer Sex website and RCT software framework were developed by Digital Life Sciences (www.digitallifesciences.co.uk). The Men’s Safer Sex website is available at 
. This article presents independent research funded by the National Institute for Health Research (NIHR). The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.
Contributorship
Julia Bailey led the study design and conduct, Rosie Webster was the trial co-ordinator, Mark Griffin and Nick Freemantle conducted the statistical analyses. Rachael Hunter, Greta Rait, Claudia Estcourt, Jane Anderson, Makeda Gerressu, Judith Stephenson, Susan Michie and Elizabeth Murray guided the design and conduct of the study and interpretation of results. All authors contributed to the submitted publication.
Declaration of Conflicting Interests
Professor Jane Anderson reports grants, personal fees and non-financial support from Gilead Sciences, personal fees from ViiV, personal fees from Jansen, personal fees from MSD, personal fees and non-financial support from BMS, outside the submitted work.
Professor Elizabeth Murray is the Managing Director of HeLP-Digital, a not-for-profit Community Interest Company which disseminates digital health interventions to the NHS in the UK. She does not, and will not in future, receive any remuneration for this work.
Ethical approval
City and East NHS Research Ethics Committee (reference number 13 LO 1801).
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by a Health Technology Assessment (HTA) grant from the National Institute for Health Research (reference number 10/131/01; ). This study was commissioned by the HTA, and the design and conduct of the research is monitored by the HTA. The HTA had no role in data collection, analysis, or interpretation.
Guarantor
Dr Julia Bailey. Anonymised data are available from Dr Julia Bailey.
Peer review
Paul Flowers, Glasgow Caledonian University, Katherine Brown, Coventry University and Julie Bayley, Coventry University reviewed this manuscript.