Developing theory-based text messages to support retention in clinical trials: A mixed methods approach

Abstract

Background: Returning a trial questionnaire is a behaviour affected by a range of psychological and contextual factors. Previously tested Short Message Service (SMS) messages to prompt questionnaire return have not addressed these factors, and have not been characterised by established taxonomies of behaviour change techniques (BCTs). Purpose: We aimed to develop acceptable theory-based SMS messages, with fidelity to four BCTs, to support participant understanding of the consequences of not returning trial questionnaires. Methods and Results: We initially developed 32 messages. Ten behaviour change experts assessed message fidelity to the intended BCT (Study 1a). All messages had appropriate fidelity to the intended BCT (mean ratings = 6.8/10 [SD = 0.6) to 7.5/10 [SD = 0.3]). Study 1b, a focus group with five patient representatives, recommended removing the BCT ‘comparative imagining of future outcomes’ (4 messages), two further messages be removed, and amendments to five messages. In Study 1c, 60 breast cancer survivors rated all remaining 26 messages as acceptable (mean = 3.8/5 [SD = 1.2] to 4.3/5 [SD = 0.8]). Twelve behaviour change experts rated the fidelity of the 26 messages to intended BCTs (Study 1d); all messages had appropriate fidelity (mean ratings = 6.1/10 [SD = 2.4] to 6.9/10 [SD = 1.4]). Conclusions: In these studies, we developed 26 SMS messages that were acceptable to the intended recipients and had sufficient fidelity to the intended BCTs. This approach could be taken to design interventions supporting behaviours needed for the successful delivery of clinical trials. The messages are available to research teams who can evaluate them in Studies within Trials.

Introduction

Identifying strategies to support retention of research participants in clinical trials is a recognised priority. ¹ Low retention rates can limit the internal and external validity of trials,^2,3 increasing the risk of research waste and reducing the likelihood of patient benefit. Poor retention is a particular problem for some trial designs, such as 2^k factorials, where missing data can threaten the integrity of the trial. ⁴

Trials utilising patient-reported outcomes use a range of strategies to improve retention and questionnaire response rates. ⁵ These include monetary rewards, newsletters, providing a pen, among many others. Randomised comparisons of these approaches indicate some may improve retention, but the certainty of these findings is moderate at best. ⁶ Most existing approaches have not construed trial retention as a behaviour, and therefore interventions are often not framed in terms of behaviour change. Short Message Service (SMS) prompts are one technique that have been shown to support behaviour change across a range of other settings, including medication adherence,^7,8 smoking cessation, ⁹ physical activity, ¹⁰ weight loss, ¹¹ and chronic disease management. ¹² SMS messages could offer a low cost and implementable strategy for encouraging questionnaire return. However, evidence supporting behaviourally focused SMS interventions in trial retention is lacking. As such, there is a need to design effective interventions to improve retention of participants in clinical trials.^13,14

A number of studies within a trial (SWATs) have investigated the effect of electronic reminders (including SMS messages) on questionnaire return.^15–20 Despite nearly 60% of surveyed UK based Clinical Trial Units using this strategy, ²¹ there is mixed evidence to support the use of SMS for improving trial retention. A Cochrane review reported the relative effect of electronic reminders sent as pre-notifications and reminders for trial retention was small. ⁶ However, relatively few trials were included within the meta-analyses, and the certainty of the evidence was low or very low. The lack of effect may be explained by the narrow focus of the SMS message content. Examples include, ‘We hope you can take a few minutes to complete this and return it to us’ ²⁰ or ‘Your answers are important; so please help by returning it as soon as you can’. ¹⁹ Such messages focus on a simple prompt, and do not consider the wide range of factors that affect questionnaire return.^22–25

Efforts are underway to improve our theoretical understanding of the barriers to trial retention.^23,24 This theoretical understanding can serve as a useful basis for the focus of the intervention. For example the determinants of trial retention have been mapped to the Theoretical Domains Framework (TDF) where key barriers include poor knowledge about the frequency of questionnaires, competing priorities, and beliefs about consequences of performing and not performing the behaviour.^23–25 Beliefs about the consequences of a questionnaire return could be a potential target for SMS messages supporting one aspect of trial retention, and would be an advance on simple prompts. In qualitative interviews with participants across five trials, some participants could not recognise any negative consequences of not returning a questionnaire, and others were unsure what difference their contribution made to a trial. ²⁴ In a separate qualitative study investigating barriers to questionnaire return, participants reported greater satisfaction with returning a questionnaire if they knew their questionnaire would positively contribute to research. ²⁵ Supporting participant beliefs about positive consequences of questionnaire return and negative impacts of non-completion may improve trial retention.

A further weakness of existing interventions designed to support questionnaire return is that they do not conceptualise questionnaire return as a behaviour. As such, there is little consideration of established approaches for developing and evaluating behaviour change interventions. ²⁶ This results in atheoretical interventions that lack generalisability. Considering questionnaire return as a behaviour, and using formal and established intervention development approaches from the field of behaviour change may encourage clinical trial specialists to specify their interventions according to existing behaviour change taxonomies. ²⁷ Such an approach would build a cumulative evidence base which could further our understanding of the mechanisms through which this behaviour can be changed, allow interventions to be compared using a standardised taxonomy, and improve the generalisability of the intervention in the long-term.

We aimed to follow an established procedure for developing a pool of theory-based SMS messages targeting specific behaviour change techniques (BCTs) to support understanding of the importance of questionnaire return in trials. We evaluated the messages for fidelity to the intended BCT in an expert sample and their acceptability to prospective recipients.

Method

We undertook a series of iterative studies to develop the SMS messages (Figure 1). We adapted our approach to develop the message content from Bartlett et al. ²⁸ to accommodate social distancing requirements in place at the time of undertaking the study. Ethical approval for all studies was granted by the University of Leeds Research Ethics Committee for the School of Medicine (MREC 20-038). Data from studies 1a, 1c and 1d will be uploaded to the University of Leeds data repository after publication of the data. Data from study 1b is not available to share because membership of the group may be known to others, and therefore the data cannot be anonymised.

Figure 1.

Overview of short message service development studies.

Identification of the behaviour change techniques

To identify candidate BCTs addressing beliefs about consequences, we drew upon a review linking BCTs to mechanisms of action derived from the TDF. ²⁹ This review linked five BCTs to beliefs about consequences. We discarded one BCT as the trial team did not consider this appropriate in this context (Information about emotional consequences [5.6]), as there are unlikely to be emotional consequences of questionnaire completion. The four remaining BCTs were included as candidates; information about health consequences [5.1], information about social and environmental consequences [5.3], pros and cons [9.2] and comparative imagining of future outcomes [9.3]. ²⁷

Message generation

To generate the SMS messages, the name, description and examples of each BCT (taken from ²⁷ ) were sent to members of the research team (SS, SG, ER, LH). Guidance was given that messages should be up to 160 characters to ensure they could be sent as a single message, generic enough to implement across different trials and should aim to improve understanding of trials and trial processes. We worked independently to each generate as many SMS messages as possible. One author (ER) collated all responses and removed messages that did not fit with the intended content or the targeted BCTs. The messages were circulated to the same members of the research team, and a group consensus was reached after minor amendments and removal of messages.

Study 1a: Expert survey of initial fidelity

Aim: Using SMS messages developed by the research team (SS, SG, ER, LH) aimed at addressing beliefs about the consequences of not returning study questionnaires, our goal was to evaluate the fidelity of the SMS messages to the intended BCTs using a sample of experts in behaviour change.

Recruitment: We identified experts in behaviour change from the United Kingdom through the research team’s academic networks and searching university websites and Twitter profiles. We identified 25 suitable participants, all of whom were invited via email. We emailed a single reminder to non-respondents after 1 week. Participants were offered £100 to participate, which included attending a half-day online workshop for a related study that will be reported elsewhere.

Procedure: A survey containing the generated messages was sent to the experts in behaviour change. Following e-consent we collected basic information about their background, including demographic information. Participants were then presented with each BCT title and a description from the BCT v1 taxonomy, ²⁷ alongside the pre-generated messages. They were asked to rate how well each message reflected the BCT in question using a single item with a 10-point rating scale: 1 (not very well) to 10 (very well). To limit the length of the survey, the message was only assessed for the BCT it was intended to target.

Analysis: Participant characteristics and backgrounds were described. The mean and standard deviations (SDs) of survey responses for each message were calculated. A priori, we considered messages rated below 5.5 by experts for removal from the message bank.

Study 1b: Patient and public involvement group

Aim: We used an established Patient and Public Involvement (PPI) group to ascertain the acceptability of the content of messages to women with experience of breast cancer.

Recruitment: Women were members of an existing PPI group for a pilot trial evaluating the feasibility of evaluating interventions to support medication adherence in women affected by breast cancer. Women were recruited into the PPI group by advertising through the Yorkshire Cancer Community, a charity supporting people affected by cancer in the Yorkshire area. Group members were paid £37.50 each to attend for 90 minutes.

Procedure: The group was attended by two members of the research team (SG, ER) who were familiar to the members. E-consent was taken prior to the meeting, and consent was confirmed verbally at the start of the meeting. All participants were sent a copy of the SMS messages 2 weeks prior to the meeting. The group were presented with the title and descriptions of the relevant BCTs and SMS messages that were retained from study 1a. The group were asked to discuss the acceptability of sending brief messages to encourage and support questionnaire completion generally, as well as comment on the specific wording of messages.

Analysis: The meeting was recorded and transcribed verbatim. One author (SG) reviewed the transcript to identify suggestions made by the PPI group on how to amend the content of the messages. Based on the suggestions, amendments to the messages were made by one author (ER) and agreed through group consensus of the research team (SS, SG, ER, LH).

Study 1c: Patient acceptability survey

Aim: We aimed to assess the acceptability of the SMS messages remaining for studies 1a and 1b to women affected by breast cancer using an online survey.

Recruitment: Sixty women with a personal history of breast cancer were recruited by a market research company (Dynata) commissioned by the research team. The women were identified on the basis that they had oestrogen receptor-positive (ER+) breast cancer, as the host trial that will evaluate these messages involves this patient group. Women received a small monetary incentive from Dynata for survey completion.

Procedure: Dynata sent out the survey link to their panel of profiled respondents who have signed up to participate in market research. People interested in participating who were sent the survey link were asked to confirm in a single screening question on the first page of the survey whether they were over 18 and have received a diagnosis of ER+ breast cancer. If they were eligible, they were asked to provide e-consent on the next page of the survey prior to being shown the survey questions. The survey contained the SMS messages that remained after study 1a and 1b. Participants were asked to rate the SMS messages using a 5-point rating scale assessing how acceptable they felt the message would be to them if they were receiving it (1 = completely unacceptable, 5 = very acceptable). The order of message presentation was randomised.

Analysis: We summarised participant characteristics. Mean scores and SDs for acceptability of each SMS message were calculated. An overall acceptability score was additionally calculated for each BCT by averaging the fidelity scores of all messages relating to each BCT. A priori, we considered messages for removal from the message bank if they were rated as below the midpoint (3) on the overall acceptability score.

Study 1d: Expert reassessment survey of fidelity

Aim: We aimed to assess the fidelity of the SMS messages remaining after any amendments were made in studies 1a, 1b and 1c to the intended BCTs.

Recruitment: We approached the pool of individuals identified but not approached for Study 1a, in addition to those who did not respond to the invitation, or who could not participate in study 1a. All were considered to be experts in behaviour change based on their social media or academic website profiles. We emailed 41 individuals and offered a £25 Amazon voucher upon survey completion.

Procedure: Participants were sent a link to the survey, which provided information about the study and the procedure for providing e-consent. Participants were asked to rate each remaining SMS message in terms of its fidelity to the intended BCT using a 10-point rating scale: 1 (not very well) to 10 (very well). They were presented with a short description of a BCT taken from the BCTTv1, and the corresponding SMS messages for that BCT. This was repeated for each BCT included.

Analysis: We summarised participant characteristics and backgrounds. The mean and SDs of survey responses for each message were calculated. We removed messages rated by the experts as scoring below 5.5 on the fidelity item.

Results

Message generation

The research team generated 32 messages for the four BCTs (Table 1, Figure 1).

Table 1.

Fidelity scores of text messages for each behaviour change technique rated by behaviour change experts (Study 1a).

	Number of messages evaluated	Fidelity of messages to BCT a , mean (SD)
Information about health consequences	6	7.4 (0.2)
Information about social and environmental consequences	17	7.5 (0.3)
Pros and cons	5	6.8 (0.4)
Comparative imagining of future outcomes	4	6.8 (0.6)

^aFidelity score ranged from 1 to 10 with higher scores indicating better fidelity to the intended BCT.

Study 1a: Expert survey of initial fidelity

Participants: Ten experts in behaviour change participated in the online survey. Eight were full-time research scientists, while two were healthcare professionals who also undertook research. The average time spent working in research was 16 years (SD = 5), and most (80%) described behaviour change as central to their work.

Messages and decision-making: No message scored below 5.5, and therefore all 32 messages were considered to have adequate fidelity for the BCT they were intending to target (Table 1, Figure 1).

Study 1b: Patient and public focus group

Participants: Five women aged between 41 and 79 (mean age = 54 years, SD = 15) participated in the focus group (Table 2). All women were daily mobile phone users, and all used SMS messages at least once a week.

Table 2.

Participant characteristics for Study 1b and Study 1c.

	Study 1b (n = 5)	Study 1c (n = 60)
Mean age (SD)	54 years (15)	51 years (16)
Ethnicity (%)
White British	5 (100.0)	49 (81.7)
Asian or Asian British	0	7 (11.7)
Black, Black British (African)	0	2 (3.3)
Black, Black British (Caribbean)	0	1 (1.7)
Mixed	0	1 (1.7)
Educational level (%)
GCSE/O-level/CSE	2 (40.0)	5 (8.3)
Vocational qualifications (NVQ1+2)	1 (20.0)	8 (13.3)
A-level or higher	0	6 (10.0)
Higher educational qualifications (below degree)	0	12 (20.0)
Degree level education	2 (40.0)	25 (41.7)
No formal qualifications	0	3 (5.0)
Still studying	0	1 (1.7)
Menopausal status
Premenopausal	1 (20.0)	26 (43.3)
Postmenopausal	1 (20.0)	29 (48.3)
Unsure	2 (40.0)	5 (8.3)
Other	1 (20.0)	0
Stage of breast cancer at diagnosis
Stage 1	0	24 (40.0)
Stage 2	3 (60.0)	16 (26.7)
Stage 3	1 (20.0)	9 (15.0)
Stage 4	0	1 (1.7)
Unsure	1 (20.0)	10 (16.7)
Frequency of mobile phone use (%)
More than once a day	5 (100.0)	45 (75)
Once a day	0	8 (13.3)
More than once a week but not everyday	0	3 (5)
Once a week	0	0
More than once a month but not weekly	0	2 (3.3)
Less than once a month	0	2 (3.3)
Frequency of SMS use (%)
More than once a day	2 (40.0)	31 (51.7)
Once a day	0	9 (15)
More than once a week but not everyday	3 (60.0)	11 (18.3)
Once a week	0	2 (3.3)
More than once a month but not weekly	0	0
Less than once a month	0	7 (11.7)

Messages and decision-making: We implemented all suggestions from the group where there was no disagreement. A total of six messages were discarded across four BCTs (Figure 1, Table 3), and exemplar quotes supporting these decisions are provided in Table 3. For the messages relating to the BCT ‘comparative imagining of future outcomes’ the group agreed this BCT should not be targeted. All four messages targeting this BCT were therefore discarded. Following suggestions from the focus group, we amended the wording of five messages. For example following a suggestion to start and finish with a positive statement where possible, ‘Completing questionnaires can be time-consuming, but by doing so you are contributing to cutting-edge research. It’s worth it to help improve patient care!’ was amended to ‘By completing your questionnaire you are contributing to cutting-edge research. We know it can be time-consuming, but it's worth it to improve patient care!’.

Table 3.

Justifications for removing messages following Study 1b.

Behaviour change technique	Message	Quote from focus group
Information about social and environmental consequences	It’s time to complete your [TRIAL NAME] study questionnaire. We need as many people as possible to fill these in – if they don’t, the trial might not continue.	‘if they don’t the trial might not continue, oh my god, you’re blaming me for the failure of [TRIAL NAME], you know, again a bit sort of urgh'.
Pros and cons	Returning your study questionnaire will help us decide whether to continue with this project. It might take time, but hopefully you’ll agree it’s worth it	‘so it’s a bit like we’re not quite sure whether we can be bothered, or whether it’s worth doing, so would you mind helping us decide whether to carry on with it. That reads as one individual not returning it quashes the whole thing, but it doesn’t'.
Comparative imagining of future outcomes	If you complete your [TRIAL NAME] questionnaire you help to increase the quality of our findings. If you don’t, there is a chance our research will not be useful.	‘Do you have to have comparative imaging of future outcomes, or…Get rid of it'. New speaker: ‘Yeah, I’d stop number one at the first sentence, I wouldn’t put the “if you don’t”. I’m trying to be succinct, yes, I would wipe the whole category [laughs]'.
	Our research can only be useful if people complete their questionnaires. If not, the trial may not to continue.
	Completing your questionnaire increases the usefulness of the trial’s findings. Think about how you will feel if you completed it now compared to delaying.
	Complete your questionnaire to know that you have contributed to important healthcare research. If you don’t, your experience won’t be counted.

Study 1c: Patient acceptability survey

Participants: Sixty breast cancer survivors participated in the online acceptability survey (Table 2). The majority were white (82%), and 42% were educated to degree level. Most women (75%) used their mobile phone at least once a day, and half (52%) used SMS messaging more than once a day.

Messages and decision-making: Table 4 shows the mean acceptability ratings for the BCTs overall, and the individual items. Overall, the messages within each BCT were considered acceptable to the breast cancer survivors. No individual messages or BCTs scored below the midpoint on the acceptability scale (3), and therefore no messages were removed as a result of this survey. Only one message scored less than four on the acceptability item ‘Not receiving enough questionnaire responses could make our results less valid which could mean the research has less impact. Please return yours’ (mean = 3.8, SD = 1.2).

Table 4.

Mean acceptability (Study 1c) and fidelity (Study 1d) ratings of text messages.

	Patient mean acceptability rating (SD) out of 5 a	Expert mean fidelity rating (SD) out of 10 b
BCT: Information about health consequences Description: Provide information (e.g. written, verbal and visual) about health consequences of performing the behaviour.	4.2 (0.7)	6.9 (1.8)
Please complete and return your [TRIAL NAME] questionnaire. The information you provide could influence patient treatments and healthcare initiatives.	4.3 (0.8)	6.8 (2.6)
Your [TRIAL NAME] questionnaire is due, return this to help research stay up-to-date, so that patients receive effective treatments and healthcare initiatives.	4.2 (0.9)	8.2 (2.1)
Health research is vital to ensure patients receive the best and most innovative care possible. Please return your [TRIAL NAME] questionnaire.	4.2 (1.0)	6.4 (2.8)
The more questionnaire responses we receive, the better the quality of research will be. This increases the impact the trial could have upon healthcare.	4.2 (0.9)	6.3 (2.3)
The information collected from the [TRIAL NAME] questionnaires will help us find out whether your programme improves health & wellbeing for patients.	4.3 (0.9)	6.4 (1.8)
If we don’t receive responses to the [TRIAL NAME] questionnaires, we won’t know whether the interventions improve health and wellbeing of patients.	4.1 (1.0)	7.0 (2.2)
BCT: Information about social and environmental consequences Description: Provide information (e.g. written, verbal, visual) about social and environmental consequences of performing the behaviour.	4.1 (0.7)	6.9 (1.4)
Please return your [TRIAL NAME] questionnaire, by doing this you are helping us to provide the NHS with up-to-date evidence on how to care for their patients.	4.3 (0.9)	7.8 (2.3)
Please return your [TRIAL NAME] questionnaire. By doing this you’re helping to advance healthcare for community members who hugely benefit from an improved service.	4.2 (0.9)	7.6 (2.5)
Charities rely on quality research to best support the lives of those affected by health conditions. Please return your [TRIAL NAME] questionnaire.	4.1 (1.0)	6.8 (2.3)
You may feel it doesn’t matter if one person’s questionnaires are incomplete, but each one ensures our research represents a wide range of people.	4.1 (1.0)	5.8 (2.4)
NHS services can only improve for everyone if people from all walks of life take part in research. Please return your [TRIAL NAME] questionnaire.	4.0 (1.2)	6.8 (2.0)
Evidence shows that over half of trials would have different conclusions if all participants completed assessments. Please return your [TRIAL NAME] questionnaire.	4.1 (1.0)	6.7 (2.3)
Research shows that over half of trials would have different results if all participants completed assessments. Please return your [TRIAL NAME] questionnaire.	4.1 (1.0)	6.4 (2.3)
The NHS relies on having evidence to deliver excellent patient care. Returning your [TRIAL NAME] questionnaire may help improve the NHS for others in the future.	4.3 (0.9)	7.4 (2.2)
Please return your [TRIAL NAME] questionnaire. Our research cannot benefit the NHS without your help.	4.3 (1.0)	7.3 (1.6)
The researchers managing this study are really grateful when questionnaires are completed, because it helps improve the quality of their findings.	4.3 (0.8)	7.2 (2.3)
The researchers managing this study are really thankful when participants complete questionnaires because it ensures their findings are comprehensive.	4.2 (1.0)	6.9 (2.2)
We really value your responses. Completing these questionnaires helps us to know if this study is useful.	4.3 (1.0)	6.8 (2.1)
It’s time to complete your [TRIAL NAME] questionnaire. It’s essential that you tell us how you’re doing - NHS practices could be shaped by your views.	4.1 (0.9)	7.5 (1.3)
It’s time to return your [TRIAL NAME] questionnaire. More questionnaires being completed helps us to be more confident in the trial’s results.	4.2 (0.9)	6.7 (2.1)
Not receiving enough questionnaire responses could make our results less valid which could mean the research has less impact. Please return yours.	3.8 (1.2)	6.0 (1.8)
If participants don’t return their [TRIAL NAME] questionnaire, the trial’s findings will not be useful for advancing patient care.	4.0 (1.1)	6.3 (2.3)
BCT: Pros and cons Description: Advise the person to identify and compare reasons for wanting (pros) and not wanting to (cons) change the behaviour (includes ‘Decisional balance’).	4.2 (0.8)	6.1 (2.4)
By completing your questionnaire you are contributing to cutting-edge research. We know it can be time-consuming, but it’s worth it to improve patient care!	4.3 (0.9)	6.3 (2.8)
Your questionnaire responses are vital to ensure people from all walks of life are represented. You may feel it is low priority, but your views are invaluable.	4.2 (1.0)	6.2 (2.8)
Think about of how you will feel when you contribute to cutting-edge research by completing your questionnaire. We know you’re busy, but your feedback is vital.	4.1 (0.9)	6.0 (3.1)
A pro to completing the questionnaire may be holding up your end of the agreement. A con may be the time it takes. We hope you agree it is worth it.	4.0 (1.0)	6.0 (2.5)

BCT: behaviour change technique.

^aAcceptability scores ranged from 1 to 5 with higher scores indicating better acceptability of the message.

^bFidelity scores ranged from 1 to 10 with higher scores indicating better fidelity to the intended BCT.

Study 1d: Expert reassessment of fidelity

Participants: Twelve experts in behaviour change participated in the online survey, all of whom were different from those participating in Study 1a. Eleven were full-time research scientists, while one was a researcher and a healthcare professional. All participants described behaviour change interventions as central to their work (7/12) or somewhat central (5/12). The average time spent working in research was 9 years (SD = 3).

Messages and decision-making: Table 4 shows the overall mean fidelity ratings from the behaviour change experts for each BCT. Overall, the messages were considered to have appropriate levels of fidelity to the intended BCT. No messages were rated below 5.5 out of 10 with regard to fidelity, and therefore no messages were removed for consideration. The highest rating message was from the ‘Information about Health Consequences’ BCT: ‘Your [trial] questionnaire is due, return this to help research stay up-to-date, so that patients receive effective treatments and healthcare initiatives’ (mean = 8.2, SD = 2.1). The lowest rating message was from the ‘Information about social and environmental consequences’ BCT: ‘You may feel it doesn’t matter if one person’s questionnaires are incomplete, but each one ensures our research represents a wide range of people’ (mean = 5.8, SD = 2.4).

Discussion

In this iterative series of studies, we developed a pool of 26 SMS messages considered to be acceptable to the intended recipients and which have sufficient fidelity to the intended BCTs as rated by experts in behaviour change. The SMS messages are available to research teams who can evaluate these messages further within SWATs to establish if these messages are effective at improving questionnaire completion, and whether specific messages are more effective than others.

Previous studies investigating SMS pre-notifications and reminders have indicated the effects of this strategy are mixed.^6,15–20 This is surprising considering the effectiveness of SMS messages to support behaviour change across a wide range of contexts.^7–12 One explanation could be that the existing interventions tested in SWATs largely focus on basic prompts and cues, and have not explicitly made use of behaviour change theory in their development. ³⁰ Therefore, BCTs implicitly targeted in existing interventions were unlikely to be theoretically informed. ³⁰ Further evaluation of SMS messages targeting BCTs that have been chosen with an explicit theoretical rationale will improve our understanding of whether this strategy is broadly effective. Comparisons between messages will help establish the incremental gains that might be observed by targeting one BCT over another.

We adapted an approach for developing SMS message content to support behaviour change, ²⁸ and focused on developing SMS messages to support participant understanding of the value of questionnaire return, and the potential issues of missing data. However, given the wide range of factors affecting questionnaire return, ²² a similar approach could be undertaken to develop messages targeting other theory-informed factors. This could include targeting determinants of trial retention such as behavioural regulation, goals and memory, using BCTs such as problem solving and action planning. A process of continuous optimisation with regard to the content of these messages may help to improve participant retention, and offer a cumulative, generalisable basis for developing the evidence base in this field.

Our study had limitations. We plan to register a SWAT to evaluate three of these SMS messages within host trials involving women affected by breast cancer. As such, the content of the SMS messages was reviewed by this population in both the patient and public focus group and patient surveys. The content of these messages could therefore be more or less acceptable to different clinical populations. Our selection of BCTs was guided by evidence of previous strategies used for addressing beliefs about consequences. ²⁹ However, this is just one factor of many that may affect questionnaire return, and further research quantifying the relative importance of known determinants of this behaviour would be a useful next step.^22–25 We evaluated the fidelity of the messages to the BCTs we had planned to target. This was a pragmatic decision due to the volume of messages we developed. However, it is likely that some of the content also targets similar BCTs, but we did not investigate this.

In summary, we used an iterative series of studies to develop a pool of SMS messages that aim to support questionnaire return within clinical trials. The messages developed use the BCTs of ‘Information about Health Consequences’, ‘Information about social and environmental consequences’ and ‘Pros and Cons’ and specifically focus on supporting participant understanding of the consequences of returning questionnaires to enhance validity and maximise patient benefit. The messages were found to be acceptable to the intended recipients, and faithful to the intended BCTs. SWATs are now needed to evaluate the effects of these messages on questionnaire return.