Implementation successes and lessons learnt from a randomised controlled trial of a complex school-based intervention

Abstract

Background

The UK has seen a recent shift towards children’s mental health being supported and treated in school settings. Several current school-based interventions focus on autism and social skills, with education professional involvement in their delivery increasing. The study of these interventions poses specific implementation challenges. This paper discusses implementation successes and learnings from the I-SOCIALISE research study which delivered and evaluated efficacy of LEGO® based therapy (now Play Brick Therapy) for autistic children and young people delivered in schools. Detailed Methods and results of the trial are reported elsewhere.

Methods

The I-SOCIALISE study was a pragmatic large-scale NIHR-funded cluster randomised controlled trial. Children and young people, their parents/guardians, and schoolteachers or teaching assistants were recruited from mainstream schools in the UK. They completed outcome measures and were randomised to receive either 12-week of LEGO® based therapy and usual support or usual support only. Various methods to achieve successful recruitment and retention were used and learnings were documented.

Results

The study recruited to time and target with successful delivery of this complex intervention in schools. Several lessons were learnt about recruitment methods, data collection, participant burden and retention, blinding, and the importance of relationships with key school contacts. Main recommendations based on these learnings are provided.

Conclusions

This study demonstrated that it is possible to undertake large scale, robust evaluation of pragmatically delivered complex school-based interventions. Recommendations are made to address the logistical challenges of undertaking research in this setting which are intended to facilitate future research.

Background

School mental health support and interventions

In recent years, the UK has seen a shift in treatment for children’s mental health away from clinics and into school environments. ¹ This change is very recent, however, and much of the mental health support and intervention work currently being implemented in schools is health promotion.^2,3

Teacher involvement in the delivery of child mental health interventions has increased in the last 20 years, ⁴ with some research trials taking place in schools. ⁵ One such area is the field of autism research where a number of studies have sought to implement interventions for improving social skills in schools.^6–11

School-based RCTs pose very specific methodological challenges that previous authors ¹² and special journal issues ¹³ have attempted to explore. These include confounding factors such as multiple interventions and support delivered in school at the same time, the challenge of exploring effects on subgroups, challenges in blinding outcome measurement, the complexities of process evaluation in a complex system, and concerns about the generalisability of findings. While this paper does not address all of these issues, they demonstrate that RCTs in this environment are not easy to conduct. They are further hindered by a lack of research discussing the common challenges and how to address them.

Specific recruitment and retention challenges

Due to the challenges of recruitment and retention in RCTs, a number of systematic reviews investigate the barriers to and strategies for recruitment,^14,15 retention, ¹⁶ and methods to improve these and increase trial efficiency. ¹⁷ Researchers are now using these as evidence to design their research.

There are specific issues with recruitment and retention in school-based research.^18–20 Considerations in school-based research include the educational and organisational needs of the school, political challenges, complex consent processes depending on the age of the child, and access to schools and school staff. Cluster designs are often recommended to avoid contamination across the trial arms, ²¹ but these can be difficult in some cases, for example in coordination of multiple consents across participants, if a school is not offered the intervention they prefer, or all participants in a school do not enter the study at the same time. Research network support in the UK is also centred on hospital or NHS services, meaning school-based recruitment and delivery support can be difficult to access and resource, leading to reliance on schools and grant funded research teams. Mental health trials also have specific issues with retention, with a systematic review reporting that treatment drop out in child mental health care can be anywhere between 28 and 75%. ²²

The I-SOCIALISE trial

The I-SOCIALISE trial (Investigating SOcial Competence and Isolation in children with Autism taking part in LEGO® based therapy clubs In School Environments) was, to our knowledge, the first large scale RCT implemented in schools to assess a psychosocial intervention delivered by teachers and teaching assistants (TAs). It recruited children and young people (CYP) from over 100 schools across three cities and surrounding areas, and sought to establish sustainable training, by training a trainer in the Local Authority, for teachers and TAs to deliver the intervention. Results from the primary analysis showed a small positive improvement in social skills as measured by the social skills subscale of the Social Skills Improvement System (SSIS) in CYP participating in LEGO® based therapy groups, as compared with CYP receiving usual support only.^23,24 Cost effectiveness benefits and good acceptability were also found.^25,26

This paper describes the trial methodology used and some of the challenges and solutions to delivery of this large multi-school RCT. Implementation of the LEGO® based therapy intervention will be covered elsewhere. We use trial process data along with the trial team’s experiences to provide important new information for others wishing to carry out research using school-based interventions.

Methods

Summary of trial design and procedures

The I-SOCIALISE research study was a pragmatic two-arm cluster RCT funded by the Public Health Research (PHR) funding stream of the National Institute for Health Research (NIHR) (PHR15/49/32; ISRCTN64852382). The trial investigated the clinical and cost-effectiveness of LEGO® based therapy on the social and emotional skills of autistic CYP. ²⁷ Positive ethical opinion was gained from the University of York Department of Health Sciences and from the Health Research Authority (HRA) (18/HRA/0101).

LEGO® based therapy is a social skills intervention where CYP build models together. Teachers or TAs in schools allocated to the intervention delivered sessions for one hour per week over 12 weeks. The intervention is described in full in the NIHR report. ²³

Participants

Autistic CYP aged 7–15 years were recruited from mainstream schools in three cities in the North of England. During the trial, the geographical area from which participants were recruited was widened to areas surrounding these cities to increase recruitment. Autism diagnosis was confirmed by parent/guardian and school reports of a confirmed clinical diagnosis and a score of 15 or higher on the Social Communication Questionnaire (SCQ). For each CYP recruited, their parent/guardian, a teacher/TA who facilitated the intervention sessions (intervention schools only) and a separate associated teacher/TAs who knew the CYP well, were also recruited to complete outcome measures. The research study required participation by three CYP in each session of LEGO® based therapy. As such, the trial allowed for the inclusion of CYP without a diagnosis of autism to make groups of three where there were not enough autistic CYP in the school. Consent was sought from these CYP and their parents/guardians, but no outcome data was collected, and they were not considered to be study participants.

Procedures

Recruitment was planned around the school term times, making sure that the 12-week intervention could be delivered in the current school term (allowing for 1 week breaks for half term). Once schools had agreed to take part, they approached the parents or guardians of eligible CYP and provided ethically approved invitation letters, participant information sheets (PIS), and reply slips. Informed consent was then gained from parents/guardians, CYP, and associated teachers/TAs by the research team. After completion of baseline measures (collected flexibly face-to-face, by post, or email link), the school was cluster randomised using remote randomisation stratified by stage of education (primary or secondary) and number of eligible CYP in the school (1–6 CYP or > six). Allocation was implemented using randomisation software provided by epiGenesys, a University of Sheffield company. EpiGenesys created a user interface to specifications provided by the trial’s randomisation statistician. Following recruitment, each school’s details were entered into the randomisation software by the trial manager. The randomisation outcome was generated without the trial manager seeing randomisation lists or having knowledge of algorithm details, which might allow them to predict the allocation of the next school recruited.

Following randomisation, relevant staff from schools randomised to the intervention provided informed consent and were trained in LEGO® based therapy. Sessions were delivered once weekly over 12 weeks either during school hours or after school.

Outcome measures were completed at baseline, 20- and 52-week after randomisation (face-to-face, by post, or online). Researchers collecting outcome data were blind to treatment allocation to limit potential bias. See Appendix 1 for a detailed trial flow chart and Appendix 2 for a table of outcome measures used in the trial.

Patient and public involvement (PPI)

The research study involved PPI from the early stages in its design and methodology. The Young Dynamos research advisory group and a representative from the National Autistic Society (NAS) provided advice on study design and the intervention and fed back on the study’s participant facing documents. They were also able to help the study team consider key implementation challenges, for example, what to do if schools had fewer than three eligible CYP. In addition to these groups, a parent of an autistic child was a member of the trial steering committee (TSC). This provided lived experience insight to assist the study team with issues arising throughout the trial.

Recruitment methods

The I-SOCIALISE trial gave study information to schools via post and email, telephone calls, and school networks. Initially only local authority (LA) schools were contacted, though once sufficient attempts had been made to contact these schools, the team contacted academies using school website contact details.

The research team was flexible in recruitment methods, speaking with schools and families over the phone or in person at times which suited them best and allowing for multiple meetings if needed.

To reduce the burden on schools, research assistants (RAs) (rather than school staff) conducted recruitment and collection of informed consent and outcome measures in schools and at participant homes. Recruitment and retention of CYP to trials requires staff time and adequate resourcing for data collection. The research team planned for an RA in each recruitment area to spend at least 2.5 days per week coordinating and undertaking recruitment and data collection.

Retention methods

Retention methods employed in this trial included careful resourcing of RAs, each with their own localities to build relationships with professionals, schools, and families. The time needed for typical outcome measure completion was overestimated to ensure sufficient time could be given should participants have questions or need support. In cases where participants were upset or simply wanted to elaborate on answers, it was important for the researcher to listen and offer appropriate compassion and understanding. All researchers interacting with participants were equipped with signposting information for parents/guardians experiencing distress and/or needing support.

Participants in the control arm of the trial were incentivised by the offer of LEGO® based therapy training after completion of their 1-year follow up data. Whilst some studies would not offer intervention training if its efficacy were still unknown (or if there was a risk of harm), the trial’s ethical approvals based on previous small RCT evidence of no harm and possible benefit allowed for this. Schools also reported that they planned to deliver the therapy as a matter of course after the trial had ended. The study team therefore felt it was important to provide training to enable appropriate and accurate delivery. To further aid participant retention, the study team also offered a prize draw for all for parents/guardians in both arms of the trial who completed the 1-year follow-up outcome measures.

Intervention delivery and training

The I-SOCIALISE trial used a pragmatic methodology to ensure feasible and continued intervention delivery by school staff following study completion, allowing it to fit in with the school’s usual support for autistic CYP. Timely delivery of both intervention training and the 12-weekly intervention sessions was important to ensure completion of the programme before the 20-week follow-up time point. It was intended that a ‘train the trainer’ model would be used, with experts in the field of LEGO® based therapy training LA education teams in intervention delivery. The LA would then train teachers and TAs in schools in their localities. This was, however, logistically difficult in practice.

Details of training session and intervention delivery are reported elsewhere.^23,24

Results

Recruitment

One hundred and three schools (N = 285 CYP) were recruited between 2017 and 2019, 98 (N = 250 CYP) of which entered the study. Fifty schools (N = 127 CYP) were randomised to the LEGO® based therapy intervention arm and 48 schools (N = 123 CYP) were randomised to usual support (see Appendix 3). CYP demographic characteristics at baseline in the intention to treat (ITT) population are shown below (Table 1).

Table 1.

CYP baseline demographic characteristics, ITT population.

		Intervention	Control
		(n = 127)	(n = 123)
Categorical variables
CYP Gender	Male	102 (80%)	92 (75%)
	Female	25 (20%)	31 (25%)
Age group at start of school year	6–10	89 (70%)	86 (70%)
	11–14	38 (30%)	37 (30%)
CYP Ethnicity	White	105 (83%)	108 (88%)
	Asian/Asian British	9 (7%)	4 (3%)
	Mixed/Multiple ethnic groups	4 (3%)	7 (6%)
	Black/African/Caribbean/Black British	5 (4%)	1 (1%)
	Other ethnic groups	3 (2%)	1 (1%)
	Prefer not to say	1 (1%)	2 (2%)
Continuous variables
CYP age	N (%)	127 (100%)	123 (100%)
	Mean (SD)	9.6 (2.3)	9.7 (2.1)
	Median (IQR)	9 (8, 11)	9 (8, 11)
	Min., Max.	7, 15	7, 15
SCQ score	N (%)	127 (100%)	123 (100%)
	Mean (SD)	25.1 (5.2)	24.2 (5.2)
	Median (IQR)	25 (22, 29)	24 (21, 28)
	Min., Max.	15, 37	15, 36
SSIS social skills (teacher)	N (%)	126 (99%)	120 (98%)
	Mean (SD)	66.1 (19.3)	64.9 (21.4)
	Median (IQR)	64 (50, 79)	66 (48, 77)
	Min., Max.	29, 117	26, 129

Method of invitation to schools

Invitation letters

Invitation letters sent by post were found to be time consuming and generated very little response from schools. Subsequent qualitative work revealed that schools receive large amounts of unsolicited emails and letters including regular requests for involvement in a range of activities and surveys. We found that letters had often not been received or read by the staff members who were responsible for deciding whether the school would participate. Addressing the post to the relevant head teacher or special educational needs co-ordinator (SENCo), through identification on the school website, seemed to generate more response.

Email contact

Sending invitations to schools via email was the most successful form of recruitment, both in response rate and overall resource required from the study team. Initially, invitations were sent to school office/administrative email addresses, however identifying the relevant staff member through the school website and emailing them directly or with an ‘FAO’ was much more successful. Email subject lines were carefully worded to gain attention and avoid the email appearing to be spam. This was typically achieved by including the study acronym and the word ‘research.’

Telephone follow-up

Following up email and postal invitations with a telephone call helped to prompt interested schools that had not yet engaged with the trial team. It also allowed researchers to speak directly with schools that had not received the initial contact to explain the research, answer any questions immediately, build a rapport with schools, and set a date for an in-person meeting to discuss the study in more detail. Using telephone calls as an initial contact method rather than to follow up email or postal invitation was found to be much less effective as relevant staff members were often either unavailable or did not have time to speak about a study they weren’t already aware of. It was found that the best time to speak to teachers on the phone was before school, during lunch, or after pupils had left at the end of the school day. Busy periods, however, differed amongst school staff members, meaning research team flexibility wherever possible was key to building rapport.

Response rates

Most schools that responded did so after receipt of only one invitation. Recruitment success was lower with schools that had been contacted multiple times (Figure 1). For this reason, during the recruitment phase of the trial, the geographical area from which the team could recruit was widened, giving the team access to more schools that had not yet been contacted.

Figure 1.

Graph showing number of invitations schools received before deciding to take part.

Over 100 schools were successfully recruited for the project. Engaging with school leadership has been highlighted as an important strategy in school engagement. ²⁸ Although liaising with senior leadership (e.g. head teachers) was key in recruitment, these same individuals rarely had time to undertake the day-to-day aspects of the research or liaise with the research team. In these instances, a key person was identified within the school who was aware of eligible CYP and often already had established relationships with them. This tended to be the SENCo, Learning Mentor or Deputy Head. This person was able to assist the research team in acting as a key point of contact and coordinating meetings for consent and data collection.

Reasons for declining to participate

The most common non-participation reason was a lack of response or ineligibility (88%) (Table 2). However, 12% of schools specifically chose not to take part, either because they were not interested or they were unable to, despite being eligible.

Table 2.

Reasons for non-participation.

Reason	n	%
No response/school stopped responding	304	65
School ineligible	106	23
School not interested	32	7
School eligible but unable to take part	19	4
School interested and eligible, but no interest from families at school	5	1

A commonly cited reason for declining participation was a lack of capacity within the school to take on additional responsibilities (e.g. contacting parents/guardians of potentially eligible CYP, attending training, completing outcome measures etc.).

Another common reason was the possibility of being randomised to the control arm as this prevented schools from carrying out any LEGO® based therapy in their school for the duration of their trial participation (1 year in total). Whilst it is unfortunate that these schools were not recruited, it demonstrates a clear understanding of what the trial would involve which may account for low post-randomisation withdrawal rates.

Though schools were asked for an estimate during recruitment, data was not systematically collected on the number of eligible CYP in each school during initial engagement, thus it cannot be said how many individuals were approached, or how many declined. As such, some non-recruited data is missing that is usually reported in trials, and, although this is common in RCTs, ²⁹ it should be considered when designing school-based research.

Retention

I-SOCIALISE achieved high levels of school and family retention and completed data. Figure 2 shows primary outcome completion rates at each time-point.

Figure 2.

Percentage of Teacher SSIS form returned, by region and time point.

Simultaneous data collection from parents/guardians and CYP typically took between 45 and 90 min. For associated teachers or TAs, it was between 30 and 60 min, and facilitator teachers or TAs tended to spend between 5 and 15 min completing post-intervention session forms. RAs were very flexible in collecting follow-up data and worked around families and school staff with competing commitments and busy schedules.

Collecting data from CYP

It was noted early on that some CYP seemed to struggle to complete their outcome measures. These were the Asher Loneliness Scale (ALS), the Multidimensional Scale of Perceived Social Support (MSPSS), and the Child Health Utility 9D (CHU9D). Researchers, parents/guardians, and teachers/TAs were typically available to help with completion, but it seemed the outcome measures may have used language or concepts that some CYP struggled to understand. This is despite PPI work carried out prior to study implementation, showing the importance of continued and multifaceted PPI work.

In Sheffield, the majority of CYP were seen at school by RAs as these schools tended to have larger numbers of participants meaning seeing as many as possible in school at once maximised efficiency. CYP were largely seen at home in Leeds and York as there were typically fewer participants per school and more flexibility could be offered. These different approaches yielded different results, as seen in Figure 3. Both teams noted low completion rates where CYP follow-up outcome measures were sent via post or left with parents/guardians or teachers/TAs after visiting them for data collection. Completing CYP outcome measures at school with an RA, as was commonly done in Sheffield, seemed to produce better completion rates (Figure 3).

Figure 3.

Percentage of CYP Asher Loneliness Scale form returned, by region and time point.

Collecting data from teachers/TAs

Many teachers/TAs preferred to complete outcome measures in their own time and requested these via post rather than having face-to-face meetings. These outcome measures, however, appeared to easily slip down the priority lists of busy teachers with already high workloads. To address this, the team began visiting schools in person to drop off and collect teacher/TA outcome measures. These were left at school with a collection date and schools were contacted the day before or day of to remind them of the imminent collection. This method greatly helped follow-up completion rates whilst allowing teachers/TAs the freedom to complete the forms without a scheduled meeting. It also allowed researchers collecting the forms to check for any missing information whilst at school where it could be completed quickly and easily. Building positive relationships with reception staff also allowed the team to offer an additional prompt for teachers through these key staff members.

New teacher at 52 weeks

The I-SOCIALISE trial was set up to have a final follow-up of 52-week after randomisation to assess the long-term effects of the LEGO® based therapy intervention. However, by the 52 week follow up point, the majority of CYP had moved to a different class in their school or had moved schools entirely. SENCos not delivering the intervention often completed associated teacher forms and were able to continue this where CYP had moved to a new class. In the majority of cases though, a new teacher had to be identified and the baseline and follow-up outcome measures for these CYP were completed by different people. Further, some new teachers had only known the CYP for a short amount of time and didn’t feel able to complete the outcome measures. In these cases, the study team took a flexible approach, liaising with schools to find the best placed teacher/TA in the new school or, in some cases, going back to the previous teacher to complete the outcome measures.

Missing data

After collecting a small number of baseline forms, it was apparent that certain components of the participant consent forms and outcome measures were easily missed or misunderstood. To combat this, the team began providing notes on the forms summarising more complex aspects of the completion guidance to ensure participants completed easily missed questions. Data completeness improved following the use of notes which saved the research team a great deal of time in following up missed or misinterpreted questions.

Many of the missed questions were in the resource use section of the parent/guardian and associated teacher outcome measures. These asked about any community and school-based resources used and informed the study’s health economics analysis. It is a common phenomenon in clinical research that resource use information for cost-effectiveness analysis is often difficult to collect. ³⁰ This may be due to difficulty with quantifying or categorising the information. It was challenging for teachers and TAs to provide the requested information about school-based resources (including current interventions and support in school), especially in secondary schools where multiple teachers are involved in the CYPs’ education and support. Some teachers found completing this outcome measure directly following two large and time-consuming outcome measures (the SSIS and the strengths and difficulties questionnaire [SDQ]) too onerous. Including explanations and making instructions as clear as possible was key here.

Blinding

Although all participant outcome measures used were self-completed measures that did not require any RA input, RAs completing follow-up data collection were blind to treatment allocation to avoid any potential bias. Where an RA became ‘unblinded’ (often accidentally by a parent/guardian or teacher), another study team member took over any further data collection visits for that school. This proved onerous where an RA had become unblinded to a school with a large number of participants as this meant a much larger amount of work for other RAs, who themselves ran the risk of becoming unblinded. In a number of cases, all RAs became unblinded to a school’s allocation and some visits had to be conducted by unblinded RAs (see Table 3 for the percentage of blinded visits).

Table 3.

Percentage of follow-up visits completed by blinded researchers by location and time-point.

	Participant type	Leeds (%)	York (%)	Sheffield (%)	Mean (%)
20 weeks	Parent/guardian	85	77	81	81
	CYP	92	96	88	92
	Teacher	84	82	100	89
52 weeks	Parent/guardian	84	74	88	82
	CYP	89	84	88	87
	Teacher	89	84	100	91

To overcome the instances of ‘unblinding’ that occurred multiple times early on in follow-up, RAs began conversations with participants by reiterating the meaning and importance of blinding and asking them not to reveal treatment allocations. On some occasions, however, providing this explanation actually prompted comments on treatment allocation by the parent/teacher, resulting in unblinding. With instances of unblinding increasing, a different approach was taken: blinded researchers would attend visits, but would no longer arrange them. With this method, the concept of blinding could be explained by an unblinded researcher prior to the visit and any questions regarding allocation could be answered. This approach also has the potential to introduce bias; thus, more research is needed into the pros and cons of the various options.

It was unfortunately not possible to blind parents/guardians or teachers/TAs completing outcome measures to treatment allocation as the nature of such a school-based intervention with CYP did not allow for this.

Intervention training

A training day for LA staff was held prior to the start of recruitment and was run by experts in the field of LEGO® based therapy. Following training, it was intended that these LA staff would train school staff in schools randomised to the intervention. Scheduling these training sessions with the LA, however, proved logistically difficult during the trial as the training had to be done quickly enough after randomisation that the intervention could be completed by the 20-week follow-up and there often was not much notice for the LA between a school becoming interested and their randomisation. To overcome this issue, trained members of the research team delivered a number of the training sessions where LA staff were unable to.

Discussion

In general, recruitment and retention to the I-SOCIALISE research study were very good, despite the COVID-19 pandemic beginning in the final months of follow-up leading to school closures. The trial delivered to time and recruitment target with no need for extensions and a single modification where the recruitment areas were widened modestly. There are some key take home messages which we hope will help others planning similar studies to achieve good recruitment and retention.

Main findings

The trial was designed to be pragmatic, to fit in with school timetables, and not be too burdensome for trial participants. It took into consideration the educational and organisational needs of the school as recommended in the literature.^18–20 Researchers can use the PRECIS-2 tool ³¹ to identify whether elements of their trial design are exploratory or pragmatic and modify elements to make them more pragmatic and fit in with the trial setting. Intervention delivery in I-SOCIALISE was intended to model possible future school roll-out. The workload required from teachers/TAs was reduced as much as possible, with the trial team undertaking the initial study information sessions, consent visits, and all data collection.

Overall attrition was low compared to reported rates in child mental health care (28%–75% ²² ). Having flexible and varied methods of communication both between the team and with schools and parents/guardians (face to face, email, text, letter) was helpful as different people responded to different approaches. It was found that some CYP were more willing and able to complete their outcome measures in school. From parent/guardian and CYP feedback it seems this may be due to a strong distinction held by many CYP between school and home, and their desire to keep anything related to school activities strictly within the school setting.

RA planning and resourcing was very important in terms of managing workload and allowing flexibility to manage occurrences such as unblinding. As with other studies, it was found that providing support to overcome the practical challenges (e.g. parental working hours) faced by families, which can impact on engagement with research, ³² improved recruitment and retention. ³³ High quality PPI was found to be very helpful indeed in generating information about the realities and attitudes of those involved in the research and key to understanding the experiences of participants.

Strengths of the study

Rapport building

The study team used telephone conversations and face-to-face meetings along with the approved PIS to explain in detail everything that would be required of participants throughout the trial. The personal nature of these techniques allowed for rapport building with schools which we believe improved engagement in the trial and overall recruitment and retention rates. The trial team’s awareness of the common barriers to participation, such as busy teaching schedules and administrative responsibilities, allowed for provision of a comprehensive explanation of the commitment required to take part in the trial. This aided school decisions on whether to take part.

Research champion

Finding a school staff member (e.g. SENCo, learning mentor) who could coordinate the school’s involvement in the trial proved to be very helpful. This person acted as a key point of contact for the research team, and although they did not necessarily provide data for the trial, they often very helpfully coordinated meetings within the school. This saved time for the research team and facilitated a sense of belonging with the research endeavour.

Flexibility and support

Use of flexible communication methods for school contact, recruitment and data collection allowed for reduced participant burden in the trial. Reducing the time and resource commitment required from school staff increased school capacity to engage and take part in the research. This also meant the trial team had more control over the data collection process. This is important when working with teams and organisations with busy schedules and other priorities.

Practical intervention delivery

A key factor in the study’s overall success was its utilisation of an intervention with a high degree of acceptability from CYP, parents/guardians and education professionals. PPI work and feasibility studies will help with understanding acceptability, as was found in the present study. The flexible nature of the intervention in terms of delivery during school hours or immediately after school helped with acceptability, but the intervention also engendered a high degree of motivation driven by CYP participants and school staff.

Limitations of the study

The recruited sample, although representative in terms of the typical gender balance associated with autism, was not representative in terms of the cultural diversity of the UK and within the autistic population.

Although the pragmatic design of the research allowed for implementation of a complex intervention in a school setting and for emphasis to be placed on its use after study completion, this design also limited the study.

The research team worked to reduce the burden on participating schools and education professionals throughout, allowing them to choose their preferred method of data completion. The majority of school staff opted to complete outcome measures themselves without meeting researchers, although the availability of advice and support was helpful for some. Although this was beneficial for the research team in terms of the resource use and logistics of face-to-face visits, the data completeness yield of meeting with staff members, wherever possible, was significant.

It was also not possible to keep assessors (teachers/TAs and parents) blind to treatment allocation as would be ideal in a large-scale RCT. This was due to the logistics around implementation of a school-based intervention with CYP and the reality that a CYP’s participation could not realistically be kept from assessors.

Additionally, as the intervention and data collection time-points were designed around the school calendar, the reality of CYP moving to new classes and schools and having outcome measures completed about them by new staff members was unavoidable.

Further, the pragmatic implementation of intervention training led to a number of logistical issues due to the tight baseline completion and randomisation requirements of the study. However, for schools considering using LE based therapy, it is worth pointing out that were this model of training delivery to be implemented in a real-world setting, these logistical issues such as timing would likely be much less impactful.

Recommendations for future research

Although the study’s overall retention rate was high and every effort was made to reduce participation burden on schools, one of the main reasons for schools declining to participate was a lack of capacity to undertake tasks required for participation such as liaising with families and completing outcome measures. This highlights the need for research teams to carefully consider the level of burden placed on schools participating in a trial and suggests that researchers should be flexible wherever possible and take on responsibility in order to remove barriers for schools. Research support infrastructure is available in many hospitals but is traditionally not yet available in schools. This is something for future consideration.

Similarly, when including CYP participants with or without mental health conditions, it is vital that their contribution to the study is considered from an inclusion but also participant burden perspective. CYP should be included in research about them wherever possible, but any outcome measures to be completed must be appropriate for them and their abilities. Appropriate provision of support to complete outcome measures should also be considered.

In terms of design recommendations, organisation is made easier using a cluster randomised design in that individual CYP within schools are not randomised and groups can be simply set up on a school-by-school basis. This methodology also has a number of strengths including reducing possible contamination factors and not allowing for different allocations of CYP within the same school.

We believe that school-based research will continue to grow in the coming years as schools are increasingly considered as settings for intervention delivery. The present paper describes experiences and insights into the implementation of school-based intervention research which the authors hope will be helpful for those in the same and similar fields. Specific recommendations for future research methods in this field have been made by previous researchers including using mixed methods ³⁴ and integrating implementation/process evaluation into trials. ³⁵ Some additional methodological recommendations based on the experiences from the I-SOCIALISE research study are summarised below.

Main study recommendations

• It is important to consider implications of working with schools early, for example whether the trial needs to be pragmatic in design and if the intervention needs to be delivered within school timetables. Consider the educational and organisational needs of the school and how these might work with the study.

• Participant burden should be considered and reduced as much as possible at every stage of the research for each type of participant.

• Participant preference should also be considered in terms of how best to complete outcome measures. For example, it was found that some CYP were more willing and able to complete their outcome measures in school potentially due to a strong distinction held by many CYP between school and home, and their desire to keep anything related to school activities strictly within the school setting.

• Sending study invitations to schools via email was the best form of recruitment for this study, both in response rate and overall resource required from the study team.

• Following up other forms of invitation with a telephone call may help to prompt interested schools that may have forgotten about the study.

• Senior leadership staff members in schools are typically very busy indeed, so finding other key staff (e.g. SENCos, learning mentors) who could liaise with families and coordinate study events can be very helpful for logistics and resource use.

• Where participant numbers in schools are large, meeting with families together at school may be most efficient for time and resource. Where participant numbers are smaller, 1:1 school or home visits may be more realistic and can offer a more personal approach.

• A personal approach to data collection, where possible, is impactful. For example, meeting with participants to complete outcome measures or physically dropping these at schools with prompts about collecting them, again in person. This can help with completion rates whilst allowing participants freedom to complete measures without a meeting. It can also allow researchers to check missing information in person without needing a meeting.

• Retention methods should be carefully considered including resourcing of the research team and effectively building relationships with various types of participants.

• Incentives for participants in a control arm, especially in RCTs with non-treatment control arms, can boost retention rates (e.g. offer of intervention training following participation).