Trial tribulations: Lessons learned from an unsuccessful pilot trial of Behavioural Activation in young people with early or emerging psychosis at a regional mental health service

Abstract

Aim

The experience of conducting an unsuccessful pilot trial of behavioural activation (BA) to improve depressive symptoms in young clients with early or emerging psychosis is reported on. We aimed to identify barriers and enablers to inform the conduct of future trials with this client group.

Methods

Clinicians completed BA training with the aim of delivering this to suitable clients. Feasibility outcomes were calculated as proportions of feasibility targets. The various stages of the pilot randomised trial were examined, informed by interviews with five young people with early or emerging psychosis and four clinicians. Interviews were content analysed.

Results

Client recruitment and retention targets were not met, with proportions of 10% and 50%, respectively. Barriers included ‘clinician self-efficacy in recruitment’, ‘staff turnover’, the ‘fragility of young people’s engagement with the service’, and ‘workload incurred from study involvement’. The enablers were ‘clients’ and clinicians’ perceived potential to contribute to new evidence’, ‘the clinician-client relationship’, and ‘opportunities to provide feedback to the research team’.

Conclusions

Recommendations for future similar studies include pre-trial stakeholder consultation; offering multiple cycles of training; support to grow clinician confidence for recruitment; accounting for increased workload; and ongoing service-research team meetings to identify solutions to emerging issues.

Introduction

Early psychosis often emerges in the late adolescence-young adulthood periods and involves positive (e.g., hallucinations) and negative (e.g., decreased motivation) symptoms and comorbid features of anxiety and depression. ¹ With an estimated 75% of young people with early or emerging psychosis experiencing depression, ² there is a need to identify effective therapeutic approaches which are manageable for services to deliver. Behavioural activation (BA) is a promising approach.^3,4 Underpinned by behavioural principles, BA is designed to promote re-engagement with rewarding activities. ⁴ Evidence suggests this intervention to be safe, effective, feasible, and acceptable in the treatment of depression in other relevant populations.^5–7 For services negotiating workforce and resource challenges, such as those in regional and rural areas, evidence showing that BA can be delivered with high fidelity by non-specialised clinicians without the need for highly extensive education and training is encouraging.^6,8 To explore the potential of BA for young people with early or emerging psychosis, in partnership with an early psychosis service in a youth mental health centre, ⁹ a pilot controlled clinical trial with a two-arm parallel-group design was undertaken to examine outcomes regarding acceptability, feasibility, and preliminary efficacy. ³

This article explores some of the challenges and opportunities that can emerge in the trialling of therapeutic interventions for this client group receiving treatment in a regional early psychosis service. This will be achieved by undertaking a fine-grained examination of the aforementioned unsuccessful pilot study to understand the factors that shaped the course of this research and what can be learned from this. Undertaking a similar process of reflection enabled Tindall and colleagues to deliver a range of helpful recommendations following an examination of their pilot trial of computerised cognitive behaviour therapy for depression in adolescents, which spanned aspects such as planning for infrastructure and communication. ¹⁰ This research seeks to similarly contribute to recommendations useful for planning future trials of psychosocial interventions for young people with early or emerging psychosis in relevant settings.

To identify barriers and enablers of conducting a pilot study of BA for young people with early or emerging psychosis, in this article we describe and examine three stages of this work: preparation; introducing the trial; and implementing the study. We outline the main results and present findings from the analysis of interviews with clinicians and young people about their experiences of being involved in the trial. The article concludes with recommendations useful in the planning and conduct of similar trials.

Materials and methods

The study was conducted at a specialist early psychosis service based in a regional hub. In addition to providing local mental health support, the service also supports young people from surrounding areas, which are largely rural, and experience high socioeconomic disadvantage.^11,12 As with other similar areas, ¹³ attracting and retaining health professionals can be challenging.

The full methods of the randomised trial are detailed in the published protocol. ³ In short, we aimed to recruit and train eight clinicians in BA and recruit 60 clients with early or emerging psychosis over 2 years. Unfortunately, after 19 months (May 2022 to February 2024), we stopped the trial due to financial pressure associated with the slow recruitment and needing to train more than the expected number of clinicians due to staff turnover. Semi-structured interviews with five clients and four clinicians explored their experiences of participating in the trial. Interviews were recorded, transcribed, and content analysed. ¹⁴ The primary aim was establishing the feasibility of a subsequent RCT. Therefore, we calculated descriptive analyses (median, range, and proportions) for all feasibility outcomes.

Participants

Inclusion criteria for clinicians

Clinicians (e.g., psychologists, social worker, support worker and nurse) were eligible if they were currently working with the service and had at least 6 months of experience doing so.

Inclusion criteria for young people

Clients at the service were eligible to participate if they:

• Were 15 years and older;

• Had experienced early or emerging psychosis for less than 5 years;

• Received at least a month of care;

• Were experiencing depression [BPRS depression item score ≥3, according to the most recently conducted routine assessment by clinical staff];

• Were able to speak and understand English; and

• Were able to provide informed consent, and if aged <18 years, the consent of a guardian was also required.

Clients were excluded if they were suicidal or known as a risk to others or themselves, or if substance misuse was a primary contributing factor.

Procedures

The three stages of work comprised preparation, introducing the trial and implementing the study.

Preparation

The preparatory work primarily concerned the formation of the research-clinical service relationship; service clinicians and the youth advisory committee were engaged as key stakeholders. Additionally, procedures were established to monitor safety and mitigate the risk of adverse effects. Clinicians were trained in cohorts and offered ongoing regular clinical supervision. Clinicians undertook an accredited ‘Professional Certificate in BA for Depression’ training program delivered online over a 10-week period in preparation for the trial. ¹⁵

Introducing the trial

Once training was completed in the first cohort, potentially eligible clients on the clinicians’ caseloads were identified, and data gathering started to establish the proportion of all clients with clinically meaningful depressive symptoms registered with the service over the study duration. Clinicians were informed which clients met the depression threshold, and if the client was deemed potentially suitable to participate, the clinician provided them with a study information flyer and ascertained their general interest in participating. If interested, clinicians referred the client to an independent research assistant to conduct the formal recruitment and consent processes. Over the course of the trial, 21 clients were approached to participate.

Immediately before recruitment, members of the research team met with the youth advisory committee of the service to discuss the study and seek feedback on recruitment and other study materials. As a result, an amendment to the ethical approval was acquired to revise the materials according to the suggestions of this committee. As an example, the language was revised in the recruitment materials and a figure incorporated to give a visual representation of the research process. This consultation enhanced the relevance and appropriateness of these materials.

Implementing the study

During assessments and at the start of BA sessions, clinicians and researchers would ask participants about their study experience and about any unexpected experiences arising from taking part. This enabled researchers to monitor for issues and respond in a timely manner. As clients received BA at their treating service centre, the clinician could activate their pre-established risk management plan in the event of any concerns about self-harm or suicide. All adverse events were recorded, coded, and reported to a trial steering committee which could cease the study if the events were determined to be potentially associated with the intervention.

Results

Feasibility outcomes

In total, six clients (21%) and 16 clinicians (100%) agreed to participate in the research. Clients were aged between 16 and 23 years, with the majority having engaged with the early psychosis service for more than 3 months. Clinician treatment fidelity was not examined owing to challenges with recruitment and the low attendance of clients at sessions. Given the small sample size, preliminary efficacy could not be calculated. We present the feasibility outcomes, followed by content analysis findings from the qualitative interviews. For a summary of which study objectives could be addressed, see Supplemental file 1.

Clinician training outcomes

While 16 clinicians (100%) who enrolled completed the training and qualified, owing to staff turnover and staff taking leave, it was necessary to train three cohorts of clinicians over the study period. As per Table 1, only four clinicians went on to engage in recruitment for the trial.

Table 1.

Number of clinicians trained, deemed competent, and engaged in recruitment.

Cohort	Time of recruitment	Number of clinicians trained and deemed competent	Nr of clinicians engaged in recruitment
1st	May–Jun 2022	7	0 (all left the service)
2nd	Apr–Jun 2023	5	4 (1 of which left the early psychosis team in early 2024)
3rd	Feb 2024	4	0 (recruitment had to be stopped)

As noted in the interviews, clinicians found the training to be well-designed, including the practicum element and resources. The self-paced structure and having time allocated during work hours aided clinicians in completing the program. Suggestions to enhance the program included having an option for shorter, more intensive training delivered in person, and increasing the focus on recruiting clients to participate. Further, the research materials provided by the research team at the outset, such as data request forms, were deemed easy to complete by clinicians and helpful for keeping track of recruitment efforts.

Proportion of clients with clinically meaningful depression

On average, 68.4% of clients receiving services at the study site met the depressive symptom criterion when measured at regular intervals, which is in approximate agreement with literature. ² Figure 1 shows assessment results at each interval.

Figure 1.

Percentage of young people with early or emerging psychosis (YPEEP) with clinically meaningful depression symptoms.

Outcome measure completion, client recruitment, and attendance

The CONSORT diagram shows the flow of participants through the study (Figure 2).

Figure 2.

Study CONSORT flow diagram.

Of the six clients who agreed to participate, three were allocated to the treatment group to receive six BA sessions. Two received three BA sessions each, and one received two sessions. All six participants completed baseline measures; of these, four (66%) completed the 6-week follow-up measures, and three (50%) completed the 3-month follow-up measures (Table 2).

Table 2.

Participant’s completion of baseline measures, behavioural activation treatment, and follow-up measures.

YP	Group allocation	Nr of sessions attended	Baseline	6 weeks	3 months
1	BA	2	Yes	No	No (withdrew)
2	TAU	2	Yes	Yes	No (lost to follow-up)
3	TAU	2	Yes	Yes	Yes
4	BA	3	Yes	Yes	Yes
5	TAU	4	Yes	Yes	Yes
6	BA	3	Yes	No (lost to follow-up)	No

One participant allocated to the BA group became unwell and withdrew from the study. It was determined that this event was unlikely to be related to the intervention by the trial steering committee.

The small sample meant that it was only possible to describe clinical outcomes rather than calculate preliminary efficacy (Table 3).

Table 3.

Clinical outcomes of the participants.

Time and outcome variable	TAU	BA
Total number of BA or TAU sessions
Median (range, n)	3 (0–3,3)	2 (2–4, 3)
Baseline – Median (range, n)
CDSS	9 (2–14, 3)	7 (6–12, 3)
WSAS	16 (1–22, 3)	19 (12–23, 3)
GAS	14 (3–24, 3)	NA (-)
SANS Global rating	7 (7–15, 3)	5 (4–6, 3)
Post-intervention – Median (range, n)
CDSS	3 (1–13, 3)	0 (0–0, 1)
WSAS	15 (0–15,3)	1 (1–1, 1)
GAS	11 (5–20, 3)	NA (-)
SANS Global rating	9 (4–11, 3)	0 (0–0, 1)
3-month follow-up – Median (range, n)
CDSS	6.5 (1–12, 2)	0 (0–0, 1)
WSAS	15.5 (14–17, 2)	1 (1–1, 1)
GAS	7 (7–7, 1)	NA (-)
SANS Global rating	9.5 (8–11, 2)	0 (0–0, 1)

Legend: CDSS: Calgary depression scale for schizophrenia; WSAS: work and social adjustment scale; GASS: Glasgow antipsychotic side-effect scale: SANS: scale for the assessment of negative symptoms.

Qualitative interview findings

Clinicians’ reasons for involvement in the study

All eligible clinicians at the service accepted the invitation to be involved in the trial (see Supplemental file 2 for further analysis detail). The opportunity for professional development and acquiring skills in BA within a recognised training program, and experience in a research study with potential to advance understanding of treatment for young people were key reasons for doing so. Clinicians also noted that there was a sense that the study was important for the service.

Clinicians’ considerations in recruitment

In deciding whether to approach an eligible client, clinicians considered whether BA would be the right option for them, at that time. This involved making judgements about the priorities for the client and whether they perceived them as likely to be favourable to participating. For example, a clinician cited the need to retain flexibility in the approach to working with a client experiencing situational crises, as they felt the structured approach and requirements of BA would not have been suitable. There was an underlying sense that engagement with clients could be fragile and perhaps not sufficiently consistent to deliver the BA protocol. One clinician highlighted how the relationship cultivated with clients facilitated recruitment.

For some less experienced clinicians, gaining confidence and struggling with self-efficacy in recruitment posed barriers to approaching clients. Offering incentives, potentially monetary incentives, to clients was suggested to help recruitment. It was also thought that the discussion about the study could be a ‘hard sell’ as some clients may not get any immediate treatment or benefit (i.e., those allocated to the control group). Practical factors such as few eligible clients in the caseload, and the impact of additional demands stemming from the research on workload, were also noted as impacting recruitment.

Young people’s decisions about whether to participate

Of the 21 eligible clients approached, six (29%) agreed to participate. Feedback was gathered from clinicians and clients to better understand their decision-making. The reasons clients gave for declining were: not being interested or not comfortable in being part of the research; reluctance to meet with researchers; a lack of time or the process was too lengthy; they doubted that BA would be helpful or were averse to cognitive behavioural therapy; challenges in obtaining a guardian’s consent; and they did not feel ready at this time. Clinicians also emphasised that the need to meet the researcher could be daunting for young people who experience social anxiety. For clients who agreed, they described being motivated by the potential to contribute to research aimed at improving care for young people like them, potential benefits for their own care, and the opportunity to experience something new.

Clinicians’ and young people’s experiences of the trial

Clients reported feeling comfortable with the trial process, including with the questionnaire used, the length of sessions, and communication about the study. Clinicians highlighted issues that they negotiated. Some reservations were expressed where they observed fluctuation in the moods of the participants during the trial and where they felt that some clients seemed to be behaviourally activated already. Challenges were encountered in satisfying the trial requirements, such as ensuring that participants completed the tools and attended the minimum number of BA sessions. Due to the low recruitment rate and attendance of participants at BA sessions, clinicians did not audio record any sessions with clients, and as such, the intervention fidelity could not be determined. Connectedly, clinicians were reluctant to introduce additional requests, such as recording sessions, as the engagement of the clients was considered to already be relatively fragile. The ongoing clinical supervision provided was considered helpful by clinicians and a space in which issues could be discussed. That said, given the small sample size, there was not always enough to discuss. Given this, dedicating an hour to supervision each month could feel burdensome, especially as there were often competing work priorities.

Clinicians’ reflections upon supervision and support

Becoming involved in the trial placed increased demands on clinicians, including the need to complete additional tasks like associated paperwork. Suggested strategies to manage this impediment were: accounting for time to complete study-related tasks within high clinician workloads, and potentially seeking help from other professionals in the organisation for processes such as recruitment. Having regular reminders about the study and check-ins from the research team were deemed useful, particularly in the context of busy caseloads. The responsiveness and adaptability of the research team to clinicians’ feedback was appreciated, as was the opportunity to meet in person and explore the challenges and possible new approaches to supporting recruitment. These additional approaches included providing tip sheets for clinicians, updating study flyers and encouraging clinicians to call the research assistant with the client present to arrange appointments for recruitment and baseline data collection.

Discussion

While the potential of BA to offer an effective, acceptable adjunct therapeutic intervention in the treatment of depression for young people with early or emerging psychosis was evident, as was the potential for delivery in resource-constrained service settings, ¹⁶ the feasibility of trialling this intervention in the client group was unknown. In the process of undertaking the feasibility trial, a number of challenges and opportunities arose, which provide important insights for those seeking to undertake similar efforts. ¹⁰

Barriers to such efforts included the perceptions of clinicians about the suitability of BA and the trial for clients, staff turnover, clinician confidence and self-efficacy in recruitment, fragility of young people’s engagement with the service, as well as additional workload associated with the research. Facilitators for clinicians included their interest in research participation and in contributing to knowledge gain, professional growth, offering accredited training at multiple time points, having a good relationship with a client prior to discussing the research with them, and opportunities for feedback to the research team. For young people, discomfort with the specific therapeutic approach associated with the research process discouraged participation, while motivating factors such as desire to experience being part of research and contribute to gaining new knowledge were facilitators.

Recruitment was a core difficulty of the trial. While addressing this issue in training, and discussions with the research team were deemed helpful by clinicians, some additional recruitment enabling strategies could be explored. Allowing for a longer recruitment period in trial design and planning may better account for some of the considerations and complexities involved in engaging young people receiving support at regional early psychosis services. Given the feedback, having regular clinician-research team meetings and adopting a case management approach to addressing recruitment issues, could also be implemented and evaluated in future research.

The BA therapy training for clinicians was a successful component of the study and suggests the feasibility of remotely delivered training for comparable studies. Retaining clinicians following the training emerged as a challenge. There was no observable or reported pattern of turnover associated with length of service. This problem should be contextualised in relation to health workforce shortages and the very high rate of turnover of staff across sectors as well as the general transience of the population in the region. While there may be some facilitating strategies, such as incentivising clinicians to stay for the duration of the trial, it is likely that factors impacting these decisions lie outside of the study, and embedding capacity for multiple training cycles at the planning stage may be worth considering for research with services operating in similar contexts.

A further strategy which could have potentially been useful in the monitoring and assessment of progression in the study would be the suggested use of a traffic light rating against pre-specified criteria. ¹⁷ This approach offers comprehensive progression criteria for the design, conduct, analysis, and reporting RCT pilot stages to enable evaluation of advancement and identify issues. For example, a red rating could be interpreted as significant problems in need of urgent attention, while, minor problems to be addressed could be rated as an amber.

Conclusion

Seven recommendations are made for the planning of psychosocial trials with young people with early or emerging psychosis in regional service contexts. (1) Young people should be engaged with (possibly via co-design) to guide the pre-trial preparation. (2) The training in the therapeutic approach could be accredited, offered at multiple timepoints, and accounted for within clinicians’ workloads. (3) The allocation of a longer recruitment period and the identification of any appropriate recruitment incentives. (4) Implementation of support designed to grow clinician confidence and self-efficacy in approaching eligible young people, perhaps in complement to the BA training. (5) Where possible, account for the increased workload, and explore options for assistance from other service staff to support recruitment. (6) Schedule regular clinician-researcher meetings to identify solutions and make adaptions where needed. (7) Finally, it is recommended that study progression criteria are pre-determined, and a traffic light system is used for monitoring and directing attention to problems.

Supplemental material

Supplemental material - Trial tribulations: Lessons learned from an unsuccessful pilot trial of behavioural activation in young people with early or emerging psychosis at a regional mental health service

Supplemental material for Trial tribulations: Lessons learned from an unsuccessful pilot trial of behavioural activation in young people with early or emerging psychosis at a regional mental health service by Janina Catalao Dionisio Murta, Bróna Nic Giolla Easpaig, Mitchell K. Byrne, Richard Gray, Martin Jones, David Mitchell, Rebecca Creek, Ellie Brown, Jing-Yu (Benjamin) Tan, Shaun Dennis, and Daniel Bressington in Australasian Psychiatry

Supplemental material

Supplemental material - Trial tribulations: Lessons learned from an unsuccessful pilot trial of behavioural activation in young people with early or emerging psychosis at a regional mental health service

Supplemental material for Trial tribulations: Lessons learned from an unsuccessful pilot trial of behavioural activation in young people with early or emerging psychosis at a regional mental health service by Janina Catalao Dionisio Murta, Bróna Nic Giolla Easpaig, Mitchell K. Byrne, Richard Gray, Martin Jones, David Mitchell, Rebecca Creek, Ellie Brown, Jing-Yu (Benjamin) Tan, Shaun Dennis, and Daniel Bressington in Australasian Psychiatry