The Implementation of Whole-School Approaches to Transform Mental Health in UK Schools: A Realist Evaluation Protocol

Abstract

Evidence suggests that mental health interventions are more effective when they consider the whole context of schools; addressing the needs of all students, their families, and staff; otherwise known as a whole-school approach (WSA). The UK Government is piloting WSAs to transform mental health and wellbeing by locating educational mental health practitioners in educational settings across England. This study aims to develop a ‘bottom-up’ understanding of the contextual factors and mechanisms that underlie WSAs in Trailblazer schools in the North East and North Cumbria, to gain insight into the facilitators and barriers of delivering a WSA, and optimal evaluation methods. To undertake a realist evaluation, we included the generation of initial programme theories from existing academic literature and policy documents; ‘theory gleaning’ interviews with NHS/local authority stakeholders, Trailblazer staff and school senior leaders; refining and development of theories; and individual interviews and focus groups with pupils, parent/carers and school staff. The findings will enable Trailblazer partners to better understand how their WSAs to mental health contain the essential components for transformation in schools in the region. This will contribute to the embedding of continuous evaluation into regional Trailblazers’ practice for participating schools, for subsequent annual waves and producing relevant findings for non-Trailblazer schools. Complementing the national evaluation of all 25 Wave 1 Trailblazer pilot sites, this study will generate an explanatory theoretical account of how to optimally design, implement and evaluate WSAs by exploring the contextual factors associated with implementation of WSAs.

Background

In England, it is estimated that one in eight young people (12.8%) aged between 5 and 19years are living with a diagnosable mental health illness (Sadler et al., 2018), with 50% of diagnosable mental health illnesses amongst adults having presented by the age of 14 (Royal College of Psychiatrists, 2010), increasing to 75% by age 18 (Kim-Cohen et al., 2003). Rather than providing interventions in schools focused on individuals, there is emerging evidence that interventions are more effective when they consider the whole context of the school, otherwise known as a whole-school approach (WSA), where mental health is considered as ‘everyone’s business’ (Mackenzie & Williams, 2018; Stirling & Emery, 2016; Wells, Barlow, & Stewart-Brown, 2003). A WSA involves a commitment to partnership working between all stakeholders including senior leadership, teaching and non-teaching staff, as well as parents/carers and the wider community (Critchley et al., 2018). WSAs focused on reducing stigma and raising awareness of mental health can lead to changes in school culture, which can improve social, education, physical health and mental health outcomes for all pupils, and particularly those at risk of, or showing, emerging symptoms of mental ill health (Fazel et al., 2014).

A scoping review conducted in 2020 (Flynn et al., 2020) identified seven completed and five in-process studies of UK-based school-wide interventions targeting children and young people, published from 2015 to June 2020. Mental health literacy initiatives utilising a range of structured educational ‘class-room based’ intervention approaches, designed to impact on knowledge and attitudes, and intentional behaviours related to the concept of mental health literacy and resilience demonstrated positive results, focussed on improving outcomes in school children. Empathy-based approaches, which were underpinned by attachment theory to promote the development of more caring and less aggressive behaviours, and those involving peer support, also appear to have promise, improving scores on measures of empathy, aggression and prosocial behaviour. The interventions reviewed primarily focused on children, and omitted non-teaching school staff and other members of the wider school community, including parents/carers. Critically, this scoping review did not identify any studies specifically evaluating the impact of WSAs on school culture, and the need to explore the long-term cumulative impact of WSAs was noted.

The English Government is piloting WSAs to transform mental health and wellbeing, via the Transforming Children and Young People’s Mental Health Provision Programme, delivered through a joint and collaborative programme led by the Department of Health and Social Care, Department for Education and National Health Service (NHS) England and NHS Improvement, with support from Health Education England and Public Health England. The programme involves successive waves, and the Trailblazers are the first of these, with funding available to train educational mental health practitioners (EMHPs)-in partnership with higher education institutions - locating them in schools and colleges across England (UK Government, 2017), as part of the Mental Health Support Teams (MHSTs). These clinicians are responsible for delivering three elements: Treatment of low-to-moderate depression and anxiety; signposting to services, including NHS mental health services; and implementation of WSAs. The NHS Long-Term Plan (NHS England, 2019a) aims that by 2023/24, 345,000 children and young people will have access to mental health support via the transformation programme. In the North East and North Cumbria (NENC), 24.7% of children under aged 16 years live in poverty (England average, 19.9%), and 2.77% of school pupils are known to have social, emotional and mental health needs (England average, 2.39%) (Noble et al., 2019).

We will undertake a realist evaluation (Pawson, 2006) to develop an understanding of the contextual factors that contribute to the design, implementation and evaluation of WSAs to mental health, to transform mental health and wellbeing across the school community. We aim to develop a ‘bottom-up’ understanding of the components and mechanisms that underlie WSAs, to gain insight into the implementation facilitators and barriers of delivering a WSA, and optimal evaluation methods. Bottom-up approaches are less directive and encourage and empower people to achieve change locally (Gifford et al., 2012).

Objectives

1. To develop a realist evaluation framework to conceptualise and assess contextual factors relevant to WSAs.

2. To undertake qualitative data collection with stakeholder groups (local authorities, CCGs, school staff, MHST staff, parents/carers and pupils) to determine acceptability, feasibility and relevance of outcomes proposed in the evaluation framework.

3. Synthesise data to understand the key factors needed to design, implement and evaluate WSAs, in ways that lead to transformation.

Research Questions

1. How do schools conceptualise WSAs? For example, do they drive culture change, raise awareness, reduce stigma, provide a language to communicate about mental health and build resilience?

2. What approaches and initiatives enable schools to transform the way mental health and wellbeing is addressed in schools? For example, signing up to a charter mark, adopting a trauma-informed approach to be compassionate about ‘challenging behaviours’ and implementing anti-stigma campaigns?

Methods

Setting

There are three NENC Trailblazer sites, which commenced in April 2020 and involve 500 Trailblazer schools and colleges. Led by our educational specialist HW, we will undertake a robust recruitment approach, working closely with our regional Trailblazer contacts to recruit participating schools and colleges that match our eligibility criteria. We will evaluate one of each of the four main types of education settings included in the Trailblazers nationally, based on ‘typical’ regional education settings (based on estimated average school sizes): a primary school (approximately 400 pupils, 16 teachers and four teaching assistants), a secondary school (approximately 1300 pupils, 40 teachers and four teaching assistants), a further education college (approximately 5000 pupils, 100 teachers and 10 teaching assistants) and a special school (approximately 120 pupils, 12 teachers and 24 teaching assistants). All pupils, parents/carers and school staff will be invited to participate.

Theoretical Approach

Realist evaluation methodology focuses on the context, mechanisms and outcomes of an intervention, which are known as context-mechanism-outcome (CMO) configurations (Pawson, 2006). The context is what conditions are needed for an intervention to activate mechanisms to yield specific outcomes, including the individual circumstances of a setting (such as the socioeconomic context and history of previous interventions); mechanisms are what is it about an intervention which may yield a particular outcome in a specific context; and outcomes are what are the impacts, both intended and unintended, produced by mechanisms being activated in a specific context. These CMO configurations are developed by engaging with a range of data to understand what aspects of an intervention are related to improved outcomes, and what contextual factors are needed to replicate the intervention in other contexts. One of the important activities in undertaking a realist evaluation is to develop a programme theory, or theories, that explain what works, for whom, under what circumstances and how. Those theories are then ‘tested’ (confirmed, refuted or refined) using the best available evidence, including empirical qualitative data collection (Greenhalgh et al., 2011).

Realist Evaluation Stages

The proposed realist evaluation has five key stages and will adhere to published guidance on the conduct and reporting of realist evaluation studies (Greenhalgh et al., 2011; Wong et al., 2016):

1. Generation of initial programme theories, from existing academic literature and policy documents.

2. ‘Theory gleaning’ interviews with key stakeholders.

3. Refining and development of theories.

4. Individual interviews and focus groups with pupils, parents/carers and school staff.

5. Potential consideration of available school/MHST process data.

Using published literature, policy documentation (including PHE’s eight principles of WSAs (UK Government, 2021) and existing findings from our earlier research (Flynn et al., 2020), we will develop initial programme theories, which aim to define and describe the contexts, mechanisms and outcomes of WSAs. These initial programme theories will then be tested through ‘theory gleaning’ interviews with key school staff and Trailblazer stakeholders (head teachers, educational mental health practitioners, mental health support team leads and school pastoral leads). These data will then be used to refine and develop our proposed WSA theories, before moving onto the next stage. We will collect data on pupil, parents/carers and staff views of WSAs through interviews (n = 40) and focus groups (n = 12). We will conduct 10 interviews and three focus groups (one each with pupils, parents and staff including some from the health and social care system) per school. Interviews and focus groups will be semi-structured to allow the research team to probe hypotheses and to explore unexpected topics raised by participants. Interview guides will be developed from the programme theory. The included sample sizes are approximate, and we will continue to collect data collection until saturation, whereby there is enough information to replicate the study, when additional new information is attained and when further coding is no longer feasible.

There still exists a need nationally to improve mental health data collected in schools (UK Government, 2018). As part of the Trailblazer programme, all EMHPs must report routinely collected process data to NHS England, which is stored on NHS England’s Mental Health Services Data Set and contains one open-ended field on schools’ progress in implementing WSAs. To support the qualitative data collected as part of our realist evaluation approach, we will also explore the feasibility of accessing these data, to inform the development of a more nuanced approach to capturing information relating specifically to WSAs.

Data Analysis

Data from the individual interviews and focus groups will be transcribed verbatim and imported into NVivo 12 qualitative analysis software to facilitate data analysis. Thematic analysis of all qualitative data (interviews, focus groups and free-text items from the mental health dataset) will be conducted in accordance with the framework provided by Braun and Clarke (Braun & Clarke, 2006) to identify key emergent themes. In contrast with other analytical approaches, thematic analysis is not directly associated with a research paradigm. The authors highlight the benefits of thematic analysis in terms of its flexibility, which allows it to be utilised within different theoretical perspectives. This approach will follow an inductive process whereby themes will be directly driven by the data as opposed to theory. These will be discussed amongst the whole research team.

Ethics and Governance

The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. All procedures involving human subjects/patients were approved by Northumbria University Ethics Committee (Ref: 32926). All data will be handled in accordance with General Data Protection Regulation (GDPR). For theory-gleaning interviews with stakeholders, and subsequent interviews and focus groups with pupils, parents/carers and staff, we will (i) ensure participant and school confidentiality, no identifiable information will appear in any reports or publications; (ii) written informed consent/assent is obtained from all children, parents/carers and staff; and (iii) adhere to guidelines for safeguarding of children.

Public Involvement

We convened meetings about a regional Trailblazer evaluation with the three Trailblazer CCG leads before they submitted their bids to NHS England to become Trailblazers sites. This involved sharing evidence on inequalities, public engagement and embedding evaluation into their bids, and shortlisting based on these criteria. All three sites demonstrated that they prioritise public engagement, evidence-informed decision-making and evaluation. Public involvement is integrated into our co-production approach with the Trailblazer sites, including visits to each Trailblazer’s multi-agency steering group meetings. These steering groups have parents and young people as members whom, along with multi-agency partners, have expressed the desire for a regional evaluation that is more in-depth than NHS England’s routinely collected data, and focuses on improving a range of outcomes for young people, not just clinical outcomes, especially for the most vulnerable families. HW and MW have taken over leading these regular discussions via the monthly regional Trailblazer meetings convened by NHS England as part of their roles training the EMHPs at Northumbria University. As part of our project management plan, we will continue to attend these meetings.

Results

This project will enable our NENC Trailblazer stakeholder partners to better understand how well, and to what extent, their WSAs to mental health contain the essential components for transformation in schools in the region. This project will contribute to the embedding of self-evaluation and continuous improvement into the regional Trailblazers’ practice for schools participating in the study, and for the subsequent annual waves planned through 2023/24 (NHS England, 2019b), as well as producing relevant findings for non-Trailblazer schools. We will seek to inform national policy, in terms of how WSAs are designed, implemented and evaluated. We will disseminate findings to our national colleagues at NHS England and the Department for Education for their consideration.

Discussion

WSAs are not well defined in the policies and peer-reviewed literature, and there is a lack of peer-reviewed evaluations. Consequently, there are variable interpretations in how schools are implementing WSAs. NHS England and the Department for Education have commissioned Birmingham University and The London School of Hygiene & Tropical Medicine to undertake a national process evaluation of Trailblazer delivery by clinicians (Ellins et al., 2021). We have partnered with these institutions and will complement their process evaluation with an in-depth realist evaluation, exploring how schools approach the implementation of the WSA element of their Trailblazer programmes. The current study will generate an explanatory theoretical account of how to optimally design, implement and evaluate WSAs by exploring the contextual factors associated with implementation of WSAs across the NENC. As part of this, we will identify potential facilitators and barriers to implementation of WSAs, and provide guidance on operational definitions of WSAs, assessment methods and evaluation. There also continues to be a need for work with young people to better understand what acceptable, effective preventative school-based mental health support looks like, and to improve access to services at the school level (Spencer et al., 2020). The findings from this evaluation will help to inform local commissioners’ transformation plans, and improve services across the NHS, social care and the third sector. This will become increasingly important in the management of COVID-19, and the expected impact on specialists working within teams in community and primary care settings, multidisciplinary teams or services working in the community or in outpatient services together with widening health inequalities (Patel et al., 2021; Wolpert et al., 2014).

This project builds on two previous Fuse (the Centre for Translational Research in Public Health) projects, which involved several members of the current research team. PROMOTE:NE provided key stakeholders and decision makers across the 12 local authorities in the North East with an easy-to-read evidence-based guide on school-based interventions to treat depression and anxiety (JN, LS and EH) (Newham et al., 2017). The second, commissioned by NHS England (Flynn et al., 2020), worked with 27 stakeholders involved in the Trailblazer Programme across the North East and North Cumbria to co-produce an evidence-based guide to inform the design, implementation, and evaluation of WSAs to anti-stigma and awareness raising mental health campaigns (DF, LS and EH).