Theory of disruption: A method for understanding context in social policy evaluations with a worked example based on the COVID-19 pandemic

Abstract

The context in which social interventions are piloted and evaluated is critical to success, and there is room for further theoretical and empirical work. Limited knowledge exists about ‘creating the conditions’ for interventions to succeed and about dealing with disruptive changes in context. This article presents a ‘theory of disruption’, to go alongside theories of change and harm. After discussing the challenge of context, and gaps in research, we present a worked example that uses a reoriented logic model to describe the impact of unexpected changes in the wider context of a recent evaluation. This considers the COVID-19 pandemic as a disruptive force on an ongoing school-based intervention. The pandemic is considered an intervention in itself, and the disruptive effects it created are mapped and discussed. The article concludes by considering other uses of the method, including to theorise and measure the effects of a wide range of changes in context.

Keywords

Theory-based evaluation context theory of change disruption process evaluation

Introduction

There is increasing recognition of the role of context in the success or failure of social policies. Yet context is difficult to define and measure, and evaluators still focus much more on interventions than the context in which they operate. In this article, we describe a novel way of examining context and present a worked example based on a recent evaluation. The study took place during the COVID-19 pandemic, and using the pandemic as an example, we aim to understand sudden changes in context that disrupt ongoing interventions. We suggest the approach described could be applied more generally, including to understand other disruptive contextual changes and address more general questions about the context in which social programmes prosper or struggle.

Context has plenty to answer for. It is common to distinguish interventions that are ‘simple’, such as school meal subsidies or tax credits, from those which are ‘complex’. The latter tend to involve multiple interacting components, feedback loops and causal pathways, and most psychosocial interventions fall into this category – from parenting programmes to obesity prevention initiatives. The distinction may be helpful for evaluation design, but in practice even ostensibly simple interventions can involve manifold pathways to impact in the real world. For example, the recent surge in cash transfer programmes in children’s social care (CSC) and related fields are beginning to demonstrate the inherent complexity in evaluating a ‘simple’ intervention in a complex setting (Sanders et al., 2023; Westlake et al., 2024a). This is because a wide range of contextual factors – structural, individual, environmental and political – tend to influence programmes in enabling and disabling ways. Several commentators have noted the need to create supportive conditions for interventions to succeed (see, for example, Durie and Wyatt, 2013; Plunkett, 2024). Across social policy areas, a lack of such conditions has been attributed to previously promising interventions failing to be effective in new settings (Cartwright and Hardie, 2012; Dixon-Woods, 2014).

This presents an opportunity for theory-based evaluation (TBE) approaches which typically acknowledge the importance of contextual factors more than other approaches. Popular examples include realist evaluation, evidential pluralism and phronesis (Flyvbjerg, 2001; Pawson, 2013; Shan and Williamson, 2021). All offer ways of studying interactions between intervention and context, by viewing context as the conditions that influence whether the underlying causal assumptions hold true in specific settings. These methods map contextual factors that might affect the logical flow from inputs to outcomes, treating context as important variables in causal chains. In recent years, TBE approaches have been incorporated into study designs that are traditionally considered more positivist, such as randomised controlled trials (RCTs; Bonell et al., 2012, 2018). While this remains contested (see Bonell et al., 2013; Marchal et al., 2013), this development is encouraged by guidance for evaluating complex interventions, which emphasises the need for both evidence of impact and ‘theories of change’ that explain the processes which lead to positive effects (Moore et al., 2015). Such theories are becoming customary in programme development and evaluation (Breuer et al., 2015), and logic model diagrams are increasingly used to depict how programmes are thought to work (Ebenso et al., 2019).

Nonetheless, the potential of TBE with regard to context is currently unrealised. Clearly, context is a challenging aspect of evaluation and this demands thorough analysis. Understandably, given the practical and financial constraints of evaluation, context is rarely, if ever, the subject of enquiry. However, making it such in cases where there is evidence of, or scope for, major disruptions to social programmes may be warranted. As we demonstrate below, this may address calls to further develop the concept in both evaluation theory and evaluation practice. Nielsen et al.’s (2022) comprehensive review found that context often remains underdeveloped in evaluations, while Greenhalgh and Manzano (2022) call for more sophisticated approaches to contextual dynamics. Before presenting our worked example, it is necessary to develop the case for why context needs more attention.

The challenge of context

Inner and outer contexts

Different types of context are discussed within various evaluative traditions, and there is a consensus that theories of change should account for context (Moore et al., 2015; Pawson and Tilley, 1997). This is because, while different traditions draw upon contrasting ontologies with regard to causality, all agree that context is important in some way. Coming from an empiricist perspective, we are more interested in pragmatic ways of understanding context in applied evaluations than in philosophical debates about the underlying mechanics. Nonetheless, evaluators of all stripes encounter challenges when trying to put this into practice. The most obvious challenge is the sheer range of contexts that may influence social programmes, which makes boundary setting difficult. Often, there are relevant contexts operating at both the immediate implementation domain (organisational capacity, staff skills, etc.) and at the broader environmental level (social, economic, political conditions). Bate (2014) terms these as ‘inner’ and ‘outer’ contexts. How should evaluators decide which contexts are relevant to explore and which to exclude?

The inner context is where most attention is paid by evaluation methods, at least in published studies. This may be because higher-level contexts are best considered in large studies spanning different regions, whereas evaluations are more often done on a smaller scale (Astbury, 2011; Campbell, 1986). For example, although the pure version of realist evaluation found in textbooks considers context as any factor outside of the formal programme that has a causal impact on the programme mechanisms (Greenhalgh and Manzano, 2022; Pawson and Tilley, 1997), the application of these methods is predominantly focused on more proximal contexts, that is, the situational backdrop to an event that creates an outcome. Contexts at higher levels, such as the conditions present within a town, region or society, tend to have a more ambiguous role in how interventions are thought to operate when these methods are operationalised. This is justifiable on a few grounds. It keeps the range of contextual factors to consider manageable, which is important since gathering rich contextual data is resource and time-consuming, and inevitable trade-offs exist between depth of understanding of the programme versus the context (Marchal et al., 2012). Furthermore, given that theories of change often serve as blueprints or roadmaps for implementation and replication, it makes sense to focus on inner context factors which are more likely to be within the control of people delivering interventions. While this may not have a bearing on how causally relevant a contextual factor is, interventions need to be practically deliverable if they are to have any impact. The programme manager of a town pilot may have the power to determine aspects of the setting or services available to participants, but they are not the architect of national policies that may influence the scheme’s ultimate success.

Deciding where to draw the line also relates to wider issues such as generalisability. Focusing too much on contextual differences might underestimate how well some interventions work across different settings from one place to another. While this has been the subject of much debate, there is evidence to suggest some interventions are more transportable than previously thought. For instance, Gardner et al.’s (2015) systematic review of parenting interventions suggested they are highly portable between cultural settings, and counterintuitively evidenced greater effects when programmes were transported to more distant cultural settings. This indicates that, if the key mechanisms of change are not reliant on cultural context, this aspect of context and the need for adaptation has been overestimated for these programmes.

However, and as Gardner and colleagues recognise, this is likely to be because the causal mechanisms involved are less affected by cultural differences, and that there may be a universality to parenting that outweighs these differences (Gardner et al., 2015). This will not always be the case, of course, and ‘. . .if there are certain characteristics that modify the effect of an intervention, and the distribution of those characteristics varies from setting to setting, then the intervention’s effectiveness would similarly vary’ (Mehrotra et al., 2019: 2, our emphasis). Indeed, other types of intervention have been shown to be highly influenced by cultural context, and therefore difficult to transport from one setting to another. Psychological interventions in humanitarian aid (Perera et al., 2020) and in HIV prevention (Vitsupakorn et al., 2023) are two examples. Moreover, other types of contextual factors may invisibly disrupt or change the way interventions operate and produce outcomes. In this vein, Thomas and Parsons (2016) argue,

When an evaluation focuses only on the link between program activities and observable results for participants, the evaluation may inadequately address the hidden factors. Frequently, these hidden factors are viewed as lying outside the boundaries of the evaluand. Yet the hidden factors can foster or constrain the project’s design, implementation, and impacts. (p.7)

All of this supports the use of theories of change and logic models to depict such relationships, since they are particularly useful in setting out causal mechanisms and the mediating and moderating influences surrounding them.

Ordinary and extraordinary change

This brings us to consider what counts as a disruptive context, and how this can be distinguished from normal variation in context. Evaluators tend to think of context in quite static terms of ‘readiness for change’ – the extent to which a host setting might facilitate an intervention – or common variations in contexts between settings or over time. In our field of CSC, it can be remarkable how much local authorities differ from each other, and there is an adage that ‘change is the only constant in Children’s Services departments’. This is a reminder that there is an important distinction between ordinary change – the variation in context that we can expect in any given setting over a certain period of time – and extraordinary change. The latter is our focus here – exogenous changes in context that can unexpectedly challenge, derail or, conversely, enhance interventions. While acute disruption can be difficult to define, for our purposes it is best thought of as events in the outer context that would warrant discussion in an evaluation report.

There are many examples where political changes have led to programmes being abandoned. Take, for example, the Ontario basic income project, which was cancelled when a new government was elected, before data could be analysed (UNESCO, 2022), or the many ongoing federal and academic programmes cancelled by the incoming US administration in early 2025 (The Guardian, 2025). Other sudden disruptions arise in the form of structural reorganisations (Forrester et al., 2013), withdrawal of funding (Meindl and Westlake, 2024) or natural and man-made disasters, such as the COVID-19 pandemic which led to the early closure of studies such as the Frank Friends trial (ISRCTN, 2022). Such changes are liable to cause chaos for social programmes which rely on a stable supply of resources in predictable conditions. This makes it important not only to create receptive conditions, but also to anticipate disruption, learn to adapt at short notice and develop flexible interventions. However, this is less well accommodated by traditional logic modelling and the existing literature on social policy evaluation. Unsurprisingly, it is more prominent in crisis and disaster management, and we return to this in the ‘Discussion’ section.

Beginning to address the problem of context

In her essay on context in health services quality improvement, Mary Dixon-Woods advocated for a mix of correlational and mechanistic methods and argued that existing approaches fall into problematic extremes. At one end, pure clinical trial approaches treat context variables as confounding influences to be controlled out and focus on average effects across populations, lacking insight into why interventions succeed or fail. At the other, as noted above, Dixon-Woods argues that realist-type approaches focus on local explanations and lack rigour in establishing causal relationships, but also struggle to identify which contextual factors are most important (Dixon-Woods, 2014).

In the decade since Dixon-Woods made this observation, these two worlds are becoming more integrated. While purists in realist evaluation claim their rejection of successionist causality absolves the need for correlational evidence (Pawson, 2013), others have used realist methods alongside correlational designs (Bonell et al., 2012). Indeed, RCTs are becoming more ‘modular’, with most What Works Centre-funded trials involving components seeking to answer different questions about effectiveness, implementation, experiences and attitudes, and government guidance encourages the use of TBE designs alongside randomised trials or quasi-experimental designs (HM Treasury, 2024). Despite these advances, the key questions Dixon-Woods poses for researchers to address remain helpful. She asks, ‘What are the best methods for investigating the influence of context on quality improvement activities?’ (Dixon-Woods, 2014: 99). In the following section, we present a theory-based approach, using a different type of logic model, as a potential method that goes some way to addressing this question.

Theories of change, harm and . . . disruption

Theories of change and their accompanying logic models (which depict the theory diagrammatically) have long been used to explain how interventions are intended or expected to produce outcomes. Traditional versions typically featured context as a row of ‘external factors’ or ‘assumptions’ somewhere in the margins of the diagram, which are characterised as a static backdrop rather than a dynamic force that might influence different parts of interventions in different ways. This has been extensively critiqued (Funnell and Rogers, 2011; Malhi et al., 2022; Moore et al., 2019). Modern logic models tend to use systemic approaches to integrate context within the systems of the intervention and its setting, showing how different contextual layers interact with programme elements (see, for example, Ebenso et al., 2019). Rather than treating context as a uniform influence, these show specific pathways of influence between contextual factors and programme elements or mechanisms of change. These more sophisticated approaches are helpful, but they have issues of visual complexity, and arguably by trying to delineate contextual influences more specifically, they are perhaps more open to the criticism that they oversimplify complex relationships.

The approach presented here has a different starting point, which puts the focus on the context as an active or dominant agent in implementation. The format of our approach is inspired by Bonell and colleagues’ work on so-called ‘dark’ logic models. They reoriented the logic model as a tool for analysing how a youth work intervention unintentionally increased teenage pregnancy rates (Bonell et al., 2015), and this has since been used to theorise other unanticipated harmful consequences (e.g. Catlow et al., 2021). Being the opposite of theories of change, we can call these ‘theories of harm’, and the proposed context-oriented theory of disruption described here completes the triad. Instead of unintended consequences owing to the intervention, the focus of a theory of disruption is unforeseen consequences owing to the context. These can be positive, negative or neutral with regard to intervention objectives and have significant and unignorable consequences. In the worked example below, we realign the logic model as a tool to analyse the effects of the pandemic on a school-based intervention in England. In this configuration, the pandemic is positioned as an intervention in itself, with mechanisms and outcomes related to it mapped.

Methodology

Before presenting our analytical approach, Table 1 defines key terms as used in this article. These definitions draw from realist evaluation and implementation science while adapting concepts for analysing disruptions specifically.

Table 1.

Key terms defined.

Term	Definition
Theory of Disruption	A method for analysing major contextual changes as interventions, mapping their effects on ongoing interventions
Mechanism	The process through which a disruption produces its effects on interventions (following Pawson and Tilley, 1997)
Disruption	An extraordinary external change meeting criteria of suddenness, scope, persistence and violation of assumptions
Inhibiting Pathway	Theorised causal chain through which disruption blocks intended intervention mechanisms
Enabling Pathway	Theorised causal chain through which disruption creates unexpected positive mechanisms
If/Then Statement	Analytical tool: ‘If [disruption] then [mechanism] produces [outcome]’

Identifying eligible disruptions

Not all contextual changes warrant analysis as disruptions, and in some cases it may be hard to distinguish extraordinary disruptions from routine contextual variation. Therefore, the following criteria may help researchers identify genuine disruptions and ensure the approach is reserved for genuinely disruptive events rather than typical implementation challenges. Disruptions qualify for this approach when they meet at least three of the following: (1) temporal suddenness – occurring with minimal (<3 months) warning; (2) multilevel scope – affecting individual, organisational and system levels simultaneously; (3) assumption violation – fundamentally contradicting the intervention’s theory of change; (4) persistence – lasting 3 months or longer; and (5) external origin – arising outside the intervention’s control.

Analytical process

In developing our theory of disruption for the worked example, we started with an existing theory of change for the intervention (Westlake et al., 2024b) and applied this six-step process:

Establish disruption characteristics: Document the disruption’s timeline, scope and nature using official records, policy documents and other available sources (see supplementary material for the results of this for our worked example).

Collect data: Gather evidence of how the disruption affected implementation through interviews with participants and stakeholders, and direct observation where possible. In our worked example this involved 45 semi-structured interviews across 21 sites. We conducted theory-driven interviews following realist principles (Manzano, 2016), specifically probing for context–mechanism–outcome configurations related to implementation. Interview questions explored participants’ explanations of how and why the pandemic affected SWIS implementation, what changed and what remained possible. This generated rich data about causal pathways that could be coded into if-then statements.

Develop if/then statements: Code qualitative data into conditional statements linking disruption characteristics to outcomes via mechanisms, using NVivo software (QSR International, 2018). For example: ‘If schools close, then social workers cannot maintain informal visibility, resulting in reduced preventative work’ (inhibitory); ‘If school closures necessitate home visits, then social workers engage with parents in their own environment, reducing power imbalances and improving relationships previously characterized by mistrust’ (enabling). Each statement captures a specific causal chain from disruption through mechanism to outcome. By specifying (1) the disruption characteristic (if), (2) the mechanism activated (then) and (3) the outcome produced (resulting in), this distinguishes the approach from general observations about contextual effects.

Consolidate into pathways: Group-related if/then statements to identify broader patterns. Statements are classified as inhibiting (blocking intended mechanisms) or enabling (creating unexpected positive mechanisms). Multiple statements often converge on similar mechanisms, leading to key pathways.

Create disruption logic model: Reorient the traditional logic model to position the disruption as the intervention, mapping its various pathways of influence on the original intervention. This visual representation shows how disruption mechanisms interact with intervention mechanisms.

Sense check with stakeholders: Review pathways with stakeholders to ensure accuracy and identify gaps.

Distinction from standard approaches

While this method draws on process evaluation principles, it differs fundamentally in focus and application. For example, realist evaluation asks ‘what works, for whom, under what circumstances’ about interventions. By contrast, the theory of disruption asks ‘what changes, through what mechanisms, with what effects’ about disruptions themselves. Rather than examining contextual moderation, it examines major contextual changes and positions them as active agents. Moreover, unlike developmental evaluation (Patton, 2011) or other adaptive evaluation approaches (Brown et al., 2017), the theory of disruption specifically analyses how external shocks reshape intervention mechanisms. It can be applied retrospectively or concurrently and complements rather than replaces process evaluation and impact evaluation.

A worked example: The ‘Social Workers in Schools (SWIS) trial’

The SWIS intervention was a different approach to doing statutory social work, including child protection and working with children in care. Social workers (SWs) remained employed by and responsible to local authorities but were based full-time at schools rather than local authority offices. This differs from ‘school social work’ in other countries, which tends to be more like counselling or early intervention work and is typically carried out by workers employed directly by schools (Westlake et al., 2025).

A quasi-experimental feasibility study ran in three local authorities from 2018 to 2020, and suggested SWIS was a promising approach to reducing the need for referrals to CSC and child protection enquiries (Westlake et al., 2020, 2024b). The SWIS trial followed the feasibility study and involved around 280,000 students, across 291 schools within 21 Local Authority areas in England. This made it the largest social work RCT in the world, and one of the largest RCTs in the English school system. An extensive ‘implementation and process’ evaluation was integrated into the trial and explored how SWIS was implemented and experienced by those involved, using a broadly theory-based approach that involved some techniques from Realist Evaluation, such as the coding of qualitative data into ‘context, mechanism, outcome’ configurations and ‘if-then’ statements (Dalkin et al., 2015). The study started shortly after the World Health Organisation declared COVID-19 a pandemic and encompassed the height of pandemic disruption (2020–2024) (Adara et al., 2023; Westlake et al., 2024b).

The trial estimated the effects of SWIS on a range of social care and educational outcomes, and found it was not effective on any of the outcomes measured. Seasonal analyses explored effects across different time periods, including the period of acute pandemic disruption, and found no patterns, so we concluded that SWIS did not work to improve the outcomes specified. We noted:

The disruptive role of COVID-19 may have presented a challenge for implementing SWIS and affected the extent to which students and social workers were in schools at certain points, and the work they were able to do, but this does not seem to have materially affected the results.

This was reinforced by the finding, within the implementation and process part of the evaluation, that SWIS was implemented relatively well, despite the disruption of the pandemic. Indeed, the nature of the intervention – being an SW located within a school, rather than a more rigid or technical intervention – gave SWIS a level of flexibility that helped it adapt to these conditions. However, this is not to say the pandemic did not affect how SWIS was implemented during the acute period of pandemic disruption, so it was important to analyse the effects it had.

The COVID-19 pandemic

The COVID-19 pandemic created unprecedented educational disruption, with England experiencing its first school closures since the advent of compulsory education in 1918. The pandemic disproportionately affected vulnerable children – those most likely to need social care support – through digital exclusion, increased domestic risks and reduced safeguarding visibility (Bradbury-Jones and Isham, 2020). After reopening, schools operated with restrictive ‘bubble’ systems, while pupils lost 2–3 months of learning and mental health difficulties increased markedly (Education Endowment Foundation, 2020; NHS Digital, 2021). These multilevel disruptions, from individual wellbeing to institutional functioning, exemplified the type of extraordinary contextual change that warrants analysis as a disruption. (Supplementary materials give a more detailed description of the pandemic disruption and detail pandemic restrictions across all 21 study sites.)

COVID-19 and SWIS: A theory of disruption

The theory of disruption presented here accompanies a theory of change that can be found elsewhere (Adara et al., 2023). The analysis was completed at an interim stage in the project, before the results of the trial were known. Data analysis within this approach is similar to the coding of mechanisms and outcomes in the development of a programme theory in realist methodology, only with the contextual change taking the place of the intervention as the focus. After thematic coding of interview data at the interim stage of analysis, each piece of data coded under the COVID-19 theme was moved to a text document and used to generate ‘if-then’ statements. These captured theories supported by the data about how the pandemic impacted on SWIS by describing how a certain mechanism produces a particular outcome under certain circumstances. The if/then statements were then divided into two categories, based on whether they identified COVID-19 as an enabling or inhibiting factor. Overlapping if/then statements in each category were consolidated into more refined theories capturing context–mechanism–outcome chains, which were taken forward to form the basis of the logic model diagrams presented here.

The net impact of social distancing measures and school closures on SWIS was negative, and school staff, SWs and their managers pointed to various ways in which the pandemic inhibited implementation. Often this manifested in key aspects of the role becoming more difficult or impossible. However, aspects of social distancing and school closures were also thought to have some positive consequences – both for the implementation of SWIS and for the wellbeing of children. The logic model that follows is presented as two flow diagrams to illustrate the inhibitory and enabling effects through various mechanisms and moderators (see Figures 1 and 2).

Figure 1.

COVID-19 disruption as an inhibitor of SWIS.

Figure 2.

COVID-19 disruption as an enabler of SWIS.

COVID-19 as an inhibitor (Figure 1)

We identified three pathways through which the pandemic inhibited SWIS or made the circumstances it operated in more challenging. Pathway A related to school closures, which could be distinguished from broader social distancing measures (Pathway B). Although most schools remained operational to some degree, serving a much smaller number of students face-to-face, overall SWs were physically present in schools far less frequently. For some this meant working elsewhere and not going onto the school site, while others were able to maintain a (reduced) physical presence. There was variation between schools, even within the same local authorities, in how open they were to having SWs on site during the closure periods.

Where SWs were required to work remotely, they struggled to feel part of the school, which affected how well they were able to learn about the school’s safeguarding needs. Unsurprisingly, remote working also meant SWs were less visible and accessible within schools and therefore less likely to be called upon by the wider school staff, as described by this school professional:

Staff are aware [of SWIS], but it’s not something that I think is, is kind of at the forefront of their minds, for the simple reason that . . . she’s not been visible . . . and because obviously there is only a relatively small number of students that she is working with in our school, it’s only certain members of staff that might be aware of that. (School safeguarding lead, local authority 8, term 2)

As well as SWs not spending as much time in schools, the students who they primarily worked with were also kept away. Although students who were known to CSC – those with child in need, child protection or care plans – were in the group still able to attend schools, we were told that many did not attend:

Quite a few [students] came in, not many open to social care, those particular families and students, as you can imagine, probably chose to stay at home, the more challenging families shall we say. So not many social care students came in, there was a couple, so other than that I was working from home, we kept in contact with all students that were open to Children’s Social Care. (School safeguarding lead, local authority 8, term 2)

Several consequences arose from this. SWs were less familiar with the school, its students and its culture, and less visible to school staff and students. This reduced visibility meant staff and students were less likely to think of seeking out SW input, and when they did they were less easily accessible. This had a knock-on effect on the amount of informal ‘preventative’ work SWs were able to do, and on the relationship building elements of the role that were central to the intervention.

Pathway B shows that even when schools reopened, social distancing measures frustrated efforts to implement SWIS as intended. Segregation into groups designed to limit interpersonal contact created logistical issues that had wider ramifications. An important drawback associated with reducing students’ freedom to move around school buildings was the decreased scope for more informal interactions with SWs and their impact on non-statutory work. Throughout the trial, interviewees emphasised the value of doing ‘preventative’ work that does not meet the threshold for statutory interventions, and often this was initiated by students approaching the SWIS worker. However, when discussing the impact of COVID, a reduction in opportunities for this was a clearly identified theme. Illustrating this, one SW explained how students had to stay ‘in their own bubbles, they’re not able to leave a classroom without being escorted’ (SW, LA11, Term 1). A team manager from a different local authority (LA) helped put this into context, by summarising the impact on what they called ‘impromptu’ contact between students and SWIS workers:

When the students were in school, they’d just see the social worker in the corridor, or they knew where the social worker was, and they’d pop in and say, ‘Hi’, and then conversations start and things come out [and the worker offers] informal support . . . Whereas now we are obviously very reliant on the line of communication being there, and sometimes with some families it takes quite a lot of effort to maintain that. (Team manager, local authority 1, Term 4/5)

This materially impacted on the type of work that was feasible for SWs to do, with group work no longer being viable where it involved mixing student bubbles and therefore requiring more of the SWs time to deliver work individually or postpone it altogether. Other public health measures created different challenges, such as how to ensure confidentiality when SWs were speaking to students while rules stated office doors needed to remain open for ventilation (LA11). One worker noted ‘I want young people to be able to just come and access my support. . . without that becoming known to their other peers’ (SW, LA11, Term 1).

Pathway C related to an increase in need and risk, both due to the pandemic directly (viral infection and the related health risks) and indirectly (e.g. economic pressures related to the pandemic, increases in mental ill health). This increased pressure on families, meant that problems were less easy and more time-consuming to identify, and meant that existing problems may be exacerbated. The overall pressure on workloads increased for SWs and school pastoral staff, who found themselves working with a wider range of families. When schools returned to a semblance of normality, some staff reported being concerned about more children and for different reasons – including an increase in mental health difficulties. This had an impact on the amount and type of work some SWIS undertook. SWIS workers focused more on addressing needs developed and exacerbated by COVID-19, and helping children return to attending school regularly. As one worker described, this was not straightforward, and often involved working with problems that are not within the remit of statutory social work:

I would say that the work that I have picked up is mostly with non-attenders . . . after lockdown, because I think for some children – being out of education for so long – it’s caused them some difficulty in coming back. So, they’ve had some challenges because of Covid. (Social worker, local authority 11, Term 1)

COVID-19 as an enabler (Figure 2)

Despite the overall negative impact, some unexpected benefits were identified. In part, these arose from the changed circumstances and ways of working forced by the pandemic. While they were not necessarily enablers of the intervention as envisaged, they made certain aspects of it (e.g. building relationships with parents and professionals) easier in some ways. The main enabling pathway (Pathway A in Figure 2) relates to school closures, which counterintuitively presented some opportunity for SWs and school staff to engage more directly with parents. In some cases, schools (and the SWIS worker attached to the school) launched an outreach programme whereby home visits were being done regularly. Being part of a crisis response, the work took on a more overtly humanitarian dimension, which was explicitly about helping families get through a difficult time and ensuring they felt supported and heard. Along with the SW safeguarding leads and school nurses undertook visits to the homes of students who were deemed vulnerable, as well as phone calls to parents and students to check on their wellbeing. They found this could improve relationships and communication with those who may previously have held negative views of the school or of CSC. The increased frequency of non-judgemental contact with parents appeared to be particularly important in enabling this:

I think in some ways, it built quite good relationships because, obviously, you know, as a team, we were all phoning parents once a week and actually, some relationships have [become stronger as a result]. So parents that maybe weren’t so engaged with us have been phoned every single week and have got to know us a little bit better and it’s not a . . . You know, quite often you’re ringing as a school to say, ‘Blah, blah’s done this, blah, blah, isn’t that terrible?’ But, actually, we were literally ringing to say, ‘Are you alright?’ Yeah, ‘Do you want us to do anything for you?’ There wasn’t any punishment, or any judgement sat behind that. (School safeguarding lead and deputy, local authority 13, Term 2)

A second pathway (Pathway B in Figure 2) relates to the broader social distancing measures. It suggests that (1) the move to virtual communication increased opportunities to liaise between professionals due to the diversification of ways to communicate. This was felt by some to lead to better working relationships, despite the fact one of the frustrations that provided a rationale for SWIS was the slow and disjointed nature of remote communication. During the pilot studies, professionals felt being physically co-located was better than email or phone communication that often went unanswered (Westlake et al., 2020, 2024b). However, during the pandemic it was felt the wider context made people more attentive to remote forms of communication, and one SW noted ‘everyone has been probably more accessible . . . more conscious of, accessing . . . the online Chat, checking emails, answering their phones’ (Local authority 21, Term 1). This echoes the work of Baginsky and Manthorpe (2022), who noted improvements in multiagency working through this mechanism. That said, among SWIS workers, important aspects of communication were felt to suffer from the reduced in-person contact:

‘I feel like I’m just doing part of the work and not all of it. I obviously have meetings virtually, with parents and other various agencies, and I can do some work on Teams, but I don’t think it’s as engaging as being able to do it one-to-one’ (SW, LA8, Term 2)

The second pathway also suggests (2) that in some instances SWs felt the constraints on student movement around the school and more structure around where they were located was a benefit. Some schools were large and SWs found it difficult to navigate the sites and locate students, so the bubble restrictions made this easier.

Discussion

By focusing on the disruptive forces at work, this exercise has highlighted several insights about how the crisis conditions impacted the intervention and how it responded. In this section we start with SWIS, reflecting on the lessons it offered about that intervention and some implications for the wider field of social work. Then we broaden our scope to consider how the theory of disruption approach builds on existing frameworks within implementation science and disaster studies, before closing with a discussion of the limitations of the approach.

Insights about SWIS and implications for social work

In addition to the wide-ranging challenges and forced adaptation described, which are perhaps unsurprising, this analysis also shows that positives can emerge from disruption. This has been noted by scholars in crisis management (James et al., 2011) and in many other areas, including economic, biological and political systems (Taleb, 2012). Notably, some aspects of the pandemic helped SWIS workers intervene in different ways and prompted them to be more creative in their efforts to help children and families during an acute time of need. We know from research we have undertaken previously (Westlake et al., 2022) that families value practical and material support, and that this can help build better engagement and relationships with their SWs. The crisis response the pandemic stimulated seems to have created some of the conditions that enabled this: increased contact with some families where the focus was on helping with material, practical and emotional support, or the provision of food and laptops to those in need. The unprecedented nature of the pandemic meant SWs and schools had to work together to develop strategies quickly. The pilots that informed the development of SWIS conveyed the sense that – when it worked well – SWIS could improve relationships with schools, students and families, and the joint problem-solving that the pandemic demanded seemed to accelerate this in some schools. This supports the notion that we should strive for adaptability in the face of external shocks as well as efficiency and efficacy (Taleb, 2012).

At a practice level, there are clear implications for the SWIS role and consequently the shape of the intervention. In the pilot studies, which informed the development of SWIS, we noted that the role was wider and less clearly defined than many social work roles. This raised questions about the remit of school-based SWs, the boundaries between their role and that of other professionals, and the spectre of ‘mission creep’ in circumstances where these issues were overlooked (Westlake et al., 2020, 2024b). This speaks of a long running debate about what constitutes social work, and what activities individual workers should undertake. Writing in 1925, Washington observed, ‘The trained social worker is prepared to find and expects to find social work extended from year to year to include activities that formerly were not considered social work at all’ (Washington, 1925). And commentary from the 1970s, when the profession was growing in terms of workforce and profile, highlighted the profession’s tendency to do ‘. . . an ever widening range of tasks not traditionally associated with their repertoire of responsibilities’ (Brennan, 1973). One way of reconciling this is to acknowledge creativity and improvisation as defining features of social work. Ferguson et al. (2022) make this case in their analysis of how social work practice changed during COVID-19, drawing a dichotomy between this ‘improvisatory practice’ and ‘institutional procedure’ (Ferguson et al., 2022). In any case, the theory of disruption indicates the pandemic created additional pressures at the boundaries of the role, pushing workers to undertake a greater range of work than planned, and supports similar findings about professional boundaries in responses to other disasters (Quarantelli, 1988).

Building on existing frameworks in implementation science and disaster studies

The theory of disruption described here can be positioned within Nilsen’s taxonomy of implementation science approaches (Nilsen, 2015). This characterises five categories of theoretical approaches with three overarching aims: process models that describe and guide implementation processes; determinant frameworks that identify barriers and enablers; classic theories borrowed from other disciplines; implementation theories developed specifically for the field; and evaluation frameworks. Our approach most closely aligns with determinant frameworks, as it systematically identifies how contextual changes impact on interventions, creating both barriers and enablers for implementation, and affecting outcomes. Moreover, it addresses several gaps that Nilsen identifies in existing approaches. While we acknowledge that existing frameworks increasingly recognise contextual complexity, comprehensive reviews suggest room for methodological development. Nielsen et al. (2022) found that context often remains underdeveloped in realist evaluations, frequently presented as lists of factors rather than as dynamic elements that interact with mechanisms. Nilsen notes that ‘most determinant frameworks provide limited “how-to” support for carrying out implementation endeavours since the determinants may be too generic’ (p.9), while ‘process models recognize a temporal sequence of implementation endeavours, whereas determinant frameworks do not explicitly take a process perspective of implementation’ (p.9). Similarly, developers of major frameworks acknowledge challenges capturing temporal dynamics and sudden external shocks (May et al., 2018; Pfadenhauer et al., 2017). Theorising disruption in this way addresses these limitations by treating major contextual changes as interventions with specific, empirically derived mechanisms rather than generic determinants. Our approach addresses this specific gap by treating extraordinary disruptions not as contextual moderators but as interventions in their own right. This responds to Nielsen et al.’s call for better methods to analyse contextual dynamics while complementing, rather than replacing, existing frameworks.

Furthermore, it moves beyond the typical assumption of linearity between the determinants and the outcomes by mapping complex interactions where disruptions can simultaneously create both positive and negative implementation effects. In doing so, it represents a hybrid that combines elements of determinant frameworks with process thinking, addressing Nilsen’s observation that ‘context lacks a unifying definition in implementation science’ (p.6). As we argued above, TBE faces a similar challenge, and the approach described here provides concrete analytical tools for understanding how dynamic contextual forces shape implementation outcomes. This reframing reveals how mechanisms that inhibit intended pathways can simultaneously create opportunities for innovation. The pandemic’s disruption of physical co-location, typically viewed as implementation failure, generated new opportunities for schools to engage parents. This suggests implementation science needs to account not just for whether interventions work as intended, but also how they transform when core assumptions are broken. The approach thus addresses what Greenhalgh and Manzano (2022) identify as a critical need: methods for understanding how context operates dynamically rather than simply moderating pre-identified mechanisms.

The fields of disaster studies and crisis management also provide useful material for understanding how organisations respond to major disruptions that are akin to the worked example. As recent work by Graveline and Germain (2022) shows, resilience theory has evolved from simple ‘bounce back’ definitions to ‘build back better’ and ‘bounce forward’, where disasters are seen as ‘an opportunity to improve, change, and thus adapt’ (p.332). The enabling pathways identified exemplify this phenomenon – these were not simply adaptations to maintain services, they were innovations that potentially enhanced the intervention beyond its original design. This work highlights the importance of adaptive capacity and suggests the inherent flexibility of the SWIS design was a key strength that facilitated its survival of the acute period of disruption (Cedergren and Hassel, 2024; Graveline and Germain, 2022). Viewing the theory of disruption, as applied to SWIS, in the context of this research shows how the pandemic functioned not only as a barrier but also as a dynamic force that stimulated innovation. As such, our analysis supports the principle that disruptions reveal both vulnerabilities and latent capacities. The pandemic exposed the fragility of physical co-location while showing untapped potential for other engagement. This suggests evaluators should analyse disruptions not merely as threats to fidelity, but as natural experiments revealing which components are essential and which are subsidiary, and what dormant alternative pathways might exist.

Finally, a theory of disruption could arguably be developed across disparate intervention types and disruption contexts. Using some hypothetical examples, Table 2 shows how different extraordinary changes might create similar opportunities, regardless of sector or setting. The analytical process remains consistent while the specific mechanisms and pathways vary by disruption type and intervention context. Future applications could therefore test the approach with other disruption types, refining the criteria and analytical process based on diverse cases.

Table 2.

Potential opportunities for future theory of disruption analysis.

Disruption	Intervention example	Evaluation challenge	Example ‘if-then’ statements	Relevant study/finding
Natural disasters	Community mental health programmes	Destroyed infrastructure	If formal services become inaccessible, then communities mobilise informal support, resulting in peers replacing professional interventions	Lowe et al. (2015) found that community-level factors in shaping post-disaster mental health were significant after Hurricane Sandy, with social support acting as a protective factor
Policy reversals	Cash transfer pilots	Premature termination, incomplete/missing data at endline	If participants face premature programme cancellation, then they may be forced to abandon medium-term commitments, which may undermine the programme theory regarding long term impact	McDowell and Ferdosi (2020) documented how Ontario Basic Income pilot participants responded to the programme’s unexpected cancellation in 2018. Some had made commitments based on the expected 3-year duration but had to abandon these plans
Economic shocks	Job training programmes	Breached assumptions, meaningless outcomes	If economic shocks displace workers from struggling industries, then training programmes are mismatched to job opportunities, resulting in negative short-term outcomes regardless of training efficacy	Heinrich et al. (2013) found some employment programmes during the 2008 recession proved ineffective, suggesting that training alone was insufficient to overcome job market challenges during an economic downturn
Organisational restructuring	Multiagency partnerships	Staff turnover, new structures, updated goals	If key personnel change, then institutional memory reduces and organisational priorities change, putting at risk ongoing programmes	Forrester et al. (2013) found that change in children’s services departments was fragile and dependent on key leadership figures

Limitations

There are potential challenges associated with the proposed approach. First, it does not avoid the underlying issues in defining context. While we have proposed key criteria herein, this is a new approach. Until more examples become available, critics may reasonably ask how disruption that warrants being treated as an intervention in itself should be distinguished from normal variation in context. Existing theory, particularly within the field of implementation science, covers a lot of ground regarding implementing programmes in different contexts (Greenhalgh, 2017; Nilsen, 2015). Whether to apply those approaches or to develop a theory of disruption should become clearer as the latter is applied in more cases. Likewise, more clarity about what types of disruption are most suited to this analysis should also emerge with more application. Theories of disruption may also be less amenable to testing than theories of change or harm.

Conclusions

In this article we have argued that TBE approaches do not yet deal well enough with context, and in particular significant disruption events – a limitation that risks constraining one of their primary claimed advantages over correlational methods. Our focus has been disruptive forces that exist in the outer or macro context of interventions but directly affect their implementation, and we used the COVID-19 pandemic as a worked example. Other publications have considered challenging contexts for evaluation, primarily from a research methodology standpoint, for instance, in humanitarian aid research (Smith and Blanchet, 2019). However, to our knowledge ours is a novel approach to understanding how a change in context affects implementation of an ongoing programme. It has a few potential uses for other social policy evaluations.

First, it could be applied post hoc to any such disruption that took place during implementation, as in the worked example. The impact of an earthquake, election, war or any other major change could be theorised in this way. Researchers occasionally publish articles describing how evaluations failed in various ways (see, for example, Dixon et al., 2014), so adopting this approach could help articulate the external disruption thought to be caused by the contextual change. The insights provided could then contribute to policy and practice decisions about what to do next, by helping to disentangle the signal of programme impact from the noise of the disruptive context. There are many published evaluations where the technique could be applied. For example, Harrison and colleagues (2003) considered why clinical guidelines failed to achieve their aims to increase compliance in an RCT where the control group increased compliance as much as the intervention group. The researchers ascribed this to changes in the broader policy context – ‘at the particular time of our study, any effects of this intervention were overtaken by the larger pressure of central government policy’ (p.152). In a subsequent study, Checkland, Harrison and Marshall (2007) argued that the ‘barriers to change’ metaphor dominating guideline implementation research focused too narrowly on individual clinician behaviour, missing the wider organisational and policy contexts that shaped how practice actually responds to such initiatives. A theory of disruption might point to ways in which the guidance and the central government policy could be better synthesised.

Second, the approach could be used to anticipate contextual changes that might impact interventions and inure them to the effects. The ‘pre-mortem’ approach developed by Klein as a risk assessment tool aims to do this, by anticipating reasons for failure at an early stage in a project (Klein, 2007). Klein’s idea is built on earlier research by Mitchell and colleagues (1989), who studied what they called ‘prospective hindsight’. The notion of ‘generating an explanation for a future event as if it had already happened’ (p.25) is clearly a way of incorporating theories of harm into programme and evaluation design, but it also has potential for ex ante theories of disruption, where the possible effects of unexpected contextual shifts could be projected. The stage at which this seems most appropriate would be when the initial programme theory was being developed in collaboration with practitioners, as is recommended in theory-based approaches (Bernheim et al., 2025).

Third, theories of disruption could be used to consider in more detail what the optimal conditions for interventions to succeed are. For example, as demonstrated in the worked example – amidst the disruption, some benefits that facilitate good practice might emerge. As noted above, unlike theories of change and theories of harm, it is harder to envisage how correlational evidence could be used to test a theory of disruption, mainly because of the responsive nature of the exercise. Perhaps the most important contribution of this approach, then, is that it addresses the problem of viewing context as a static and somewhat passive backdrop to interventions, and not the dynamic and active agent it invariably is. The more techniques available to researchers for understanding how interventions interact with the wider context, the more likely we are to design and implement programmes that attain the policy goals they set out to achieve.

Supplemental Material

sj-docx-1-evi-10.1177_13563890261439289 – Supplemental material for Theory of disruption: A method for understanding context in social policy evaluations with a worked example based on the COVID-19 pandemic

Supplemental material, sj-docx-1-evi-10.1177_13563890261439289 for Theory of disruption: A method for understanding context in social policy evaluations with a worked example based on the COVID-19 pandemic by David Westlake, Melissa Meindl and Verity Bennett in Evaluation

Supplemental Material

sj-tiff-2-evi-10.1177_13563890261439289 – Supplemental material for Theory of disruption: A method for understanding context in social policy evaluations with a worked example based on the COVID-19 pandemic

Supplemental material, sj-tiff-2-evi-10.1177_13563890261439289 for Theory of disruption: A method for understanding context in social policy evaluations with a worked example based on the COVID-19 pandemic by David Westlake, Melissa Meindl and Verity Bennett in Evaluation

Footnotes

ORCID iDs

David Westlake

Melissa Meindl

Verity Bennett

Funding

The authors disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This study was funded by What Works for Children’s Social Care. CASCADE also receives funding from Health and Care Research Wales.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Supplemental material

Supplemental material for this article is available online.

David Westlake is a Principal Research Fellow at Cardiff University’s Children’s Social Care Research and Development Centre (CASCADE), specialising in evaluation research in Children’s Social Care. He is a leading proponent of causal evaluation, particularly experimental and quasi-experimental methods.

Melissa Meindl is a Research Fellow at Cardiff University’s School of Medicine, specialising in social policy evaluation and implementation research. Her work focuses on understanding contextual factors in social interventions and developing methodological approaches for complex evaluations.

Verity Bennett is a Research Fellow at Cardiff University’s Centre for Social Care and Artificial Intelligence Learning (SCALE), specialising in child safeguarding, and evaluating complex interventions with a particular focus on the use of Artificial Intelligence in social care.

References

Adara

Ayayo

Munnery

, et al. (2023) The Social Workers in Schools Trial: An E valuation of S chool B ased S ocial Work. Project Report. London: What Works for Children’s Social Care.

Astbury

BGP

(2011) The Practice of Realistic Evaluation: A Conceptual and Empirical Review. Melbourne, VIC, Australia: University of Melbourne, Melbourne Graduate School of Education.

Baginsky

Manthorpe

(2022) Multiagency working between children’s social care and schools during COVID-19: Case study experiences from English local authorities and international reflections. Journal of Integrated Care 30(2): 134–45.

Bate

(2014) Context is everything. In: Bate

(ed.) Perspectives on Context: A Selection of Essays Considering the Role of Context in Successful Quality Improvement. London: The Health Foundation, 1–30.

Bernheim

Stabler

Hayes

, et al. (2025) Applying a realist evaluation to an intervention in children’s social care: A worked example from the Safeguarding Family Group Conference study. Evaluation 31(1): 49–69.

Bonell

Fletcher

Morton

, et al. (2012) Realist randomised controlled trials: A new approach to evaluating complex public health interventions. Social Science & Medicine 75(12): 2299–306.

Bonell

Fletcher

Morton

, et al. (2013) Methods don’t make assumptions, researchers do: A response to Marchal et al. Social Science & Medicine 94: 81–2.

Bonell

Jamal

Melendez-Torres

, et al. (2015) ‘Dark logic’: theorising the harmful consequences of public health interventions. Journal of Epidemiol Community Health 69: 95-98.

Bonell

Melendez-Torres

Quilley

(2018) The potential role for sociologists in designing RCTs and of RCTs in refining sociological theory: A commentary on Deaton and Cartwright. In: Kawachi

Subramanian

Mowat

(eds) Randomized Controlled Trials and Evidence-based Policy: A Multidisciplinary Dialogue, vol. 210. Amsterdam: Elsevier, 29–31.

10.

Bradbury-Jones

Isham

(2020) The pandemic paradox: The consequences of COVID-19 on domestic violence. Journal of Clinical Nursing 29(13–14): 2047–9.

11.

Brennan

(1973) The practitioner as theoretician. Journal of Education and Social Work 9(1): 5–12.

12.

Breuer

Lee

De Silva

, et al. (2015) Using theory of change to design and evaluate public health interventions: A systematic review. Implementation Science 11(1): 63.

13.

Brown

Curran

Palinkas

, et al. (2017) An overview of research and evaluation designs for dissemination and implementation. Annual Review of Public Health 38: 1–22.

14.

Campbell

(1986) Relabeling internal and external validity for applied social scientists. New Directions for Program Evaluation 1986(31): 67–77.

15.

Cartwright

Hardie

(2012) Evidence-based Policy: A Practical Guide to Doing It Better. Oxford: Oxford University Press. Available at: https://doi.org/10.1093/acprof:osobl/9780199841608.001.0001

16.

Catlow

Bhardwaj-Gosling

Sharp

, et al. (2021) Using a dark logic model to explore adverse effects in audit and feedback: A qualitative study of gaming in colonoscopy. BMJ Qual Saf 31: 704–5.

17.

Cedergren

Hassel

(2024) Building organizational adaptive capacity in the face of crisis: Lessons from a public sector case study. International Journal of Disaster Risk Reduction 100: 104235.

18.

Checkland

Harrison

Marshall

(2007) Is the metaphor of “barriers to change” useful in understanding implementation? Evidence from general medical practice. Journal of Health Services Research & Policy 12(2): 95–100.

19.

Dalkin

Greenhalgh

Jones

, et al. (2015) What’s in a mechanism? Development of a key concept in realist evaluation. Implementation Science 10: 49.

20.

Dixon

Biehal

Green

, et al. (2014) Trials and tribulations: Challenges and prospects for randomised controlled trials of social work with children. The British Journal of Social Work 44(6): 1563–1581. https://doi.org/10.1093/bjsw/bct035

21.

Dixon-Woods

(2014) The problem of context in quality improvement. In: Dixon-Woods

(ed.) Perspectives on Context: A Selection of Essays Considering the Role of Context in Successful Quality Improvement. London: The Health Foundation, 87–100.

22.

Durie

Wyatt

(2013) Connecting communities and complexity: A case study in creating the conditions for transformational change. Critical Public Health 23(2): 174–87.

23.

Ebenso

Manzano

Uzochukwu

, et al. (2019) Dealing with context in logic model development: Reflections from a realist evaluation of a community health worker programme in Nigeria. Evaluation, Program Planning 73: 97–110.

24.

Education Endowment Foundation (2020) Impact of School Closures on the Attainment Gap: Rapid Evidence Assessment. London: Education Endowment Foundation. Available at: https://educationendowmentfoundation.org.uk/education-evidence/evidence-reviews/school-closures-rapid-evidence-assessment

25.

Ferguson

Kelly

Pink

(2022) Social work and child protection for a post-pandemic world: The re-making of practice during COVID-19 and its renewal beyond it. Journal of Social Work Practice 36(1): 5–24.

26.

Flyvbjerg

(2001) Making Social Science Matter: Why Social Inquiry Fails and How It Can Succeed Again (trans. Sampson

). Cambridge: Cambridge University Press.

27.

Forrester

Westlake

McCann

, et al. (2013) Reclaiming Social Work? An Evaluation of Systemic Units as an Approach to Delivering Children’s Services. Luton: University of Bedfordshire.

28.

Funnell

Rogers

(2011) Purposeful Program Theory: Effective Use of Theories of Change and Logic Models. San Francisco, CA: Jossey-Bass/Wiley.

29.

Gardner

Montgomery

Knerr

(2015) Transporting evidence-based parenting programs for child problem behavior (age 3–10) between countries: Systematic review and meta-analysis. Journal of Clinical Child & Adolescent Psychology 45(6): 749–62.

30.

Graveline

Germain

(2022) Disaster risk resilience: Conceptual evolution, key issues, and opportunities. International Journal of Disaster Risk Science 13(3): 330–41.

31.

Greenhalgh

Manzano

(2022) Understanding ‘context’ in realist evaluation and synthesis. International Journal of Social Research Methodology 25(5): 583–95.

32.

Greenhalgh

(2017) How to Implement Evidence-based Healthcare. New York: John Wiley & Sons.

33.

Harrison

Dowswell

Wright

, et al. (2003) General practitioners’ uptake of clinical practice guidelines: A qualitative study. Journal of Health Services Research & Policy 8(3): 149–153.

34.

Heinrich

Mueser

Troske

, et al. (2013) Do public employment and training programs work? IZA Journal of Labor Economics 2(6): 1–23.

35.

HM Treasury (2024) The Magenta Book: Central Government Guidance on Evaluation. London: UK Government.

36.

ISRCTN (2022) ISRCTN72047541. ISRCTN Registry.

37.

James

Wooten

Dushek

(2011) Crisis management: Informing a new leadership research agenda. Academy of Management Annals 5(1): 455–93.

38.

Klein

(2007) Performing a project premortem. Harvard Business Review 85(9): 18–19.

39.

Lowe

Sampson

Gruebner

, et al. (2015) Psychological resilience after Hurricane Sandy: The influence of individual- and community-level factors on mental health after a large-scale natural disaster. PLoS ONE 10(5): e0125761.

40.

McDowell

Ferdosi

(2020) The experiences of social assistance recipients on the Ontario basic income pilot. Canadian Review of Sociology/Revue canadienne de sociologie 57(4): 681–707.

41.

Malhi

Karanfil

Merth

, et al. (2022) Five ways to get a grip on the shortcomings of logic models in program evaluation. Evaluation and Program Planning 91: 102042.

42.

Manzano

(2016) The craft of interviewing in realist evaluation. Evaluation 22(3): 342–60.

43.

Marchal

van Belle

van Olmen

, et al. (2012) Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research. Evaluation 18(2): 192–212.

44.

Marchal

Westhorp

Wong

, et al. (2013) Realist RCTs of complex interventions – An oxymoron. Social Science & Medicine 94: 124–8.

45.

May

Cummings

Girling

, et al. (2018) Using normalization process theory in feasibility studies and process evaluations of complex healthcare interventions: A systematic review. Implementation Science 13(1): 80.

46.

Meindl

Westlake

(2024) Evaluation of the Family Drug and Alcohol Court in Wales pilot: Final Report. Cardiff: Cardiff University.

47.

Mehrotra

Petersen

Geng

(2019) Understanding HIV program effects: A structural approach to context using the transportability framework. Journal of Acquired Immune Deficiency Syndromes 82(suppl. 3): S199–205.

48.

Mitchell

Russo

Pennington

(1989) Back to the future: Temporal perspective in the explanation of events. Journal of Behavioral Decision Making 2(1): 25–38.

49.

Moore

Audrey

Barker

, et al. (2015) Process evaluation of complex interventions: Medical Research Council guidance. BMJ 350: ph1258.

50.

Moore

Evans

Hawkins

, et al. (2019) Advancing complexity science in healthcare research: The logic of logic models. BMC Medical Research Methodology 19(1): 111.

51.

NHS Digital (2021) Mental Health of Children and Young People in England, 2021—Wave 2 Follow Up to the 2017 Survey. Leeds: NHS Digital. Available at: https://digital.nhs.uk/data-and-information/publications/statistical/mental-health-of-children-and-young-people-in-england/2021-follow-up-to-the-2017-survey

52.

Nielsen

Lemire

Tangsig

(2022) Unpacking context in realist evaluations: Findings from a comprehensive review. Evaluation 28(1): 91–112.

53.

Nilsen

(2015) Making sense of implementation theories, models and frameworks. Implementation Science 10(1): 53.

54.

Patton

(2011) Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use. New York: Guilford Press.

55.

Pawson

(2013) The Science of Evaluation: A Realist Manifesto. London: SAGE.

56.

Pawson

Tilley

(1997) Realistic Evaluation. London: SAGE.

57.

Perera

Salamanca-Sanabria

Caballero-Bernal

, et al. (2020) No implementation without cultural adaptation: A process for culturally adapting low-intensity psychological interventions in humanitarian settings. Conflict and Health 14(1): 46.

58.

Pfadenhauer

Gerhardus

Mozygemba

, et al. (2017) Making sense of complexity in context and implementation: The Context and Implementation of Complex Interventions (CICI) framework. Implementation Science 12(1): 21.

59.

Plunkett

(2024) Are policies like medicines? Not really, no. Time for a new push on empiricism. Medium, 29 October. Available at: https://medium.com/@jamestplunkett/are-policies-like-medicines-1962f8daf495

60.

QSR International (2018) NVivo Qualitative Data Analysis Software, version 12. Available at: https://www.qsrinternational.com/nvivo-qualitative-data-analysis-software/home

61.

Quarantelli

(1988) Disaster crisis management: A summary of research findings. Journal of Management Studies 25(4): 373–85.

62.

Sanders

Hume

Vallis

(2023) Trial Protocol – Additional Financial Assistance for Care Leavers Trial. London: Centre for Homelessness Impact and King’s College London.

63.

Shan

Williamson

(2021) Applying Evidential Pluralism to the social sciences. European Journal for Philosophy of Science 11(4): 96.

64.

Smith

Blanchet

(2019) Research Methodologies in Humanitarian Crises. London: Elrha.

65.

Taleb

(2012) Antifragile: Things That Gain from Disorder. New York: Random House.

66.

The Guardian (2025) Trump Administration to Cancel $1bn in Biden-era School Mental Health Grants. The Guardian, 30 April. Available at: https://www.theguardian.com/us-news/2025/apr/30/trump-administration-school-mental-health-grants

67.

Thomas

Parsons

(2016) Culturally responsive evaluation meets systems-oriented evaluation. American Journal of Evaluation 38(1): 7–28.

68.

UNESCO (2022) On how Ontario trialed basic income. Available at: https://community.unesco.org/inclusivepolicylab/s/thinkpiece/2012-MCVJJWQITDZBHJBGXDOMWJ4PVUQY

69.

Vitsupakorn

Pierce

Ritchwood

(2023) Cultural interventions addressing disparities in the HIV prevention and treatment cascade among Black/African Americans: A scoping review. BMC Public Health 23(1): 1748.

70.

Washington

(1925) What professional training means to the social worker. Annals of the American Academy of Political and Social Science 121: 165–9.

71.

Westlake

Pallmann

Lugg-Widger

, et al, (2025) A cluster randomised controlled trial of social workers in schools in England. Research on Social Work Practice. https://doi.org/10.1177/10497315251338227

72.

Westlake

Holland

Sanders

, et al. (2024a) The basic income for care leavers in Wales pilot evaluation: Protocol of a quasi-experimental evaluation. PLoS ONE 19(10): e0303837. https://doi.org/10.1371/journal.pone.0303837

73.

Westlake

Melendez-Torres

Corliss

, et al, (2024b) Social workers in schools: A feasibility study of three local authorities. Journal of Social Work 24(5): 607–626. https://doi.org/10.1177/14680173241258927

74.

Westlake

Pallmann

Lugg-Widger

, et al. (2022) The SWIS trial: protocol of a pragmatic cluster randomised controlled trial of school based social work. PLoS ONE 17(6): e0265354. https://doi.org/10.1371/journal.pone.0265354

75.

Westlake

Melendez-Torres

Corliss

, et al. (2020) Social Workers in Schools: An Evaluation of Pilots in Three Local Authorities in England. Project Report. London: What Works Centre for Children’s Social Care. Available at: https://whatworks-csc.org.uk/wp-content/uploads/WWCSC_Social-Workers-in-Schools_pilot-study_full-report_May-2020.pdf

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

6.87 MB

0.00 MB

0.02 MB