Abstract
There has been a growing academic recognition of the increasing significance of psychologically – and behaviourally – informed modes of governance in recent years in a variety of different states. We contend that this academic research has neglected one important theme, namely the growing use of experiments as a way of developing and testing novel policies. Drawing on extensive qualitative and documentary research, this paper develops critical perspectives on the impacts of the psychological sciences on public policy, and considers more broadly the changing experimental form of modern states. The tendency for emerging forms of experimental governance to be predicated on very narrow, socially disempowering, visions of experimental knowledge production is critiqued. We delineate how psychological governance and emerging forms of experimental subjectivity have the potential to enable more empowering and progressive state forms and subjectivities to emerge through more open and collective forms of experimentation.
Introduction: The experimental state and psychological governance
Our story begins in the offices of the Job Centre Plus in Loughton, Essex. During 2012, the UK Government utilised Loughton’s Job Centre to trial a new strategy for getting people back to work. The trial involved splitting-up 2000 job seekers into two randomised groups. The first of these two groups continued to follow the established Job Centre procedures, while the second were subjected to a new system. The new procedure involved a three-step process. The first step ensured that job seekers actively talked to an official about seeking employment during their first visit to the Job Centre (previously this conversation was often delayed by up to two weeks) (Haynes et al., 2012). The second step used a series of ‘commitment devices’, which established what the client would do over the coming two weeks to seek employment. The third saw those clients who were still searching for work after eight weeks ‘building their psychological resilience and wellbeing’ through ‘expressive writing and strengths identification’ processes (Haynes et al., 2012). The results of this trial saw those job seekers in the ‘treatment groups’ being 15–20% more likely than those in the ‘control group’ to be off benefits 13 weeks after visiting the Job Centre.
This story could be interpreted as innocuous. This trial is, after all, a well-meaning attempt to improve the employment prospects of job seekers. It is, however, our contention that this trial represents a distillation of an emerging and significant form of state practice, which involves three key components: (1) the utilisation of randomised, experimental trials within the delivery of public policy; (2) the application of new behavioural insights within the design of public policy; and (3) actively supporting the development of new forms of ‘psychological capital’ within target populations. We claim that these interconnected practices reflect the emerging apparatus of a psychologically oriented and experimentally grounded state (OECD, 2017; Whitehead et al., 2014; World Bank, 2015).
The idea of a psychologically oriented state is not new (Foucault, 2008; Nolan, 1998; Rose, 1985, 1999). The insights of the psychological and behavioural sciences have been shaping governance since at least the 19th century. Over the last 20 years, however, these insights have become increasingly influential within a series of governments throughout the world (Foucault, 2008; Heukelom, 2012; Jones et al., 2013; Leggett, 2014; Mettler, 2011; Whitehead et al., 2014). To date, psychological governance has been understood to possess two key characteristics. First, it has been described as a style of government that is particularly sensitive to the emotional (or more-than-rational) aspects of the human condition. Authors such as Nolan (1998) and Ecclestone and Hayes (2008) have emphasised the therapeutic nature of the psychological state, as it seeks to grapple with the ‘fragile’ emotional conditions of its subjects. Second, work on psychological governance has examined how the state is increasingly and deliberately operating in a covert fashion. Mettler (2011) has developed a submerged state thesis to describe the growing emphasis being placed on the manipulation of behaviours through subtle nudges and ‘choice architectures’. We are primarily interested in this second form of psychological state in this paper.
We claim that work in this area tends to neglect a defining aspect of psychological governance: its experimental form. The psychologically oriented state is increasingly characterised by two experimental tendencies: (1) the innovative desire to develop and test novel, and often, counter-intuitive ways of governing and (2) the emphasis it places on knowledge that is grounded within experimental techniques. These practices embody a form of experimental government that both promotes experiments in government and seeks to govern through the processes of experimentation. In part, this experimental ethos reflects the transference of preferred methodological techniques from the psychological and behavioural sciences into government. The shift to more experimental governance may also reflect a broader transition taking place in how economies and societies are organised and reproduced (Elliott and Radford, 2015; Thrift, 2011). More will be said about the nature of experiments (and the variations that exist in their definition) later in this paper. Throughout this paper, however, we primarily explore experiments that involve three key features: (1) an artificial element of set-up; (2) attempts to induce change by processes of external control; (3) a focus on the measurement of observable effects (Gross and Krohn, 2005: 64).
This paper seeks both to develop new critical insights into the impacts of the psychological sciences on public policy and to consider more broadly the changing experimental form of modern states. Emerging forms of psychological government provide a focal point through which to unpack the political, bureaucratic, and ethical implications of states that are oriented towards more formal systems of experimentation. We do not claim that emerging forms of psychological statehood are either the first or only way in which states have expressed experimental ambitions (see Pearce and Raman, 2014; Sabel and Zeitlin, 2008). We do, however, maintain that the psychological state embodies a potential harbinger of governmental systems whose structure, ways of knowing and modes of practice are based on extended mobilisations of experimental methods.
The discussion in this paper draws on extensive documentary research and over 100 interviews conducted with policy-makers, academics, and other parties associated with the use of behavioural insights throughout the public, private and non-governmental sectors. 1 This research was funded by the UK’s Leverhulme Trust and Economic and Social Research Council. The interviews conducted as part of these projects were wide ranging, but a proportion of our interviewees drew attention to the more experimental aspects of using behavioural insights to inform public policy. It is these contributions, along with the policy-related literature that elaborates on the use of experimental methods, which inform the arguments made in this paper.
The paper begins by charting the emergence of the psychological state and how it has been theorised. In this section, we critique these theories for neglecting the psychological state’s experimental form. Drawing on the insights of critical sociologies of experimental trials (Ansell and Geyer, 2017; Cartwright, 2007; Gross and Krohn, 2005; Pallet, 2012), we explore the notion of the experimental state and suggest that it provides important perspectives on hitherto overlooked epistemological, constitutional and ethical dimensions of psychological government. The next section considers how psychological experimentation is reshaping the apparatuses of states, highlighting the relationship between experimental government and flat bureaucratic structures. The final section considers the application of formal experimental trials within public policy and how they promote governing through experimentation. We argue that emerging forms of experimental governance are predicated on very narrow, often socially disempowering visions of experimentation, which find their origins in the natural sciences. We conclude the paper by delineating how emerging forms of experimental subjectivity have the potential to produce more empowering subjectivities and potentially progressive ‘beta’ forms of statehood (Christiansen and Bunt, 2012).
From psychological governance to experimental states
Critical perspectives on psychological governance
To the extent that states and governments have taken an active interest in the behaviour of their citizens, they have always been psychological in orientation (Rose, 1999). We contend, however, that the psychological sciences are now having an increasingly significant influence on the operation of states. What unites these psychological sciences is the emphasis they place on what is observable and quantifiable in human behaviour, and their tendency to see behaviour as an independent variable within the determination of social action. The emergence of the kinds of psychological state practices we are primarily interested in commenced in the early 2000s (Jones et al., 2013). Commonly referred to as libertarian paternalism, nudges, and/or behavioural insights, these techniques seek to identify and exploit cognitive biases and unconscious heuristics to produce desired policy outcomes. These new approaches can now be seen in a series of policy areas including low carbon living, personal finance, public health, taxation and welfare provision (see Thaler and Sunstein, 2008). While these novel, psychologically oriented, policies first emerged in the US and UK, they can now been found in a series of states throughout the world including Australia, Canada, Denmark, the Netherlands, Germany, Singapore, and Guatemala, Japan, the Lebanon among many others (Whitehead et al., 2014). Related policies are also being actively promoted by prominent international organisations such as the World Bank, the OECD, the Global Economic Forum, and the European Commission (e.g. World Bank, 2015).
Established critiques of the psychological state have examined its implications for democratic norms and established assumptions about human autonomy (Gill and Gill, 2012; John et al., 2011: 22). Concerns have also been raised about the ways in which the submerged state tends to redefine the boundaries that demarcate the zones where state intervention within people’s personal lives is deemed to be legitimate (Mettler, 2011; Sunstein, 2014).
The experimental state
Understanding the nature of psychological governance, and their associated critiques, is an important starting point from which to explore emerging experimental state practices. We contend that the experimental nature of the psychological state relates to what we may term: (1) the ethos of the state and (2) the technical practices of governments. The experimental state that we explore in this paper, therefore, is both a metaphorical mobilisation of experimental cultures (Guggenheim, 2012) and a set of actual practices. At a metaphorical level, the ideal of an experimental state is deployed to suggest innovative styles of government, which, while aspiring to the goals of experimentalism, can never be purely experimental themselves. The metaphorical posturing towards experimentalism does, however, have real implications for emerging forms of statehood and the ability of state agencies to support experimental practices.
The imbrication of the state with an ethos and practice of experimentation has had a history that is independent from psychological governance (Greenberg et al., 2003). In a metaphorical sense, there has been an ongoing programme devoted to the development of more innovative and experimental styles of government in states throughout the world. The neoliberal reinvention of government, expressed in the orthodoxies of New Public Management and the notion of advanced liberal governmentalities (Harvey et al., 2005), has clearly promoted the experimental pursuit of more efficient ways of delivering government services (Mitchell, 2005; Osborne and Gaebler, 1992; Rose, 1999). Similarly, experimentalist governance (Sabel and Zeitlin, 2008), which has become an increasingly popular system of government in the EU and the US over the last 20 years, is a form of problem solving government that exploits multi-level systems of governance to promote local policy experimentation. The experimentalism of neoliberalism and experimentalist governance has one clear thing in common with more psychologically oriented visions of government: a continual emphasis on improvements in things that can be measured (Davies, 2014). They do, however, differ in important ways. The experimentalism of neoliberal states seeks to facilitate the spread of market forces within the public sector and often involves the artificial creation of markets in places where they have been previously restricted (Davies, 2014; Peck, 2010). The experimentalism of the psychological state is of a different order. In the psychological state, emphasis is placed on the limitations of market-oriented rationalities, and experimentation is encouraged in areas that uncover the predictably irrational and counter-intuitive forces that shape behaviours and practices within and beyond the state (Millo and Lezaun, 2006). Moreover, the formally orchestrated experimentalism of the psychological state raises ethical and constitutional issues that are relatively unproblematic in neoliberalism and experimentalist governance systems.
In more a practical context, it is important to acknowledge that states have played an important role in the organisation and delivery of scientific experiments that predate the forms of psychological statehood discussed in this paper. The Lanarkshire School Milk Experiment of 1930 is often cited as a forerunner of the more systematic government trials that would emerge in the later 20th century. The Milk Experiment tested the health impacts of providing free milk to pupils in the Scottish county of Lanarkshire and involved some 20,000 students, half of whom were given free milk and half who were not (John, 2013). The emergence of welfarist government regimes in North America and Europe in the post Second World War period also resulted in the establishment of another wave of state orchestrated public experiments (John, 2013). State experiments were utilised during this period to test the actual impact of welfare provisions such as housing benefits. The emergence of more neoliberal systems of government in the US in the 1980s and 90s was then witness to the further popularisation of public policy trials. The US, for example, saw the development of formal policy experiments to test the effectiveness of various welfare to work and job training schemes, as well as education initiatives (John, 2013).
Early iterations of experimental methodologies possessed little or no direct connection with psychologically oriented systems of state practice, but did provide a ‘trial of strength’ for novel, but relatively uncontroversial policy initiatives (Millo and Lezaun, 2006: 180). The emergence of the psychological state has, however, become associated with arguably the most sustained applications of experimental trials in public policy history (Haynes et al., 2012; John et al., 2011; Pearce and Raman, 2014; Sunstein, 2013). Some of the more sustained engagements with experimentation have taken place in the UK, with state-sponsored experiments occurring in a range of policy contexts. The Department of Work and Pensions ran a large-scale randomised control trial (RCT) in 2003 exploring incapacity claimants (Haynes et al., 2012: 11). The Education Endowment Fund and the Department of Education are running the largest public policy RCT currently in operation in the UK. But it is the work of the UK’s behavioural insights team (BIT) that has arguably extended the use of experiments furthest into the everyday work of government. In addition to providing guidelines on the construction and use of RCTs (Haynes et al., 2012), the BIT has conducted experimental trials on energy labelling initiatives, Court Service fine repayments, as well as the aforementioned Job Seeker experiments (Haynes et al., 2012). At a more strategic level, the UK government has also established a series of What Works Centres (bringing together academics and policy makers), which are focusing on addressing crime reduction, local economic growth, ageing and early intervention. Many of these centres are promoting the use of controlled experimental forms of intervention in policy analysis (it is significant that the What Works initiative is being headed by David Halpern, who is also the CEO of the BIT). 2
In the US, too, the Social and Behavioural Sciences Team (SBST) have been running a series of experimental trials to test the efficacy of behavioural insights in policy areas including health coverage, farm subsidies and retirement saving (SBST, 2015). The Departments of Labour and the Interior have also stated an interest in the wider application of design experiments and RCTs in the US (Sunstein, 2013: 189). Nor is the use of experimental trials in public policy evaluation limited to Western governments. The use of RCTs in public policy was pioneered by economists such as Esther Duflo in the context of poverty alleviation measures in the global south (Banerjee and Duflo, 2011) – this work is now supported institutionally by the influential Abdul Latif Jameel Poverty Action Lab (J-Pal). More recently, the UK’s BIT has worked with the World Bank and the Guatemalan Tax Authority to develop policy trials on tax compliance. Given the fact that the World Bank’s 2015 World Development Report (entitled Mind, Society and Behaviour) explicitly endorses the behavioural sciences (World Bank, 2015), it appears likely that experimental trials will become an increasingly important aspect of the work of international development organisations in coming years (Karlan et al., 2009).
The contemporary rise of experimental methods within public policy in part represents a deliberate attempt to provide a robust defence of what are often perceived as controversial behavioural policies. We can tease out two key themes here. First, there is a recognition on the part of some practitioners that the evidence base available to support the rolling out of potential interventions is thin and that trials, especially RCTs, are needed to plug this gap, as this testimony from a leading exponent of the use of behavioural insights in the UK shows: I think what I’ve found striking is how thin the evidence base [is] … and that most of the evidence in books like Nudge is tiny examples, tiny experiments with groups of undergraduates, which are very misleading if you’re trying to provide healthcare to working class communities in Britain or France or whatever. (Civil Servant, UK, interview 2011)
Second, agencies promoting the psychological sciences within public policy development have been quick to cast their work as primarily pragmatic, evidence based, ‘what works’ government, as opposed to psychologically informed policy-making. As one key advocate of experimental trials in the UK put it: ‘the main strength … [is] … it’s evidence-based, that for us is the key strength. It’s not based on our intuition or beliefs about what might work’ (Civil Servant, UK, interview 2011). Emphasising such a pragmatic set of reasons allows the proponents of behavioural interventions to dodge a series of more contentious questions surrounding the ethics of using behavioural insights to frame public policy; a subject that has a subject that has been discussed at length in the academic literature (Jones et al., 2011; Leggett, 2014).
The broad point made by many commentators is that an almost essential connection exists between behaviourally informed state interventions and different forms of experimentation, ideally RCTs (John et al., 2011). As noted unequivocally by the BIT (Haynes et al., 2012: 4), ‘[r]andomised controlled trials (RCTs) are the best way of determining whether a policy is working’. RCTs are deemed to be the crucial element in the BIT’s effort to promote a virtuous policy circle of testing, learning and adapting. It is striking that such views have become commonplace in different states. One highly placed individual in the Netherlands stated that psychological governance and RCTs had to be viewed as a ‘common package: one could not have one without the other’ (Civil Servant, Netherlands, interview 2014). Similar sentiments were voiced in Australia, where behaviour change was seen as part of a broader shift to a more evidence-based version of policy-making and one in which RCTs were viewed as the ‘gold standard’ for testing effectiveness (cf. Cartwright, 2007). In Singapore, too, an intrinsic connection was made between RCTs and behavioural interventions: ‘[t]he best kind of policies is the RCTs combined with behavioural insights, right? So you have a … behaviourally-inspired intervention. But before you roll it out, you do it as an experiment, right?’ (Civil Servant, Singapore, interview 2015).
Towards a critical theory of the experimental state
In this section, we develop a critical analysis of emerging forms of experimentalism within government. In particular, we move beyond accounts of state experiments that depict experimentation as benign practices of truth production. We seek to build upon existing technical critiques of the practices and assumptions of experimental practices (and in particular RCTs) by positioning these analyses within a broader consideration of governmental power and citizenship (Guggenheim, 2012). In doing this, we do not suggest that experimental trials are not helpful within certain policy evaluation situations. We assert, however, that experimental analyses have a specific, and necessarily limited, set of applications, and that they raise a series of issues concerning ethics, power, and citizenship that are rarely acknowledged (Ansell and Geyer, 2017; Cartwright, 2007).
It is important to position our critical project within the broader fields of political and sociological inquiry that are currently focused on experimentalism. Work within sociology and Science and Technology Studies have questioned the extent to which experiments (whether natural or social) actually embody the forms of objective detachment they claim (Gross and Krohn, 2005; Guggenheim, 2012). Our concern within this paper is not, however, to expose the sociological bias in governmental experiments but to reflect upon the limitations that emerge from attempting to apply experimental techniques taken from the natural and medical sciences within public policy contexts. The critical perspectives we develop draws attention to how experimental state practices embody the contingent mobilisation of a particular form of experimentation. Danzier (1992) has demonstrated that the psychological utilisation of experimental trials reflects the adoption of a scientific methodology of experimentation that is tightly connected with the measurement of stimulus and response. This experimental paradigm lies in opposition to the early holistic studies of crowd psychology and ‘field theory’, which examined ‘whole situations’ using ‘quite an elaborate constellation of observations’ (Danzier, 1992: 318; see also Gross and Krohn’s (2005) reflections on the societal experiments of Chicago School sociologists). While the preferred RCT methodologies of many state authorities reflect a hybrid of the controlled, laboratory studies of stimulus and response psychology, and more holistic field experiments, they ultimately tend towards the epistemological closures that are associated with medical and natural science methodologies, and thus negate more open-ended observations. Critics have questioned whether one can identify causal laws within social circumstances; suggested that complex and unpredictable social processes militate against real control on behalf of the experimenter; and asserted that the application of experiments on society raises ethical issues (Gross and Krohn, 2005: 64).
Beyond these epistemological and ethical concerns, however, we are also concerned with two perspectives that are as yet not covered within the critical literature. First, we explore what government-sponsored experiments mean for state–citizen relations and the broader function of state systems. Second, and in light of the sociological critiques of scientific experimental methods outlined above, we consider alternative ways in which governmental experiments could be delivered. In this context, we draw on work from the critical public engagement literature (Pallet, 2012: 11–14) and related discussions about open and collective experimentation (Felt and Wynne, 2007; Stilgoe, 2016), which both outline alternative ways of envisioning large-scale state experimentation. With a focus on the active participation of subjects in the construction of scientific knowledge, an aversion to the detachment of the researcher from the object of study, and a sensitivity to the complexity of everyday life, such approaches to open and collective experimentation provide more empowering and epistemologically interesting forms of experimental governance (Ansell and Geyer, 2017). We outline these forms of open and collective forms of experimentation in the penultimate section of this paper.
The apparatus of the experimental state
Recent policy debates about the need to create a more experimental form of government make much of the existence of a virtuous circle between innovation, experimentation and organisational forms. While we acknowledge that innovation and experimentation are not the same thing, in this section, we examine how experimentalism is said to support innovation and how innovative practices often require experimental activities. Organisational theory seeks to understand how to create the conditions, whereby organisations can innovate (Kelman, 2005; Nooteboom, 2000). The whole backdrop to such a concern is the fact that organisations are deemed to be resistant to innovation, with reasons including ‘the power of routines, psychological factors and standard operational procedures, which tend to benefit those in power’ (John, 2013: 9; World Bank, 2015). A recent report on so-called ‘i-teams’ (Innovation Teams), published jointly by NESTA (National Endowment for Science Technology and the Arts [UK]) and Bloomberg Philanthropies, makes clear that; [b]ureaucracies exist to bring predictability and order’ and, more problematically, that the ‘natural stance of bureaucracies is to stifle ideas’ (Pittick et al., 2014: 3).
Organisational theory states that a number of conditions should be met in order to enable innovation to occur, including the existence of less hierarchy, longer term forms of performance evaluation, senior managers who can protect and nurture innovators, separate funding streams, a separate physical space and, ideally, a low staff turnover (John, 2013: 11). One of the most famous examples of such innovative organisations was the so-called ‘skunkworks’, the nickname used for Lockheed-Martin’s Advanced Development Projects Division, which designed Lockheed’s P-80 Shooting Star during the Second World War. Such developments have been mirrored in other corporations, with Apple’s Texaco Towers team being another notable example. Skunkworks and other innovation teams purportedly show how organisational structures and relations, as well as space, can be managed in order to create the appropriate conditions for different kinds of experimental innovation to emerge (cf. Thrift, 2004).
Numerous examples of this productive connection between organisational makeup and innovation have emerged in recent years. One of the most significant in the UK, according to John (2013), has been the aforementioned BIT. A recent contribution by John (2013) draws attention to the fact that the BIT’s role in innovating governmental practices in the UK derives from its peculiar organisational status. For instance, despite the fact that those working within the BIT were, up until recently, part of the civil service and followed its procedures, it was also characterised by a relatively ‘flat’ management structure, with steering being ‘light touch’ (John, 2013: 13). Similarly, the BIT was allowed a certain latitude in relation to its goals, being famously allowed by politicians ‘to fail’ (Benedictus, 2013). The BIT has succeeded in innovating policy because it had backing from powerful patrons, including David Cameron, the then Prime Minister and Sir Gus O’Donnell, the erstwhile Cabinet Secretary (chief civil servant) of the UK Government. The BIT is not housed in a separate building, as were the original skunkworks, but in most other respects it follows the norms associated with those units charged with promoting innovation within organisations (John, 2013: 14–15). Furthermore, it is clear that this is a norm that is being replicated in other governmental contexts. Notable examples include the Barcelona Urban Lab, the unit created by Barcelona’s municipality to turn the city into an ‘urban laboratory’ (John, 2013: 15) and the Centre for Public Service Innovation in South Africa, whose aim is to ‘create an innovation culture across the South African government’ (John, 2013: 22).
Moreover, a virtuous circle is said to exist between particular kinds of organisational structure, the innovation that they foster and the role that is played by RCTs in providing the evidence base upon which this innovation can happen. The BIT (Haynes et al., 2012: 12) argue: ‘Encouraging variation needs to be matched by mechanisms that identify and nurture successful innovations … In public services … RCTs and multi-arm trials may play a powerful role here, especially where these results are widely reported and applied’.
The i-teams report also emphasises the need to ‘[r]elentlessly measure impacts, quantify successes, and be sure to stop what isn’t working’ (Pittick et al., 2014: 7). Again, a close connection is made between formal experiments, quantification, the robustness of evidence that feeds into policy innovation and the organisational structures that underpin such innovation.
The key challenge, for many, is ensuring that the work of these innovation units meshes effectively with the more ‘traditional’ forms of government exercised elsewhere (Christiansen and Bunt, 2012: 3). Issues of path dependency, as well as the existence of more traditional forms of political authority, can mean that there is an uncomfortable organisational connection between innovation and stasis, between governmental skunkworks and more conventional governmental structures, or between RCTs and more orthodox forms of policy development and evaluation (John, 2013: 5, Galley et al., 2013). There are different ways of addressing this particular challenge. One option is to house innovators in a central and cross-cutting governmental department, in the hope that innovative solutions to old problems can be mainstreamed across all government departments. The problem with such structures is that they are still dependent on the existence of an appetite for novel solutions in other government departments and, as some of our UK interviewees suggested to us, this is not always the case.
Another option is to embed behavioural innovation units in different departments, so that more experimental attitudes are distributed more evenly across government. While this may lead to a relative lack of coordination across government, it has more potential to create a situation in which ownership over innovation is more widespread. The Netherlands are developing a form of psychological state structure based on this model (Joint Research Centre, 2016). A final option is to position the innovators outside of state bureaucracies, while still ensuring that there are sufficient communication channels to ensure that new behavioural insights are fed into the policy-making process. The more recent history of the BIT fits into this organisational pattern. Since 2014, it has been re-constituted as a ‘social purpose company’, jointly owned by the UK Government, NESTA and its employees. The additional advantage of adopting such a structure is that it has enabled the BIT to more easily promote its interventions in other states (e.g. in Guatemala, Australia, the US and Japan).
These organisational developments have two implications for a critical theory of the experimental state. First, they emphasise how the desire to drive innovation (and, at least, metaphorically, experimentalism) in public policy appears to require new forms of post-bureaucratic structures in government. Second, they raise important questions of accountability. As systems of government exhibit flatter bureaucratic form, it is easy for precise lines of accountability to become blurred. Therefore, while skunkwork governments could help to deliver on the promise of post-bureaucratic systems to ‘take back power over the things that matter to us from the anonymous, distant bureaucrats in government’ (Hilton, 2015: 20), they could just as easily result in power being lost within the distributed networks of innovation units and behavioural teams.
Governing through experimentation: RCTs and experimental subjects
Having considered the relationship between experimentalism and state structures, this section analyses the impacts of experimental state practices on political subjectivity (Jones et al., 2011). Focusing specifically on the proliferation of government-backed RCTs, we consider the implications of this now popular experimental technology for the subjectivities of state personnel and citizens.
Critical reflections on behavioural trials
Although alternatives exist (John et al., 2011, Parsons, 2002), the main experimental method employed by states in evaluating behaviourally informed policy interventions are RCTs (Pearce and Raman, 2014). In the UK alone, for example, the BIT is conducting approximately 130 experimental trials a year, some of which have cohorts as large as fifteen thousand people (Partington, 2017). First popularised within the field of clinical health and medicine, RCTs are now offered as the most reliable way of determining the effect of particular policy interventions. When applied in policy, as opposed to clinical contexts, RCTs tend to be field based. According to Druckman et al. (2006: 627), field-based RCTs ‘take advantage of naturally occurring political contexts while simultaneously leveraging the inferential benefits of random assignment’. In addition to randomised assignment, RCTs also involve the development of a control. Controls can take different forms, but generally they are a group who are subject to a policy initiative that has the specific treatment that is under review removed. The control group is seen to be important because it allows experiments – at least theoretically – to discount external (population level or general contextual) factors as causes for any observed difference between the different trial groups (Goldacre, 2008: 7).
While it is hard to oppose the development of more rigorous evidence bases for public policy-making, the promotion of experimental methodologies within government raises some important epistemological and ethical issues. Epistemologically, while controlled experimentation is now being promoted as a ‘gold standard’ for research in the public policy-oriented social sciences, it is far from a universally accepted methodology, particularly in relation to the more plural explanations that are often pursued within the social sciences (Ansell and Geyer, 2017; Cartwright, 2007). Some of our respondents, for example, recognised the difficulties in making the leap from the RCTs used in medical science out into the ‘real world’ of policy experimentation. One of our interviewees from the Netherlands described this ‘experimental gap’ (Millo and Lezaun, 2006: 179): And, we think … we like to do both some experimental studies to understand what’s going on, but I very much like … and most people in our lab do once in a while to go outside and deal with normal people and see how they respond … But, you know, it’s almost all the time that you use students, and they are a very specific group of people. (Academic, Netherlands interview 2014)
Various ‘externalities’, obviously, may play a crucial role in affecting the decisions made by individuals. RCTs are, by definition, ‘narrow’ in their outlook and, as Cartwright (2007: 12) has maintained, ‘their results are formally valid for the group enrolled in the study, but only for that group’. Testing the ‘external validity’ of behaviourally informed interventions is thus problematic (Cartwright, 2007: 11).
A second challenge revolves around a potential incompatibility between RCTs and some of the fundamental tenets of behavioural insights. One of the mainstays of the literature on behavioural insights is that different individuals or groups react to interventions in different ways. There is a need to separate policy audiences into different ‘segments’, defined according to particular characteristics (Pykett et al., 2014). Recognising this fact leads to fundamental problems for field-based RCTs. If there is no such thing as a homogeneous population that can be randomly separated into two more or less identical groups – a control group and another that is subject to an intervention – then the experimental logic that underpins RCTs becomes difficult to sustain. This point was made by an advocate of behavioural insights in the UK: ‘Almost everything in behaviour change tells you [that you] have to actually segment your population quite a lot – that what works for one group won’t work for another … It means that randomised control trials are almost impossible in this field’ (Civil Servant, UK, interview 2011).
A third challenge centres on more temporal issues. RCTs may be able to isolate the drivers of short-term behavioural shifts, but are less reliable when it comes to understanding long-term patterns of social transformation (Davies, 2014). One of our interviewees in Australia referred to StartSmart, an intervention aimed at teaching young people to deal with money in a responsible manner. One of the key challenges, according to this individual, was how to measure the longer term impacts of the scheme on ‘behaviours, in many cases, that won't necessarily surface or be evident until later in life’ (Civil Servant, Australia, interview 2013). RCTs were not necessarily the most appropriate approach in this context.
Experimentally orchestrated forms of public policy also raise ethical issues. There has been some discussion of the unfairness of the division of society into groups who receive a new intervention and those who do not, leading to the creation of control or placebo citizens who do not receive the same access to potentially more effective policy interventions than their counterparts. One Dutch respondent equated the allocation of people into control groups as being ‘like depriving patients of working medicines’ (Civil Servant, Netherlands, interview 2014). Similarly, a UK interviewee questioned how one could justify such an approach, since it ‘involve[d a] different kind of offer and deal for different citizens’ (Civil Servant, UK, 2011 interview). Indeed, in Finland, it was suggested that the desire to utilise RCTs would require a change in the state constitution, to allow the government to treat ‘treatment’ and ‘control’ groups in the population in different ways (Breckton, 2015).
Another ethical issue relates to the way in which individuals are recruited into trials. While in the social and behavioural sciences people are recruited into experimental trials on a voluntary basis, it is less clear that this will be the case within public policy initiatives. Concern has been raised about the fact that, in some cases, individuals are not aware that they are taking part in a state-sanctioned trial. The practical benefits of doing so are clear, as explained by the following quote by a Dutch advocate of RCTs: ‘we don’t talk about experiments while they’re on-going … if people are taking part in an experiment, it will actually affect their behaviour and they will act differently’ (Civil Servant, Netherlands, interview 2014). And yet, there are obvious issues here relating to the lack of informed consent associated with such experiments. The issue of consent is only heightened when it comes to the potential use of social media platforms to conduct large scale RCTs on unwitting participants (Whitehead et al., 2017). The lack of openness represented here echoes the much larger concerns voiced by many critics of psychological governance (Mettler, 2011).
A further apprehension related to trial recruitment revolves around issues of state coercion. In the behavioural trial discussed in the introduction to this article, concerns were raised that participants in the experiment were worried that their benefits would be cut, or lost, if they did not participate (Sanders, 2014). While it may be possible to get people to volunteer for public policy trials, it is difficult to completely disassociate this voluntary act from the existential threat that non-compliance may generate in the mind of the subject. There is also a danger that experimental trials of this kind unintentionally target certain, lower income groups, from lower income areas, who are most dependent on government policies and thus more likely to feel compelled to participate in experiments of this kind. This raises the possibility of governing through experimentation: where vulnerable segments of the population are governed through the ongoing threat of non-participation in trials.
It is in relation to such concerns that Sunder-Rajan (2007: 85) has argued persuasively for the analysis of experimental trials to focus on the broader contexts within which they operate. The freedom to volunteer to partake in drugs trials in India, for instance, cannot be divorced from ‘prior moments of violence that provide the inducement to sign an informed consent form’. We are not suggesting that the utilisation of RCTs within psychological state practices reflects the same ethical problems associated with the clinical trials currently being visited upon vulnerable communities in places like Mumbai. We maintain, however, that the uneven use of RCTs on vulnerable communities should be recognised, and the conditions under which the consent of economically vulnerable participants is obtained scrutinised carefully.
Advocates of experimental forms of government in general, and RCTs more specifically, suggest that through a mix of obtaining informed consent and post-intervention briefing that most of the ethical concerns mentioned above can be addressed (John et al., 2011: 38). We are less sanguine. In the context of an increasingly complex, non-hierarchical skunkworks style of psychological state apparatus, monitoring the ethical legitimacy of experimental research will be difficult and open to exploitation. This has implications not only for the treatment of research subjects but also for how the knowledge that is gained from trials is used (with certain policy trials now involving partnerships between government departments and private sector companies who would be keen to exploit the commercial value of data). While there has been discussion in places such as the UK and the Netherlands of developing assessment panels to analyse the academic validity of RCTs, there is far less evidence of the formation of ethical committees that could be used to validate experimental research in the public sector.
From experimental subjects to experimental citizens
A final context for thinking more critically about the notion of the experimental state is in relation to experimental subjectivity, an idea that emerges from the reflections of Dobson (2014: no page). Dobson proposes the experimental subject as a pejorative term, set against the idea of conventional citizenship: ‘for nudgers, people are not citizens involved in the co-creation of policy, but experimental subjects to be prodded and poked in the petri dish of the behavioural economist's imagination’.
In contradistinction to Dobson, we maintain it is possible to construct a progressive vision of experimental citizenship. From the work of Irwin (2006) on public engagement within studies of biotechnology to Sunder-Rajan’s (2007) analysis of the forms of ‘experimental subjects’ that have emerged in transnational pharmaceutical trials, there is a growing recognition that it is possible to imagine a more empowering process of citizenly engagement in the experimental process (Pallet, 2012). This work rejects the idea that experimental subjects are ‘innocent citizens’ who enter experiments as impartial participants (Irwin, 2006). If experimental participants are truly impartial innocents, perhaps all we can learn from them is encoded in their observed behaviour. But if, as studies of the sociology of science suggest, experimental subjects (not to mention experimenters themselves) are never neutral arbiters of knowledge, there may be more creative ways in which we can construct trials.
We suggest there is a need for a form of participatory behavioural experiment, within which participants are both subject to a trial, but also active members in its construction and evaluation. This is a vision of the experimental state within which we can talk not only of the rights of experimental citizens but also the responsibility of citizens to contribute to the production of scientific knowledge alongside different ways of knowing – some ‘expert’ and others ‘lay’ – helping to produce what might be termed ‘heterotopic’ experimental knowledges (Laborde, 2015). This is also a vision of an experimental subject who is able, as Thrift (2011: 18) has put it, to rekindle a sense of imaginative play and curiosity associated with childhood, with these qualities having the potential to ‘speak back into the all-encompassing ambitions of [both] the security-entertainment complex’ and the experimental state in unexpected ways.
In this vision of the experimental state, the belief that we can randomise and control social reality is abandoned. Instead, sophisticated understandings of the impacts of policies on individuals and communities are produced. These forms of collective experimentation could, among other things, produce controlled data (if that were deemed necessary and/or appropriate), but would involve a broader system of knowledge gathering that Irwin and Michael (2003) have described as an ‘ethno-epistemic assemblage’. The idea of the ethno-epistemic assemblage recognises that the production of knowledge always involves the coming together of different groups, which include scientists, experts and lay communities. This idea also recognises the influences that each group has on the construction of knowledge and should result in the figures of the detached and objective external expert, and innocent participant, receding from view.
There are strong political and epistemological arguments for the construction of more collective and open forms of public experiments. In his analysis of the UK Mobile Telephone Health Research programme, Stilgoe (2016) praises the open forms of experimentation it promoted on the basis that it helped to ease public concerns about the technology (whereas closed system of public policy experimentation, often, increase anxieties). He also reflects upon the ways in which more open forms of experimentation can be used to develop a combined interest in scientific uncertainties and legitimate public concerns, while also being open to elements of surprise. The fact that collective experimentation is more able to observe a range of unexpected outcomes than controlled trials speaks to their epistemological value. According to Ansell and Geyer (2017: 156), theories of complex systems suggest that our world is made up of orderly, complex and disorderly systems. While RCTs can be effective in analysing orderly systems, with binary outcomes, they are unable to reveal the nature of complex systems that are marked by contingency and evolutionary dynamics. The problem here, then, is not so much the use of RCTs but rather their misapplication within circumstances that are defined by structural complexity and uncertainty. In situations that are defined by such complexity, a much broader set of qualitative and quantitative methods are required. The danger with applying RCTs in such circumstances is not only that they may be epistemologically limited, but that they can frame situations of complex uncertainty as ones of simplified control (Gross and Krohn, 2005: 69). Open and collective experimentation thus simultaneously facilitates new forms of participatory democratic opportunities while utilising multiple public perspectives as contexts in and through which to observe and analyse complex systems. These collective approaches portray progressive interplays between state systems and experimental methods and illustrate how post-bureaucratic forms of the experimental state can be rethought in more radically decentralised forms.
While the notions of collective experiments and ethno-epistemic assemblages may appear to be optimistic, they have been mooted as part of a psychologically oriented state in the Netherlands. One group, which has been advising the Dutch state on its use of behavioural insights, envisaged forms of collaborative experimentation: ‘So, we say that if you have these controversial topics, you should involve citizens more and civic organizations more and collaborate with them … Experiments with people, not just about people or around them or for them but with them’ (Academic, Nertherlands, interview 2014).
It is our contention that developing experiments that engage people, rather than simply carrying out experiments on them, could militate against many of the ethical and intellectual critiques of experimental statehood, while still preserving the evaluative intent of trial-based policy systems. It is pertinent to note here that RCTs have been popularised in association with a particular brand of psychological governance – often referred to as nudge or libertarian paternalism – which tends to take a fairly dim view of human nature. For libertarian paternalists, the human ability to engage in deliberation and effective decision-making is limited (Jones et al., 2013). Other models within behavioural psychology are, however, more optimistic about human decision-making capacity (Gigerenzer, 2002; Rowson, 2011; Whitehead et al., 2017). It is around these models of cognition and psychology – which emphasize humans’ nuanced capacities to know – that alternative models of experimental statehood and citizens can be most effectively developed.
Conclusion: On experiments and beta-statehood
Our goal in this article has been to shine a light on a previously neglected aspect of the emergence of the psychologically orientated state, namely its growing and more systematic use of experimentation. The growth in experimentation is witnessed most explicitly in governments’ affinity towards RCTs, but it is also connected to a growing emphasis that is being placed on innovation more broadly within public administration.
As well as representing specific responses to particular policy challenges or social ills, we claim that the growth of the experimental state represents a qualitative shift in the theory and practice of government. Davies (2014), in this respect, has cogently argued that new systems of emergency government have emerged as a way of responding to the challenges of the financial crisis. We contend that the development of the so-called experimental state is part and parcel of this process. Advocates of experimentation, for instance, argue strongly that it is the most effective way of addressing a so-called ‘pivot point’ in the history of governance. They maintain that ‘public sectors around the world are facing a challenge of reinvention with very little knowledge about how to do it’ (Christiansen and Bunt, 2012: 5). The need for reinvention has come about for two main reasons. First, there is a tension between recognising that the neoliberal government of ‘old’ has helped to create a series of social ills that require more intervention while, at the same time, realising that full-blown state intervention is politically unpalatable. Second, there is a need to design more effective forms of intervention at the same time as making large-scale savings to the cost of government. In all this, we may be witnessing somewhat of an existential crisis among state practitioners as they come to realise, belatedly, that they have lost their capacity to know, predict, control and discipline their populations and economies in effective and political acceptable ways. A behaviourally informed and experimentally predisposed state is said to provide a technocratic and seemingly neutral solution to these conundrums.
Advocates of more experimental forms of statehood go even further when they argue that experimentation is leading to the formation of a new kind of state or, at least, to a recasting of the ethos of government. They maintain that we may be witnessing the evolution of a form of ‘beta-state’, which is constantly mutating and open to different kinds of input – from civil servants, academics and the general public alike – and one which is in a constant process of becoming: The concept of ‘beta’ is relevant here. An established principle in technology development, beta versions are an early prototype version of a platform, tool or web presence. Beta is a powerful idea to apply to public policymaking. It changes expectations of performance and permanence of public services, given the signal of early-stage development and ongoing learning. Beta not only welcomes feedback, but proactively encourages challenges and critique from the public, potential users, colleagues, partners, experts and other relevant actors. (Christiansen and Bunt, 2012: 18)
There are, however, certain hurdles that may hinder such a progressive form of public engagement with the state, not least the need to engage with the scientific discourses and practices that are being valorised by the state. It appears reasonable to suggest that the emphasis on such things as RCTs is making it increasingly more challenging to inform, amend or counter public policy. NGOs working on public policy issues, for instance, often do not possess the resources to run their own RCTs, which would allow them to engage with the experimental state on a more equal footing. More worryingly, how does one argue against the findings produced by an RCT, especially when that is increasingly being deemed to represent the ‘gold standard’ of policy evaluation? RCTs, in this respect, could well be closing off any meaningful route to the kinds of interactions and dialogue that would characterise the beta state.
Finally, there is another somewhat disturbing tendency associated with the growth of the experimental state and that is its foregrounding of certain forms of academic knowledge over and above others. Increasingly, it is the academic knowledge of the psychological, neurological and behavioural sciences that are being valued, with those of the social sciences potentially being downplayed and marginalised (Ansell and Geyer, 2017). We do want to contribute to a turf war between these two sets of academic visions of the nature of humanity and behaviour. There is, potentially, much to be gained by developing public policy solutions that are underpinned by sophisticated behavioural and psychological theories. And yet, there is also much to lose, if one focuses solely on these kinds of explanations, not least the significance of human interactions as influences of behaviour, the structural factors that make certain behaviours more possible or desirable than others, and the impact that places, spaces and environments have on those behaviours.
Footnotes
Acknowledgements
We appreciate the comments and suggestions provided by the editors and the three anonymous referees.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This article is based on research undertaken in two projects funded by the Leverhulme Trust (F/00 424/L) and the ESRC (ES/L003082/1). We are grateful for the financial support.