Abstract
There is a rational assumption built into some research projects that policy contexts are influenced by the quality of the evidence. This is, at best, only somewhat true some of the time. Through policy ethnographies, two education researchers working in the context of sustainability discuss their experiences with evidence-based policy. Central to both accounts is how critical messages about such issues as race, wellbeing and sustainability can become diluted and even lost. In the existing ‘politics of unsustainability’, and at a time of ‘post-truth’ politics, these accounts also show the limits of evidence-based policy.
We argue that those working with ‘the evidence’ need to be open about how evidence-based approaches can end up supporting the ‘status quo’. Moreover, while approaches such as knowledge mobilisation emphasise the relational qualities of policy contexts, and the importance of simple compelling narratives for decision-makers, they, like many other practices, do not sufficiently theorise the power structures surrounding knowledge and the policy context. In addition to the careful use of evidence, we argue that there needs to be greater emphasis on building healthy policy ecologies – including far more emphasis on building critical and creative policy alternatives, especially in areas like sustainability and education.
Keywords
Introduction – power, evidence and policy ecologies
Many education policy contexts emphasise the importance of ‘evidence-based’ or ‘evidence-informed’ policy as a rigorous and scientific approach to the complex domain of policy-making (Lingard, 2013). While this point of view can seem self-evident on the surface, after all it seems inherently desirable to have decision-makers considering ‘what works’ and ‘the best evidence’, such assumptions fail to provide enough insight into the power structures that underpin the policy process.
In this paper, the two researchers, Robert and Arjen, with strong interests in the role of education in fostering a more sustainable world, share personal accounts of bringing evidence to policy. A critical discussion of evidence-based policy precedes our stories. This discussion focuses on the political and ideological context of policy making and questions the extent to which this context might be considered broadly ‘rational’ – that is to say, a process framed by a technical and objective approach to ‘the facts’ (Simons et al., 2009). We draw on a critical policy perspective (Prunty, 1985; Taylor, 1997) to argue that policy is a domain of power, where evidence is just one of many forces that may (or may not) influence decision making. In this regard, we draw particular attention to the way ‘post-truth’ politics is an example of how irrational and powerful elements can undermine rational forms of analysis such as evidence-based policy (Peters, 2017; Stratford, 2017).
The personal stories provided by the two authors are forms of auto-ethnography (Denshire, 2014). Our accounts are intended to illuminate the limits of evidence-based policy, especially the extent to which critical messages get ‘lost’, while ‘business as usual’ gets to carry on. Our presentations reflect two very different policy contexts – Robert’s story is drawn from education evaluation and policy in New Zealand, and Arjen’s from his experiences evaluating the Decade for Education for Sustainable Development (Wals, 2009, 2012) and contributing to the 2016 Global Education Monitoring Report (UNESCO, 2016) – our accounts share many similarities, especially in terms of how critical messages can be distorted by decision-makers.
In drawing on our experiences, we consider the implications for an evidence-based approach to Education for Sustainable Development (ESD). While we accept that there is always a role for evidence in policy discussions (see Biesta, 2010 for a good discussion of what counts as evidence), as well as approaches that attempt to build relationships with decision-makers – such as ‘knowledge mobilisation’ (Moss, 2013) – we also emphasise the need for an ecological understanding of policy-making, one that draws on a range of strategies and not just those typically described as ‘evidence-based’ or even ‘evidence-informed’ encounters with decision-makers. Indeed, we argue that it is not just better forms of evidence that are needed, but a more political understanding of knowledge and more healthy policy ecologies. While there is not space in this paper to provide a detailed theorising of what makes up a healthy policy ecology, the point is made that when we come to understand policy as a domain of power, then it is important to, on the one hand, emphasise the need for rigorous theoretical work to clarify and challenge the values and structures surrounding evidence and, on the other, also understand how policy alternatives – developed as competing policies – need to be developed in competition to the work of ‘decision-makers’. Policy alternatives, we argue, help us to go beyond the power structures and narrow political imaginations of decision-makers and consider more fully just what might ‘be possible’ in what Ron Barnett has described as the ‘best of all possible worlds’ (Barnett, 2010).
By using the term policy ecologies, we wish to emphasize the relational aspect of policy and policy-making in times where, in at least in some parts of the world, we are witnessing a shift from governmentality towards governance. This relational aspect is manifested in governance arrangements that span multiple actors, often stakeholders and boundary-crossing between sectors and levels, and involve continuous negotiation, framing and reframing of the issues a policy seeks to address. Governmentality is a Foucauldian concept referring to the generation of different subjectivities facilitated by particular techniques and rules which include investigations of the typical Foucauldian knowledge/power nexus (Amos, 2010).
The adjective ‘healthy’ is used not only to refer to policy ecologies where these relations and interactions are relatively open and transparent, and not dominated or even corrupted by certain powerful groups, but also to the possibility of transgression and the disruption of unhealthy patterns or routines that hinder more fundamental change.
The bankruptcy of evidence-based policy in the post-truth era
Recent political events have shown a high level of disregard for ‘the truth’. In the English-speaking world, the UK’s Brexit campaign and the never-ending US presidency of Donald Trump reflect a trend towards politicians ‘doubling down’ on telling outrageous lies. The famous ‘Brexit bus’, for example, claimed that the UK was sending £350m to the European Union every week. This fabrication, along with many others, has been pointed to as complicit in mobilising a considerable section of the voting public – including, it has been suggested, those left behind in the ongoing neo-liberalisation of Britain since Margaret Thatcher’s election in 1979 (Economist, 2016).
Post-truth politics extends beyond the English-speaking world, as the citizens of Russia, The Philippines, Poland and Turkey can attest (Economist, 2016). Politicians such as Vladimir Putin, Recep Tayyip Erdoğan and Jair Bolsonaro can be added to a post-truth collection of politicians that includes Donald Trump, Boris Johnson, Nigel Farage and Tony Abbott. While there might be ongoing academic debates about the nature of post-truth politics (Stratford, 2017), there is enough political truth-dodging on display to suggest that the idea of evidence-based policy seems like a ‘big ask’ – much like Donald Trump’s claim that he is going to tax the wealthy (Kiely, 2017; Qiu, 2017). Nevertheless, there is still a considerable level of support for the idea that ‘evidence-based’ policy is the (only) ‘rational’ approach we need to take, and an approach which will somehow limit the extent to which policy-making is based on ideology, belief, instinct and outright fabrications.
In New Zealand, where Robert lives, a rationalist and evidence-based approach has been championed by Sir Peter Gluckman, who up until recently held the role of the Prime Minister’s Chief Science Advisor. Gluckman’s (2011) view emphasised that ‘scientists’ should act in the role of the ‘honest broker’ when providing evidence to the policy context. Rather than operating as advocates for particular ideas, scientists should serve the values-driven policy-makers of the day by making ‘the facts’ as clear as possible: Democratic societies make decisions and policy based on many inputs, including fiscal considerations, societal values, prevailing public views, and the ideals and vision of the government of the day. But democratic governments want to make good decisions and at the base of such decision making should be the use of high quality information and evidence, both in developing new policies and in evaluating current policies. (Gluckman, 2011: 5)
Certainly, Gluckman’s approach did not seem to ameliorate the post-truth political tactics of the New Zealand government during his time as Chief Science Advisor. For example, John Key, who was the Prime Minister of New Zealand when Gluckman made the above statement, helped deliver countless post-truth political moments. In response to media questions about New Zealand’s rising inequality and declining environment, John Key said that the facts were open to interpretation and that ‘experts were like lawyers’ given that it was always possible to find another who will give a different opinion (Peacock, 2016).
In Arjen’s home country of The Netherlands, a similar political entanglement to knowledge is in play. In 2016, Arjen was one of the 64 sustainability-oriented professors who signed a petition to urge the Dutch government to meet agreed upon CO2 reduction targets by, among other things, closing all the country’s coal burning facilities. In recoiling from such expertise, one Member of Parliament, from the Liberal Party of Prime Minister Mark Rutte, dismissed the scientifically argued plea as ‘a cheap story’, while another Liberal MP referred to the renowned Dutch Meteorological Institute (KNMI), which has helped provide the key data on climate change, as a ‘bunch of charlatans’. In the 2019 provincial elections, a young party called ‘Forum for Democracy’, a party that trivializes climate change and demands that the government create a ‘reporting point’ where students can report scientist who indoctrinate a leftish agenda, received the most votes.
Re-considering evidence within healthy policy ecologies
In the face of such political disregard for facts, expertise and evidence, rationalists might argue that they have no control over the values and judgements of decisions-makers. Rationalists are also right to point out that evidence is both a necessary and (at least potentially) an influential aspect of policy-making (Oliver et al, 2014, Bennett and Jessani, 2011). In this regard, there are numerous examples of how evidence has (eventually, in some cases) led to policy change, including the laws governing seatbelts, anti-smoking policies and international agreements about CFC proliferation. Importantly, these are issues where political consensus and will were suitably mobilised, albeit that in examples such as tobacco regulation, those with power were certainly able to politicise the evidence and massively slow down justified changes to the law. Something of this complexity can also be seen in arguments around climate change (and climate change denial) where there have been up front political attempts to manoeuvre around the evidence. Bruno Latour (2015) has charted, for example, how climate deniers have mis-used scepticism about climate science to falsely claim there are two ‘sides’ to the climate debate. Latour points out that the over-reliance on a depoliticised ‘evidence-based’ approach by scientists is part of the problem. Latour argues that by not recognising the entangled, political nature of their knowledge, scientists have ceded too much political ground to those exploiting scientific deference to certainty. Instead he advocates for embracing the political context of knowledge and getting beyond a (Holocene-Modernist) fact-value dichotomy to establish a science-with-politics perspective.
The importance of the science-with-politics perspective needs to be put alongside those examples which demonstrate how ‘evidence-based’ policy can be a tool for maintaining the political status quo. For example Sarah Maddison (2012), in discussing Australian policy approaches to Aboriginal and Torres Strait Islanders, has pointed out that the use of evidence has historically been shaped by particular mainstream values or attitudes towards the indigenous population and justified on the basis that this was ‘for their own good’. Importantly, Maddison’s work is a reminder that there is no such thing as value-free evidence and that ‘Eurocentric’ positions can become a default approach in evidence-based policy. In this regard, the points raised by both Latour and Maddison also reflect years of argument from critical policy about the discursive nature of policy and the extent to which policy processes tend to legitimise established forms of power and authority (Ball, 1993; Ball, 2015; Codd, 1988; Olssen et al., 2004; Prunty, 1985).
In the face of the political nature of knowledge, ‘evidence-based policy’ may be softened by its more nuanced cousin: ‘evidence-informed policy’. In basic terms, the shift from evidence-based to evidence-informed policy signals a move away from hard and fast rules about the importance of objective evidence and towards more of an acceptance of the extent to which the policy process is also a domain linked to values and social context (Head, 2013). While this is a welcome softening of a technocratic and positivistic approach to evidence and politics, it does not fully clarify the entangled nature of ‘evidence’ and still assumes that decision-makers are the only ‘political’ bearers of value judgements. In a similar vein, some rationalists might also point out that, just as researchers need support in developing their messages to a policy-maker audience, decision-makers themselves need to develop skills in reading and responding to evidence. In some cases, for example, scientists are also encouraged to develop rapport with policy makers, developing clearly presented material, including persuasive policy narratives or compelling short stories, which can help their evidence have an impact on the values, beliefs and political considerations of decision-makers. Such ideas underpin the growing field of knowledge mobilisation (Organisation for Economic Co-operation & Development, 2007, Moss, 2013, Cooper et al, 2009).
Knowledge mobilisation recognises something of the subjectivity of the policy context by positioning the relationships between knowledge ‘producers’ and ‘users’ as both a form of politics and a context for policy change. An example of knowledge mobilisation can be found in the work of the Canadian Social Sciences and Humanities Research Council (SSHRC). The Council defines knowledge mobilisation as: The reciprocal and complementary flow and uptake of research knowledge between researchers, knowledge brokers and knowledge users—both within and beyond academia—in such a way that may benefit users and create positive impacts within Canada and/or internationally, and, ultimately, has the potential to enhance the profile, reach and impact of social sciences and humanities research. (SSHRC, 2019)
Like many funding bodies, the Council wants knowledge to transcend the work of knowledge producers and carry over into the work of knowledge users. Canada’s Sustainability and Education Policy Network (SEPN) has been well supported by the SSHRC and the influence of their funding can be seen in SEPN’s guidelines for Knowledge Mobilisation. Through these guidelines it is clear that SEPN’s focus is on using their research to create impact on decision-makers as well as educational practitioners (SEPN, 2019). In particular, SEPN’s Knowledge Mobilisation objectives focus on these three goals: increasing the use of research-based tools in policy-making and practice within education types, identifying points of entry for action on Canadian sustainability issues to mobilize target audiences through education, and, lastly, increasing expertise through multi-sector research partnerships and training.
What is not made clear in the definitions and guidelines about knowledge mobilisation from SSHRC and SEPN is the extent to which the political context of knowledge is more than a persuasive argument from ‘knowledge producers’. Indeed, in depicting an essentially relational interpretation of the knowledge context, questions also need to be asked about the political structures surrounding the knowledge context and the power structures underpinning all politicians, including those forces in the extreme via post-truth politicians and climate deniers. When asking such questions, the limitations of rationalism in a post-truth context are highlighted by asking how persuasive an argument would have to be to convince Donald Trump that climate change is a hoax?
Moreover, despite the political irrationality exemplified by post-truth politics, it is important to remember that sustainability, as well as Education for Sustainability argued for by organisations such as SEPN, carries with it its own political subjectivity that is variously contested across a wide range of policy contexts. Drawing on critical approaches to sustainability, there is reason to doubt that this concept has the political depth to actually change the deep (power) structures underpinning education, social, economic and political systems (Blewitt, 2015). This is a familiar issue and is linked to questions that about the lack of a suitable theoretical base for sustainability education as well as the connections that have been made between neo-liberalism and sustainability education (Kopnina, 2014). John Blewitt’s analysis is worth recalling here especially. In discussing the lost opportunity for a Green New Deal following on from the Global Financial Crisis (GFC), Blewitt made the point that ‘institutionalized EfS [has] continued to demonstrate a weakness that came with decades of accommodation, compromise and collusion’ (Blewitt, 2013).
Broadening this discussion from education in the context of sustainable development to sustainability per se, Ingolfur Blühdorn’s work provides an important insight into the political nature of evidence. Through his ideas about the ‘politics of unsustainability’ (Blühdorn, 2007, 2011, 2013, 2016). Blühdorn identifies how the current political system is locked into a politics of simulation that normalises both the identification of a global environmental crisis and the failure of policy-makers to adequately respond to this crisis. Blühdorn connects this inaction to a shift towards an ‘objectivist’ approach to environmental politics linked to ‘sustainability’. This approach has involved a move away from older-style subjectivist values towards environmental protection and towards more objectivist and ‘evidence-based’ traditions of sustainable development and ecological modernisation. Significantly, and as a part of this objectivist approach, the idea of evidence is necessarily filtered through the (typically undisclosed) value-laden neo-liberal assumption that economic growth is non-negotiable and that technological fixes will ‘somehow’ decouple economic activity from current and future environmental damage (Fedrigo-Fazio et al, 2016; Jackson, 2016; Stratford, 2019; Victor, 2008).
Without fully critiquing the ‘green growth’ agenda here, or restating the case that the current economic and social assumptions have put the planet on a dire trajectory (Steffen et al., 2015), it is important to point out that Blühdorn’s arguments about the politics of unsustainability underline the importance of Latour’s science-with-politics perspective. Both writers inform about what else is needed beyond ‘evidence-based’ interactions with decision-makers by asking what sorts of political actions should be carried out with evidence. The concept of policy ecologies is useful here. Drawing on a critical policy perspective, and incorporating the central role that power plays in policy decision making, we argue that a range of processes needs to be used to explore how evidence can support meaningful change. There is not space in this article to explore in depth the parameters and dynamics of policy ecologies, but perspectives from the critical policy literature provide a suitable starting point. Bob Lingard, for example, has explained how, beyond a narrow idea of evidence, there is an important role for deeper theoretical questioning, such as that which can (hopefully) come from academics. This importance extends from the idea that critical and scholarly work can challenge the assumptive worlds of policy-makers (Lingard, 2013). Lingard reminds us that there are always deeper issues at play than just ‘what works’ (Biesta, 2010) and that developing critical and theoretical perspectives is vital in asking deeper questions about the power structures surrounding a policy context (See also Ball, 1995).
A similar point of view has also been put forward by Sandra Nutley (Nutley, 2013) who, drawing on the work of Martin Rein (1976), has reiterated that policy as well as policy research is always more than being ‘evidence-based’ but exists on number of levels:
It is important to reinforce the points made above with reference to what is known about the practice of evidence-based policy, especially in the context of education in relation to sustainable development. Regrettably, much ESD-policy and ESD-policy research, especially in the UN-context, is still limited to developing indicators and showing some kind of impact on conventional ‘measurable’ outcomes (e.g. socio-emotional, cognitive and behavioural ones) of ESD activities, which essentially consolidates a neo-liberal interpretation of education that lies at the heart of the current sustainability crisis (Huckle and Wals, 2015).
The idea that a depoliticised approach to ‘evidence-based’ policy can lead to the ongoing endorsement of the status quo has also influenced the emerging field of policy ethnography. Alex Stevens, for example, has used an ethnographic approach to explore his time as a participant-observer with a British team of civil servants working on social policy and criminal justice (Stevens, 2011). Steven’s paper concludes that, at least in his own case, the British civil servant he was working with did undertake policy discussions based on evidence, but ‘few use(d) available evidence that challenge(d) the contemporary distribution of power’ (Stevens, 2011: 255).
So what else is needed besides evidence-based policy? This is especially pertinent, we suggest, for ESD and other sustainability-oriented activities that have sought to be ‘evidence-based’. Before such a question is answered however, the experiences of Robert and Arjen are presented as additional ethnographic evidence. Following these examples, the idea of policy ecologies is returned to and the idea that there needs to be more paradigm challenging approaches for policy – using Nutley’s language – or, from our point of view, more alternative policy forms which can provide a far more diverse policy ecology for ESD than currently exists.
A note on the following policy ethnographies
The following stories from Robert and Arjen are broadly consistent with the experiences of Alex Stevens. As with Stevens’ account, it is important to point out that our auto-ethnographic stories are very much told from our individual perspectives. They are not simply accounts of ‘what happened’, but our particular understandings of what happened. Managers and co-workers at the time may have different understandings. In the case of Robert, some of the events are also being recalled over more than a decade in New Zealand’s public service, and the added dimension of selective or poor recall should be considered in critically evaluating this account. That said, it is important not to dismiss these accounts out of hand. The questions each narrative raises about the potential for evidence-based policy is consistent both with the account given by Stevens and the critical questions we ask about the role of potentially critical projects like ESD. This discussion will be returned to in the implications section following the stories.
The experiences of Robert: being significant, but not too significant, in New Zealand’s public sector
I was a member of New Zealand’s Public Service between March 2003 and August 2014. During that time, I worked for two different agencies. While both of these agencies were involved in education policy and evaluation, there were no opportunities for me to contribute to Education for Sustainable Development (ESD) projects. Sadly, and in line with some of Arjen’s findings, the Decade of Education for Sustainability (DESD) had only a limited impact in the New Zealand context and therefore got little more than a brief mention in much of my work. My story does connect however to ideas about indigenous (Māori) education in New Zealand, and a concept with some resonance to sustainability: that of wellbeing.
The first agency I worked for was Te Puni Kōkiri (TPK), the Ministry of Māori Development (Māori are the indigenous people of Aotearoa/New Zealand). I had a number of roles at TPK, including that of the Manager in a small team that evaluated education outcomes for Māori, and as a ‘Policy Manager’, overseeing and working in a range of diverse policy projects but with a focus on educational and evaluation projects. Like many Public Service workers, my time at TPK was complicated by a restructure, a process which caused a considerable amount of change in the organisation. I went from working in a dedicated evaluation team to a policy team that tried to better link up evidence from public service evaluations with TPK policy development. While I left TPK before I could directly observe whether or not this ambition came to fruition, I did have time to learn some other important aspects about providing evaluation evidence in a policy setting.
One of these was how little impact evidence could have on a policy process. For example, my efforts to brief the Minister of Māori development about evaluation findings tended to be a miserable failure (if indeed the aim was ministerial engagement and policy change). On one occasion I was instructed by his office to walk with him to the elevator, as he made his way to the Cabinet Committee (that was the venue for policy-makers to discuss the report). As I made the short walk to the elevator there was time enough for the Minister to ask if everything was okay with the service being evaluated. ‘Yep’, was my hesitant reply as the elevator door closed – a one word briefing. On another occasion, I was asked to ensure that any face-to-face briefing with the Minister was reduced to a series of three or four bullet points because the Minister would not be reading either the report or the multi-page briefing. Needless to say I was barely able to make these points on a day when the Minister was very busy and getting ready to address a large national protest that had just arrived outside parliament that morning.
It might be tempting to criticise my performance as an evaluator or the Minister’s performance as a decision-maker in the above examples. This seems to me unfair given the expectations the system placed on both parties. While I was dutifully asked to provide ‘persuasive stories’ of how services might be developed to improve outcomes for Māori, the Minister himself was very busy managing a variety of politics at a time when there was both a conservative back-lash against so-called ‘race-based’ policy and the fact that the Labour Government’s Māori members were under tremendous pressure regarding a controversial law about land rights (the Foreshore and Seabed Act, 2004). On reflection, my work pragmatically channelled itself towards the sorts of short and persuasive stories necessary for a fast-working policy sector. This is not always to say that such a context means that this system is as good as we can get, or that poor practice didn’t occur. What it does underline is that evidence and policy are entangled in complex ways and require an understanding of political context, power and possibility.
In July 2005 I started a new role at the Education Review Office (ERO), New Zealand’s agency for evaluating the quality of schooling and early childhood education. My role at ERO was in preparing national evaluations about the quality of education in New Zealand, a role which regularly saw me presenting evaluation evidence to policy staff at other agencies. By the time I started at ERO I understood that a ‘useful’ evaluator was someone who could use evidence to indicate ‘significant’ issues but who would not be overly focused on issues deemed to be beyond one’s control (managers could be relied on to help you understand the difference). As I came to learn, and as was also evident in Alex Steven’s example, there was an ideological balancing act to perform in conducting ‘evidence-based’ work, a balance that involved having useful findings for policy makers while also not questioning deeper issues connected to such issues as inequality and institutional racism.
By most measures, my work at ERO was well regarded. I was given many additional responsibilities, including periods working as the manager of the evaluation team. I was told by my superior, when I had left, that I could have taken more management responsibilities if I had not decided to work only 4 days a week. While in my role at ERO, I wrote national evaluation reports on secondary schooling, inclusive education, science education, students in alternative settings (teen-parent schools, and other alternative provision), the education of Māori and Pacific students and school professional development. In general, these evaluations were highly evidence-based, including drawing on New Zealand’s iterative Best Evidence Synthesis (Alton-Lee, 2003, Alton-Lee, 2015). Nevertheless, it was also clear to me what could and could not be included in the findings of evaluations and the recommendations that were to be made to policy. The findings and recommendations were typically prepared so that ‘meaningful’ recommendations and change could occur (though not paradigm-changing, drawing on Nutley’s framework). Typically these developments included significant alterations to policies and operations, although they usually did not have significant fiscal implications and did not lead to the transformation of policy paradigms (especially those that could be politically embarrassing).
In my final throes at ERO however, I did make the mistake of pushing ‘too hard’ for a new paradigm via the evidence of an evaluation. In this instance I had drafted three-quarters of a report about wellbeing in secondary schools. In the lead up to the evaluation I had been closely involved in preparing ERO’s draft indicators of student wellbeing (Education Review Office, 2013). The indicators were developed around the idea of schools developing into communities of wellbeing, where wellbeing would become the central rationale for teaching and learning as well as systems of student support. These indicators were informed by a variety of enlightened views and evidence, including Māori world views that saw wellbeing (hauora) as a complex outcome linked to student interconnections with family and as a result of physical, spiritual and mental dimensions. This was in contrast to a mainstream view which tends to place wellbeing primarily in the domain of pastoral care and with little relationship to the curriculum (Gibbons et al, 2017). In terms of the framework provided by Nutley, such an approach to wellbeing drew on evidence in a way that attempted to go beyond the status quo assumption that wellbeing could be reduced to measurable support structures and develop a new paradigm for thinking about schooling.
When I left ERO to begin my doctorate, I had (naively) assumed that the findings and conclusion of this report would be left essentially intact. This was not the case, and the argument that I had made about schools focusing on wellbeing as a central element of their curriculum was watered down when it came to publication. Essentially, the critical messages prepared about using wellbeing as a way of understanding how the curriculum could reflect the aims, aspirations and development of every student had been replaced by a more traditional view of wellbeing as a way of supporting students to achieve (in a conventional curriculum). Wellbeing outcomes were made subservient to ‘academic’ outcomes in the report, and while schools were asked to find a ‘balance’ between these two forms, the media coverage of this report (featuring quotes from ERO management) focused on the idea that assessment was providing high levels of stress for students (Education Review Office, 2015). The idea of schools reorienting curricula to develop healthy human beings was pushed to the background. Ironically, just as I had been so successful at ensuring that evidence-based messages did not overstep any political expectations, my colleagues at ERO had also ensured that their wellbeing report did not question the fundamental structure of schools to focus on ‘academic’ achievement, often at the expense of wider issues of student happiness and development.
The experiences of Arjen – between being and being strategic: navigating force-fields in sustainability education policy research
Between 2008 and 2013 I worked as a policy-analyst/researcher within the UNESCO Decade of Education for Sustainable Development (UN-DESD) . My role was to help assess the significance of the DESD across the globe using a protocol developed by the Monitoring and Evaluation Expert Group (MEEG), which consisted of a group of researchers familiar with ESD and monitoring and evaluation. Using data from UNESCO’s regional offices, national ESD Focal Points as well as ESD experts and practitioners – mainly through NGOs – I worked on two reports: one on policies and structures that countries put in place to accommodate the development and implementation of ESD and another on the forms of teaching, learning and capacity-building that are emerging across the globe within the context of ESD. More recently I joined UNESCO’s Global Education Monitor (GEM) Team for almost six months to help the team finalize its most recent report which was launched in September of 2016: Education for people and planet: creating sustainable futures for all (UNESCO, 2016).
Taking on these jobs was somewhat delicate. On the one hand I had been appointed as a UNESCO Chair in Social Learning for Sustainable Development (although I preferred to speak of ‘sustainability’ rather than sustainable development), which may be seen as an ESD bias that I would bring to the reports. On the other hand I had been rather critical of ESD and sustainable development (Wals and Jickling, 2002; Jickling and Wals, 2008), so becoming a UNESCO Chair as a part of the DESD and taking on this task could be seen by my critical friends as ‘selling out’ or ‘being co-opted’ by the ESD train, which they saw, at best, as a distraction from what really needed to be done in education and, at worst, as an amplification of economic globalization. By accepting the job I had to navigate carefully so as to remain critical and somewhat distant of ESD, while also maintaining the trust and confidence of the UNESCO people who commissioned me to do the global monitoring. At the same time, I had to preserve my academic integrity and try to deliver reports that displayed some form of scientific rigour and quality. Achieving this balance would be a daunting task, so it turned out (although I could have known this ahead of time). Of course UNESCO wanted to show the impact of its Decade of ESD and had an interest in seeing ESD continue beyond the decade. The information sources I was provided with tended to be familiar with ESD and sympathetic to ESD’s goals. Early on, I realized there would be a pro-ESD-bias among this group and a tendency to advocate for the importance of both ESD and the UN-DESD. This made me extra attentive to those who raised critical comments about ESD and its effectiveness. I thought it was my duty to also pay close attention to the critical remarks that were raised and felt that I was able to bring these comments to the report most of the time, although not always and not always in important stages of the report writing.
Let me illustrate this with two critical incidents that occurred during the finalization of the second report, that focused on teaching, learning and capacity-building (UNESCO, 2012). The report contained many vignettes and quotes from various countries illustrating the trends and types of practices occurring within different ESD contexts. This included different accounts of ESD teaching and learning taking place across a range of contexts, such as early-childhood, primary, secondary, vocational and post-secondary education. In preparing this report, I selected several critical quotes indicating that in some instances the ‘uptake’ of ESD was low or even superficial. When the report was finished, it was edited in Paris for consistency, clarity and regional balance and returned to me for final approval. Much to my surprise, I noted that many of the critical quotes, questioning the uptake of ESD, had been removed by the UNESCO ESD research officer, who is known to be very sympathetic to, and even a strong advocate of, ESD. Below are examples of quotes that I had carefully selected and included in the report which were eliminated (sources have been anonymized) : No transformative approach has been allowed, just tweaking. Education remains a political ball. It continues to be ad hoc, small scale and without a clear mandate that [ESD] has to happen. Mainstream education won’t do it unless they are told they have to. Mainstream education has been habituated to directives from government, so some ESD is under the radar if it is happening. Economic growth continues to dominate everything with little reflection about the values and thinking that got us to where we are today. (Source: Key Informant Survey, a United Kingdom-based school network) In the UK, pressure from the Higher Education Funding Council for England, and equivalent Scottish and Welsh funding authorities, has secured some traction in most institutions. There is, however, still a lot of ‘greenwash’, and too many institutions that promote themselves as having sustainable credentials when in reality they are promoting small and insignificant elements of activity. (Source: Global Monitoring Evaluation Survey, someone from a university-based knowledge transfer network) [The] financial situation [is] forcing attention to financial sustainability – the context of HE will be particularly unstable as we charge students higher fees. Continued focus on the environmental issues at the expense of the wider ESD considerations. Lack of sharing between institutions as increase on competitive mode of operation increases. Also, even within ESD, some key champions have been competitive and self-serving - building their own areas rather than more inclusive approaches e.g. sharing, supporting and disseminating.
The second ‘move’ UNESCO made was to create a glossy synopsis of the second report (UNESCO, 2012). This report presented highlights of the UN-DESD but watered down the critical intent of my detailed report and failed to represent the questions that had been raised about the effectiveness of ESD. It was this glossy summary that was distributed at the Rio ‘Plus 20’ meeting in 2012. In addition, this report also included a history of ESD that started with Our Common Future (1987) and the United Nations Conference on Environment and Development (1992), but which ignored earlier landmark events like the Environment and Human Development conference in Stockholm (1972) and the Tbilisi UNESCO/UNEP conference on Environmental Education, with its foundational declaration on the meaning of environmental education which essentially contains a description of what many people nowadays call ESD. I urged the UNESCO staff to include this part of the history but was again unsuccessful in securing a more critical version.
In my view, the glossy summary report was a misrepresentation of both the detailed findings and the history of ESD. I did not want to be listed as the author of the ‘best of’ version of the report and the ‘credits’ went to the consultant who was hired to write this text, which, in fairness, was clearly written and nicely illustrated (and translated in several languages, unlike the more critical full report). Despite my frustration and complaints, I was still asked to present the summary report at the Rio Plus 20 conference via a high-powered panel event featuring UNESCO’s Director-General, Irina Bokova, as the chair. This gave me the opportunity to raise some critical points, albeit diplomatically phrased, that had been downplayed in the censored reports. My critical friends in academia, who take issue with ESD, were not convinced that the discourse had shifted towards a more critical transition-oriented one. Frustrated as I had been involved in the process, I did sense however that within the UNESCO ESD group, people were beginning to engage with more critical perspectives but were constrained by the larger bureaucracy of which they are part.
When I was asked to contribute to the 2016 Global Education Monitoring Report (UNESCO, 2016) report during its final stages, I again had to ask myself: will I be able to be critical and to bring transitional perspectives to this major report? Being near or within UNESCO circles, I judged that I could have more impact in working on the report and attempting to change the dominant discourse than if I had stayed outside of the process and attempted to raise questions from a distance. When I joined the GEM2016 team as an external consultant in December of 2015, I encountered a certain logic that I found highly problematic. I had to be diplomatic and sensitive – especially during one-on-one critical conversations with team-members or during team meetings or at lunch – to avoid being dismissed as overly critical, naïve or too radical. When I read about People-Planet-Profit (as the GEM team used initially), I then needed to ask ‘who or what got the P for Profit in there?’. Why not just People and Planet? If you need to have something more explicitly related to economy in there, then let us at least use the P of Prosperity? When I read that education for all is important because it will lead to economic development, which was also seen as a prerequisite for ecological and environmental sustainability, I then felt I had to question this logic. For example I asked: is education there to serve the economy first and foremost? Is all education by definition good or can it be highly problematic? When the ‘world of business’ jumps on the sustainable development, sustainability and ‘green’ bandwagon, what do we need to understand about what their underlying motives entail? When we read about sustainable development we should also ask whether we ‘must always develop, grow and expand?’ Or can sustainability represent an alternative to development? Are there any references in the report to ethics? The non-human world? Indigenous ways of knowing and being in the world? How important are disruptive questions as part of our critical thinking towards sustainability?
Overall, these were just a few areas of contention or attention that I felt needed to be discussed in the team. To their absolute credit, these deeper questions were responded to by the GEM team and the discourse of the report did shift to some extent. Now that the report is out it is perhaps useful to ask: Does the GEM 2016 report signify a change from the dominant neo-liberal agenda that sees education as an extension and a driver of the globalizing economy and its push for infinite growth, innovation and expansion? Current models of economic growth cause environmental destruction For education to be transformative in support of the new sustainable development agenda, ‘education as usual’ will not suffice. Expand education on global citizenship, peace, inclusion and resilience to conflict. Emphasize participatory teaching and learning especially in civic education. Invest in qualified teachers for refugees and displaced people and teach children in their mother language. Incorporate education into the peace-building agenda. Mobilize domestic resources, stop corporate tax evasion and eliminate fossil fuel subsidies to generate government revenue for fundamental needs such as education and health. Promote the value of indigenous livelihoods, traditional knowledge and community-managed or -owned land through actions such as land conservation and locally relevant research. Engage community elders in curricular development and school governance, produce appropriate learning materials and prepare teachers to teach in mother languages. Incentivize universities to produce graduates and researchers who address large-scale systemic challenges through creative thinking and problem-solving. Promote cooperation across all sectors to reduce policy-related obstacles to full economic participation by women or minority groups, as well as discrimination and prejudice that also act as barriers.
The question that arises from the inclusion of such discourse is the extent to which such language actually marks a changed paradigm away from unsustainable forms of economic growth. As can be seen in my work on the UN-DESD, established ideas and cherished paradigms tend to resist contradictory evidence when it comes to the time for research and policy staff to make changes and publish reports. No doubt this report on its own does not signify the last word on this matter, but it is perhaps possible that ongoing action and critical engagement, along with the development of viable policy alternatives, will help the GEM team and others at UNESCO to help us all make the sorts of changes that will support transitional forms of education to flourish.
Towards healthy policy ecologies
There are many questions that might be asked in light of the stories we have provided about our experience of bringing evidence to policy. Of course, we reiterate that our stories are anecdotal, and that ethnography itself has some limitations, albeit that it also brings considerable potential for insight. However, despite the anecdotal nature of our accounts, it also needs to be emphasised that much of the clamour for evidence-based policy seems insensitive to politics of knowledge, especially in the post-truth times we live in. As Stevens (2011) points out, evidence-based policy too often asserts itself as a ‘given’ when, by its own standards, there has been insufficient examination or evidence to prove its worth in helping to transform the unsustainable policies of the Global North.
The accounts we have provided also reflect our roles as policy insiders. Following on from our ethnographies, there are lessons to be learnt about how other critically-minded insiders might balance their relationships with critical colleagues who are outside the policy process. Both insiders and outsiders need to consider how the ‘ecology’ within and surrounding policy contexts, and the inter-connected actors within them, can use evidence to do more than reinforce the assumptive worlds of decision-makers and not just transcend Blühdorn’s politics of unsustainability, but transgress them in ways that lead to the sorts of alternatives needed for transformational economic, social and educational policy.
The creation of alternative forms of policy thinking – as a way of getting beyond evidence-based approaches with decision-makers – registers the political nature of knowledge. It also begs questions about what is needed to create a healthy policy ecology – one that enables boundary-crossing, challenges power imbalances and can help create ideas that lead to transformation. Following on from Blühdorn, this is especially important because so much of what occurs in the name of sustainability policy does not yet fully attend to issues connected to our addictions to ongoing economic growth, resource use and environmental degradation. When thinking about EfS or ESD, it also has to be conceded, following Blewitt (2015), that something more than mainstream sustainability education might also be needed for the sake of the planet.
We will end by offering some future possibilities for healthy policy approaches that might gather their own political power and acceptance as the resistance to normalised unsustainability and the concern about runaway climate change and other manifestations of systemic global dysfunction grows. Beyond evidence-based policy and drawing on the work of Nutley (2013) and Rein (1976), we argue that at least part of a healthy policy ecology, in an age of post-truth, involves considering how education policy might operate without a neoclassical imperative for growth (and an obsession with employability). Subsequently, it could draw on strong (heterodox) versions of ecological economics (Raworth, 2016, 2017; Spash, 2012, 2013) to help provide a liberating alternative basis for potentially transformative varieties of ‘ecological’ education (Wals, 2019). Furthermore, in referring to the work of Ashley Jay Brockwell (Brockwell, 2019), there might be merit in the utilization of values-based indicators that require the input of all actors in a deliberative process establishing what counts and whether and how what counts can be, if not measured, made visible.
In addition to the invaluable contributions of critical scholars in environmental and sustainability education, such perspective(s) could also be critically developed in relation to the emerging discourse on the ecological university (Barnett, 2018; Stratford, 2019). In turn, critical policy thinkers could be a valuable addition to such a policy ecology too, as could those who have worked on developing theoretical and philosophical perspectives on ecology and the Anthropocene (Hamilton et al., 2015), as well as those who have already attempted to identify alternative policy positions based on the planet’s predicament. One example of such alternative policy thinking has recently been observed in the Lancet Committee’s work identifying the key features of a globally healthy diet (Willett et al., 2019).
While accusations of merely sketching utopia might follow this tentative outline of an alternative policy ecology for education in the Anthropocene, we would conclude by reconnecting to the main points raised in this paper. Beyond the rationalism seen in ‘evidence-based’ approaches to policy, there remains an urgency to find solutions to the politics of unsustainability. The need for a sophisticated approach to the politics of knowledge is, furthermore, accentuated at a time of post-truth politics, in part at least because democracies around the world are actively looking for alternatives to (the neo-liberal) status quo. Instead of limiting our work as scholars to ‘honest broking’, we also need to be actively contributing to new political approaches to knowledge and governance and developing new policy alternatives. Potentially, such a liberated approach to policy might especially allow those working in and around EfS/ESD to go beyond the imaginations of policy-makers to develop educational ideas that encourage more of the much-needed thinking and change the planet so desperately needs.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
