Abstract
Histories of statistics and quantification have demonstrated that systems of statistical knowledge participate in the construction of the objects that are measured. However, the pace, purpose, and scope of quantification in state bureaucracy have expanded greatly over the past decades, fuelled by (neoliberal) societal trends that have given the social phenomenon of quantification a central place in political discussions and in the public sphere. This is particularly the case in the field of education. In this article, we ask what is at stake in state bureaucracy, professional practice, and individual pupils as quantification increasingly permeates the education field. We call for a theoretical renewal in order to understand quantification as a social phenomenon in education. We propose a sociology-of-knowledge approach to the phenomenon, drawing on different theoretical traditions in the sociology of knowledge in France (Alain Desrosières and Laurent Thévenot), England (Barry Barnes and Donald MacKenzie), and Canada (Ian Hacking), and argue that the ongoing quantification practice at different levels of the education system can be understood as cultural processes of self-fulfilling prophecies.
Keywords
Introduction
We live in an age of quantification. In our everyday lives as university professors, parents, and citizens, we encounter statistics at almost every crossroad as, for instance, assessment and productivity scores, grades, and ratings. Course assessment scores are used to evaluate the quality of our teaching, tracks of student production and publication points are reported and generate income for our university, and equally importantly, we ourselves contribute to the production of statistics through our assessment of students, systems, and leaders. As parents, we notice the same trend in our children’s everyday lives at school. Their skills and performances are regularly assessed and given scores, and they are increasingly required to partake in assessment activities themselves, assessing their own performance and their teachers, school, and peers. Accordingly, we notice how both our students’ and children’s attention is turned towards assessment criteria. The question of interest to them seems more often to be how they are assessed than what they actually learn. In the age of quantification, a specific mindset seems to be nurtured – that of the
In this article, we address this quantification turn as a social phenomenon. We ask
The statistical society
Statistics, measurement, and quantification have been crucial for the development of modern societies. Science, the state, the market, and the economy as we know them today would not have been possible without tools for measurement and quantification. We could say that quantification has helped enable these institutions – and vice versa – and that the state, science, and the market have brought about different quantification processes (Diaz-Bone and Didier, 2016: 7). At the same time, the scope of quantification has evolved dramatically over the past two decades, and a range of statistical overviews and results from assessments, measurements, and tests are being produced at an ever-increasing rate. Technological advances are contributing to continual growth in quantitative ‘small’ and ‘big’ data. These quantifications influence how others (institutions and individuals) see us, how we see ourselves, and how we shape our lives. A simultaneous construction of statistics and society is taking place (Sætnan et al., 2011), as well as a simultaneous construction of statistics and individual self-understanding.
The relation between statistics and society has been central in the sociology of science studies (Desrosières, 1998; Hacking, 2010; Mackenzie, 1981). We argue, however, that the tendency to quantify and assess is particularly strong in the field of education, fuelled by a global ‘education as social investment’ paradigm (e.g. Esping-Andersen, 2002; Heckman, 2006), as epitomised in the annual
Our analysis takes a slightly different path by looking at quantification as a social phenomenon. Observing that the prevailing zeal for quantification and assessment is influencing policy formation, bureaucratic practices, everyday school life, and even the research programmes that allocate money for analysing this phenomenon, we ask
The demand for quantification and standardisation in education is international in origin. In his studies of the development of European education policy, Romuald Normand explores how scientific expertise and politics have worked together in establishing new indicators and classifications (Normand, 2008, 2010). He argues that the Organisation for Economic Co-operation and Development (OECD) has spearheaded the development of a mindset in which the world is regarded as an ‘educational laboratory’ (Normand, 2008: 665). Standardisation and the need for comparatively quantifiable parameters require a shift of focus at the national level, rendering the testing of basic skills a top priority. This has created a global mapping and testing regime in schools, as exemplified by international tests such as PISA, TIMSS, PIRLS, and compulsory mapping tests in primary and lower secondary schools. The performative nature of the new standards and classifications will make up new pupils (Gorur, 2018; Popkewitz and Lindblad, 2018). This tendency is also evident in Norway, despite the strong standing of the public ‘unity school’ (
According to the Norwegian Directorate for Education and Training, the overall idea behind the new mapping regime is to identify pupils who fall below a ‘defined limit of concern’ in order to initiate intervention. 4 Hence, the political ambition remains in line with the egalitarian principles of the welfare state, at least in rhetoric. Seen in a different light, it is an interesting example of how an economic rationale, rooted in a global competitive education industry, is transformed into ‘good intentions’ in compliance with the ideals of an egalitarian welfare state.
In Norway, as in other countries, structures and institutions are created for quantification, accountability, and assessment in the education system, and simultaneously, a culture for this purpose is established in bureaucracy and schools. Radhika Gorur argues that large-scale assessments and comparisons are growing and that more assessments – especially regional and national assessments – are being developed (Gorur, 2017a: 263). She reminds us that scholars like Porter and Desrosières have demonstrated that quantification is intimately linked to statecraft and is an indispensable technology of regulation (Gorur, 2017b: 350). With reference to Ian Hacking, she points out that numbers not only describe but also
The emerging sociology of quantification
Histories of statistics and quantification have demonstrated that systems of statistical knowledge participate in the construction of the objects measured (Desrosières, 1998; Hacking, 2010; MacKenzie, 1981). However, the pace, purpose, and scope of quantification in state and private bureaucracy have expanded greatly over past decades, fuelled by (neoliberal) societal trends that have given the social phenomenon of quantification a central place in political discussions and in the public sphere. The extensive body of research on the neoliberal turn and its effect on education illustrates how this also permeates research and scholarly debate.
At the frontier of both European and American sociology, ‘the sociology of quantification’ is discussed as an emerging field in the social sciences, drawing heavily on the French tradition associated with Robert Salais, Laurent Thévenot, and Alain Desrosières (Diaz-Bone and Didier, 2016; Mennicken and Espeland, 2019; Thévenot, 2019). Desrosières (1998: 1) introduces the topic as follows: Unemployment, inflation, growth, poverty, fertility: these objective phenomena and the statistics that measure them support description of economic situations, denunciation of social injustices and justifications for political actions. They are inscribed in routinized practices that, by providing a stable and widely accepted language to give voice to the debate, help to establish the reality of the picture described. What these two otherwise very different situations have in common is constituted by the perverse effects of the feedback of indicators on the actors whose actions have been quantified. (Desrosières, 2015: 336)
A similar line of reasoning can be traced to the Edinburgh school, notably the work of Barry Barnes. Barnes (1995) argues that knowledge is embedded in traditions, classifications, and conventions, and that these traditions exist To know that a practice exemplifies a norm is to know that most people take the practice to exemplify the norm. For example, to know that [in the United Kingdom] driving on the left is a norm is to know that most people indeed regard it as normative. A collective will make the claim that driving on the left is normative into a true claim by coming to believe it (collectively). Thus, in a perfectly clear and straightforward sense, knowing of norms is knowing what is generally known. And thus knowledge of norms is ‘about’ itself. A normative order exists as a distribution of self-referring and self-validating knowledge. It exists, therefore, just so long as it is (collectively) known to exist. A normative order is a gigantic self-fulfilling prophecy. (Barnes, 1995: 60) Once in possession of the conventions of their traditions, members will apply them and perpetuate them. Knowing what counts as ‘the same thing’ they will go on and themselves identify further cases of ‘the same thing’ […] In a nutshell, future use of our conventions of classifications is underdetermined and indeterminate. It will emerge as we decide how to develop the analogy between the finite number of our existing examples of things and the indefinite number of things we shall encounter in the future. This view of classification is sometimes labelled is not money by virtue of its physical and chemical properties alone; it is money because it is believed to be a medium of exchange and store of value, and that belief is validated by the practice it informs. Our shared belief that the pieces of paper we call ‘dollars bills’ are money leads us to treat those pieces of paper in ways that make them constitute money. (MacKenzie, 2007: 66)
Building on the same line of reasoning, the Canadian philosopher Ian Hacking describes how (statistical) knowledge enables classification systems that ‘make up’ people. His approach to performativity is to analyse how classifications of people interact with the people classified. He asks: how do classifications of people affect the people classified, how do people change by virtue of being classified, and how do the ways in which people change have a sort of feedback effect on our systems of classifications themselves (Hacking, 2004a: 99)? In several of his works, Hacking studies how statistics have created categories that become normative and subsequently lead people to modify their behaviour. The advancement of statistics and the calculus of probability in the 1800s triggered an avalanche of printed numbers (Hacking, 2004a: 100, 2010: 2). Statistics acquired new applications, and statisticians were concerned with finding order and patterns in the social world. Society could be predictable, even if it was not governed by universal laws. During the nineteenth century, it became possible to see that the world might be regular and yet not subject to universal laws of nature. A space was cleared for chance. The erosion of determinism made little immediate difference to anyone. Few were aware of it. Something else was pervasive and everybody came to know about it: the enumeration of people and their habits. Society became statistical. A new type of law came into being, analogous to the laws of nature, but pertaining to people. These laws were expressed in terms of probability. They carried with them the connotations of normalcy and of deviations from the norm. The cardinal concept of the psychology of the Enlightenment had been, simply, human nature. By the end of the nineteenth century, it was being replaced by something different: normal people. (Hacking, 2010: 1)
Statistics create new ways of being. Accordingly, statistics contribute to what Hacking terms ‘making up people’ (Hacking 2004a). Vast amounts of statistical data were used in the calculations of the average, and the average came to be regarded as the norm. The idea of the average influenced perceptions of the normal, and when more aspects of human behaviour were studied and made the object of statistical studies and calculations of averages, this feedback had effects: social actors modified their behaviour to align with statistical perceptions of what was normal and, by implication, what was ‘good’. Consequently, according to Hacking, the construction of new categories changes not only ideas of society but also how people describe each other and how they see themselves. Thus, the classification of people into categories constructs the reality it is intended to describe. In this way, classification processes contribute to making up people.
We argue that the strong theory of performativity and self-fulfilling prophecy (Barnes, Desrosières, Hacking, MacKenzie, Thévenot) should be taken as a point of departure in studies of the culture of classification in the education system in order to address what is at stake. In the following, we identify three critical areas in which the notion of the benefits of the quantification turn is challenged. Pertaining to the context of the education system in the Norwegian welfare state, we question what is at stake in state bureaucracy, in professional practice, and for individual pupils.
Following the classifications: What is at stake?
What is at stake in state bureaucracy?
How are statistical tools constructed, used, and justified for the purpose of quantification and measurement? What might be the implications? Many studies attempt to measure effects, often based on the Campbell Collaboration’s models for meta-studies, which use statistical methods to combine results from several independent studies of the same problem. In Norway, this is used by, among others, the Knowledge Centre for Education (see Lillejord et al., 2015). The same rationale lies behind the wish of the National Commission on Gender Equality in Education to measure and assess children’s potential and performance based on register data (NOU, 2019: 3). The report can serve as an example of a set of underlying assumptions that are commonly used to justify measurement and quantification. First, the relatively poorer academic performance of boys compared with girls is portrayed as a social problem
The report provides an example of how modern classification systems are constructed with the dual ambition of, in this case, helping boys who perform poorly academically while simultaneously improving economic efficiency. The process has a parallel in the construction of statistical knowledge in nineteenth-century Britain. Donald Mackenzie (1979, 1981) showed how statistical knowledge was produced to satisfy particular class interests in Victorian England. Mackenzie’s starting point is the structural position of the mathematical statistician Kart Pearson as a professional middle-class person. Pearson developed his thinking by mobilising what nineteenth-century British middle-class culture had to offer (morals, philosophy, Darwinism, eugenics, mathematics), and he creatively put this together and developed his statistics. He became a eugenicist during the 1880s, asking whether the sick and the violent should have the right to reproduce and create new sick and brutal humans. His socialist solution (in a social Darwinistic sense and interpreted on a collective level – survival of the fittest group) was government intervention in childbearing. His thinking was ‘condensed into his eugenics’. He regarded eugenics – i.e. the idea that natural selection must be replaced with artificial selection – as an integrated part of politics (social change was to be controlled from above and with the knowledgeable classes seated at the controls); at the same time, it was an application of his moral philosophy (only the scientifically educated can be moral) to human reproduction and a science to be developed along the lines of his epistemology; finally, the necessity of a programme of national eugenics was, he felt, a direct consequence of the application of evolutionary theory to the contemporary world of international competition. Thus, behind the construction of statistical tools, such as multiple regression analysis and Pearson’s R, is a rather elitist and hygienic demand for ‘normality’.
The analogy is, of course, provoking, exactly like Desrosières’ (2015) comparison of Soviet statistics and New Public Management, mentioned above. However, the resemblance between the use of statistical data in early eugenics research to justify policy and actions, which today are regarded as dehumanising, and the contemporary ‘trust in numbers’, should not be ignored. The current demand for and reliance on evidence is indeed politicised, albeit with the intention of doing good: identifying people who cause concern in order to help them or compensate for shortages. Despite its good intentions, this practice involves sorting people into categories that are constructed based on aggregate data. This, in turn, makes it necessary to turn to the practice of the people responsible for putting the policy developed in the state bureaucracy into practice.
What is at stake in teachers’ professional practice?
How do the numbers, classifications, and test results ‘travel’ from state bureaucracy to the teachers’ professional practice, and vice versa, and how are statistical classifications and conventions translated into social classifications and conventions in schools and classrooms? In order to understand whether and how statistical classifications are translated into social conventions in the everyday work of teachers, we need to explore the culture of the profession. Previous research points to the fact that professional teachers are carriers of the cognitive ideals of both knowledge and moral virtues. Parsons’ pattern variables – five dichotomous and variable options, mutually exclusive, where one set of variables is instrumental and the other expressive – are instructive for understanding the ambivalent character of teachers (Parsons, 1939). The ambivalence between knowledge and morality in teachers’ habitus is well known from Norwegian studies (Skarpenes, 2011; Thue, 2019; Østrem, 2010). From studies of professions, we know that this ambivalence is not unique to teachers but characteristic of many professional groups in the wake of the neoliberal turn (Bourdieu, 1999; Evetts, 2009; Zacka, 2017). The ‘impossible mission’ of teachers is to reconcile policy from above with the needs, skills, and concerns of individual pupils. In professional practice, the act of classifying people usually involves some sort of assessment of individual pupils, relying on a set of standardised indicators or assessment criteria. The accountability against which teachers are held up relies, to a large extent, on their usage of assessment tools. Such tools and indicators appear as objective entities, and this apparent objectivity is usually perceived as an important prerequisite for fair and equal treatment. However, standardised assessment tools also necessitate professional discretion. In the presentation of assessment results, such as grades, however, discretionary judgements are concealed, rendering the results into objective realities. These apparent objective realities are then, in turn, used for policy development, as outlined above, reinforcing the system for assessment and classification. The paradox is striking.
In a culture of accountability, it is how teachers’ actions are documented and justified that is at stake. Discussing the connections between accountability and responsibility with reference to Bauman’s concepts of morality and ethics,
6
Biesta (2004) argues that the accountability culture seems to be governed by ethics and to elicit behaviour that suits the accountability
The question at stake, then, is how the implementation of quantification influences teachers’ practice. Do they become ‘docile instrumentalists’ or ‘disobedient moralists’? And how, in turn, does that influence the ideology of the Norwegian unity school, notably the principles of egalitarianism and inclusion and the
What is at stake for the pupils?
How are numbers, classifications, and categories involved in the production of new subjectivities and forms of personhood? How does classification affect the behaviour and self-perception of the pupils themselves? Undoubtedly, the new culture of classification and evaluation in education has played a part in the construction of new youth categories and, undoubtedly, these categories will affect the pupils classified (Skarpenes and Nilsen, 2014). In the past decade or so, categories depicting deviant groups such as ‘ADHD pupils’, ‘school dropouts’, ‘workplan heroes’, ‘too-good-for-their-own-good girls’, as well as normative categories such as ‘generation obedient’, the ‘generation without rebellion’, and ‘generation performance’ have reached the public sphere in Norway. While some of these categories depict ideological understandings, others rely on statistics. For instance, ‘school dropouts’ is a category used for the approximately 20–30% of young people who quit upper secondary school before the exams. The alleged increase in dropout rates has caused moral panic and influenced political priorities (Vogt, 2017). In a similar vein, and as mentioned above, boys have been singled out as a category of concern due to their relatively poorer grades compared with girls (NOU, 2019: 3, Vogt, 2018). What these examples have in common is that categories of deviance are created based on the premise of relatively poor statistical reasoning, while simultaneously constructing broad categories of pupils of concern. Once classified as a category of concern, the people whom the category is ascribed become people of concern. A relevant question thus becomes ‘How do classifications affect the people who are classified?’ In line with the theory of self-fulfilling prophecies, there is reason to suspect that people live up to the categories with which they are identified. Moreover, the categories become an integrated part of institutional practice. Hacking (2004a, 2004b) points out that socially constructed categories not only have labelling effects on an individual level but happen at an institutional level and are maintained through the very institutions of which they are part. For instance, a child with severe autism is unlikely to be aware of his ascribed category, yet the category is integrated in institutional practices that come to shape the child’s everyday life and self-perception. The interaction between the individual and the category thus takes place at an institutional level.
Davide Sparti (2001)
7
developed Hacking’s point about ‘making up people’ in a way that brings several important points to the education debate. Sparti argues that identity is created in contexts of collective forms of classification and that, with reference to Axel Honneth, it is dependent on the acknowledgement of others. More specifically, identity will be viewed as a set of social categories used to classify individuals into different types. Consequently, being a specific type of person is partly determined by the categories used to describe that type (Sparti, 2001). When social groups start grouping activities into categories and the use of these categories becomes widespread, a classification process takes place whereby verbs are replaced by nouns: those who drink become
In Norway, pupils who score poorly on certain tests are categorised as ‘pupils of concern’ (
Final reflections
The prevailing zeal for quantification and assessment is influencing policy formation, bureaucratic practices, everyday school life, and the construction of pupil identities. Structures and institutions are created for quantification and assessment in the education system, and a culture for this is simultaneously established in bureaucracy and schools. We have argued that the use of statistical knowledge and classifications in the educational system participates in the construction of the objects that are measured. Moreover, we have shown how classifications of pupils might affect the pupils classified and have a feedback effect on the statistical systems of classifications themselves. Concepts such as ‘looping effects’, ‘feedback’, ‘bootstrapped induction’, and ‘Barnesian performativity’ are drawn from different traditions in the sociology of knowledge, and we have argued that the concepts add theoretical vigilance to the understanding of educational changes as cultural processes of self-fulfilling prophecies.
The Norwegian public school, the so-called unity school, is formed on a tradition of egalitarianism and inclusion. These ideals do not seem to sit well with the current global education industry. From a social welfare perspective, the quantification turn thus challenges some of the core ideals of the welfare model, despite intentions of ‘doing good’, by, for instance, identifying children below a ‘defined limit of concern’. Moreover, it might also challenge the
For society at large, however, what is at stake are some of the core values of the welfare state of which the unity school is a cornerstone. The public discourse in the wake of the Covid-19 pandemic is a thought-provoking example. As home-schooling became the norm and exams were cancelled, a sense of moral panic emerged in Norway, invoking terms such as ‘lost generation’ to signify how much the young generation had lost in terms of learning due to the pandemic. What is revealed by this discourse is a narrow understanding of learning, in which whatever organised and planned activity that takes place in a classroom and can be assessed is defined as learning, in contrast to naturally occurring and unpredictable events, such as a global pandemic. If the world is an educational laboratory, as suggested by Normand (2008), then the pandemic should offer a unique learning opportunity, encapsulating both practical knowledge, such as cooperation, cleanliness, and health care, as well as theoretical knowledge of, for instance, virus mutation, risk assessment, infection control, policymaking, social economy, and statistical prediction. At the very least, the pandemic has taught us that the world can still be unpredictable, despite our sophisticated statistics. When we are faced with such a crisis, it is arguably the ability to cooperate and think in new ways, combined with a well-functioning democratic leadership, that matters – abilities that are integral to
Footnotes
Declaration of conflicting interests
The author(s) declare no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
