The Organisation for Economic Co-operation and Development’s International Early Learning and Child Well-being Study: The scores are in!

Abstract

This is the fourth colloquium for Contemporary Issues in Early Childhood on the Organisation for Economic Co-operation and Development’s International Early Learning and Child Well-being Study, and marks the recent publication by the Organisation for Economic Co-operation and Development of reports on the first round of this study. In it, the authors discuss what the results tell us, what they do not and what might come next. They conclude by supporting the need for comparative studies of early childhood education, but argue that the International Early Learning and Child Well-being Study is not the way to go.

After four years of development work, testing 7000 children and spending millions of dollars, pounds and euros, the results of the first round of the Organisation for Economic Co-operation and Development’s (OECD’s) International Early Learning and Child Well-being Study (IELS) were published in March 2020 with three national reports – one for each participating country (England, Estonia and the USA) – a full report and a summary report (OECD, n.d. b). The main findings are also presented in a recording of a webinar (EduSkills OECD, 2020).

To recap, the IELS is a cross-national assessment of five-year-olds on four ‘early learning domains’ (early literacy and numeracy skills, self-regulation, and social and emotional skills), based on ‘developmentally-appropriate, interactive stories and games delivered on a tablet device’ (OECD, 2020a: 96) and supplemented by information (individual background, home learning environment, early childhood education and care experience, children’s skills) from staff and parents using questionnaires. This is the fourth report on the IELS that we have contributed to Contemporary Issues in Early Childhood, with the aim of providing readers with information and a critical perspective on this highly consequential initiative by the OECD (for earlier reports, see Moss et al., 2016; Moss and Urban, 2017, 2018). In it, we discuss what the results tell us, what they do not and what might come next.

What do the results tell us?

The answer, in a nutshell, is very little that we did not know already – there is nothing new here. Reading the reports, we discover that:

• There are gender and socio-economic differences in test performances: girls do better on emergent literacy and socio-emotional skills, and poorer children do worse on all measures, reiterating much previous work (e.g. Burger, 2010; Voyer and Voyer, 2014).

• ‘What parents do is pivotal for their children’ (OECD, 2020a: 12), and ‘children from advantaged families, on average, have more learning opportunities’ – for example, children from ‘advantaged families’ are four times as likely to live in families with more than 100 children’s books (OECD, 2020c: 7). This again reiterates much previous work (e.g. Smees and Sammons, 2018).

• There are differences in children’s test performances across the three countries, with children from Estonia performing best overall. Estonia also ‘had the smallest differences amongst children based on their socio-economic backgrounds whereas the greatest differences were found in the United States’ (OECD, 2020c: 6).

• There are substantial differences between the three participating countries on demographic and socio-economic indicators, and also for early childhood policies and provision, with Estonia by far the smallest in population (with just 54,000 children aged five years and under compared to 24 million in the USA) but with the lowest levels of inequality and child poverty, and the best-developed policies and provision.

• Despite being a rich country and producing copious research and publications on early childhood education and care, which the OECD reports rely on heavily, early childhood policy and provision in the USA and the conditions of young children in that country are poor. To take just a few examples from the national report for the USA (OECD, 2020b): the distribution of wealth is ‘highly unequal’ (22); the country is ‘unique among OECD countries in having no statutory entitlement to paid maternity, paternity or parental leave’ (24); ‘the early childhood landscape is highly fragmented’ (26); ‘childcare is expensive’ (27), yet ‘many [early childhood educators] have salaries so low they are eligible for or receive public financial assistance’ (28); and ‘[a]ttendance at centre-based ECEC programmes is highest among children from high-income families’ (31).

But the reports, and the whole IELS exercise, lead to two other conclusions. First, given enough time, expertise and money, it is possible to construct and conduct standardised tests of young children in different countries; the issue is not whether we can do this, but why? Second, as throughout the development of the IELS, these latest reports make no reference or response to the many and varied reservations raised about the study (including in our three previous colloquia reports); provide no discussion of why so few countries agreed to participate; and offer no explanation of or justification for the IELS’s paradigmatic and disciplinary positioning (discussed further below). In short, the OECD deals with criticism and questioning by simply ignoring it.

What do the reports not tell us?

The answer, again in a nutshell, is an awful lot. Reading the reports, we discover nothing about:

• The rationale for the selection of the three countries that participated, whose five-year-olds are in different types of setting (e.g. primary school in England, preschool in Estonia, and school or preschool in the USA). To what useful questions could comparison between them provide answers?

• How many children were selected for testing in each country, and how the selection of children was undertaken in the USA – an important consideration given the size of that country, its federal structure and the large differences existing between the 50 states (the US national report vaguely refers to ‘a representative sample of five-year-olds enrolled in registered school or preschool settings in each participating country’ (OECD, 2020b: 19), which begs more questions than it answers).

• Possible explanations for ‘disruptive behaviour’, reported to be more prevalent for boys than girls. Attributing it to a ‘lack of self-regulation’ and ‘inhibition skills’ individualises a phenomenon without any indication of possible systemic or contextual explanations. What exactly are these young boys ‘disrupting’? What cultural and/or class biases and institutional constraints underly expectations about what characterises approved (non-disruptive) behaviour?

• How the overall early childhood education and care systems in each country function, so ignoring an emerging global ‘systemic turn’ and recognition of the importance of ‘whole-systems approaches’. The World Bank’s Systems Approach for Better Education Results (SABER) initiative, while subject to fundamental critique (e.g. Klees et al., 2020; Steiner-Khamsi, 2013), acknowledges that any early childhood programme is embedded in complex cultural, economic and political systems, which need to be taken into consideration when evaluating their impact (Powers and Paulsell, 2018). The Group of Twenty Leaders Summit, too, commits to building ‘international consensus on government responsibility for a “whole-systems” approach to ECD/ECEC [early childhood development/early childhood education and care] policies’ (Think20, 2019: 4; Urban et al., 2019).

• The ‘culture’ of early childhood education and care in the three countries, which has been described as ‘an intricate weave of traditions and influences, theories and concepts, social constructions and images (of the child, the worker, the parent, the centre), procedures and practices, shaping understandings of what services are about and what constitutes “good” work in them’ (Moss, 2018: 25).

• How early childhood systems address questions of diversity and (in)equality among children, families and communities beyond the prevailing deficit model in which diversity is seen as a problem that needs fixing – for example, multilingualism framed as deficiency in the dominant language.

• What might explain the reported national differences in test performance – for example, why it might be that ‘Estonia had the smallest differences amongst children based on their socio-economic backgrounds whereas the greatest differences were found in the United States’ (OECD, 2020a: 12). Might this be because Estonia has a stronger welfare state and lower inequality, as the work of Wilkinson and Pickett (2009) suggests? More generally, the full report contains no final chapter offering discussion and conclusions.

• How the OECD’s stated rationale for the IELS – that ‘[c]ountries will make faster progress on improving children’s early learning experiences if they are able to learn from other countries and systems’ (OECD, 2020a: 26) – might actually happen. How and what might countries learn from the first round of the IELS? For example, what might England learn from Estonia and vice versa? How might the USA’s federal Department of Education brief President Trump?

What next?

The IELS, as noted above, makes no attempt to explain or justify its paradigmatic or disciplinary positioning, which is, respectively, positivism and developmental psychology. It is also, as we noted in an earlier article (Moss and Urban, 2018), permeated by a strong ‘anglophone’ orientation, whether considering the contractors undertaking the work, the experts advising on the study or the countries participating. For us, the whole exercise confirms Loris Malaguzzi’s view of ‘Anglo-Saxon testology’, ‘which is nothing but a ridiculous simplification of knowledge, and a robbing of meaning from individual histories’ (Malaguzzi, 1990, quoted in Cagliari et al., 2016: 378).

Equally disturbing is to see how the IELS is part of a vast project by the OECD – a growing web of international large-scale assessments measuring national education performance by applying common and decontextualised indicators. The ambition is to include ever more countries and ever more ages in this web, which, in addition to the IELS, includes:

• The Programme for International Student Assessment (commonly known as PISA), a triennial international programme of testing 15-year-olds in reading, mathematics and science, which began in 2000 and is now into its seventh round; the results from this 2018 round of testing were published in December 2019. It tests a sample of half a million students who represent 28 million 15-year-olds in 80 countries and economies. PISA is now firmly established, with wide participation and high levels of interest in its results from politicians, policymakers and the media.

• The PISA-based Test for Schools, a ‘PISA-like’ test that may be used to ‘benchmark’ how well an individual school or school district compares with others or with those countries that are ‘PISA winners’.

• PISA for Development, a version of PISA using ‘enhanced PISA survey instruments that are more relevant for the contexts found in middle- and low-income countries but which produce scores that are on the same scales as the main PISA assessment’ (OECD, 2018). In this project, the OECD also defines supposedly globally valid competencies that are needed for young people in all developing countries. The results are intended also to be used as benchmarks for development assistance from the World Bank and other donors. The OECD has so far ignored any criticism of PISA for Development, including how its claims of universality contradict diverse communities’ rights to self-determination in education as enshrined in the United Nations Declaration on the Rights of Indigenous Peoples (United Nations, 2007).

• The Study on Social and Emotional Skills, an assessment of social and emotional skills among 10- and 15-year-old children across the ‘big five’ domains – task performance, emotional regulation, collaboration, open-mindedness and engaging with others. After field-testing in 2018, the main study, to be undertaken in ten cities in nine countries, is scheduled for 2019–2020 (OECD, n.d. d).

• The Starting Strong Teaching and Learning International Survey (TALIS Starting Strong), an assessment of ‘professional development; pedagogical beliefs and practices; and working conditions, as well as various other leadership, management and workplace issues’ for the early childhood workforce (OECD, n.d. c). It is ‘part of the OECD’s long-term strategy to develop early childhood education and care data’ (OECD, n.d. c), with nine countries participating in the first round of data collection in 2018, leading to a first report published in 2019.

• The Programme for the International Assessment of Adult Competencies (PIAAC), a programme of assessment and analysis of adult skills, including a Survey of Adult Skills that measures ‘adult proficiency in key information processing skills – literacy, numeracy and problem solving – and gathers information and data on how adults use their skills at home, at work and in the wider community’ (OECD, n.d. a). The first cycle collected survey data between 2011 and 2018 in 38 countries/economies, with data collection for a second cycle scheduled for 2021–2022 in 30 countries.

Through this expanding collection of international large-scale assessments, the OECD strives to establish itself as the global arbiter and governor of education – defining standards, measuring indicators, drawing comparisons and encouraging benchmarking, and offering prescriptions for improving performance. The OECD has no formal legal power over education. Instead, it exerts great influence by this growing use of comparisons, statistics and indicators – an exercise of what might be called ‘soft power’ (Sjøberg, 2019).

For this web of measurement to work and ensure the OECD’s position as the global kingpin of education, each newly developed international large-scale assessment needs to grow, adding countries until the assessment achieves wide coverage and a high public profile, following the trajectory of PISA. From this perspective, the IELS has got off to a shaky start, with just three countries signing up. So, the immediate goal for the OECD has to be more signings for the ‘next cycle of IELS’, which, according to the OECD’s (n.d. e) website, is scheduled to start preparation in 2020. Indeed, in one of the webinars organised by the OECD to launch the IELS findings, Andreas Schleicher, the OECD’s Director of Education, spoke about how PISA itself started small, how the initial round of the IELS is an invitation to other countries to participate, and of how it is now ‘up to other countries to take up the challenge to see how their [five-year-old] children are faring’. So, expect the OECD to lobby national governments hard to participate in the next round of the IELS.

Conclusion

In conclusion, we return to arguments we have made earlier. We strongly believe in the importance of comparative studies of early childhood education – but equally strongly believe that the IELS is not the way to go. For, apart from Malaguzzi’s rebuke of ‘Anglo-Saxon testology’, the IELS is also an example of comparison as a ‘mode of governance’, and of what Nóvoa (2018: 551) terms ‘“dataism”, the religion of data’, which entails a belief in ‘global solutions imposed by data and evidence on “what works” and “where the best results are” . . . [an approach] based on the false idea of consensus on the aims of education and the paths to achieving them’. Nóvoa calls instead for comparative studies to be part of a ‘science of difference’, an endeavour that should provoke thought by encounters with difference and recognition of the world’s rich diversity and complexity – a science, too, that should remind us that education is not primarily a technical endeavour (of standards and indicators, measurement and management) but a political endeavour about meanings, purposes, values and ethics. What the IELS reports confirm is the OECD’s reluctance to delve into the complexity and diversity that Nóvoa, and many others, value, preferring to reduce education to easily measured (and governed) common outcomes, and exemplifying what Orr (1996: 699) calls a ‘culture of fast knowledge’ in which ‘only that which can be measured is true knowledge’.

The publication of the IELS reports comes at a time of unprecedented and rapidly unfolding global crises. Questions of how to secure humanity’s survival on a finite planet have moved from hypothetical to urgent with a convergence of climate crisis and global pandemic. For early childhood education to make a meaningful contribution to sustainability and social and ecological justice, we will have to shift our focus from assessing narrow and predetermined ‘early learning outcomes’ to pedagogies of uncertainty, exploration and, to borrow Freire’s (2004) term, ‘untested feasibility’.

In that spirit, we welcome responses from readers to the IELS, to the OECD’s wider agenda of global assessment, and to this and previous articles we have contributed to the journal.