Abstract
This colloquium brings information about a second cycle of OECD's International Early Learning and Well-being Study (IELS) to the early childhood community, and offers a further critique of the approach to comparative research that the IELS embodies.
Keywords
Through this expanding collection of international large-scale assessments, the OECD strives to establish itself as the global arbiter and governor of education – defining standards, measuring indicators, drawing comparisons and encouraging benchmarking, and offering prescriptions for improving performance. The OECD has no formal legal power over education. Instead, it exerts great influence by this growing use of comparisons, statistics and indicators. (Moss and Urban, 2020: 169)
The story so far
The Organization for Economic Cooperation and Development (OECD), with its membership of 38 mostly rich countries, is a major player in the field of education. A key part of its strategy to become a ‘global arbiter and governor’ in this field has been the development and deployment of ILSAs – international large-scale assessments. The best known of these is the ‘Programme for International Student Assessment’ (PISA). PISA is a triennial international programme of testing 15-year-olds in reading, maths and science, which began in 2000; the results from the eighth round of testing, involving 690,000 students from 81 countries and economies, were published in December 2023.
PISA is now firmly established, with wide participation and high levels of interest in its results from politicians, policymakers and the media. But PISA is just one of several ILSAs purveyed by OECD. As well as PISA offshoots, including PISA for Development to encourage and facilitate PISA participation by low- and middle-income countries, there are the Study on Social and Emotional Skills, Starting Strong Teaching and Learning International Survey, Programme for the International Assessment of Adult Competencies and the International Early Learning Study (IELS). The IELS is a cross-national assessment of 5-year-olds on four ‘early learning domains’ (early literacy and numeracy skills, self-regulation, and social and emotional skills), based on ‘developmentally-appropriate, interactive stories and games delivered on a tablet device’ (OECD, 2020: 96) and supplemented by information (individual background, home-learning environment, early childhood education and care experience, children's skills) from staff and parents using questionnaires.
The ‘first cycle’ of IELS began in 2016, and only three countries participated – England (the only one of the four countries that constitute the UK to volunteer), Estonia and the United States. Little information about the recruitment of children for this initial run of IELS has been made available, except that nationally representative samples of 5-year-old children would be tested. There followed four years of development work, testing of children, data analysis and finally three national reports – one for each participating country – and a full report plus summary report (OECD, 2020). The main findings are also presented in a recording of a webinar (EduSkills OECD, 2020).
The introduction of this new ILSA by OECD attracted considerable critical comment (cf. Carr et al., 2016; Moss et al., 2016; Moss and Urban, 2020 Urban & Swadener, 2016). Much of this appeared in Contemporary Issues in Early Childhood. Invited to respond by the journal, OECD chose not to do so.
The next chapter
Commenting on the final reports produced by OECD on the ‘first cycle’ of IELS, we concluded that for this web of measurement to work and ensure the OECD's position as the global kingpin of education, each newly developed international large-scale assessment needs to grow, adding countries until the assessment achieves wide coverage and a high public profile, following the trajectory of PISA. From this perspective, the IELS has got off to a shaky start … So, the immediate goal for the OECD has to be more signings for the ‘next cycle of IELS’. (Moss and Urban, 2020: 169)
Given the paucity of the IELS website, we contacted OECD for more information on this second cycle of IELS. OECD responded that ‘the second cycle … involves the same international consortium as Cycle 1, led by ACER and that includes IEA and cApStAn … The domains measured will be the same as Cycle 1, and largely in the same ways’ (personal communication, OECD). They added that ‘OECD's websites are currently being updated, and so, beyond what you will have already found, here, there are no additional sources to point you towards for public information; so while there will be more information published in due course, … none is currently available’. Readers should bear in mind that the second cycle of IELS began more than two years ago.
OECD's reply listed participants in the ‘second cycle of IELS’. This time round there are five countries (England, Korea, Malta, Netherlands, UAE) and regions or provinces from four other countries (Azerbaijan (Baku), Belgium (Flanders), Brazil (three states), China (Hangzhou)). In addition, Switzerland is participating in the Field Trial, but will not continue to the main study. Of the three countries that participated in the first cycle, only England has chosen to go again.
A closed book
Despite its long-running story, OECD's IELS remains a closed book, in two main ways. First, the pervasive lack of information, epitomised by a website that remains uninformative well into the Study's second cycle. This is symptomatic of a wider issue: OECD's continuing failure to engage with the early childhood community, making no attempt to inform or to consult.
But OECD and its IELS remain a closed book in another way. The Study has been criticised on many grounds. To take just three examples: its disinterest in context, with no information collected on the wider political, social, cultural and pedagogical context in participating countries; the absence of rationale (apart from a willingness to participate and, hence, to pay) for the inclusion of participating countries, and consequently no obvious point to comparing them; and the paucity of the results from the first cycle, telling us ‘very little that we did not know already’ (Moss and Urban, 2020: 166). These and many other criticisms have been raised by distinguished researchers from different countries – but elicit no response from OECD.
In short, OECD carries on regardless, ignoring serious questions raised about the IELS, apparently impervious to criticism. We are left wondering, too, what the countries and regions that have signed up for the second cycle think about the criticisms levelled at IELS. Are they aware of them? Have they discussed them with OECD and, indeed, their own early childhood communities? If so, how do they respond to them?
Sadly, given past experiences, it is not surprising to see the OECD's persistent refusal to engage with the field and any scholarship that does not support its inflexible agenda. It is bewildering, however, that the organisation that seeks to position itself at the forefront of global early childhood policy appears to be oblivious of the field it aims to govern. Beginning in the early 2000s the global debate on how to best support young children, from birth, and their families and communities has taken a ‘systemic turn’ (Urban, 2022), a recognition that early childhood institutions are embedded in complex social, cultural, historical, political and economic contexts. Any meaningful attempt at improving, or governing, them requires whole-systems understandings and approaches; globally, 76 countries have already adopted integrated, multisectoral early childhood policy frameworks (Vargas-Barón, 2015; Vargas-Barón et al., 2022). Especially when ‘research on improving the quality of ECE services tends to neglect larger political, public policy, cultural, and economic contexts that shape complex ECE systems’ (Britto et al., 2014: 245), the OECD's insistence (or at least the insistence of the Directorate for Education and Skills, the part of OECD that does ILSAs like the IELS) on testing individual children as a measure of quality of early childhood education appears increasingly out of touch.
Indeed, the need to adopt a broader approach has been acknowledged by another part of the OECD, its Development Centre, when reflecting on what can be learned from the disruptive effects of the COVID-19 pandemic. The Centre finds that making crises-ridden systems ‘more inclusive, more sustainable, more resilient and more responsive’ requires ‘active participation of citizens in new forms of collective action at the local, national and international level’ (OECD, 2021: 9). One example of how such ‘thought-provoking’ ‘collective action’ can be employed to evaluate – and value – the educational practices in early childhood settings can be found in the Nordic countries (who decided to steer clear of IELS). National policies are built on trust in local decision-making, and on systematic conversations between educators, families, and policymakers. ‘The purpose is to assess the system, not the individual child’ (Urban et al., 2023).
Disappointing, could do better
In our last contribution on IELS, when the Study's reports had just been published, we argued that the IELS is a wasted opportunity: ‘we strongly believe in the importance of comparative studies of early childhood education – but equally strongly believe that the IELS is not the way to go’ (Moss and Urban, 2020: 169). OECD might have started out from the proposition that there are different ways to undertake comparative studies, each inscribed with very different paradigms and purposes and each adopting very different methodologies. Instead, OECD simply went with more of the same, adding another ILSA to its stock and pursuing a positivistic ‘science of solutions’ that seeks to elaborate generic one-size-fits-all solutions about ‘what works’ and ‘best practice’, while simplifying and reducing alternative … Instrumental purposes such as these, transform comparative education into a ‘system of governance’, not only because they are increasingly driving education policy, but also because they contribute to the creation of mechanisms of prediction and control. (Sousa and Moss, 2022: 404)
framed within a paradigm (or paradigms) valuing diversity, complexity, and critical thinking [and that] works to build pluralistic and contextualised understandings of policy, pedagogy, and practice … with emphasis given to constructing understandings and explanations of local experiences and realities. Such studies suggest a purpose for comparative research that is very different to solutionism: to provoke thought and questioning, re-consideration and re-assessment, what Nikolas Rose (1999, 20) refers to as ‘interrupting the fluency of [dominant] narratives … and making them stutter. (Ibid.: 403)
This theme is developed by Joe Tobin when he explains how the types of ethnographic comparative studies with which he works have the purpose of ‘challenging taken-for-granted assumptions, expanding the menu of the possible, and illuminating the processes of global circulation of early childhood education policies and practices’ (Tobin, 2022: 298). In short, to think.
We hope readers of this Colloquium will share the information it contains with colleagues in the early childhood field. We hope, too, they will use it to stimulate critical discussion about the role of OECD in education, the methods it has chosen to adopt, and what part comparative research might play in the process of reconceptualising early childhood education.
Footnotes
Funding
The authors received no financial support for the research, authorship and/or publication of this article.
