Abstract
Reviewed by: John Ainley, Australian Council for Educational Research, Camberwell, Australia
Large-scale assessments have emerged as part of school education in many countries, including Australia, over recent decades. These assessments have become important elements in the management of education systems and take a variety of forms. Their introduction in education systems has typically generated public commentary, controversy and debate but surprisingly little scholarly analysis.
This publication is focused largely on the Australian National Assessment Program in Literacy and Numeracy (NAPLAN). Although this is a welcome addition to deliberations about national assessments in school education, it is a limited view of national testing. The Australian National Assessment Program includes, as well as NAPLAN, three Australian sample-based assessments in Civics and Citizenship, Information and Communication Technology Literacy and Science Literacy and the Australian components of three international assessments: The Programme for International Student Assessment, the Trends in International Mathematics and Science Study and the Progress in International Reading Literacy Study. Some of these are mentioned briefly in the final chapter but generally they are ignored. As a consequence, the volume is concerned with NAPLAN testing, rather than national testing, in Australia. This constraint means that some important issues such as the extent to which sample-based testing might fulfil some of the purposes of NAPLAN and how best to establish a basis for time series measures of achievement are not discussed.
The publication begins with a chapter that succinctly sketches the origins and context of NAPLAN and ends with an interesting chapter presenting perspectives on the ways in which data such as those from NAPLAN can should and do interact with policy and practice. Between these bookends is a series of chapters that cover various aspects of NAPLAN: Properties of the tests, influences on the work of teachers and leaders in schools and impacts on students.
Three chapters in the book are concerned with the NAPLAN tests and the reporting of test results. One of these focuses on the precision (or reliability) with which NAPLAN measures achievement at individual, school and system level and how the psychometric and statistical properties of the tests constrain the inferences that can be drawn from the test results. A second discusses the appropriateness of the tests for the range of purposes to which they are put (validity) and whether such an instrument can serve multiple purposes. A third discusses the related issue of the categories of language background used to disaggregate NAPLAN results. The breadth of the categories used predates NAPLAN, but it remains an issue that is currently under review in several states and nationally.
Several chapters concern the possible influences of NAPLAN on the work of schools and teachers. One canvasses issues associated with how the reporting of school-level data on the My School website shapes the work of schools and teachers. A second examines positive and negative perceptions expressed by teachers responding to a survey in two states of the impacts of NAPLAN testing on their work. Although negative views predominate, the chapter does point to the complexities of these perceptions. A third chapter uses an institutional ethnography approach to examine the impact of NAPLAN on teachers’ and leaders’ work. A fourth chapter is concerned with the place of NAPLAN in initiatives across a cluster of schools with relatively high proportions of students from low socioeconomic backgrounds to work collaboratively. A fifth chapter reports on a school in which NAPLAN performance was initially poor and where NAPLAN became the focus of a school improvement program.
Four chapters focus on the perceived influences of NAPLAN on students. One discusses teachers’ perceptions of the effects of NAPLAN testing on student wellbeing, including stress, based on an online survey of a national sample of around 8000 teachers. It argues that there are impacts on student wellbeing which arise from the ways in which the assessment program is implemented. A second chapter, based on a quite different method, analyses students’ drawings of their NAPLAN experience and their comments on those drawings. It concludes that some children experienced anxiety about NAPLAN, but the extent of that anxiety depended on the context in which NAPLAN was implemented. A third chapter is based on interviews with students in Years 5 and 7 which concludes that although they had negative feelings about the tests, they wanted the tests to facilitate their learning. A fourth chapter examines the extent to which NAPLAN serves its purpose of identifying students ‘at risk’ in order to target interventions. It suggests that NAPLAN provides an improvement on teacher judgement in terms of identification but that it does not help with the introduction of interventions.
These chapters address a range of important issues and raise interesting questions. As would be expected, they vary in their scope and methods as well as in the extent to which one could generalise confidently from the reported results. However, readers of the book would not be rewarded with a comprehensive or even representative account of national testing in Australia as a result of the book’s nearly exclusive focus on NAPLAN. Moreover, the view of NAPLAN testing is predominantly from schools.
This has the consequence of overlooking broader perspectives. One of these consequences concerns time series analyses of achievement at jurisdictional and system level. For example, the general improvements in reading achievement in Year 3, and later in Year 5, over time compared to the lack of a comparable improvement in numeracy over the same period raise important questions. The substantial increases in reading achievement, and to a lesser extent in numeracy, in Years 3 and 5 in Queensland are also important results to emerge from NAPLAN. Similarly, little attention is given in this collection – although an example is mentioned in one chapter – to analyses that involve linking NAPLAN data to other student-level data to improve understandings of the correlates of achievement. Finally, the uses of NAPLAN results – usually based on data from a succession of years rather than one year – by education authorities to target resources and support to schools are not mentioned.