Sage Journals: Discover world-class research

Abstract

This paper examines changes in demographic and socioeconomic inequalities in student achievement over the school career, and the extent that these inequalities are accounted for by other influences such as, region and socioeconomic background (where appropriate), school differences and prior achievement. The data analysed are from a longitudinal cohort of Victorian government school students in Years 3, 5 and 7 between 2008 and 2012. The most important finding is the dominant influence of prior achievement which substantially reduces demographic and socioeconomic differences. The strong effects of prior achievement hold even after differences between schools and socioeconomic background have been taken into account. Therefore, policy positions and theories of student performance that give primacy to the socioeconomic resources of families when students are at school, or schools themselves, are not supported. The genesis of demographic and socioeconomic inequalities in student achievement occurs prior to Year 3 and point to the importance of factors operating in the preceding years.

Keywords

Student achievement NAPLAN longitudinal data gender gaps indigenous students immigrant students regional differences socioeconomic inequalities

Introduction

National assessments of educational achievement aim to provide evidence about the levels of student achievement in identified curriculum areas (for example, in the areas of reading and mathematics) for particular grade levels (Postlethwaite & Kellaghan, 2008). Examples include the National Assessment of Educational Progress (NAEP) in the United States, the Key Stages assessment program in the United Kingdom and the National Assessment Program (NAP) in Australia. Ideally, national assessments of student achievement allow schools, teachers and parents to monitor the performance of individual students. Furthermore, national assessments of student achievement provide data on demographic, socioeconomic, school and other differences which can contribute to policy development.

International assessments of student achievement allow comparisons between different educational systems aiming to identify what policies promote both higher achievement and equity. The major international assessments are Trends in International Mathematics and Science Study (TIMSS), Progress in International Reading Literacy Study (PIRLS) directed by the IEA (International Association for the Evaluation of Educational Achievement) and the Organisation for Economic Co-operation and Development's (OECD’s) Program for International Student Assessment (PISA). PIRLS focuses on reading in Year 4, TIMSS on mathematics and science in grades 4 and 8, and PISA on student achievement in reading, mathematics and science of 15 year olds. PIRLS has been conducted on a five-year cycle since 2001, TIMSS on a four-year cycle since 1995 and PISA a three-year cycle since 2000. In TIMSS, PIRLS and PISA, student achievement is measured on scales set at a mean of 500 and a standard deviation of 100.

Not only do national and international assessments monitor student performance and education systems, but student achievement is strongly predictive of subsequent educational and post-school outcomes. Student performance in PISA has strong effects on school completion (or the converse dropping out), student performance in the final year of schooling and post-school study (Marks, 2007, 2010b; OECD, 2010a). This is within the wider literature of the effects of cognitive test scores collected in adolescence or earlier on subsequent educational and labour market outcomes (Jencks et al., 1979; Korenman & Winship, 2000; McIntosh & Munk, 2007; McNiece, Bidgood, & Soan, 2004; Marks, 2014; Marks & McMillan, 2003).

The Australian NAP

Since 2008 the performance of Australian students across a variety of subject areas (or domains) has been measured by the NAP. The program has two components: the annual National Assessment Program in Literacy and Numeracy (NAPLAN) and triennial NAP assessments in Science Literacy, Civics and Citizenship, and Information and Communication Technology (ICT) Literacy. NAP follows separate testing programs conducted by most states and territories. NAPLAN tests, which are the subject of this paper, are conducted each year for all students across Australia in Years 3, 5, 7 and 9. Students in the same year level are assessed on the same test items in the assessment domains of numeracy, reading, writing, spelling and grammar. The assessments are undertaken every year in the second full week in May (ACARA, 2009, p. iv).

The NAPLAN tests are developed collaboratively by the Australian Curriculum, Assessment and Reporting Authority (ACARA), federal, state and territory education departments, and Catholic and independent education authorities. The test administration authority in each state and territory is responsible for delivery of the NAPLAN tests, and for test administration, data capture and reports. The content of the NAPLAN tests broadly assesses the literacy and numeracy skills intended to be developed through the curriculum within the states and territories. In addition, the types of questions and the formats used are familiar to teachers and students across Australia. Protocols for test administration ensure consistency in the administration of the tests by all test administrators and schools across Australia (ACARA, 2010a, p. iv).

The reading, language conventions (spelling and grammar) and numeracy tests consist of a mix of multiple choice items and short answer items. The multiple choice items are of standard format with a number of possible answers (usually four) which require students to select the best answer among the distractors. The short answer items are closed-constructed response items that generally involve a numeric answer, a word or a short phrase. The number of items ranges from 25 for spelling and grammar to between 30 and 40 for reading and numeracy (in Years 3 and 5); while the Year 7 and 9 tests include over 60 numeracy items and between 40 and 50 reading items. All items are dichotomously scored (correct or incorrect). For writing, students across the four-year levels need to write an essay in response to the same prompt. Persuasive writing replaced narrative writing in 2011. In 2008, the scripts were rated based on 10 criteria each comprising between three to seven categories (see ACARA, 2008).

NAPLAN tests are equated so that one year’s results can be compared with those for other years. Equating is the process by which the test items from two or more tests are placed on the same measurement scale. Equating involves selecting a sample of students who sit an additional equating test, as well as their respective grade level test, so that new items can be located on the common scale and used in subsequent years’ tests (ACARA, 2008, 2010b). The logic of modern test theory is that the probability of a student correctly answering a particular item is a function of student ability and the difficulty of the item (Hambleton, Hariharan Swaminathan, & Rogers, 1991). So if the abilities of a sample of students are known, then the difficulties of a new set of items can be calculated.

Each NAPLAN scale describes the development of student achievement from Year 3 through to Year 9 with the scale ranging from 0 to 1000 (called scaled scores). Student scores in each of the five domains across the four-year levels are standardized to have a mean of 500 and a standard deviation of 100. Lower year-level students score lower on the scale for a particular domain than higher year-level students. Scaled scores within a given domain are consistent across Year levels and over-time. For example, a scaled score of 540 in a particular domain has the same meaning in terms of skills and understanding for Year 5 students as Year 3 students, although their position relative to their peers and their expected performance are quite different. Similarly, the same score indicates the same level of skills across calendar years (from 2008). ACARA notes that the scaled scores between domains should not be compared, although the scales are standardized in the same way. Since persuasive writing replaced narrative writing in 2011, ACARA (2011, p. iv) warns that caution is required in comparing persuasive writing in 2011 or 2012 with performance in narrative writing in previous years.

Purpose of present study

The purpose of this study is to examine demographic and socioeconomic inequalities in student achievement between Years 3 and 7. Specifically, this paper examines:

Gender, Indigenous, language background, regional and socioeconomic inequalities in student achievement in Years 3, 5 and 7.

The extent that gender, Indigenous, language background, regional and socioeconomic inequalities change over the school career.

The extent that gender, Indigenous, language background, regional differences in achievement can be accounted for by socioeconomic background, region (where appropriate), differences between schools, and prior achievement.

The extent that socioeconomic inequalities in student achievement are mediated by differences between schools.

The extent that the effects of prior achievement on student achievement can be accounted by socioeconomic background and differences between schools.

There are several advantages in analysing NAPLAN data compared with data from other studies. First, student achievement is measured using item response theory techniques which ensure that the test items provide measures of student performance that are as accurate as possible (ACARA, 2008). Second, several domains are assessed for the same year level of students. Third, NAPLAN tests are administered annually to students in Years 3, 5, 7 and 9 allowing the measurement of change over time. Fourth, within each domain, the tests are scored on the same scale so student progress can be more easily ascertained. As regards data from other studies that could be analysed to address the aforementioned issues, PIRLS only assesses reading literacy at the Year 4 level TIMSS only assesses mathematics and science at both Year 4 and Year 8 levels, but scores are not on the same scale across Year levels. PISA which derives tests from a literacy perspective in terms of what students should be able to do in an applied manner rather than test the intended curriculum, tests performance in reading, mathematics and science but of 15-year-old students only. Finally and importantly, while PIRLS, TIMSS and PISA are sample studies, estimates from NAPLAN are from population data so are not subject to the uncertainties associated with sampling.

Background literature

Demographic differences in student achievement

Gender

In most countries, girls perform better than boys in reading but worse in numeracy (Marks, 2008a; Nowell & Hedges, 1998). In a meta-analysis of national and international achievement studies in reading conducted between 1970 and 2002, Lietz (2006) found that female secondary students scored, on average, 0.19 standard deviations higher than their male peers, regardless of age or language of instruction. The gender gap tended to be larger in NAEP assessments, PISA and in more recent national assessments in Australia.

Gender differences in mathematics favouring boys are generally small and declining (Baker & Jones, 1993; Friedman, 1989; Hanna, 2000). More recent comparisons of TIMSS data indicate little change in the gender gap for mathematics (Neuschmidt, Barth, & Hastedt, 2008). Analyses of the PISA data suggest an inverse relationship across countries in the gender gaps for reading and mathematics (Guiso, Monte, Sapienza, & Zingales, 2008; Marks, 2008a; Stoet & Geary, 2013). The authors of the most recent and comprehensive study on the issue conclude that across countries ‘sex differences in mathematics were consistently and strongly inversely correlated with sex differences in reading’ (Stoet & Geary, 2013, Abstract).

There is also evidence that gender gaps increase through the school career. In their review article, Buchmann, Diprete, and McDaniel (2008, p. 322) note a ‘growing male advantage in math scores and growing female advantage in reading scores as they move through school’.

In the PIRLS study of Year 4 students, the gender gap for reading in Australia favouring girls was 17 score points (or 0.17 of a standard deviation). According to the TIMSS study, in Year 4, the gap was in the other direction in mathematics (6 score points), but the difference was not statistically significant (Thomson, Hillman, Wernert, et al., 2012, pp. 19, 43–44).

According to the 2011 TIMSS study, the mean score in mathematics for boys in Year 8 in Australia was 9 score points higher than that for girls, although again the difference was not statistically significant (Thomson, Hillman, & Wernert, 2012, pp. 20–21).

The 2009 and 2012 Australian PISA studies showed gender gaps in reading (among 15-year-olds) were 35 and 37 score points, respectively, (favouring girls) which are only slightly larger than the gender gap in PISA 2000 (33 score points) but larger than that for PIRLS Year 4 students (17 score points). In mathematics, the gender gap (favouring boys) was 10 score points in 2009 and 12 score points in 2012 (Thomson, De Bortoli, & Buckley, 2013, p. 27, 176; Thomson, De Bortoli, Nicholas, Hillman, & Buckley, 2010, p. 55, 124, 181). Therefore, the gender gap in reading favouring girls is probably increasing over the school career and but little change over-time; the smaller gender gap in mathematics favouring boys appears to be stable.

Indigenous students

Indigenous students in Australia perform at a substantially lower level than non-Indigenous students. In the Year 4 PIRLS study, the mean score of Indigenous students for reading was 57 score points lower than that of non-Indigenous student. According to the 2011 TIMSS study of Year 4 mathematics, the difference was 64 score points (Thomson, Hillman, Wernert, et al., 2012, pp. 26, 50–53). A longitudinal study of Indigenous students in primary school found that Indigenous students improved at much the same rate as non-Indigenous students. The gap in average achievement between Indigenous students and their non-Indigenous peers evident at the start of Year 3 remained until the end of primary school (Purdie, Reid, & Buckley, 2011, p. 55). The gap was between 0.7 and 0.9 of a standard deviation (70–90 score points).

In the 2002 TIMSS study of Year 8 students, Indigenous students scored about 85 score points lower than non-Indigenous students in mathematics. The Indigenous–non-Indigenous gap for Year 8 had remained static or possibly increased between 1994 and 2002 (Thomson, 2007, pp. 218–220). In the most recent TIMSS study, the Indigenous–non-Indigenous gap among Year 8 students had declined slightly to 71 score points (Thomson et al., 2012, p. 29).

In the PISA 2000 study, the mean score for Indigenous students was 80 score points lower for reading and science and 85 score points lower for mathematics than non-Indigenous students (Bortoli & Cresswell, 2004, p. 10). A larger study that included Indigenous students from the 2000, 2003 and 2006 PISA studies found an average gap of 82 score points for reading, 80 to 86 for mathematics and 75 for science (Bortoli & Thomson, 2009, p. 11, 25, 41). For PISA 2012, the Indigenous–non-Indigenous gap had increased to 80 points for mathematics from 76 score points in the 2009 PISA assessment. For reading, the gap was 87 score points slightly higher than the 82 score point gap in 2009 (Thomson et al., 2013, p. 52, 185, 201).

Language background

In most countries, immigrant students perform less well than native students. However, this is not the case in Australia.¹ From middle secondary school to the final year of school, students with Language Backgrounds Other than English (LBOTE) tend to increase their performance relative to their peers (Marks, 2010a).

According to PIRLS, in Year 4 LBOTE students exhibited mean scores in reading 18 score points lower than students with an English language background. In TIMSS 2011, Year 8 LBOTE students outperformed English language background students by 17 score points in mathematics although the difference was not statistically significant (Thomson et al., 2012, p. 30). For Year 4 mathematics, the difference was 13 score points but again was not statistically significant (Thomson, Hillman, Wernert, et al., 2012, p. 27, 52).

In PISA 2009 and 2012, LBOTE students exhibited lower scores in reading (16 and 9 score points) but again the difference was not statistically significant. The two groups had similar mean scores in mathematics. Measures of immigrant status based on country of birth showed that foreign born and first generation students had higher scores in both reading and numeracy (Thomson et al., 2013, pp. 39–41, 188–189; Thomson et al., 2010, pp. 67–68, 192–193).

Region

There are also regional differences in student achievement in Australia. In the Year 4 PIRLS reading tests, the mean score for students attending metropolitan schools was 14 score points higher than that for the 27% of students attending ‘provincial’ schools and 56 score points higher than the 1% of students attending remote schools. For Year 4 mathematics in TIMSS, the results were similar. The mean score for metropolitan students was 16 score points higher than that for provincial students and 64 score points higher than for the remote group (Thomson, Hillman, Wernert, et al., 2012, p. 28, 53).

For Year 8 mathematics, metropolitan students scored 27 score points higher than students attending provincial schools and 64 score points higher than students attending remote schools (Thomson et al., 2012, pp. 31–32). In an analysis of the academic achievement in Western Australian students in Years 8 to 10, Young (1996) concluded that students attending rural schools had lower levels of science and especially mathematics achievement even after taking account socioeconomic and social background differences.

Cresswell and Underwood (2004, p. 10) found only small differences in PISA performance between students living in metropolitan and inner regional areas but larger differences (about half-a-standard deviation) between metropolitan and students living in remote or very remote regions. In the 2012 PISA study students attending metropolitan schools scored, on average, 25 score points higher in mathematics than students attending provincial schools. For reading, the difference was 30 score points (Thomson et al., 2013, p. 36, 184). Marks, Underwood, Rothman, and Brown (2011, p. 39, 44) found that students living in the northern and eastern suburbs of Sydney exhibited significantly higher Australian Tertiary Admissions Ranks than students living in other areas of NSW, although the differences declined somewhat when controlling for socioeconomic background and more substantially more when controlling for prior performance.

Socioeconomic inequalities in student achievement

The OECD’s (2007, p. 333) PISA study of 15 year olds routinely constructs broad measures of socioeconomic background denoted as the PISA index of Economic, Social and Cultural Status. For the 2006 PISA study, it comprised the highest of father’s or mother’s occupational status, father’s or mother’s educational attainment, and data from approximately 20 wealth, educational and cultural items. There were similar broad (but not identical) composite measures for the other (2000, 2003, 2009, 2012) PISA studies. In Australia, this broad measure accounted for 11.3% of the variation in student achievement in science in 2006, 12.7% in reading in 2009 and 12.3% in mathematics in 2012 (OECD, 2007, p. 184, 2010b, p. 55, 2013, p. 34). These are equivalent to (multiple) correlations of around 0.35. The magnitude of these measures of association indicate that socioeconomic background is only moderately associated with student achievement.

There are plausible arguments that the association between achievement and socioeconomic background differ by subject domain. Cultural capital theory would imply that subjects with a higher level of cultural content – literature and other humanities – would be more closely associated with socioeconomic background than mathematics or science. Coleman (1975, p. 382) argued that ‘reading achievement is more fully an outgrowth of home influences than are either of the other two subjects (mathematics and science), less a function of what takes place at school’.

The influence of socioeconomic background is not consistently stronger for one type of achievement test compared to others. Using Coleman’s achievement measures, White (1982) reported similar correlations between socioeconomic background and reading and mathematics. Sirin’s (2005, p. 433) meta-analysis reported a larger average correlation for socioeconomic background with mathematics (0.35) than for reading literacy (0.32) or science (0.27). Analysing student achievement in 30 countries using data from PISA 2000, Marks (2008b) found similar amounts of variance explained for reading and mathematics scores (for most countries) for models comprising differing combinations of father’s and mother’s educational attainment and occupational status.

Ability and student achievement

There are both theoretical arguments and empirical evidence for the contention that student performance in achievement tests is strongly related to students’ general cognitive ability.

Cognitive ability and achievement are conceptually similar. In PISA, literacy is defined generally as ‘concerned with the capacity of students to apply knowledge and skills in key subject areas and to analyze, reason and communicate effectively as they pose, solve and interpret problems in a variety of situations’ (OECD, 2007. p. 16). One prominent definition of intelligence is ‘a very general mental capability that, among other things, involves ability to reason, plan, solve problems, think abstractly, comprehend complex ideas, learn quickly and learn from experience’ (Gottfredson, 1997, p. 13). Rindermann (2008, p. 128) maintains there is no important theoretical difference between student achievement and ability tests since they both assess ‘thinking and knowledge’. Similarly, Baumert, Lüdtke, Trautwein, and Brunner (2009, pp. 3–5) points out that like intelligence tests, reading and mathematical assessments involve reasoning and making logical inferences.

There are also sizable correlations between student achievement and measures of cognitive ability. In a meta-analytic study of close to 3000 empirical studies of school learning, Walberg (1984, p. 23) computed an average correlation of 0.71 between various IQ measures and academic achievement. More recently, Duckworth, Quinn, and Tsukayama (2012, p. 443) reported correlations between 0.7 and 0.8 for IQ measured in grade 4, and grade 5 and 9 achievement tests for students attending New York public schools. For New Zealand, the correlation between IQ measured at ages 8 and 9 with academic performance at age 13 was 0.83 (Fergusson, Horwood, & Boden, 2008, p. 285). Gottfredson (2008, p. 554) concludes that cognitive ability correlates also at around 0.8 with performance in standardized academic achievement tests. The correlations between general intelligence (g) extracted from the Armed Services Vocational Aptitude Battery and the Scholastic Assessment Test and the American College Readiness Assessment (ACT) are both around 0.8. Standard intelligence tests show correlations between 0.55 and 0.81 with the ACT (Frey & Detterman, 2004; Koenig, Frey, & Detterman, 2008). Ability also tends to be quite stable across the educational career, from adolescence to adulthood and over the lifespan (see Marks, 2014, pp. 55–56). For Australia, Warren and Haisken-DeNew (2013, p. 21, 24) report that the addition of a preschool measure of what the authors refer to as ‘innate ability’,² increased the variance explained in Year 3 student performance (in NAPLAN) from between 11 and 15% (depending on the achievement domain) to about 25%.³

Baumert et al. (2009) point out that large-scale assessments do not simply measure ability, although general ability is important. They make three points: there are conceptual differences between achievement and ability tests; there is some evidence for separate verbal and mathematical latent dimensions; and different achievement domains have distinct learning and developmental pathways. Carroll’s (1993) well-known hierarchical model of human cognitive ability has general ability at the apex (g), a range of broad abilities at the second level and specific subject-related abilities at the third level.

Data, measures and methods

Data

The data analysed comprise the Victorian government school sector NAPLAN data for Year 3 students in 2008, Year 5 students in 2010 and Year 7 students in 2012. Data from students attending non-government schools were not included in the data supplied. The data include NAPLAN performance, unique school and student identifiers and demographic characteristics of students from enrolment data. Years 3 and 5 are, respectively, the 4th and 6th years of primary school (there is a preparatory year) and Year 7 is the first year of secondary school. Student identification numbers allow linkage of the three data sets creating a longitudinal cohort comprising only students with valid NAPLAN data for at least two of the three years.

Achievement measures

Table 1 presents means, standard deviations and the number of cases for each test in each year for the longitudinal cohort.⁴ It shows the expected increase in mean scores within domains, little change in the variation (indicated by the standard deviation) in numeracy, an increase in the variation in writing in Year 7 compared to Years 3 and 5, a decrease in the variation in reading, spelling and grammar in Year 7 compared to Years 3 and 5.

Table 1.

Summary statistics of the achievement domains by year level.

	N of cases	Mean	Standard deviation	No. of schools
Year 3 (2008)
Numeracy	41,277	416.14	72.53	1325
Reading	41,321	415.82	82.24	1325
Writing	41,213	421.25	68.64	1325
Spelling	41,286	411.18	75.15	1325
Grammar	41,286	423.40	87.69	1325
Year 5 (2010)
Numeracy	40,271	501.78	72.41	1315
Reading	40,400	497.91	77.61	1315
Writing	40,294	492.50	67.46	1315
Spelling	40,382	490.97	67.62	1315
Grammar	40,382	506.98	83.52	1315
Year 7 (2012)
Numeracy	33,703	534.72	73.01	413
Reading	33,763	536.81	68.40	413
Writing	33,837	512.20	73.47	413
Spelling	33,891	534.96	70.59	413
Grammar	33,891	538.29	70.53	413

Table 2 presents the correlations between domains and the over-time correlations within domains. It is readily apparent that student achievement in one domain is moderately to highly correlated with achievement in the other four domains. Few of the inter-domain correlations are below 0.5. The strongest correlations are between reading and grammar (0.65 < r < 0.75). Although the spelling and grammar test items were included in the same test, their inter-domain correlations (0.65 < r < 0.70) are not dissimilar in magnitude to that between numeracy and reading (0.65 < r < 0.70). The weaker inter-domain correlations often involve writing (0.45 < r < 0.65). This may be due to the different format of the writing tests to the other tests or more error in the measurement of writing performance. The change from narrative to persuasive writing in 2011 is not reflected by a different pattern of correlations between writing and the other domains in 2012, than in 2008 or 2010. The sizable inter-domain correlations are consistent with the contention that there is a common underlying general ability factor to student performance across these domains.

Table 2.

Inter-domain and over-time correlations for student achievement.

	Inter-domain					Over-time
	Numeracy (same year)	Reading (same year)	Writing (same year)	Spelling (same year)	Grammar (same year)	Year 5 (2010) same domain	Year 7 (2012) same domain
Year 3 (2008)
Numeracy	1.00	0.69	0.47	0.57	0.65	0.71	0.73
Reading		1.00	0.55	0.64	0.72	0.72	0.70
Writing			1.00	0.63	0.56	0.52	0.48
Spelling				1.00	0.68	0.83	0.81
Grammar					1.00	0.60	0.64
Year 5 (2010)
Numeracy	1.00	0.65	0.47	0.55	0.61	1.00	0.79
Reading		1.00	0.54	0.58	0.67	1.00	0.74
Writing			1.00	0.62	0.57	1.00	0.55
Spelling				1.00	0.66	1.00	0.85
Grammar					1.00	1.00	0.65
Year 7 (2012)
Numeracy	1.00	0.67	0.53	0.60	0.69	–	1.00
Reading		1.00	0.58	0.60	0.74	–	1.00
Writing			1.00	0.63	0.61	–	1.00
Spelling				1.00	0.68	–	1.00
Grammar					1.00	–	1.00

The over-time correlations are even stronger suggesting a high degree of stability in student performance. The over-time correlations for spelling are very strong (r > 0.80), followed by numeracy (0.70 > r > 0.80) and reading (0.70 > r > 0.75), grammar (0.60 < r < 0.65), and writing (0.45 > r > 0.55). There is no evidence that the over-time correlations ‘drop off’ sharply with the time elapsed between tests. The correlations in student achievement between Years 3 and 7 are comparable to, or in some cases larger than, the correlations between tests taken two years apart. The change from narrative to persuasive writing in 2011 is not reflected in lower correlations between persuasive and narrative writing tests than between different narrative writing tests. In fact, the correlation between persuasive writing for the Year 7 students in 2012 with narrative writing when they were in Year 5 in 2010 is slightly stronger (r = 0.55) than the correlation between narrative writing for the Year 5 students in 2010 with narrative writing when they were in Year 3 in 2008 (r = 0.52).

Other measures

The measure of socioeconomic background (SES) used in this study is a composite of parents’ occupation and education. Information on parents’ occupation and education were obtained from enrolment records.⁵ The measures of parents’ occupation comprise the following categories:

Senior management, qualified professionals

Other business managers associate professionals

Tradesmen/women, clerks and skilled office, sales and service staff

Machine operators, hospitality staff, assistants, labourers and related workers

Not in paid work in last 12 months

Not stated or unknown (missing data).

The measures of parents’ education for each parent were constructed from two variables: highest school level attained and post-school qualification resulted in the following ‘years of education’ ordinal measure:

15 Bachelor degree or above

13 Advanced diploma/diploma

12 Year 12 or equivalent

11 Year 11 or equivalent

10 Year 10 or equivalent

9 Year 9 or equivalent or below

Missing data were not included in the education measures.

Parental education and occupation were combined into a sheaf variable (Whitt, 1986). A sheaf variable is a combination of several variables that theoretically relate to the same concept. The variance explained (R square) by a sheaf variable is exactly the same as in an analysis comprising the constituent variables. The first stage in constructing the sheaf variable was to obtain estimates from a regression analysis of combined NAPLAN score on continuous measures of father’s and mother’s education and categorical measures of father’s and mother’s occupation group (including missing data for occupation). The sheaf variable was calculated by multiplying the estimates by the values of parental occupation and education and summing the products for each individual student. The resulting sheaf variables were then standardized to a mean of zero and a standard deviation of one.

Data on individual students’ gender, Indigenous status and language background were obtained from enrolment records. The postcode of the students’ address was used to allocate students to local government areas (LGAs) and subsequently to eight metropolitan and five non-metropolitan regions.⁶ The inner-east and inner-south metropolitan regions include Melbourne’s wealthier suburbs and formerly working class suburbs closer to the city which have become gentrified. The inner-north and inner-west metropolitan regions were traditional working class areas also undergoing gentrification. The relationship between LGA and regions is summarized in Table 3. There were instances of students attending a school in Victoria but living across the border in NSW or SA. They were allocated to the nearest Victorian region.

Table 3.

Correspondence between local government areas and regions.

Code	Region	Local government areas (from postcode)
1	Inner west	Hobsons Bay, Maribyrnong, Melbourne
2	Outer west	Brimbank, Moonee Valley, Melton, Wyndham
3	Inner north	Banyule, Darebin, Moreland, Yarra,
4	Outer north	Hume, Nillumbik, Whittlesea
5	Inner east	Boroondara, Manningham, Whitehorse,
6	Outer east	Cardinia, Knox, Maroonda, Yarra Ranges
7	Inner south	Bayside, Glen Eira, Kingston, Monash, Port Phillip, Stonnington
8	Outer south	Casey, Frankston, Greater Dandenong, Mornington Peninsula
11	Barwon and south west	Colac-Otway, Corangamite, Glenelg,Greater Geelong, Moyne, Naracoorte and Lucindale (SA), Queenscliffe, Surf Coast, Warrnambool, Southern Grampians
12	Grampians	Ararat, Ballarat, Golden Plains, Hepburn, Hindmarsh, Horsham, Moorabool, Northern Grampians, Pyrenees, West Wimmera, Yarriambiack
13	Loddon and Mallen	Buloke, Balranald (NSW), Campaspe, Central Goldfields, Gannawarra, Greater Bendigo, Loddon, Macedon Ranges, Mildura, Mount Alexander, Murray (NSW) Swan Hill, Wakool (NSW), Wentworth (NSW). Mount Beauty, Marysville (Not in LGA).
14	Hume (North East Victoria)	Albury (NSW), Alpine, Benalla, Berrigan (NSW), Greater, Indigo, Mansfield, Mitchel, Moira, Murrindindi, Shepparton, Strathbogie, Towong, Wangaratta, Wodonga
15	Gippsland	Bass Coast, Baw Baw, East Gippsland, Latrobe, South Gippsland, Tumbarumba (NSW), Wellington. French Island (not LGA)
99		Missing

Frequencies are for students with valid data on numeracy tests in Years 3, 5 and 7.

The LBOTE data were based on the administrative field that identifies whether the main language spoken at home was English. Although the LBOTE category is broad, including a great diversity of non-English speaking background students (see Creagh, 2014), the Victorian data only contained dichotomous measures. Schools were identified with standard identification numbers for each NAPLAN test year.

Methods

The data are analysed using multiple regression. The first row in a panel (tables are divided into panels for the separate year) reports the unadjusted (bivariate) regression coefficients. For categorical variables these coefficients are equal to the differences in the mean scores between the reference group and other groups. Subsequent rows report the regression coefficients adjusting for other factors (where appropriate): region, school differences, socioeconomic background or combinations of these variables. The last row for the Year 5 and Year 7 panels present the regression coefficients net of prior achievement in that domain. In multiple regression, the regression coefficient is the predicted change in the outcome variable for a one unit change in the predictor variable, net of the other predictor variables. In other words, it is the effect of the predictor variable on the outcome variable ‘purged’ of its associations (correlations) with the other predictor variables. The ‘purged’ variables are not correlated with each other. This is why the coefficients obtained in multiple regression are commonly referred to as ‘independent’ or ‘net’ effects, or reported as the effect ‘other things being equal’. Only the coefficients for the variables of interest are shown in the tables, not the regression coefficients for the control variables. Note that statistical significance levels are not reported in this paper since the analyses are performed on population data not on data from a sample.

Fixed effect models

Fixed effect models are used to control for the effects of (measured or unmeasured) differences between schools. Fixed effect models are increasingly used in educational research to control for unobserved effects (Blanden & Gregg, 2004; Jæger, 2011; Mouw, 2006). Differences between schools (controlled for by fixed effect models for schools) include resources; staff–student ratios and school size; teacher qualifications, experience, quality and morale; school sub-cultures and anything else that might differ between schools.

Mathematically, the fixed effect model for schools is

Y_{ij} = α_{j} + β x_{ij} + e_{ij}, i = 1 \dots ., n; j = 1, \dots . J

where

Y_{ij}

refers to the achievement score of student ‘i’ in school ‘j’,

x_{ij}

are variables that vary between individuals and

α_{j}

is the effect for school ‘j’. The term

α_{j}

is equivalent to a set of zero-one dummy variables for all schools (bar one). Therefore, the vector of coefficients β for the x variables is net of all differences between schools. The analyses reported later in this paper which control for school differences, use fixed effect models to adjust for differences between schools.

Fixed effect models are based on the several of the same assumptions as Ordinary Least Squares regression: the x variables are measured without error; the expectation of the error term is zero; the error is constant for different values of the x variables, the error term is uncorrelated with the intercepts ( $α_{j}$ ) and uncorrelated with the x variables, and the intercepts are exogenous. However, it does not assume that the intercepts are uncorrelated with the x variables. This contrasts with the more familiar random effects model used in multilevel modelling of student and school effects where the vector of school effects ( $α_{j}$ ) is uncorrelated with the x variables (Allison, 2005, pp. 5–10; Bollen & Brand, 2010).

Results

Gender

Table 4 presents the results for the analyses of gender differences for girls compared to boys. For numeracy in Year 3, the average score for girls is about 13 score points less than that for boys, 16 score points in Year 5 and 14 score points in Year 7. The gender gap in reading favouring girls is about 10 score points which changes little between Years 3 and 7. There is a much larger gender gap for writing favouring girls at 25 score points in Year 3 increasing to 34 score points in Year 7. The magnitudes of the spelling and grammar gender gaps are between the reading and writing gender gaps, and both increased only marginally between Years 3 and 7 by about 2 score points. Therefore, the results provide only limited support for the contention that gender gaps increase over the school career; only substantially for writing.

Table 4.

Gender differences in achievement, unadjusted and net of prior achievement for Years 3, 5 and 7.

Achievement domain	Numeracy	Reading	Writing	Spelling	Grammar
Year 3
Unadjusted difference (Girls)	−13.2	9.6	25.1	13.2	17.1
Year 5
Unadjusted difference (Girls)	−15.9	9.9	26.4	14.4	20.1
Net of Yr 3 achievement	−6.6	3.4	14.0	4.5	10.2
Year 7
Unadjusted difference (Girls)	−14.4	9.5	34.1	15.0	19.1
Net of Yr 3 achievement	−4.7	3.9	21.9	5.1	10.3
Net of Yr 5 achievement	−1.7	3.1	19.2	2.5	8.3

Tests taken in 2010. Prior achievement is in respective domain.

Controlling for prior achievement indicates that the gender gaps in numeracy, reading and spelling largely reflect gender differences in achievement measured two years earlier. The larger net effects for gender on writing and grammar are due to the weaker over-time correlations for writing and grammar (see Table 2).

Indigenous status

Indigenous score about 30 to 40 score points lower than non-Indigenous students (Table 5). The Indigenous–non-Indigenous gaps are not consistent across domains. In Year 3, the gap was largest for grammar and smallest for writing; in Year 5, it was largest for numeracy and smallest for spelling; and in Year 7, it was largest for numeracy and writing and smallest for reading. The Indigenous–non-Indigenous gaps reported here for NAPLAN in Victoria are substantially smaller than the gaps reported in national sample studies such as PISA and TIMSS. This may reflect different demographics for Indigenous students in Victoria than for Indigenous students in other states or may be attributed to other factors.

Table 5.

Indigenous–non-indigenous differences in achievement, unadjusted and net of region, SES and school differences for Years 3, 5 and 7.

Achievement domain	Numeracy	Reading	Writing	Spelling	Grammar
Year 3
Unadjusted difference (Indigenous)	−35.9	−39.1	−27.9	−39.1	−46.8
Net of regional differences	−30.9	−33.7	−24.2	−32.0	−41.1
Net of SES differences	−17.6	−17.6	−19.4	−25.7	−27.2
Net of school differences	−23.7	−24.3	−17.7	−26.7	−31.1
Net of region, SES and schools	−15.6	−15.0	−15.9	−22.1	−22.3
Year 5
Unadjusted difference (Indigenous)	−42.2	−41.1	−35.2	−29.4	−41.6
Net of regional differences	−35.0	−35.1	−28.8	−23.2	−35.3
Net of SES differences	−21.4	−20.2	−22.0	−17.4	−19.4
Net of school differences	−27.2	−26.8	−21.8	−18.0	−27.2
Net of region, SES and schools	−17.7	−15.0	−14.5	−13.3	−14.7
Net of Year 3 achievement	−17.8	−14.2	−20.0	−0.8	−14.1
Year 7
Unadjusted difference (Indigenous)	−42.6	−34.6	−39.0	−34.7	−37.9
Net of regional differences	−34.7	−29.9	−31.2	−27.5	−31.9
Net of SES differences	−28.8	−25.0	−27.8	−23.0	−25.8
Net of school differences	−27.2	−23.7	−24.5	−21.6	−25.5
Net of region, SES and schools	−22.3	−22.4	−21.7	−17.4	−20.5
Net of Year 5 achievement	−10.7	−11.2	−19.9	−8.5	−17.9

SES = Socioeconomic Background. School Differences are modelled by fixed effects for individual schools. Prior achievement is in respective domain.

The Indigenous–non-Indigenous achievement gap cannot be attributed to regional differences since controlling for region decreases the gap by only about 10%. Controlling for socioeconomic background reduced the Indigenous–non-Indigenous gaps in achievement by between a third and a half reflecting the association between Indigenous status and socioeconomic background. A similar finding was found for schools, indicating that part of the Indigenous–non-Indigenous gap is associated with differences in the schools attended. Controlling for prior achievement (for achievement in Years 5 and 7) indicates that a large proportion of the Indigenous–non-Indigenous achievement gap, between one-half and three-quarters, can be attributed to differences in prior achievement.

Language background

In no domain is the mean achievement level of LBOTE students substantially lower than that of other students. Between Years 3 and 7, LBOTE students’ performance in numeracy improves to a greater extent than that of students whose language background is English (Table 6). In Year 3 numeracy there is no difference, in Year 5 LBOTE students score on average 14 score points higher; and in Year 7, 21 score points higher than students from an English language background. An increase in the gap between LBOTE and non-LBOTE students is also evident for writing and to a lesser extent, spelling. For grammar, the small gap favouring English background students in Year 3 was reversed in later years. For reading, there was little change. So in most domains, LBOTE students make greater progress than non-LBOTE students.

Table 6.

Language background differences in achievement, unadjusted and net of region, SES, school differences and prior achievement for Years 3, 5 and 7.

Achievement domain	Numeracy	Reading	Writing	Spelling	Grammar
Year 3
Unadjusted difference (not English)	−0.6	−5.5	6.3	20.5	−5.3
Net of regional differences	−3.9	−9.9	3.3	14.6	−10.7
Net of SES differences	−3.8	−8.7	3.9	18.4	−9.0
Net of school differences	−0.1	−4.9	4.4	14.8	−4.9
Net of region, SES and schools	−3.3	−8.2	1.3	12.4	−8.5
Year 5
Unadjusted difference (not English)	14.0	3.1	15.1	24.9	11.1
Net of regional differences	8.0	−3.2	8.8	19.1	5.1
Net of SES differences	10.6	−0.7	11.8	22.6	6.9
Net of school differences	8.8	−0.6	8.8	18.0	6.8
Net of region, SES and schools	6.0	−3.7	6.2	16.0	2.4
Net of Year 3 achievement	14.7	7.3	12.1	9.5	14.5
Year 7
Unadjusted difference (not English)	20.7	−0.7	23.9	29.2	7.0
Net of regional differences	12.9	−6.9	13.9	21.7	−0.2
Net of SES differences	20.2	−1.5	23.0	28.6	6.5
Net of school differences	11.4	−5.2	11.8	19.7	0.3
Net of region, SES and schools	11.4	−5.3	11.9	19.7	0.4
Net of Year 5 achievement	10.3	−1.0	15.1	7.4	1.9

SES = Socioeconomic Background. School differences are modelled by fixed effects for individual schools. Prior achievement is in respective domain.

For achievement in each domain in Years 5 and 7, controlling for region reduces differences in achievement between LBOTE and non-LBOTE students indicating that LBOTE students tend to live in regions with higher levels of student achievement. Similarly, controlling for school differences reduces the difference indicating that, on average, LBOTE students attend higher performing schools. Controlling for socioeconomic background makes little difference to language background differences in achievement. Controlling for prior achievement does not consistently reduce language background differences confirming that LBOTE students tend to improve their relative performance in most domains.

Region

Tables 7 and 8 present the findings for the analyses of regional differences for numeracy and reading. The figures denote the differences in average achievement for students living in each region relative to students living in the outer southern metropolitan region. Generally, the inner metropolitan regions exhibit higher mean achievement levels than other regions. There is a tendency for non-metropolitan regions to show lower mean achievement levels, but this is not always the case.

Table 7.

Regional differences in numeracy achievement, unadjusted and net of SES, language background, school differences and prior achievement for Years 3, 5 and 7.

Year level	Year 3					Year 5						Year 7
Year level	Unadjusted	Net of SES	Net of LBOTE	Net of schools	Net of SES, LBOTE and schools	Unadjusted	Net of SES	Net of LBOTE	Net of schools	Net of SES, LBOTE and schools	Net of Year 3 achievement	Unadjusted	Net of SES	Net of LBOTE	Net of schools	Net of SES, LBOTE and schools	Net of Year 5 achievement
Metro-inner West	19.7	7.5	20.2	7.1	18.1	23.5	13.3	22.4	14.7	35.5	8.4	37.2	24.7	35.4	23.5	45.9	17.9
Metro-outer West	−8.5	−6.9	−7.8	7.9	31.8	−4.9	−2.9	−6.5	12.6	47.7	−1.1	−0.2	1.9	−2.9	15.9	45.4	2.2
Metro-inner North	21.5	9.1	21.8	6.5	19.1	18.7	6.5	18.1	9.4	29.8	3.0	24.7	9.1	23.8	14.6	33.6	9.6
Metro-outer North	4.1	1.2	4.4	0.1	16.5	3.4	0.4	2.5	5.9	29.8	−0.6	−2.8	−6.4	−4.2	2.6	26.0	−6.2
Metro-inner East	41.7	18.8	42.2	4.2	1.1	49.8	27.7	48.7	7.6	5.6	19.4	53.2	27.3	51.5	2.1	−3.0	13.2
Metro-outer East	16.3	12.0	15.8	2.0	0.3	12.0	7.7	13.2	−1.3	−2.1	2.3	7.7	2.1	9.7	−6.4	−6.4	−0.7
Metro-inner South	29.0	15.5	29.7	1.4	1.2	32.0	18.9	30.3	−3.6	−2.3	9.7	39.0	23.7	36.3	−1.8	0.4	12.5
Barwon and South West	7.1	7.2	6.3	6.9	8.8	−2.7	−2.1	−0.8	9.6	39.2	−5.2	0.4	−1.5	3.5	13.6	18.4	4.2
Grampians	5.8	7.7	4.9	7.2	3.0	0.4	2.9	2.6	21.4	32.8	−0.8	−2.0	1.6	1.4	23.6	16.3	−0.6
Loddon and Mallee	7.5	5.3	6.6	9.8	10.5	−1.7	−3.3	0.4	4.6	22.0	−4.2	0.8	−1.3	4.1	20.9	31.5	3.9
Hume (NE)	−3.1	−1.7	−3.9	19.5	17.6	−11.5	−9.8	−9.7	21.1	29.9	−7.1	−13.2	−11.3	−10.4	23.7	34.0	−2.7
Gippsland	−0.4	0.9	−1.3	−10.3	−10.1	−4.3	−2.0	−2.1	−10.6	−2.8	−1.2	−10.7	−8.5	−7.2	−6.8	−14.0	−5.5
PCode missing	6.3	5.9	5.9	−7.3	12.9	0.6	−2.3	1.7	1.0	32.0	−2.4	1.7	−1.8	3.5	−0.2	17.5	2.1

Reference category is ‘Metro-Outer South’. SES = Socioeconomic Background. School differences are modelled by fixed effects for individual schools. Prior achievement is in respective domain.

Table 8.

Regional differences in reading achievement, unadjusted and net of SES, language background, school differences and prior achievement for Years 3, 5 and 7.

Year level	Year 3					Year 5						Year 7
Year level	Unadjusted	Net of SES	Net of LBOTE	Net of schools	Net of SES, LBOTE and schools	Unadjusted	Net of SES	Net of LBOTE	Net of schools	Net of SES, LBOTE and schools	Net of Year 3 achievement	Unadjusted	Net of SES	Net of LBOTE	Net of schools	Net of SES, LBOTE and schools	Net of Year 5 achievement
Metro-inner West	27.1	14.1	28.3	11.4	21.1	29.0	15.0	29.2	18.2	19.9	10.2	31.7	17.4	32.0	15.2	19.5	13.3
Metro-outer West	−2.9	−0.6	−1.1	10.9	21.7	−2.0	0.8	−1.6	13.6	20.4	−0.8	−0.9	0.7	−0.4	12.1	22.6	0.6
Metro-inner North	30.6	14.0	31.2	12.8	26.6	29.0	14.3	29.1	20.1	26.9	8.1	26.1	11.3	26.3	14.5	24.8	7.6
Metro-outer North	8.4	6.3	9.3	5.4	22.9	6.2	3.7	6.4	18.6	24.9	0.1	1.0	−2.8	1.3	11.8	23.0	−2.8
Metro-inner East	48.3	21.5	49.5	8.2	3.7	46.5	21.3	46.8	3.0	2.2	13.5	43.4	18.3	43.7	7.1	6.1	13.7
Metro-outer East	18.8	14.5	17.5	0.8	4.5	13.1	9.0	12.8	−1.1	8.4	1.1	12.8	7.8	12.4	0.6	4.5	4.2
Metro-inner South	31.3	14.4	33.0	−0.6	0.5	31.4	16.1	31.8	0.0	5.0	9.6	28.6	12.4	29.0	−4.8	−1.7	8.6
Barwon and South West	12.4	10.8	10.3	13.1	−17.3	1.2	2.3	0.8	8.8	−1.6	−6.2	5.1	4.2	4.5	12.5	5.5	4.1
Grampians	11.6	12.7	9.3	9.4	−12.3	2.2	5.4	1.7	22.1	−0.4	−4.6	1.8	4.0	1.2	14.5	0.7	0.2
Loddon and Mallee	10.7	7.5	8.5	12.7	25.1	7.2	5.5	6.7	13.9	17.6	1.0	9.0	6.2	8.4	18.8	28.8	4.1
Hume (NE)	−0.3	2.2	−2.2	22.0	39.3	−6.2	−3.7	−6.7	19.9	33.8	−5.2	−6.2	−4.7	−6.7	16.1	27.5	−2.4
Gippsland	3.2	5.1	0.9	−35.2	−20.7	−0.5	1.5	−1.1	−18.3	0.0	−1.7	−4.8	−2.9	−5.4	−9.2	−4.7	−4.7
PCode missing	7.1	6.0	5.9	−6.9	22.3	2.9	1.6	2.6	2.6	27.5	−1.4	4.5	2.8	4.2	1.4	12.8	2.5

Reference category is ‘Metro-Outer South’. SES = Socioeconomic Background. School differences are modelled by fixed effects for individual schools. Prior achievement is in respective domain.

Some of the regional differences increased over the school career. For numeracy this occurred among students living in the metropolitan inner west, inner-east and inner south regions. In some regions, student achievement in numeracy declined relative to the outer southern metropolitan region: Hume (North East Victoria), Gippsland and to a lesser extent the other non-metropolitan regions. Regional differences in reading tended not to increase over the school career.

When controlling for socioeconomic background, regional differences in both numeracy and reading achievement declined by about half in the inner metropolitan regions, to a lesser extent in the outer metropolitan areas and had little impact for the five non-metropolitan regions. This reflects the relationship between socioeconomic background and students’ region. Controlling for the language background of the students had no substantial impact on regional differences in achievement. Controlling for school differences had a strong impact on achievement differences associated with region. For example, for numeracy in Year 3, controlling for school differences reduced the difference for the metropolitan inner east region (compared to the outer southern region) for 42 to 4, for the inner south region from 29 to 1. The same pattern was found for Years 5 and 7 and for reading. Therefore, regional differences are often strongly associated with school differences.

Regional differences are very much reduced when controlling for prior achievement. This suggests that bulk of regional differences in student achievement can be attributed to differences in the characteristics of students between regions. Only a few regions, notably the inner metropolitan regions, show sizable differences after controlling for prior achievement. This means that students living in these regions make greater achievement gains than students living in other regions.

Socioeconomic background and prior achievement

Table 9 presents the correlations of socioeconomic background with achievement in Years 3, 5 and 7.⁷ The correlations are consistent with the literature with socioeconomic background accounting for between 9 and 16% of the variation in student achievement, without considering the impact of other influences on achievement.

Table 9.

Correlations between socioeconomic background and achievement for Years 3, 5 and 7.

Achievement domain	Numeracy	Reading	Writing	Spelling	Grammar
Year 3	0.32	0.32	0.22	0.23	0.29
Year 5	0.33	0.34	0.26	0.24	0.30
Year 7	0.37	0.37	0.30	0.26	0.34

The correlations of socioeconomic background are substantially weaker for spelling and, to a lesser extent, writing than for the other domains. There is no indication than the correlations between socioeconomic background and aspects of English literacy (which would seem to be more influenced by the home environment) are stronger than that between socioeconomic background and numeracy (which would seem to be more influenced by schools). The correlations between achievement and socioeconomic background increase in all domains between Year 3 and Year 7.

Table 10 presents the estimates for the effects of socioeconomic background with and without controls for school differences and prior achievement. The effects of socioeconomic background are for a one standard deviation difference. Its effects are weakest for writing and spelling and stronger for numeracy, reading and grammar.

Table 10.

Effects of SES on achievement, unadjusted and net of school differences and prior achievement for Years 3, 5 and 7.

	Numeracy	Reading	Writing	Spelling	Grammar
Year 3
Unadjusted effect of SES	23.9	27.0	15.5	17.7	25.9
Net of schools	19.6	21.7	12.1	13.6	20.6
Year 5
Unadjusted effect of SES	25.0	27.0	18.0	16.4	25.5
Net of schools	19.7	21.6	13.4	13.0	21.2
Net of Year 3 achievement	8.8	9.5	10.5	2.7	11.2
Year 7
Unadjusted effect of SES	27.5	24.8	21.3	17.8	24.2
Net of schools	19.5	18.5	14.5	12.2	18.2
Net of Year 5 achievement	9.6	9.0	11.9	4.8	12.0
All years (standardized)
Unadjusted effect of SES	0.36	0.36	0.28	0.26	0.33
Net of schools	0.28	0.29	0.21	0.21	0.27
Net of prior achievement in domain	0.13	0.13	0.16	0.05	0.15

Effects are for a one standard deviation difference in socioeconomic background in Panels 1 to 3. In final panel the coefficients are standardized.

SES = Socioeconomic Background. School differences are modelled by fixed effects for individual schools. Prior achievement is in respective domain.

The effects of socioeconomic background on achievement decline by up to one-third when controlling for school differences. However, this does not mean that school differences account for up to a third of socioeconomic inequalities in student achievement. The results presented for Years 5 and 7 and in the last panel show that net of prior achievement, socioeconomic background effects on achievement are comparatively weak. The effects of socioeconomic background on achievement decline by between one-half (writing and grammar) and three-quarters when controlling for prior achievement. Again, this pattern reflects the weaker over-time correlations for writing and grammar. The last line of the last panel suggests that socioeconomic background accounts for only a very small proportion of the variation in student achievement (up to 3%), net of prior achievement.⁸

Prior achievement

Table 11 presents the effects of prior achievement on achievement in the respective domain. The standardized effects are the predicted change in achievement (in standard deviation units) for a one standard deviation difference in prior achievement (measured two years earlier). The unadjusted relationship between prior achievement and achievement is strong, with prior achievement in the respective domain accounting for between 28% of the variation in achievement (for Year 5 writing) to 74% of the variation (for Year 5 spelling). The impact of prior achievement was strongest for spelling followed by numeracy, reading, grammar and writing. The effects of prior achievement remains substantial for writing, even though there is more error associated with the assessment of student writing.

Table 11.

Effects of prior achievement on student achievement with and without controls for SES and school differences.

	Numeracy	Reading	Writing	Spelling	Grammar
Prior achievement in respective domain (Year 5 on Year 3)
Unadjusted effect	0.72	0.69	0.53	0.76	0.59
Net of schools	0.70	0.67	0.50	0.75	0.57
Net of SES	0.68	0.65	0.49	0.75	0.54
Net of schools and SES	0.68	0.64	0.47	0.75	0.54
Prior achievement in respective domain (Year 7 on Year 5)
Unadjusted effect	0.80	0.65	0.60	0.86	0.55
Net of schools	0.76	0.62	0.54	0.84	0.52
Net of SES	0.77	0.61	0.56	0.85	0.51
Net of schools and SES	0.75	0.60	0.53	0.83	0.50
Prior achievement in respective domain (Years 5 and 7)
Unadjusted effect	0.75	0.73	0.54	0.83	0.63
Net of schools	0.72	0.71	0.50	0.82	0.60
Net of SES	0.71	0.69	0.50	0.82	0.58
Net of schools and SES	0.71	0.68	0.48	0.81	0.58

Standardized effects for prior achievement.

Prior achievement is in respective domain. SES = Socioeconomic Background. School differences are modelled by fixed effects for individual schools.

The striking feature of Table 11 is that the effects of prior achievement are not substantially reduced when taking into account school differences or socioeconomic background. Controlling for school fixed effects reduced the magnitudes of the coefficients only marginally. This was the case for achievement in individual Years and in the combined analyses of Years 5 and 7 (third panel). This means that differences between schools have little impact on the effects of prior achievement on student achievement. School differences do not explain the effects of prior achievement on achievement. Similarly, controlling for students’ socioeconomic backgrounds made very little difference to the effects of prior achievement. This indicates that the strong effects of prior achievement on achievement cannot be attributed to socioeconomic background. Together, school differences and socioeconomic background only marginally reduce the strong relationships between prior achievement and achievement.

Conclusion and discussion

In the analyses presented here, prior achievement typically reduces the effects of demographic and socioeconomic factors on student achievement, often very substantially. The major exceptions are language background, the inner metropolitan regions in numeracy and reading, where these students experience greater achievement growth. The effects of prior achievement are also weaker in the analyses of writing, and to a lesser extent grammar, where the over-time correlations are weaker. Adams (2012, p. 18, 32) focusing on student gain scores in NAPLAN also found that after taking into account students’ initial scores, there is little variability in performance that is associated with student background characteristics and concluded ‘by far the most profound predictor of performance is past performance’.

The review of the literature suggests that cognitive ability has a major role in student performance in achievement tests. From these analyses of NAPLAN performance, there is substantial indirect evidence that general cognitive ability is involved. The sizable inter-domain correlations are suggestive of a common underlying factor that accounts for the similarity of student performance across domains. The even stronger over-time correlations within domains most likely reflect the stability of cognitive ability among individual students. The strong effects of prior achievement cannot be accounted for by schools or socioeconomic background. If schools were the main influence on variations in student achievement, then the effects of prior achievement would be very much reduced when controlling for school differences. Recall that the fixed effects for schools control for all differences between schools: their resources, practices and cultures. Similarly, if prior achievement was largely a function of socioeconomic background then controlling for socioeconomic background would substantially reduce the effects of prior achievement. However, controlling for both school differences and socioeconomic background made little difference to the strong effects of prior achievement on student achievement. The most plausible explanation for these findings is that student ability is the major influence on student achievement with schools and socioeconomic background playing much lesser roles. Therefore, assertions commonly made in policy debates and in academia that the variation in student achievement can be largely attributed to students’ socioeconomic backgrounds and/or the schools they attend are incorrect.

The key to understanding many of the social inequalities in student performance is to understand the influences on early school achievement and childhood ability. Future research should focus on the relative importance of innate ability (if that can be measured adequately), pre- and peri-natal health, infant health and nutrition, early childhood stimuli, parenting and preschool attendance. Understanding these relationships much better than we do today should help reduce demographic, socioeconomic and other social disparities in student achievement.

Footnotes

Declaration of conflicting interests

None declared.

Funding

This research was undertaken as part of a research partnership with the Victorian Department of Education and Early Childhood Development (DEECD). This paper uses unit record data from the NAPLAN administrative data collection, provided by DEECD. The findings and views reported in this paper are those of the authors and should not be attributed to DEECD or any other branch of the Victorian government.

Notes

References

ACARA (2008) NAPLAN 2008 central analysis technical report: Draft, Melbourne, Australia: Australian Council for Educational Research.

ACARA. (2009). NAPLAN Achievement in reading, writing, language conventions and numeracy. Sydney, Australia: Australian Curriculum, Assessment and Reporting Authority.

ACARA. (2010a). NAPLAN achievement in reading, writing, language conventions and numeracy: National report for 2010. Sydney, Australia: Australian Curriculum, Assessment and Reporting Authority.

ACARA. (2010b). Submission to the senate education, employment and workplace relations committee: Inquiry into the administration and reporting of NAPLAN testing. Sydney, Australia: Australian Curriculum, Assessment and Reporting Authority.

ACARA. (2011). NAPLAN achievement in reading, persuasive writing, language conventions and numeracy: National report for 2011. Sydney, Australia: Australian Curriculum, Assessment and Reporting Authority.

ACARA. (2012). NAPLAN achievement in reading, persuasive writing, language conventions and numeracy: National report for 2012. Sydney, Australia: Australian Curriculum, Assessment and Reporting Authority.

Adams, R. (2012). NAPLAN reporting: Measures and models for reporting gain over time. Sydney, Australia: COAG Reform Council.

Allison

P. D.

(2005) Fixed effects regression for longitudinal data using SAS, Cary, NC: SAS Institute.

Baker

D. A.

Jones

D. P.

(1993) Creating gender equality: Cross-national gender stratification and mathematical performance. Sociology of Education 66: 91–103.

10.

Baumert

Lüdtke

Trautwein

Brunner

(2009) Large-scale student assessment studies measure the results of processes of knowledge acquisition: Evidence in support of the distinction between intelligence and student achievement. Educational Research Review 4(3): 165–176.

11.

Blanden

Gregg

(2004) Family income and educational attainment: A review of approaches and evidence for Britain. Oxford Review of Economic Policy 20(2): 245–263.

12.

Bollen

K. A.

Brand

J. E.

(2010) A general panel model with random and fixed effects: A structural equations approach. Social Forces 89(1): 1–34.

13.

Bortoli, L. D., & Cresswell, J. (2004). Australia’s indigenous students in PISA 2000: Results from an international study (ACER Research Monograph No 59). Camberwell, London: Australian Council for Educational Research.

14.

Bortoli

L. D.

Thomson

(2009) The achievement of Australia’s indigenous students in PISA 2000–2006, Camberwell, London: ACER Press.

15.

Buchmann

DiPrete

T. A.

McDaniel

(2008) Gender inequalities in education. Annual Review of Sociology 34(1): 319–337.

16.

Carroll

J. B.

(1993) Human cognitive abilities: A survey of factor-analytic studies, New York, NY: Cambridge University Press.

17.

Coleman

J. S.

(1975) Methods and results in the IEA studies of effects of school on learning. Review of Educational Research 45(3): 355–386.

18.

Creagh

(2014) A critical analysis of problems with the LBOTE category on the NAPLAN test. The Australian Educational Researcher 41(1): 1–23.

19.

Cresswell, J., & Underwood, C. (2004). Location, location, location: Implications of geographic situation on Australian student performance in PISA 2000 (ACER Research Monograph no. 58). Camberwell, London: Australian Council for Educational Research.

20.

Duckworth

A. L.

Quinn

P. D.

Tsukayama

(2012) What no child left behind leaves behind: The roles of IQ and self-control in predicting standardized achievement test scores and report card grades. Journal of Educational Psychology 104(2): 439–451.

21.

Fergusson

D. M.

Horwood

L. J.

Boden

J. M.

(2008) The transmission of social inequality: Examination of the linkages between family socioeconomic status in childhood and educational achievement in young adulthood. Research in Social Stratification and Mobility 26(3): 277–295.

22.

Frey

M. C.

Detterman

D. K.

(2004) Scholastic assessment or g? The relationship between the Scholastic Assessment Test and general cognitive ability. Psychological Science 15(6): 373–378.

23.

Friedman

(1989) Mathematics and the gender gap: A meta-analysis of recent studies in sex differences in mathematical tasks. Review of Educational Research 59(2): 185–213.

24.

Gottfredson

L. S.

(1997) Mainstream science on intelligence: An editorial with 52 signatories, history, and bibliography. Intelligence 24(1): 13–23.

25.

Gottfredson

L. S.

(2008) Of what value is intelligence? In: Prifitera

Saklofeske

D. H.

Weiss

L. G.

(eds) Wisc-Iv clinical assessment and intervention, (2nd ed. Amsterdam, the Netherlands: Elsevier, pp. 545–563.

26.

Guiso

Monte

Sapienza

Zingales

(2008) Culture, gender, and math. Science 320(5880): 1164–1165.

27.

Hambleton

R. K.

Hariharan Swaminathan

Rogers

(1991) Fundamentals of item response theory, Newbury Park, CA: Sage.

28.

Hanna

(2000) Declining gender differences from FIMS to TIMSS. Zentralblatt für Didaktik der Mathematik [International Journal of Mathematics Education] 32(1): 11–17.

29.

Jæger

M. M.

(2011) Does cultural capital really affect academic achievement? New evidence from combined sibling and panel data. Sociology of Education 84(4): 281–298.

30.

Jencks

Bartlett

Corcan

Crouse

Eaglesfield

Jackson

Williams

(1979) Who gets ahead? The determinants of economic success in America, New York, NY: Basic Books.

31.

Koenig

K. A.

Frey

M. C.

Detterman

D. K.

(2008) ACT and general cognitive ability. Intelligence 36(2): 153–160. .

32.

Korenman

Winship

(2000) A reanalysis of the bell curve: Intelligence, family, background and schools. In: Arrow

Bowles

Durlauf

(eds) Meritocracy and economic inequality, Princeton: Princeton University Press, pp. 137–178.

33.

Lietz

(2006) A meta-analysis of gender differences in reading achievement at the secondary school level. Studies in Educational Evaluation 32(4): 317–344.

34.

McIntosh

Munk

M. D.

(2007) Scholastic ability vs family background in educational success: Evidence from Danish sample survey data. Journal of Population Economics 20(1): 101–120.

35.

McNiece

Bidgood

Soan

(2004) An investigation into using national longitudinal studies to examine trends in educational attainment and development. Educational Research 46(2): 119.

36.

Marks

G. N.

(2007) Do schools matter for early school leaving? Individual and school influences in Australia. School Effectiveness and School Improvement 18(4): 429–450.

37.

Marks

G. N.

(2008a) Accounting for the gender gap in reading and mathematics: Evidence from 31 countries. Oxford Review of Education 34(1): 89–109.

38.

Marks

G. N.

(2008b) Are father's or mother's socioeconomic characteristics more important influences on student performance? Recent international evidence. Social Indicators Research 85(2): 293–309.

39.

Marks

G. N.

(2010a) Improvements over the educational career of immigrant students. Australian Journal of Education 54(2): 133–154.

40.

Marks

G. N.

(2010b) What aspects of schooling are important? School effects on tertiary entrance performance in Australia. School Effectiveness and School Improvement 21(3): 267–287.

41.

Marks

G. N.

(2014) Education, social background and cognitive ability: The decline of the social, Abingdon, Oxon: Routledge.

42.

Marks

G. N.

McMillan

(2003) Declining inequality? The changing impact of socioeconomic background and ability on education in Australia. British Journal of Sociology 54(4): 453–471.

43.

Marks

G. N.

Underwood

Rothman

Brown

(2011) Career moves: Expectations and destinations of New South Wales senior secondary students, Sydney, Australia: NSW Board of Vocational Education and Training.

44.

Mouw

(2006) Estimating the causal effect of social capital: A review of recent research. Annual Review of Sociology 32: 79–102.

45.

Neuschmidt

Barth

Hastedt

(2008) Trends in gender differences in mathematics and science (TIMSS 1995–2003). Studies in Educational Evaluation 34(2): 56–72.

46.

Nowell

Hedges

L. V.

(1998) Trends in gender differences in academic achievement from 1960 to 1994: An analysis of differences in mean, variance and extreme scores. Sex Roles 39(112): 21–43.

47.

OECD (2007) Science competencies for tomorrow's world Vol. 1. Paris, France: Organisation for Economic Co-operation and Development.

48.

OECD (2010a) How does academic ability affect educational and labour market pathways in Canada, Paris, France: Organisation for Economic Co-operation and Development.

49.

OECD (2010b) PISA 2009 results: Overcoming social background. Equity in learning opportunities and outcomes Vol. II. Paris, France: Organisation for Economic Co-operation and Development.

50.

OECD (2013) PISA 2012 results: Excellence through equity giving every student the chance to succeed Vol. II. Paris, France: Organisation for Economic Co-operation and Development.

51.

Postlethwaite

T. N.

Kellaghan

(2008) National assessments of educational achievement, Paris, France: UNESCO.

52.

Purdie

Reid

Buckley

(2011) Literacy learning: What works for young indigenous students? Lessons from the longitudinal literacy and numeracy study for indigenous students. Literacy Learning: The Middle Years 19(3): 51–59.

53.

Rindermann

(2008) Relevance of education and intelligence at the national level for the economic welfare of people. Intelligence 36: 127–142.

54.

Sirin

S. R.

(2005) Socioeconomic status and academic achievement: A meta-analytical review of research. Review of Educational Research 75(3): 417–453.

55.

Snijders

T. A. B.

Bosker

R. J.

(2012) Multilevel analysis: An introduction to basic and advanced multilevel modelling, (2nd ed.). Los Angeles, CA: Sage.

56.

Stoet

Geary

D. C.

(2013) Sex differences in mathematics and reading achievement are inversely related: Within- and across-nation assessment of 10 years of PISA data. PLoS One 8(3): e57988.

57.

Thomson, S. (2007). Achievement of Australia's junior secondary school indigenous students: Evidence from TIMSS. In H. Wagemaker (Ed.), The second IEA international research conference: Proceedings of the Irc-2006 (Vol. 1 Trends in International Mathematics and Science Study). Amsterdam: International Association for the Evaluation of Student Achievement.

58.

Thomson, S., De Bortoli, L., & Buckley, S. (2013). PISA 2012: How Australia measures up. The PISA 2012 assessment of students’ mathematical, scientific and reading literacy. Melbourne, VIC: Australian Council for Educational Research.

59.

Thomson, S., De Bortoli, L., Nicholas, M., Hillman, K., & Buckley, S. (2010). Challenges for Australian education: Results from PISA 2009. The PISA 2009 assessment of students’ reading, mathematical and scientific literacy. Melbourne, VIC: Australian Council for Educational Research.

60.

Thomson

Hillman

Wernert

(2012) Monitoring Australian year 8 student achievement internationally: TIMSS 2011, Melbourne, VIC: Australian Council for Educational Research.

61.

Thomson

Hillman

Wernert

Schmid

Buckley

Munene

(2012) Monitoring Australian year 4 student achievement internationally: TIMSS and PIRLS 2011, Melbourne, VIC: Australian Council for Educational Research.

62.

Walberg

H. J.

(1984) Improving the productivity of America’s schools. Educational Leadership 41: 19–27.

63.

Warren, D., & Haisken-DeNew, J. P. (2013). Early bird catches the worm: The causal impact of pre-school participation and teacher qualifications on year 3 national NAPLAN cognitive tests (Melbourne Institute Working Paper Series no. 34/13). Melbourne: Melbourne Institute for Applied Economic and Social Research.

64.

White

K. R.

(1982) The relationship between socio-economic status and academic achievement. Psychological Bulletin 91(3): 461–481.

65.

Whitt

H. P.

(1986) The sheaf coefficient: A simplified and expanded approach. Social Science Research 15(2): 174–189.

66.

Young

D. J.

(1996) Rural and urban differences in student achievement in science and mathematics: A multilevel analysis. School Effectiveness and School Improvement 9(4): 386–409.

Demographic and socioeconomic inequalities in student achievement over the school career

Abstract

Keywords

Introduction

The Australian NAP

Purpose of present study

Background literature

Demographic differences in student achievement

Gender

Indigenous students

Language background

Region

Socioeconomic inequalities in student achievement

Ability and student achievement

Data, measures and methods

Data

Achievement measures

Other measures

Methods

Fixed effect models

Results

Gender

Indigenous status

Language background

Region

Socioeconomic background and prior achievement

Prior achievement

Conclusion and discussion

Footnotes

Declaration of conflicting interests

Funding

Notes

References