The changes in attitudes of 15-year-old Australian students towards reading,mathematics and science and their impact on student performance

Abstract

This paper investigates changes in 15-year-old Australian students’ attitudes towards reading mathematics, and science, as well as their relationships with students’ performance in these respective domains over the period of 2000–2018. multigroup confirmatory factor analysis was used to examine the measurement invariance of the construct of attitudes over the cycles of the Programme for International Student Achievement. In addition, structural equation modelling was used to explore mediating effects of attitudes on performance in the three domains. The results indicated that there were significant declines in student performance for all three domains. The levels of enjoyment of the subject domains were relatively low for reading and mathematics but relatively high for science. There were upward trends in the levels of enjoyment of mathematics and science over time, while levels of enjoyment of reading did not change significantly. Levels of instrumental motivation were relatively high and had slight upward trends for both mathematics and science. There were positive and significant relationships between attitudes and student performances.

Keywords

Attitudes enjoyment instrumental motivation reading mathematics science Programme for International Student Achievement

Introduction

The Programme for International Student Achievement (PISA) is a triennial worldwide study by the Organisation for Economic Co-operation and Development (OECD) that was first performed in 2000. Over the last two decades, 43, 49, 57, 65, 66, 72 and 79 countries and regions participated in PISA 2000, 2003, 2006, 2009, 2012, 2015 and 2018, respectively (OECD, 1999, 2003, 2006, 2009, 2013, 2017, 2019). PISA provides comprehensive and reliable indicators of students’ capabilities. It assesses 15-year-old students’ skills and knowledge in reading, mathematics and science, as well as their attitudes and other background variables. PISA can be used by policymakers in the participating countries to refine their educational policies (Schleicher, 2019).

Educators have long argued the importance of a literate society for the development of a nation. The positive relationship between cognitive skills, measured by mathematics and science scores on available international studies including PISA, and economic growth was reported by Hanushek and Woessmann (2010). Their report indicated that relatively small improvements in the cognitive skills of a nation’s labour force lead to increased innovation and productivity and, subsequently, long-term economic viability. In addition, there are other social benefits resulting from a society with strong reading, mathematical and scientific literacies, including the capability to tackle personal, social, environmental, health and wellbeing issues (Jaremus et al., 2019; Sadler & Zeidler, 2009). However, while the benefits have been acknowledged, the general decline of student engagement in mathematics and science has also been recognised (Bennett & Hogarth, 2009; Bybee & McCrae, 2011; Kennedy et al., 2014; Sjaastad, 2012). In Australia, there is evidence of a downward trend in the achievement of students in international surveys, such as the Trends in Mathematics and Science Study (Thomson et al., 2016) and the PISA (Morsy et al., 2018; Thomson et al., 2017, 2019).

Hattie’s (2009, 2012, 2015) meta-analytical studies on performance reported that around half of the learning variance was related to students’ background factors, including previous knowledge, motivation, commitment and, more significantly, the student’s attitudes towards a subject, amongst others. The investigation of students’ attitudes towards reading, mathematics and science forms a substantive and significant feature of numerous studies by the education research community over the past five decades (Crouch et al., 2018; Di Martino & Zan, 2011; Dutton, 1954; Kolić-Vehovec et al., 2014; McConney et al., 2014; McConney & Perry, 2010; McLeod, 1992; Neale, 1969). Consequently, this paper focuses on students’ attitudes towards mathematics, science and reading over time. Further, it investigates changes in the levels of students’ attitudes over the past 20 years and the associations between attitudes and student performance.

Research questions

Do the measurement models of attitudes towards reading, mathematics and science hold across groups and time?

Do attitudes towards reading, mathematics and science change over this period?

Do the relationships between attitudes and performance change over time?

Literature review

Attitudes towards reading, mathematics and science

Attitude is one of the key concepts in social psychology. It is generally defined as the tendency of an individual’s reaction towards a stimulus, either positively or negatively (Fishbein & Ajzen, 1977). Attitudes are commonly developed during the early stages of childhood and are influenced by parents, siblings and peers (Petty et al., 2003). The importance of attitude and its positive development towards a subject has been highlighted in research (Farooq & Shah, 2008; Hattie, 2009, 2012, 2015; Tapia & Marsh, 2004). Attitudes have a strong effect on students’ emotional response and behaviour, which in turn plays a role in determining their interest in, enjoyment of, and engagement with the subject. Subject-related interest is an intrinsic motivational preference that affects continuity and intensity of engagement with learning. The level of engagement and the amount of effort exerted, which is also contingent upon the kind of attitude and behaviour adopted by students, can also influence students’ levels of performance in a subject.

Attitudes towards reading

The literature indicates that there is a significant positive association between attitudes towards reading and academic achievement (Netten et al., 2011; Ozonder, 2015; Petscher, 2010). Research in this field has been extended further to find out which aspects of attitudes towards reading are most significant in predicting reading performance (Smith et al., 2018). Some studies have found that intrinsic motivation is the aspect that has the strongest positive correlation with reading achievement. This intrinsic motivation can be reflected, for example, in the levels of enjoyment or interest in engaging in a task (Dhanapala & Hirakawa, 2016). However, other studies have also indicated that rather than the intrinsic aspect, the extrinsic or instrumental motivation aspect has a stronger positive correlation with reading ability (Kim, 2011; Lin et al., 2012).

Attitudes towards mathematics

The concept of attitudes towards mathematics can be traced back to a study by Neale (1969). He defined mathematical attitude as “a liking or disliking of mathematics, a tendency to engage in or avoid mathematical activity, a belief that one is good or bad at mathematics, and a belief that mathematics is useful or useless” (p. 632). This concept of attitudes towards mathematics was well received and widely recognised (Ma & Kishor, 1997; Zan & Di Martino, 2007). More recently, attitudes towards mathematics have been examined extensively (Haladyna et al., 1983; Hannula, 2002; Hart, 1989; Ma & Kishor, 1997). Hart (1989) considers attitudes towards mathematics as the complex emotions and beliefs that an individual has in relation to mathematics. These complex emotions are then followed by an individual’s behaviour towards mathematics, which may include anxiety about and fearfulness of the subject. Haladyna et al. (1983) define attitudes towards mathematics as “a general emotional disposition towards the school subject of mathematics” (p. 20). Ma and Kishor (1997) include students’ affective responses to the difficulty as well as the importance of mathematics. Hannula (2002) states that attitudes towards mathematics can also be seen as an emotional disposition towards the subject. Hemmings and Kay (2010), in their study using a sample of Australian Year 10 students, found that attitudes towards mathematics were significantly and positively associated with mathematics performance.

Attitudes towards science

Relationships between student attitudes towards science and science performance have been explored extensively, particularly in science education research over the past two decades (Fonseca et al., 2011; McConney et al., 2014; Tytler & Osborne, 2012). The results so far show that there is no consensus in this area. Some studies reported that there is a positive relationship between the two constructs (Singh et al., 2010; Tran, 2011). In contrast, many studies indicate that only a weak relationship exists between student attitudes towards science and science performance (Osborne et al., 2003). Analysis of PISA results has suggested that more positive attitudes towards science do not necessarily correspond to a higher science performance (Bybee & McCrae, 2011; McConney et al., 2014). Fonseca et al. (2011) in their cross-national study of PISA 2006 science performance found that, generally, students in countries with better performance were less positive in their attitudes towards science than students in lower performing countries. Despite these inconclusive results, the important role of attitudes towards science in science performance for Australian students is worth exploring.

Methodology

Sample

Sampling in PISA was carried out using a two-stage sampling framework in most countries, including Australia. In the first stage, schools were selected based on a probability that was proportional to the school’s enrolment of 15-year olds. During the second stage, students were sampled within sampled schools. The number of schools and students selected to participate in PISA 2000–2018 is presented in Table 1.

Table 1.

Australian sample: PISA 2000–2018.

Sample	PISA cycle
Sample	2000	2003	2006	2009	2012	2015	2018
Schools	231	321	356	353	775	758	740
Students	6200	12,500	14,000	14,000	14,481	14,530	14,273

Source: OECD (1999, 2003, 2006, 2009, 2013, 2017, 2019).

PISA, Programme for International Student Achievement.

Variables

In PISA studies, attitudes as a construct consist of several sub-constructs. In this study, longitudinal analyses were undertaken and only those sub-constructs used across multiple cycles, also referred to as the trend constructs, were included. In 2000, 2009 and 2018, the trend construct measured for reading is engagement in reading (also labelled as enjoyment of reading). For mathematics and science, two trend constructs related to attitudes towards each domain were investigated. For mathematics, this included interest in and enjoyment of mathematics, and instrumental motivation towards mathematics. For science, this included enjoyment of science, and instrumental motivation towards science. To simplify the names of these constructs, engagement in reading, interest in and enjoyment of mathematics and enjoyment of science are labelled as enjoyment of reading, enjoyment of mathematics and enjoyment of science for the remainder of this text, respectively. The names, description and codes of the variables included in this study are presented in Appendix 1. Only items that are common to all cycles under investigation were included in this study.

Hypothesised model

The hypothesised models investigated in this study are presented in Figures 1 and 2. In Model 1, it is hypothesised that the levels of students’ attitudes towards a particular subject domain, as measured by enjoyment of reading, as well as instrumental motivation for mathematics and science, will influence their respective literacy performance. A series of multigroup structural equation modelling (SEM) analyses was undertaken to investigate the changes in the magnitude and direction of the relationships amongst enjoyment, instrumental motivation and literacy performance for reading, mathematics and science over time. In Model 2, data from different cycles were combined into a single analysis, and PISA cycle as a variable was added to the model to investigate trends in the levels of enjoyment, instrumental motivation and literacy performances over time. For reading, PISA cycle is coded as 0 for PISA 2000, 1 for PISA 2009 and 2 for PISA 2018. For mathematics, PISA cycle is coded as 0 for PISA 2003 and 1 for PISA 2012. For science, PISA cycle is coded as 0 for PISA 2006 and 1 for PISA 2015.

Figure 1.

Hypothetical Model 1: trends in the relationships between attitudes and performance.

Figure 2.

Hypothetical Model 2: trends in the levels of attitudes and performances.

Methods of analyses

There were three stages of data analyses in this study. In the first stage, a multigroup confirmatory factor analysis (M-CFA) was adopted to test the measurement invariance of the enjoyment and instrumental motivation constructs. The purpose of this M-CFA was to confirm that the same underlying constructs were being measured across groups and across time (Byrne, 2012; Milfont & Fischer, 2010). Based on Milfont and Fischer’s (2010) guidelines, a series of CFA analyses was first performed on each sample independently to confirm that the factor structure of enjoyment of reading, enjoyment and instrumental motivation of mathematics, as well as enjoyment and instrumental motivation of science fit their respective data sets well. Afterward, following the suggestion of Bialosiewicz et al. (2013), four levels of measurement invariance were tested. First, configural invariance was tested in which all factor loadings and item intercepts were freely estimated amongst multiple samples. This model can be considered as the baseline model. Second, metric invariance was tested by constraining all factor loadings to be equal between groups, while allowing the intercepts of the indicators to be freely estimated. This model is commonly referred to as weak invariance. Third, scalar invariance was tested by requiring that the intercepts for the indicators were also constrained to be equal across groups. This model is commonly referred to as strong invariance. Lastly, the final level of invariance, the strict invariance, was undertaken by constraining the factor variances to be equal.

As for the chi-square comparison between models, following the Mplus manual (Muthén & Muthén, 2017), the following steps were used to compute a chi-square difference test using the MLM or MLR (Satorra–Bentler) chi-square:

Compute the difference test scaling correction c_d

c_{d} = \frac{(d_{0} \times c_{0} - d_{1} \times c_{1})}{{(d}_{0} - d_{1})}

where d₀ is the degrees of freedom in the nested model, c₀ is the scaling correction factor for the nested model, d₁ is the degrees of freedom in the comparison model and c₁ is the scaling correction factor for the comparison model.

2. Compute the Satorra–Bentler scaled chi-square difference test $χ_{d}^{2}$ as follows

χ_{d}^{2} = \frac{(χ_{0}^{2} \times c_{0} - χ_{1}^{2} \times c_{1})}{c_{d}}

In the second stage, a multigroup SEM (M-SEM) approach, as presented in Figure 1, was used. A series of M-SEM analyses was undertaken to investigate the changes in the strength and the direction of the relationships between attitudes and performances. In the third stage, the differences of the estimates amongst the PISA cycles and the trends in the levels of enjoyment, instrumental motivation and performances for reading, mathematics and science across time were examined by analysing a series of SEM models as presented in Figure 2. In addition, overall estimations of the relationships between these attitude components and performance after controlling for the differences mentioned earlier were also sought.

The fit of the CFA and SEM models was assessed based on five goodness-of-fit statistics which include normed or relative chi-square (χ²/df), Comparative Fit Index (CFI), Tucker Lewis Index (TLI), Root Mean Square Error of Approximation (RMSEA) and Standardised Root Mean Square Residual (SRMR). There is no consensus for the normed or relative chi-square regarding the acceptable ratio for this statistic. However, current recommendations range from as high as 5.0 (Wheaton et al., 1977) to as low as 2.0 (Tabachnick & Fidell, 2007). The CFI and TLI values range between 0.0 and 1.0, with values closer to 1.0 indicating a good fit. A value greater than 0.90 is needed (Hu & Bentler, 1999). As for the RMSEA and SRMR, the consensus amongst authorities in this area is to use a cut-off value close to 0.06 (Hu & Bentler, 1999) or a strict upper limit of 0.07 (Steiger, 2007).

Mplus 8 statistical programme was used for both CFA and SEM analyses. In conducting the analyses, the cluster and the stratification options were employed to consider the two-stage sampling framework used in PISA studies instead of using the replicate weights. This approach was taken because chi-square estimate is not available when replicate weights are used. In this article, schools were regarded as clusters and the strata variable was used for the stratification option. The data imputation command in Mplus also makes it possible to use plausible values which provides appropriate standard errors and χ² estimates (Muthén & Muthén, 2017). From 2000 until 2012, PISA provided five plausible values for students’ performances. However, since 2015, an additional five plausible values have been provided. Therefore, the analyses of SEM Model 1 and Model 2, which require a similar number of imputations for the cycles included in the analyses, only five plausible values could be used. The Mplus default option for dealing with missing values was adopted in the study. With this option, Mplus uses all data that are available to estimate the model using full information maximum likelihood (Muthén & Muthén, 2017).

Results

Measurement invariance

To answer the first research question, a series of CFA analyses was performed on each sample independently and fit indices of these separate CFA models are presented in Table 2. The results show that, in general, the model fit was acceptable for all samples. The CFI and TLI values were mostly above 0.9 and only the TLI values for reading in PISA 2000 and PISA 2015 were below 0.9, but still above 0.8. The RMSEA values were acceptable except for reading for all cycles in which the values were slightly above 0.1. The SRMR values were all below 0.05. The χ² values are sensitive to sample size. Considering the sample sizes used in this study are relatively large, most of the time significant results were observed.

Table 2.

Model fit of the independent CFA models.

Measurement model for attitudes towards	PISA cycle	χ²	df	χ²/df	CFI	TLI	RMSEA	SRMR
Reading	2000	418.559	5	83.712	0.945	0.891	0.127	0.036
	2009	946.585	5	189.317	0.969	0.938	0.116	0.025
	2018	1672.553	5	334.511	0.923	0.847	0.163	0.037
Mathematics	2003	903.879	19	47.573	0.982	0.974	0.061	0.031
Mathematics	2012	673.778	19	35.462	0.983	0.975	0.060	0.026
Science	2006	973.983	26	37.461	0.989	0.984	0.051	0.012
Science	2015	934.540	26	35.944	0.986	0.981	0.052	0.013

CFA, confirmatory factor analysis; PISA, Programme for International Student Achievement; CFI, Comparative Fit Index; TLI, Tucker Lewis Index; RMSEA, Root Mean Square Error of Approximation; SRMR, Standardised Root Mean Square Residual.

The next step was to assess the measurement invariance using M-CFA to confirm that the same underlying construct was being measured across groups and across time. As mentioned earlier, four levels of measurement invariance were tested and the fit indices along with their changes are presented in Table 3. The changes in the CFI and TLI values were all less than 0.01. Changes in the RMSEA and SRMR values were also less than 0.01, except for the strict models of reading and science, where the SRMR values increased by 0.026 and 0.029 for reading and science, respectively, when the models were compared to their scalar invariance models. As mentioned earlier, due to the large sample sizes, the changes in the adjusted χ² values using the Satorra–Bentler method described above are still found to be significant. Despite this, considering the sample sizes, it can then be argued that all the constructs behave in the same way across time and groups. They all can be considered as having a strong measurement invariance. In other words, the measurement models of attitudes towards reading, mathematics and science hold across groups and time. Therefore, the comparisons between occasions can be made meaningfully.

Table 3.

Model comparisons for measurement invariance models.

No.	Model	Comparative model	$χ^{2}$	df	$χ_{d}^{2}$	Δ df	p	CFI	Δ CFI	TLI	Δ TLI	RMSEA	Δ RMSEA	SRMR	Δ SRMR
	Reading
1	Configural		3485.244	24				0.940		0.925		0.117		0.049
2	Metric	2-1	3619.485	28	175.598	4	0.000	0.938	–0.002	0.933	0.008	0.110	–0.007	0.053	0.004
3	Scalar	3-2	3917.823	33	320.749	5	0.000	0.933	–0.005	0.939	0.006	0.106	–0.004	0.052	–0.001
4	Strict	4-3	4083.718	35	194.066	2	0.000	0.930	–0.003	0.940	0.001	0.105	–0.001	0.078	0.026
	Mathematics
1	Configural		1574.358	38				0.982		0.974		0.061		0.029
2	Metric	2-1	1632.355	44	57.997	6	0.000	0.982	0.000	0.977	0.003	0.057	–0.004	0.031	0.002
3	Scalar	3-2	1927.570	52	295.215	8	0.000	0.978	–0.004	0.977	0.000	0.057	0.000	0.039	0.008
4	Strict	4-3	1974.770	55	47.200	3	0.000	0.978	0.000	0.978	0.001	0.056	–0.001	0.041	0.002
	Science
1	Configural		1905.586	52				0.987		0.982		0.052		0.013
2	Metric	2-1	2141.852	59	242.8513	7	0.000	0.986	–0.001	0.982	0.000	0.051	–0.001	0.021	0.008
3	Scalar	3-2	3004.805	68	739.5677	9	0.000	0.980	–0.006	0.979	–0.003	0.057	0.006	0.028	0.007
4	Strict	4-3	3253.181	71	199.7322	5	0.000	0.978	–0.002	0.978	–0.001	0.058	0.001	0.057	0.029

CFI, Comparative Fit Index; TLI, Tucker Lewis Index; RMSEA, Root Mean Square Error of Approximation; SRMR, Standardised Root Mean Square Residual.

Measurement model results: Model 1 and Model 2

Before the SEM models for hypothesised Models 1 and 2 were analysed, measurement models of each of hypothesised model for reading, mathematics and science were examined to estimate the Pearson product moment correlation coefficients. The correlation coefficients of the constructs are reported in Table 7. After this step, the SEM models were developed by adding the causal paths. The results for the measurement model as well as the structural model are discussed in the following sections. The descriptive statistics for each of the items and the factor loadings for the measurement models of Model 1 and Model 2 are recorded in Tables 4, 5 and 6 for reading, mathematics and science, respectively.

Table 4.

Attitudes towards reading: enjoyment of reading.

	Descriptive statistics						Model 1			Model 2
Enjoyment	2000		2009		2018		2000	2009	2018
Enjoyment	Mean	SE	Mean	SE	Mean	SE	Loading	Loading	Loading	Loading
I read only if I have to (–)	2.751	0.017	2.666	0.008	2.434	0.008	0.803	0.852	0.822	0.835
Reading is one of my favourite hobbies	2.168	0.016	2.175	0.009	2.139	0.009	0.793	0.830	0.827	0.823
I like talking about books with other people	2.095	0.015	2.18	0.009	2.111	0.009	0.713	0.737	0.727	0.730
For me, reading is a waste of time (–)	3.020	0.015	2.982	0.008	2.865	0.008	0.752	0.789	0.778	0.780
I read only to get information that I need (–)	2.596	0.016	2.607	0.008	2.46	0.008	0.732	0.776	0.740	0.757

Note: All factor loadings are significant at p < 0.05.

Table 5.

Attitudes towards mathematics: enjoyment of mathematics and instrumental motivation.

	Descriptive statistics				Model 1		Model 2
	2003		2012		2003	2012
	Mean	SE	Mean	SE	Loading	Loading	Loading
Enjoyment of mathematics
I enjoy reading about mathematics	2.082	0.007	2.201	0.007	0.760	0.780	0.770
I look forward to my mathematics lessons	2.215	0.008	2.381	0.008	0.850	0.861	0.855
I do mathematics because I enjoy it	2.216	0.008	2.292	0.008	0.904	0.904	0.905
I am interested in the things I learn in mathematics	2.474	0.007	2.546	0.007	0.817	0.844	0.830
Instrumental motivation to learn mathematics
Making an effort in mathematics is worth it because it will help me in the work that I want to do later on	3.073	0.006	3.141	0.006	0.820	0.841	0.830
Learning mathematics is worthwhile for me because it will improve my career	3.134	0.006	3.16	0.006	0.853	0.856	0.854
Mathematics is an important subject for me because I need it for what I want to study later on	2.969	0.007	2.983	0.007	0.821	0.836	0.828
I will learn many things in mathematics that will help me get a job	2.982	0.006	3.047	0.006	0.784	0.807	0.794

Note: All factor loadings are significant at p < 0.05.

Table 6.

Attitudes towards science: enjoyment and instrumental motivation.

	Descriptive statistics				Model 1		Model 2
	2006		2015		2006	2015
	Mean	SE	Mean	SE	Loading	Loading	Loading
Enjoyment of science
I generally have fun when I am learning <broad science> topics	2.582	0.012	2.644	0.011	0.866	0.887	0.876
I like reading about <broad science>	2.359	0.012	2.464	0.011	0.839	0.855	0.848
I am happy doing <broad science> problems	2.451	0.012	2.678	0.011	0.846	0.925	0.884
I enjoy acquiring new knowledge in <broad science>	2.712	0.012	2.774	0.011	0.886	0.915	0.902
I am interested in learning about <broad science>	2.655	0.012	2.725	0.011	0.895	0.918	0.908
Instrumental motivation to learn science
It will help me in the work I want to do later on	2.783	0.011	2.861	0.009	0.887	0.903	0.895
I need this for what I want to study later on	2.604	0.011	2.728	0.009	0.901	0.929	0.915
It will improve my career prospects	2.733	0.011	2.803	0.009	0.904	0.919	0.911
It will help me get a job	2.679	0.011	2.711	0.009	0.893	0.874	0.882

Note: All factor loadings are significant at p < 0.05.

Table 7.

Structural model results for Model 1.

Dependent variable	Independent variable	Unstandardised estimates						Standardised estimates
		2000		2009		2018		2000			2009			2018
		b	SE	b	SE	b	SE	r	β	SE	r	β	SE	r	β	SE
Reading
Reading	Enjoyment	61.113	2.400	65.797	1.281	51.731	1.293	0.440	0.440	0.016	0.539	0.539	0.008	0.403	0.403	0.009
Mathematics		2003		2012				2003			2012
Ins. Motiv.	Enjoyment	0.605	0.012	0.660	0.011			0.591	0.591	0.008	0.660	0.660	0.007
Mathematics	Enjoyment	21.483	2.535	25.641	2.743			0.199	0.131	0.015	0.236	0.166	0.018
	Ins. Motiv.	18.404	2.136	16.314	2.625			0.192	0.115	0.013	0.215	0.106	0.017
Science		2006		2015				2006			2015
Ins. Motiv	Enjoyment	0.750	0.011	0.494	0.011			0.668	0.668	0.007	0.490	0.490	0.010
Science	Enjoyment	59.557	2.258	48.482	1.416			0.431	0.414	0.015	0.380	0.385	0.010
	Ins. Motiv.	3.416	1.814	–1.364	1.419			0.303	0.027	0.014	0.178	–0.011	0.011

Note: Shaded estimates did not reach statistical significance (i.e. p > 0.05).

Reading

A large proportion of 15-year-old Australian students read only to get information they need (43% responding agree or strongly agree in 2000, 43% in 2009 and 53% in 2018) or only if they had to (38% in 2000, 42% in 2009 and 54% in 2018), and about 23% in 2000, 26% in 2009 and 32% in 2018 of the students reported that reading was a waste of time. Around one-third (33% in 2000, 35% in 2009 and 33% in 2018) of the students considered reading to be one of their favourite hobbies. After reversing the responses for the negative statements, the mean estimates along with the standard error of the mean for the five items included to measure the enjoyment of reading are presented in Table 4. The factor loadings are all above 0.7, indicating that the five items are very good reflectors of the enjoyment of reading construct.

Mathematics

All the items used to measure attitudes towards mathematics were positively worded. However, the initial coding scheme were 1 for strongly agree, 2 for agree, 3, for disagree and 4 for strongly disagree. For easy interpretation using a positive scale, these responses were reversed. The higher the values, the more positive the attitudes are. Less than half of 15-year-old Australian students enjoyed reading about mathematics (27% responding agree or strongly agree in 2003 and 33% in 2012) or did math because they enjoyed it (36% in 2003 and 45% in 2012). About 35% in 2003 and 38% in 2012 of the students reported that they looked forward to their mathematics lessons. Slightly more than half (50% in 2003 and 54% in 2012) of the students were interested in what they learnt in mathematics. However, Australian students reported a relatively high level of instrumental motivation. Their agreement (including strongly agree and agree responses) with all four items was around and above 75%. The estimated means along with the standard error of the mean for the eight items used to measure enjoyment of mathematics and instrumental motivation to learn mathematics are presented in Table 5. The factor loadings are all above 0.7 indicating that the items used to reflect both constructs are very good.

Science

The items used to measure attitudes towards science were positively worded and the initial coding scheme were similar to the items used to measure attitudes towards mathematics therefore these responses were reversed. More than half of 15-year-old Australian students had fun when they were learning science (58% in 2006 and 64% in 2015) or they were happy doing science problems (50% in 2006 and 66% in 2015). About 43% in 2006 and 51% in 2015 of the students reported that they liked reading about science. More than half (61% in 2006 and 66% in 2012) of the students were interested in learning about science. In addition, Australian students also reported a relatively high level of instrumental motivation. Their agreement towards all four items ranged from 55% to 70%. The means along with their standard errors for the nine items included to measure the enjoyment of science and instrumental motivation to learn science are presented in Table 6. The factor loadings are all above 0.7 indicating that the five items are good reflectors of the enjoyment of science construct.

Structural model results: Model 1

The structural model results of the multigroup SEM models, Model 1, are presented in Table 7. The results indicated that there were positive and significant correlations between attitudes and performance for all three domains of reading, mathematics and science. The path coefficient for enjoyment of reading and reading performance was 0.440 (SE = 0.016) in 2000. It became stronger in 2009 (β = 0.539, SE = 0.008) but became weaker again in 2018 (β = 0.403, SE = 0.009) explaining about 18%, 26% and 13% of the variances of reading performance in 2000, 2009 and 2018, respectively.

There were positive and significant correlations between enjoyment of mathematics as well as instrumental motivation and mathematics performance. SEM results indicate that for the effect of enjoyment on mathematics performance, the path coefficient increased from 0.131 (SE = 0.015) in 2006 to 0.166 (SE = 0018) in 2012, respectively. The path coefficients from enjoyment to instrumental motivation increased from 0.591 (SE = 0.008) to 0.660 (SE = 0.007). After controlling for the enjoyment, the path coefficients between instrumental motivation and mathematics performance decreased from 0.115 (SE = 0013) to 00.106 (SE = 0017). Both predictors explained around 4.8% and 6.0% of variances of the mathematics performances in 2003 and 2012, respectively.

There were positive and significant correlations between both enjoyment of mathematics and instrumental motivation with mathematics performance, but these correlations became weaker over time. The results presented in Table 7 indicated that the path coefficient between enjoyment and scientific literacy performance decreased from 0.414 (SE = 0.015) in 2006 to 0.385 (SE = 0.010) in 2015. The effect of enjoyment on instrumental motivation was strong but decreased over time from 0.668 (SE = 007) in 2006 to 0.490 (SE = 0.010) in 2015. After controlling for enjoyment, the path coefficients between instrumental motivation and scientific literacy performance were no longer significant in either cycle. The models explained about 17% and 15% of the variances of scientific performances in 2006 and 2015, respectively.

Structural model results: Model 2

In the last stage, hypothesised Model 2 was analysed to investigate the differences of the estimates amongst the PISA cycles and the trends in the levels of enjoyment, instrumental motivational and performance in reading, mathematics and science across time and groups. The results of these analyses are presented in Table 8 while the three SEM models of reading, mathematics and science along with their standardised results are presented in Figures 3, 4 and 5, respectively.

Table 8.

Structural model results for Model 2.

Dependent variable	Independent variable	Unstandardised		Standardised
Dependent variable	Independent variable	b	SE	r	β	SE	p
Reading
Enjoyment	Cycle	–0.088	0.009	–0.075	–0.106	0.011	0.000
Reading	Cycle	–5.152	1.581	–0.070	–0.049	0.015	0.000
	Enjoyment	59.099	0.846	0.467	0.465	0.006	0.000
Mathematics
Enjoyment	Cycle	0.087	0.012	0.070	0.070	0.010	0.000
Ins. Motiv	Cycle	–0.017	0.008	0.030	–0.014	0.007	0.039
	Enjoyment	0.631	0.009	0.623	0.624	0.005	0.000
Mathematics	Cycle	–31.726	3.112	–0.146	–0.160	0.015	0.000
	Enjoyment	23.245	1.834	0.202	0.144	0.011	0.000
	Ins. Motiv.	17.679	1.646	0.196	0.111	0.010	0.000
Science
Enjoyment	Cycle	0.087	0.015	0.057	0.057	0.010	0.000
Ins. Motiv	Cycle	0.009	0.010	0.038	0.005	0.006	0.385
	Enjoyment	0.603	0.008	0.569	0.568	0.006	0.000
Science	Cycle	–29.440	2.783	–0.118	–0.141	0.013	0.000
	Enjoyment	54.183	1.227	0.396	0.396	0.008	0.000
	Ins. Motiv.	1.714	1.126	0.233	0.013	0.009	0.128

Note: Shaded estimates did not reach statistical significance (i.e. p > 0.05).

Figure 3.

Standardised results for reading (Model 2).

Figure 4.

Standardised results for mathematics (Model 2).

Figure 5.

Standardised results for science (Model 2).

In general, using the combined data, the results indicated that enjoyment has positive effects on reading (β = 0.465, SE = 0.006), mathematics (β = 0.144, SE = 0.011) and science (β = 0.396, SE = 0.006) performances. The effect of instrumental motivation on mathematics performance was positive (β = 0.111, SE = 0.010). However, the effect of instrumental motivation on science performance was not significant. There were strong positive effects of enjoyment on instrumental motivation (β = 0.624, SE = 0.005) for mathematics and (β = 0.568, SE = 0.006) for science.

The trends in enjoyment, instrumental motivation and literacy performances can be inferred from the effects of cycle on these constructs. Results in Table 8 indicate that the level of enjoyment of reading declined slightly but significantly over time (β=-0.106, SE = 0.011). The level of enjoyment for both mathematics (β = 0.070, SE = 0.010) and science (β = 0.57, SE = 0.010) increased slightly but significantly over time. Instrumental motivation remained constant over time for both mathematics and science. It was also estimated that, taking into account the effect of attitudes, there were linear declines in a nine-year period by 5.152 score points (b = –5.151, β = –0.049, SE = 0.015) for reading, by 31.726 score points (b = –31.726, β = –0.160, SE = 0.015) for mathematics and by 29.440 score points (b = –29.440, β = –0.141, SE = 0.013) for science. Based on PISA 2018 results, these score point differences can be associated with years of schooling. Thomson et al. (2019) indicated that one year of schooling is equivalent to, on average, around 33, 28 and 27 score points for reading, mathematics and science, respectively. Therefore, accounting for effects of attitudes on performance and their trends over time, the linear declines for these nine-year periods for reading, mathematics and science are equivalent to 2, 13.5 and 13 months of schooling, respectively.

Model fit

The model fit indices for the measurement and SEM models for all domains are recorded in Appendix 2. The results showed that in general the model fit was acceptable for all models. The CFI and TLI values were mostly above 0.9. The RMSEA values were acceptable and SRMR values were all below 0.05.

Discussion and conclusion

The present study investigated the trends in student attitudes and literacy performances in reading, science and mathematics over the past 20 years using the data from the seven cycles of PISA studies. It has been reported in various studies that in general, there is a steady decline in reading, mathematics and science literacy performances of Australian 15-year-old students (Thomson et al., 2011, 2013, 2019). From the first assessment of reading as a major domain in 2000–2018, when reading was again a major domain, the average score has declined by 26 score points, equivalent to almost nine and a half months of schooling. Mathematical literacy was a major domain for the first time in 2003 when Australia achieved a mean score of 524. A consistent decline over the intervening cycles has resulted in an overall drop of 33 score points, equivalent to around 14 months of schooling. A downward trend was also observed in scientific literacy, from an average of 527 score points in 2006 to 503 score points in 2018, equivalent to almost 11 months of schooling. When the effects of attitudes on performance and their trend over time were taken into account, the adjusted linear declines for every nine-year period for reading, mathematics and science are estimated to be equivalent to 2, 13.5 and 13 months of schooling, respectively. To put this into perspective, according to Hanushek and Woessmann (2010), a change of 25 points in PISA has been estimated to be worth AUD3.6 trillion to Australia’s GDP. Morsy et al. (2018) reviewed various plausible explanations for these declining trends. These include, amongst others, the declining teacher quality where teachers are teaching out-of-field, the increase in privatisation of Australia’s education that causes a shift of students from government to Catholic and Independent schools and the decrease in time spent on a subject. However, they found no conclusive answer as to why, on average, students in Australia are performing worse over time. More research will be needed to investigate this further.

With regards to attitudes, the first research question centred on whether the same underlying constructs, in this case enjoyment and instrumental motivation, were being measured across groups and across time. From the results of the measurement invariance testing, it was found that these constructs behave in the same way over time. With this confirmation, the trends in attitudes and their effects on performances across PISA cycles can be made meaningfully.

In addressing the second and the third research questions regarding the changes in attitudes and their impacts on performance over time, two hypothetical SEM models were analysed. The results reveal that Australian 15-year-old students’ attitudes towards reading, which are expressed in terms of their enjoyment of reading, are relatively low. Around half of 15-year-old Australian students read only to get information they need or only if they had to, and one-third of the students reported that reading was a waste of time in 2000. The proportions of students that hold these negative attitudes appear to be growing over time. In terms of the impact of attitudes on performance, enjoyment of reading had a relatively strong effect on reading performance in 2000. This relationship became stronger in 2009. However, in 2018, it became slightly weaker than that of 2000, but still positive. This positive association between enjoyment of reading and reading performance is consistent with previous findings (Netten et al., 2011; Ozonder, 2015; Petscher, 2010).

For mathematics, the proportions of students agreeing and strongly agreeing to the majority of the items used to reflect the enjoyment of mathematics are below 50%, which indicates that the majority of students did not enjoy mathematics subject as such. In contrast, for most of the items related to instrumental motivation, around 75% of students indicated that they are aware of the importance of mathematics in their current and future life. As for science, the proportions of students agreeing or strongly agreeing to the items used to reflect the enjoyment of science are slightly better those for mathematics and are all above 50%. However, for items used to measure instrumental motivation towards science, the proportions of students agreeing or strongly agreeing are lower than those for mathematics, ranging from 55% to 70%. Despite declining students’ literacy performances in the domains of mathematics and science, students’ attitudes towards these two subject domains have become increasingly positive over time.

Attitudes towards mathematics, reflected by the two constructs of enjoyment of mathematics and instrumental motivation, are significantly and positively associated with students’ mathematics performance, consistent with the findings of Hemmings and Kay (2010). However, in general, these positive associations were getting weaker over time and are not as strong as the association between attitudes towards reading and reading performance. In terms of the impacts of attitudes towards science on science performance, it was found that enjoyment had a moderate positive effect on science performance. This association became weaker over time. However, after controlling for enjoyment, the effect of instrumental motivation on performance was not significant in either 2006 or 2015. Similar results have been reported by Bybee and McCrae (2011), McConney et al. (2014) and Osborne et al. (2003).

Having described the results above, it worth noting that conducting longitudinal analyses using PISA data is challenged by a range of issues. Each round of the survey focuses on a major domain, either reading, mathematical or scientific literacy. From its inception in 2000, definitions of reading, mathematical and scientific literacies have changed slightly to reflect changes in the contexts of our lives, so comparisons from one 9-year cycle to the next are not always straightforward. The instruments used and the constructs included to measure attitudes have also changed over time. Only a small number of trend constructs were included, which limits what aspect of attitudes can be included in this study. In interpreting the magnitude of the effects mentioned earlier, the exclusion of other important factors, such as cognitive ability or prior performance (Marks, 2017), economic, social, and cultural status (ESCS) (McConney & Perry, 2010; Perry & McConney, 2010), ethnicity (Song et al., 2014) and rurality (Sullivan et al., 2014, 2018), may inflate the real effect of attitudes on performance. Nonetheless, from the above findings, it can be argued that students can develop effective study habits if they find it enjoyable. Enjoyment is positively influenced by students’ interest and motivation to learn. It increases their desire to engage further with topics they are working on; to follow up the topic and acquire further knowledge and understanding. It impacts on how often and how willingly they engage with the subject. Positive emotions, such as enjoyment of learning, are also assumed to strengthen students’ instrumental or extrinsic motivation. Students’ enjoyment of learning develops out of their previous experiences in learning and function in new situations as expectations or predictions about further experiences. These attitudes, in turn, positively influence student performance. Therefore, these results point to the importance of both early experiences with learning processes and the maintenance of a sense of fun and excitement while learning the subject in supporting our attempt to lift student performance.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

I Gusti Ngurah Darmawan

References

Bennett

Hogarth

(2009). Would you want to talk to a scientist at a party? High school students’ attitudes to school science and to science. International Journal of Science Education, 31(14), 1975–1998.

Bialosiewicz

Murphy

Berry

(2013). An introduction to measurement invariance testing: Resource packet for participants. http://comm.eval.org/HigherLogic/System/DownloadDocumentFile.ashx?

Bybee

McCrae

(2011). Scientific literacy and student attitudes: Perspectives from PISA 2006 science. International Journal of Science Education, 33(1), 7–26.

Byrne

B. M.

(2012). Structural equation modeling with Mplus: Basic concepts, applications and programming. Taylor & Francis.

Crouch

C. H.

Wisittanawat

Cai

Renninger

K. A.

(2018). Life science students’ attitudes, interest, and performance in introductory physics for life sciences: An exploratory study. Physical Review Physics Education Research, 14(1), 010111. https://doi.org/10.1103/PhysRevPhysEducRes.14.010111

Dhanapala

K. V.

Hirakawa

(2016). L2 reading motivation among Sri Lankan university students. Reading Psychology, 37(2), 202–229.

Di Martino

Zan

(2011). Attitude towards mathematics: A bridge between beliefs and emotions. ZDM. The International Journal on Mathematics Education, 43, 471–482.

Dutton

W. H.

(1954). Measuring attitude toward arithmetic. Elementary School Journal, 55(1), 24–31.

Farooq

M. S.

Shah

S. Z.

(2008). Students’ attitude towards mathematics. Pakistan Economic and Social Review, 46, 75–83.

10.

Fishbein

Ajzen

(1977). Attitude–behavior relations: A theoretical analysis and review of empirical research. Psychological Bulletin, 84(5), 888–918.

11.

Fonseca

Valente

M. O.

Conboy

(2011). Student characteristics and PISA science performance: Portugal in cross national comparison. Procedia – Social and Behavioral Sciences, 12, 322–329.

12.

Haladyna

Shaughnessy

J. M.

(1983). A causal analysis of attitude toward mathematics. Journal for Research in Mathematics Education, 14(1), 19–29.

13.

Hannula

M. S.

(2002). Attitude towards mathematics: Emotions, expectations, and values. Educational Studies in Mathematics, 49(1), 25–46.

14.

Hanushek

E. A.

Woessmann

(2010). The high cost of low educational performance: The long run economic impact of improving PISA outcomes. PISA, OECD Publishing.

15.

Hart

L. E.

(1989). Describing the affective domain: Saying what we mean. In McLeod

D. B.

Adams

V. M.

(Eds.), Affect and mathematical problem solving: A new perspective (pp. 37–45). Springer-Verlag.

16.

Hattie

(2009). Visible learning a synthesis of over 800 meta-analyses relating to achievement. Routledge.

17.

Hattie

(2012). Visible learning for teachers: Maximizing impact on learning. Routledge.

18.

Hattie

(2015). The applicability of visible learning to higher education. Scholarship of Teaching and Learning in Psychology, 1, 79–91. https://doi.org/10.1037/stl0000021

19.

Hemmings

Kay

(2010). Prior achievement, effort, and mathematics attitude as predictors of current achievement. The Australian Educational Researcher, 37(2), 41–58. https://doi.org/10.1007/BF03216921

20.

L. T.

Bentler

P. M.

(1999). Cut off criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55.

21.

Jaremus

Gore

Fray

Prieto-Rodriguez

(2019). Senior secondary student participation in STEM: Beyond national statistics. Mathematics Education Research Journal, 31, 151–173. https://doi.org/10.1007/s13394-018-0247-5

22.

Kennedy

Lyons

Quinn

(2014). The continuing decline of science and mathematics enrolments in Australian high schools. Teaching Science, 60(2), 34–46.

23.

Kim

K. J.

(2011). Reading motivation in two languages: An examination of EFL college students in Korea. Reading and Writing, 24(8), 861–881.

24.

Kolić-Vehovec

Zubković

B. R.

Pahljina-Reinić

(2014). Development of metacognitive knowledge of reading strategies and attitudes toward reading in early adolescence: The effect on reading comprehension. Psihologijske Teme, 23(1), 77–98.

25.

Lin

Wong

K. K.

McBride-Chang

(2012). Reading motivation and reading comprehension in Chinese and English among bilingual students. Reading and Writing: An Interdisciplinary Journal, 25(3), 717–738.

26.

Kishor

(1997). Assessing the relationship between attitude towards mathematics and achievement in mathematics: A meta-analysis. Journal for Research in Mathematics Education, 28(1), 26–47.

27.

Marks

G. N.

(2017). Is SES really that important for educational outcomes in Australia? A review and some recent evidence. The Australian Educational Researcher, 44(2), 191–211. https://doi.org/10.1007/s13384-016-0219-2

28.

McConney

Oliver

M. C.

Woods-McConney

Schibeci

Maor

(2014). Inquiry, engagement, and literacy in science: A retrospective, cross-national analysis using PISA 2006. Science Education, 98(6), 963–980. https://doi.org/10.1002/sce.21135

29.

McConney

Perry

L. B.

(2010). Socioeconomic status, self-efficacy, and mathematics achievement in Australia: A secondary analysis. Educational Research for Policy and Practice, 9(2), 77–91. https://doi.org/10.1007/s10671-010-9083-4

30.

McLeod

D. B.

(1992). Research on affect in mathematics education: A reconceptualization. In Grouws

D. A.

(Ed.), Handbook of research on mathematics learning and teaching (pp. 575–596). Macmillan.

31.

Milfont

T. L.

Fischer

(2010). Testing measurement invariance across groups: Applications in cross-cultural research. International Journal of Psychological Research, 3(1), 111–130. https://doi.org/10.21500/20112084.857

32.

Morsy

Khavenson

Carnoy

(2018). How international tests fail to inform policy: The unsolved mystery of Australia’s steady decline in PISA scores. International Journal of Educational Development, 60, 60–79. https://doi.org/10.1016/j.ijedudev.2017.10.018

33.

Muthén & Muthén. (2017). Mplus user guide. Muthén & Muthén.

34.

Neale

(1969). The role of attitudes in learning mathematics. The Arithmetic Teacher, 631–641.

35.

Netten

Droop

Verhoeven

(2011). Predictors of reading literacy for first and second language learners. Reading and Writing: An Interdisciplinary Journal, 24(4), 413–425.

36.

OECD. (1999). Measuring student knowledge and skills: A new framework for assessment.

37.

OECD. (2003). PISA 2003 assessment framework: Mathematics, reading, science, and problem-solving knowledge and skills.

38.

OECD. (2006). Assessing scientific, reading and mathematical literacy: A framework for PISA 2006. https://doi.org/10.1787/9789264026407-en

39.

OECD. (2009). PISA 2009 assessment framework key competencies in reading, mathematics and science.

40.

OECD. (2013). PISA 2012 assessment and analytical framework: Mathematics, reading, science, problem solving and financial literacy. https://doi.org/10.1787/9789264190511-en

41.

OECD. (2017). PISA 2015 assessment and analytical framework: Science, reading, mathematic, financial literacy and collaborative problem solving (revised ed.). https://doi.org/10.1787/9789264281820-en

42.

OECD. (2019). PISA 2018 assessment and analytical framework. https://doi.org/10.1787/b25efab8-en

43.

Osborne

Simon

Collins

(2003). Attitudes towards science: A review of the literature and its implications. International Journal of Science Education, 25(9), 1049–1079.

44.

Ozonder

(2015). Prospective ELT students’ foreign language reading attitudes and motivation. Procedia Social and Behavioral Sciences, 199, 722–729. https://doi.org/10.1016/j.sbspro.2015.07.603

45.

Perry

McConney

(2010). School socio-economic composition and student outcomes in Australia: Implications for educational policy. Australian Journal of Education, 54(1), 72–85. https://doi.org/10.1177/000494411005400106

46.

Petscher

(2010). Meta-analysis of the relationship between attitudes towards reading and achievement in reading. Journal of Research in Reading, 33(4), 335–355.

47.

Petty

Fabrigar

Wegener

(2003). Emotional factors in attitudes and persuasion. In Davidson

Scherer

Goldsmith

(Eds.), Handbook of affective sciences (pp. 752–772). Oxford University Press.

48.

Sadler

T. D.

Zeidler

D. L.

(2009). Scientific literacy, PISA, and socioscientific discourse: Assessment for progressive aims of science education. Journal of Research in Science Teaching, 46(8), 909–921.

49.

Schleicher

(2019). PISA 2018: Insights and interpretations. OECD Publishing.

50.

Singh

Granville

Dika

(2010). Mathematics and science achievement: Effects of motivation, interest, and academic engagement. The Journal of Educational Research, 95(6), 323–332. https://doi.org/10.1080/00220670209596607

51.

Sjaastad

(2012). Sources of inspiration: The role of significant persons in young people’s choice of science in higher education. International Journal of Science Education, 34(10), 1615–1636.

52.

Smith

S. A.

Briggs

J. G.

Pothier

(2018). Exploring variation in reading comprehension among young adult Spanish–English bilinguals: The role of environmental language contact and attitudes toward reading. International Journal of Bilingualism, 22(6), 695–716. https://doi.org/10.1177/1367006917690913

53.

Song

Perry

L. B.

McConney

(2014). Explaining the achievement gap between Indigenous and non-Indigenous students: An analysis of PISA 2009 results for Australia and New Zealand. Educational Research and Evaluation, 20(3), 178–198. https://doi.org/10.1080/13803611.2014.892432

54.

Steiger

J. H.

(2007). Understanding the limitations of global fit assessment in structural equation modeling. Personality and Individual Differences, 42(5), 893–898.

55.

Sullivan

McConney

Perry

L. B.

(2018). A comparison of rural educational disadvantage in Australia, Canada, and New Zealand using OECD’s PISA. SAGE Open, 8(4). https://doi.org/10.1177/215824401880579.1

56.

Sullivan

Perry

L. B.

McConney

(2014). How do school learning environments differ across Australia’s rural, regional, and metropolitan communities? The Australian Educational Researcher, 41(5), 521–540. https://doi.org/10.1007/s13384-014-0144-1

57.

Tabachnick

B. G.

Fidell

L. S.

(2007). Using multivariate statistics (5th ed.). Allyn and Bacon.

58.

Tapia

Marsh

G. E.

II (2004). An instrument to measure mathematics attitudes. Academic Exchange Quarterly, 8(2), 16–21.

59.

Thomson

De Bortoli

Buckley

(2013). PISA 2012: How Australia measures up. Australian Council for Educational Research (ACER).

60.

Thomson

De Bortoli

Nicholas

Hillman

Buckley

(2011). Challenges for Australian education: Results from PISA 2009: The PISA 2009 assessment of students’ reading, mathematical and scientific literacy. Australian Council for Educational Research (ACER).

61.

Thomson

De Bortoli

Underwood

(2017). PISA 2015: Reporting Australia's results. Australian Council for Educational Research (ACER).

62.

Thomson

De Bortoli

Underwood

Schmid

(2019). PISA 2018: Reporting Australia’s results: Volume I student performance. Australian Council for Educational Research (ACER).

63.

Thomson

Wernert

O’Grady

Rodrigues

(2016). TIMSS 2015: A first look at Australia’s results. Australian Council for Educational Research (ACER).

64.

Tran

N. A.

(2011). The relationship between students’ connections to out-of-school experiences and factors associated with science learning. International Journal of Science Education, 33(12), 1625–1651.

65.

Tytler

Osborne

(2012). Student attitudes and aspirations towards science. In Fraser

Tobin

McRobbie

C. J.

(Eds.), Second International handbook of science education (pp. 597–625). Springer.

66.

Wheaton

Muthén

Alwin

F. Summers

(1977). Assessing reliability and stability in panel models. Sociological Methodology, 8(1), 84–136.

67.

Zan

Di Martino

(2007). Attitude toward mathematics: Overcoming the positive/negative dichotomy. The Montana Mathematics Enthusiast, 3, 157–168.