Abstract
Students’ information and communication technology (ICT) skills have been shown to be a valuable resource for their educational performance, although to varying degrees across countries. Our paper adds to existing knowledge to understand how and why country-specific ICT characteristics make a difference to math performance benefits students glean from their ICT skills, thereby adding to a general understanding of how digital divides at different levels are interrelated. We home in specifically on two features that reflect a country’s ICT promotive environment, respectively national ICT access in education and government promotion of ICT. We specifically elaborate on how these country level features augment the learning and signaling function of student ICT skills. Multilevel linear regression analyses were conducted on a sample (PISA 2018) of 248,720 students across 43 countries, enriched with contextual information from the Networked Readiness Index 2016 and World Bank’s data. Congruent with past research our results indicate that student ICT skills serve as an education resource. This, our study reveals, is indeed dependent on the country’s ICT promotive environment, as students’ ICT skills translate to greater math performance gains in countries with higher levels of ICT access in educational environments.
Keywords
Introduction
Skills in information and communication technology (ICT) are vital for a person’s functioning and life chances in contemporary information-rich societies. This is especially so for today’s youth, for whom the application of ICT skills is evidently important in educational learning, and spills over to professional and leisure domains. Prior research has demonstrated that ICT skills are beneficial for student’s educational performance (Loh, Kraaykamp and van Hek, 2023; Pagani et al., 2016). It also revealed that despite narrowing disparities in ICT access and ICT skills among students (Gorski, 2005; OECD, 2021), the benefits students derive from ICT skills differ between countries (Arpacı et al., 2021; Odell et al., 2020). This is an important observation as it evinces that national conditions and efforts do matter, and have implications on how important students’ ICT skills are for their educational performance (OECD, 2021; Novak et al., 2018). As ICT grows ever more interwoven into all facets of life, it thus is important to gain knowledge on how ICT promotion in students’ country of residence relates to how important ICT skills are in their educational career.
In this paper, we focus on math performance and identify specific aspects of countries’ ICT promotive climate which may influence the extent to which students derive benefits in math performance from their ICT skills. Among the key domains in learning, students’ ICT skills are likely most advantageous for math learning. Firstly, ICT is prominently used as a learning tool in math education (Bray and Tangney, 2017; Trouche et al., 2012). Secondly, ICT skilfulness overlaps with competencies important to math such as structural and logical thinking, information management and problem solving (Geraniou and Jankvist, 2019).
While country characteristics have been recognized as important, there are scarcely any studies, to our knowledge, that examined the country dependency of student’s ICT skills in its consequences for math performance. Park and Weng (2020) is an exception, finding that ICT skills yield greater benefits to educational performance as a construct of math, reading and science, in more affluent countries when compared to less affluent countries. However, little theoretical elaboration was given as to why this would be the case. Such an endeavour would require a more pointed identification of which aspects of the country context matter. On country characteristics, much work has been devoted specifically to ICT country conditions, illustrating how digital differences between countries are interrelated with digital divides at the individual level, highlighting how individual, social and national factors intersect to impact each other (see for e.g. Ghobadi and Ghobadi, 2015; OECD, 2021, Van Dijk, 2005). There have also been studies such as Hu et al. (2018) and Skryabin et al. (2015) that have examined how country-level ICT development influences math, reading and science performance, though they did not go on to investigate how country-level ICT development also moderates the relationship between ICT skills and subject performance. Strikingly, the moderating role played by ICT in educational environments, the key site of learning and influence for students, is hardly featured. Looking to the vast body of work that has examined differences in national ICT policies and ICT adoption (see Resta and Laferrière, 2008; Van Dijk, 2020 for a more in-depth overview, and Kozma, 2008 for a review of educational ICT policies across countries), we posit that ICT access in schools and governmental stimulation of ICT application are key aspects of a promotive national ICT climate relevant to students benefiting from their personal ICT skills. These features seem relevant for two theoretical reasons. First, countries differ in the provision of a general opportunity structure for learning with ICT which pertains to physical access to ICT and a related infrastructure (e.g. broadband) (OECD, 2021). For students, this most obviously relates to country-specific ICT access opportunities in schools. A second country-feature refers to the valuation of ICT skills – which exemplifies a positive attitude toward ICT and how valued ICT is within countries. How valued, or indeed necessary, ICT skills are perceived to be, is expressed not only through the infrastructure available at school, but also by governmental policies driving the access to, integration and usage of ICT. Together, available opportunities to actually put ICT skills to use and how valued ICT skills are within a country could influence student’s potential to reap learning benefits from their personal ICT skills.
In sum, this paper investigates the extent to which the benefits that students derive from their ICT skills in terms of math performance depend on a country’s ICT promotive environment. Our paper attempts to make two key contributions. To begin, it is one of the first efforts to conduct an empirical investigation of the possible moderating role of countries’ ICT promotive environment on the math performance benefits students derive from their ICT skills. As part of our efforts, we also provide elaboration on the underlying theoretical mechanisms. Next, we identify country characteristics that are specifically directed at ICT promotion, rather than applying a single measure of internet penetration, index of usage, or applying a single measure of general development of a country (e.g. GDP) (Hu et al., 2018; Park and Weng, 2020; Skryabin et al., 2015). More specifically, we consider both infrastructural and policy aspects of a country’s ICT promotive environment that seem relevant to student learning in secondary education. In all, this paper adds to existing knowledge to understand how and why country-specific ICT characteristics make a difference to the math performance benefits students glean from their ICT skills, thereby adding to a general understanding of how digital divides at different levels are interrelated (Ghobadi and Ghobadi, 2015).
To answer our research question, we employ data from the most recent iteration of the OECD’s Programme for International Student Assessment (PISA) conducted in 2018, henceforth PISA 2018. PISA surveys 15 year-old students and houses information on their ICT skills, social background and educational performance. We draw data on country factors from the World Economic Forum’s (WEF) Networked Readiness Index (NRI) 2016, the World Bank’s data repository, and through aggregation of PISA 2018 information. Our sample consists of 248,720 students from 43 countries that participated in the PISA ICT Familiarity questionnaire and are represented in the NRI.
Theory
ICT skills and math performance
ICT skills strike as especially useful to math learning and therefore beneficial to math performance given the high utilization of ICT in math classes (Bray and Tangney, 2017; Trouche et al., 2012), and the overlap of ICT skilfulness with math competencies such as structural and logical thinking (Geraniou and Jankvist, 2019). However, there is a lack of work on how students’ ICT skills benefit their math performance specifically. Most studies have thus far focused foremost on students’ ICT access and usage instead (Loh, et al., 2023; Pagani et al., 2016), and some studies subsumed math with other subjects as measure of educational performance (e.g. Park and Weng, 2020). Nevertheless, there are a few empirical studies that illustrate that self-reported ICT skills (Loh, et al., 2023; Guzeller and Akin, 2014; Hu et al., 2018; Skryabin et al., 2015) and also content-specific ICT skills (for e.g. information ICT skills in Pagani et al., 2016) are beneficial to math performance. Three papers are of note here. Using PISA data, Hu et al. (2018) and Loh et al. (2023) illustrate that self-reported ICT skills benefit math performance, even when other ICT facets, such as access and usage, are taken into account. Pagani et al. (2016), with in-depth standardized tests of students’ information ICT skills, also find these to be beneficial to math performance. Taken together, these studies indicate that ICT skills are advantageous, or at least positively associated with math performance. The precise mechanisms underlying the math learning benefits of ICT skills, however, are scarcely explicated. In lieu of this, we draw on literature on ICT skills and educational performance more broadly. On this front, prior research suggests that ICT skills are beneficial to educational performance through serving as a learning resource and by signalling positive learner characteristics. The same likely holds true, even when only math is examined.
ICT skills serve as learning resource as they facilitate greater autonomy and adjustment to contemporary learning practices and environments (Passey et al., 2004, Pittard et al., 2003). Research also suggests that ICT skills facilitate digital informal learning more broadly, including math learning (Mehrvarz et al., 2021). For instance, students who possess ICT skills can better tap into the learning opportunities available online (Pagani et al., 2016). It has also been argued that students with ICT skills feel more confident in digitalized learning environments and feel more comfortable using ICT for school, and are therefore more likely to engage with ICT for learning purposes (Kim et al., 2008). Moreover, students who are more skilled with ICT are also more frequent users of ICT and better able to sidestep potential negative impacts of ICT (Van Dijk, 2020). They can therefore better capitalize on how ICT intersects places of learning to further engage in learning outside of the school environment (Livingstone, 2012), and learn efficiently, perhaps even more so than with traditional media (Hochberg et al., 2020; Sung et al., 2016).
Student’s ICT skills also serve as a signalling resource; as a marker of capability, aptitude, and attitude, and it parallels cultural capital in that sense. Cultural capital in the classroom points at students’ attributes that connect to teachers’ expectations, and are often associated with high social class activities (Bourdieu 1984; Lareau, 1987; Paino and Renzulli 2013). Students who demonstrate cultural capital, by way of exhibiting cultural knowledge, competencies, and behaviour, are recognized by teachers as more intelligent and gifted, and therefore tend to do better in school than those who do not (Kraaykamp and Notten, 2016). Similar mechanisms are likely at work when it comes to ICT skills, where teachers may recognize ICT skills as a marker of capability and motivation. Intertwining ICT skills and cultural capital even further, some scholars argue that ICT skills are a contemporary digital marker of cultural capital; students who display ICT skills are not only demonstrating their skills, but are presenting themselves as culturally competent and inspired members of the digital or information-age society (e.g. Paino and Renzulli, 2013; Tondeur et al., 2011). Along this line of argument, teachers dole out positive sanctions for displays of ICT skills, or provide more attention and support – social support or actual ICT know-how, to encourage students to use their ICT skills in educational settings and for educational tasks. This positive reinforcement likely leads to better educational performance (Skryabin et al., 2015). Initial positive performance may even accumulate into a self-fulfilling prophecy, when students are motivated by positive feedback put in more effort in their learning (Voyer and Voyer, 2014; Wigfield and Eccles, 1994). This signalling mechanism may also work the other way around – a lack of ICT skills signals a lack of ability and motivation. Indeed, prior research illustrates that teachers too easily and often wrongly assume that everybody has the resources, incentives and guidance needed to develop ICT skills, and thus show little understanding for students with low ICT skills (Broos, 2005). Poor teacher interactions and evaluations may then feed a negative loop of demoralization and in turn weaken a student’s performance.
Based on the above arguments and previous empirical evidence, we expect that:
National ICT environment, ICT skills and math performance
Whether students’ ICT skills serve as a learning and signalling resource might vary as students find themselves with more or with fewer opportunities to put their ICT skills to use, or because their ICT skills are more or are less valued (Ilomäki and Rantanen, 2007). These conditions are expressed through a country’s ICT opportunity structure in education and a country’s policy in advancing a ICT promotive environment.
National ICT access in education
National ICT access in education pertains mainly to the provision of ICT infrastructure in schools. This is a cornerstone of national educational ICT policies and is often referred to in terms of the computer-to-student ratio in empirical work. While students develop ICT skills mostly in the home environment, and therefore through personal access and effort (Biagi and Loi, 2013; Selwyn, 2010), it is this robust infrastructure in schools that forms the platform for students to translate their personal ICT skills into educational gains (Berge et al., 2009; Claro et al., 2012).
Countries with greater ICT access in education provide more educational opportunities for students to utilize their ICT skills (OECD, 2021), thereby augmenting its function as a learning resource. Greater ICT access in education affords more opportunities to digitalize education and to employ ICT more heavily in classroom learning, in curricula and in pedagogical practices. For example, teachers in countries with more advanced ICT facilities at school are more likely to use ICT as a learning tool – by using its features for assignments, grading and in class (Inan and Lowther, 2010; Tondeur et al., 2008). Students in countries with greater ICT access in education are therefore more likely to make use of their ICT skills for educational tasks – ranging from simple tasks such as accessing and submitting assignments through their school online portals, to more complex tasks such as information gathering and bricolage, and doing statistics. They are thus more likely to see educational performance gains from their ICT skills. In contrast, similarly ICT-skilled students in a country with a less developed ICT infrastructure in education would experience fewer opportunities to profit from ICT skills for educational purposes.
A country’s educational ICT infrastructure might also influence how pertinent and well-regarded student ICT skills are, and thereby augment its signalling function. On this front, we again draw from cultural capital arguments – more specifically on how the advantage of demonstrating cultural capital would vary depending on the evaluations of relevant gatekeepers. In education, teachers are the key gatekeepers who recognize and respond according to student’s cultural capital. The value of cultural capital is, however, context dependent and is considered valuable in education because schools reflect and are responsive to the cultural orientations of the dominant class (Lareau, 1987; Kingston, 2001). A similar argument can be made with how the valuation of student ICT skills depends on how prevalent ICT is in educational settings. Within the educational environment, the provision of ICT opportunities normalizes the expectation for students to use, and indeed, be competent in ICT. The linkage between student ICT skill and competence is therefore likely more prominent and salient in countries where national ICT access in educational is higher. In such countries, students “have no reason” to be poorly skilled in ICT as ICT skills are integrated into the curriculum, and students are expected to have mastered these skills already at home and in earlier stages of education (Lewis et al., 2019). Therefore, in countries with greater national ICT access in education, students ICT skills signal favourable learner characteristics, such as capability and motivation, more strongly as it aligns with the prevalent expectations that teachers have of a competent student in a digitalized educational environment. In contrast, a similarly ICT-skilled student in a country with a less developed ICT infrastructure in education is less likely to receive the same level of recognition as ICT skills are less of a salient marker of competence and motivation. We therefore expect that:
Governmental policy in promoting ICT implementation
The impact of student ICT skills might not solely vary by a country’s ICT opportunity structure in education but could also differ because of a country’s value climate regarding the importance of ICT. In some countries, the discourse around ICT as a means of capital enhancement, progress and growth will inevitably give teachers and students the sense that ICT skills are important. Teachers’ and students’ valuation and expectations regarding ICT skills could be swayed by government ICT promotion efforts. Government promotion efforts often take the form of a national strategy or plan with a clear policy framework (Anderson et al., 2015). National policies that highlight positive outcomes of ICT use create awareness, and inspire students to use ICT and develop ICT skills – particularly because they observe positive consequences in the domains of labour market, education, health and community (van Dijk, 2020). These policies thus support trust in and the prestige of ICT and emphasize it as a means of individual capacity and employability building with positive consequences for a country’s economy. It does so through implementation – encouraging wider integration and use of ICT in education and through national digitalization efforts, and discourse – by speaking of ICT in positive terms, and associating concepts of progress and ensuring a competitive edge with ICT (Salajan, 2019). A particularly pertinent theme for secondary school students is how ICT use and ICT skills prepare them for the future workforce and are related to the capacities and motivations to use ICT to tackle real-world problems. Often, national policies refer to ICT skills, the development of ‘twenty-first century competencies’, and lifelong-learning skills as a key means of supporting knowledge creation, innovation and entrepreneurialism in a ‘knowledge economy’ (Anderson et al., 2015; Kozma, 2008). We therefore expect that teachers and students in countries with greater government promotion of ICT are influenced by this normative framing and rousing associations. This happens through direct communication and exposure to the rhetoric (Jones, 2003), and through opinion leaders such as school principals, colleagues and peers (see Two-step Flow of Communication Model by Katz and Lazarsfeld, 2017). In such a context, student ICT skills function as a salient marker of competency, and teachers would be more selective towards skilled students and see skilled students as being readier for the future, more likely to fare well in the digital age, and generally more capable based on their ICT skills. As such, we expect that:
Our hypotheses are summarized in the following conceptual model (Figure 1):

Overview of hypotheses.
Data, measures and analytical strategy
Data
This study utilized data from the 2018 iteration of the OECD’s PISA survey, henceforth PISA 2018, from the World Economic Forum’s (WEF) Networked Readiness Index (NRI) 2016, and from the World Bank’s data repository. PISA 2018 assessed the academic skills and knowledge of students, ranging from 15 to 16 years old. The 2018 sample consists of 612,004 students across 79 countries. The PISA ICT Familiarity Questionnaire was optional, and completed by 49 countries in 2018. The mandatory student questionnaire and optional ICT Familiarity Questionnaire were used together to obtain information on student math performance, ICT skills, demographic background and previous performance. Data from the PISA school questionnaire, WEF NRI 2016 and World Bank data repository was utilized to derive country-level measures for the main analyses and robustness checks. We retained only countries with valid data on the number of computers with internet available in schools, government ICT promotion efforts, and GDP per capita. After these selections, the sample consisted of 261,134 students from 44 countries. To further improve comparability, we included only students with valid responses to ICT skill items, and relevant control variables. We therefore had to drop all 5,945 students from Japan, due to missing information on previous performance, and 6,469 students across other countries with incomplete information. Our final sample consists of 248,720 students from 43 countries. Descriptive statistics of all measures are provided in Table 1. Descriptive statistics of key variables by country are provided in Appendix A.
Descriptives.
PISA 2018; N students = 248,720; N countries = 43.
All country variables were grand-mean centred for the analyses. Per country key student- and unstandardized country-variables are provided in Appendix A.
Measures
Student-level variables
Student math performance was provided by PISA. Students’ performance was measured with a set of 10 plausible values for each subject. According to the PISA data analysis manual (OECD, 2009), using one plausible value provides an unbiased estimate of population parameters. It is demonstrated that using one plausible value does not yield a significant difference in the mean estimates or in the standard error estimates with a sample size over 6400 students (OECD, 2009). In this paper, we focus on math performance, and utilize the first plausible value for math.
Student ICT skills were captured using the five available items on perceived ICT competence from the PISA ICT Familiarity Questionnaire. Direct assessments of student’s ICT skills would have been preferrable though not feasible for large scale surveys such as PISA. Nevertheless, prior research suggests that students who report a higher sense of general ICT competency also do display more advanced ICT skills (Fraillon et al., 2014). We therefore proceeded to employ the ICT items available in PISA 2018. Students were asked to indicate the extent to which they agreed or disagreed with five statements. These were: ‘If I need new software, I install it by myself’, ‘I read information about digital devices to be independent’, ‘I use digital devices as I want to use them’, ‘If I have a problem with digital devices, I start to solve it on my own’ and ‘If I need a new application, I choose it by myself’. Students’ responses were on a four-point Likert scale (α = 0.86) ranging from Strongly Disagree (1) to Strongly Agree (4), which we recoded to range from 0 to 3. A confirmatory factor analysis revealed that items load evenly and acceptably on a common factor (RMSEA = 0.144, CFI = 0.953, SRMR = 0.040). For ease of interpretation, we took the average score across the five items to calculate students’ ICT skills. The resulting measure ranges from 0 to 3, with a mean of 1.86.
As for student control variables, we included parental education, gender, age, immigration background, and indicators of previous performance as these have repeatedly proven to be key influences for student performance. For parental education, the PISA 2018 questionnaire had students indicate their parents’ education level, which was then categorized according to the ISCED scale, and subsequently translated into number of years based on the average number of years respondents in a country spent to attain that educational level. This scale ranges from 3 to 18 years. We recoded it to 0 to 15 for reasons of interpretation. For student’s gender, we utilized a dummy variable, girl, where 1 indicates a girl. Next student’s age, which initially ranged from 15.08 to 16.33, was recoded to range from .08 to 1.33 (mean = 0.79) to afford us a meaningful and interpretable zero, with 0 indicating exactly 15 years old. We also controlled for student immigration background (0 = native; 1 = migrant background). Grade, type of education, and repeat, were further included to capture past performance. Student’s grade is relative to the modal grade, where 0 stands for a student who was at the modal grade. Type of education reflects a student is enrolled in a general, pre-vocational, vocational or modular educational track, and was recoded to 0 for vocational and 1 for general education. Lastly, with repeat, we control for student’s grade repetition.
Country-level variables
National educational ICT access was derived aggregating information from the PISA 2018 school questionnaire. School principals were asked to indicate the number of students and the number of computers with internet access available for student use. Educational ICT access was initially considered at the school-level, however, too little variance was found between schools within a country. Earlier research already indicated that ICT access in terms of computer-to-student ratio is noted to have little between-school variability regardless of school size (Luu and Freeman, 2011). As such, we calculated the computer-to-student ratio in schools, and subsequently aggregated it to the country level. We then centered the measure of average computer-to-student ratio across all countries (0.56 = 0). The resulting measure ranges from −0.51 to 0.43.
Our measure of government promotion of ICT was derived using data from NRI 2016. Executives were asked, as part of the WEF’s Executive Opinion Survey, ‘To what extent does the government have a clear implementation plan for utilizing ICTs to improve your country’s overall competitiveness?’, and indicated their answer on a scale of 1 = not at all – there is no plan to 7 = to a great extent – there is a clear plan. The WEF sampled executives from companies of various sizes and from various sectors and industries in order to closely reflect the structure of the country and its economy (Schwab, 2018). We took the average within countries and recoded it to range from 0 to 6. The measure was then centred on the average across all countries (3.08 = 0), and ranges from −1.16 to 1.85.
GDP per capita in US dollars was retrieved from the World Bank data repository. This measure was divided by 1000, and subsequently centred on the average across all countries (35.60 = 0). The resulting measure ranges from −27.33 to 60.96.
As one would expect country-level ICT variables to be highly correlated with each other and with GDP per capita, a correlation matrix was computed between the variables to check for any correlation coefficients of concern (Table 2). GDP per capita was indeed highly correlated with both aspects of national ICT promotive environment; this is reasonable as both also reflect aspects of national wealth and development level. Furthermore, GDP per capita held a Variance Inflation Factor (VIF) of 2.45, leaning close to the 2.50 threshold for multicollinearity issues. In light of this, GDP per capita was excluded as a control variable in our main analyses, and used in robustness checks only.
Correlation matrix.
Analytical strategy
We performed three-level multilevel linear regression analysis, with students nested within schools within countries to reflect the nested nature of the PISA data, and to avoid statistical and interpretational problems (Goldstein, 2011; Hox, 2010). Including the school-level ensures that country-level aspects do not reflect compositional difference of schools between countries. The analysis was performed in Stata 15.6 and Mplus 8.4. First, an empty model was estimated with variation at all levels to partition the variance in math performance scores into within- and between-group components (Model 0). Next, student variables were added (Model 1), followed by national ICT environment variables (Model 2) to examine their impact on student math performance. A random slope of student ICT skills was then included to ascertain if it indeed had a significant variance component (Model 3). Finally, the cross-level interactions between national ICT environment variables and student ICT skills were added one by one (Model 4) before testing them together (Model 5).
Results
The results of the empty model (Model 0, Table 3) showed that between country variance accounted for 24.8% of the variability of math performance scores, while between school differences accounted for 47.7% (ICC Country = 0.248; ICC Country|School = 0.477).
Multilevel regression on student’s math performance of ICT skills (unstandardized coefficients).
N students = 248,720; N schools = 12,288; N countries = 43.
p < 0.001, **p < 0.01, *p < 0.05.
ICT skills, national ICT environment and math performance
With our first hypothesis, we expected students with more ICT skills to have better math performance. Our results (Model 1, Table 3) indeed show that students with greater ICT skills score better on the math performance test (B = 12.671). We therefore accept hypothesis 1. The impact is not trivial either. We observe that a student with an average level of ICT skills (mean = 1.863, rounded up to 2) would do, on average, 25.342 points better in math compared to a student who indicated to have no such skills at all. Comparing Model 1 to the null model, we conclude that student ICT skills, demographic background, and previous performance contribute to explaining variations in student math performance, reducing variation by 26.2%, 34.0%, and 8.2% at the country-, school-, and student-level respectively.
Next, we included national ICT access in education and government promotion of ICT. Our results (Model 2, Table 3) indicate national ICT access in education has an overall positive impact on student math performance (B = 92.693). This main country effect indicates that students who are educated in countries with higher ICT access in education attain better math performance scores. A country’s governmental ICT promotion effort showed no significant impact on student’s math performance although the estimate points in a positive direction.
Moderating effect of national ICT environment
Next, we included a random slope for student ICT skills to account for country variability in benefits to math performance derived from student ICT skills. Our results (Model 3, Table 3) indicate the between-country variance in the influence of student ICT skill on math performance is 21.464.
With our second hypothesis, we examined if national ICT access in education and government promotion of ICT influenced the degree to which students derive educational benefit from their ICT skills. Our results (Model 4a and b, Table 4) suggest that students’ gains in math performance from their ICT skills do depend on both national ICT access in education and government promotion of ICT. To accept hypothesis 2 without controlling for the other interaction term comes with possible inflation of Type I error rates (Brutus et al., 2013). We therefore also tested them together, and found that only the moderation effect of national educational ICT access remained statistically significant (Model 5, Table 4). Thus, only hypothesis 2a is accepted. According to our results, a student gains 13.025 points in math performance per unit of student ICT skills in a country with average national ICT access in education (mean = 0.563). Students in countries with above average (1 S.D. higher) national ICT access in education stand to gain 20.905, that is 7.880 more, in math performance scores from their ICT skills compared to equally ICT-skilled students in countries with average national ICT access in education. This is illustrated clearly by the diverging lines in Figure 2 below. Inclusion of this cross-level interaction explains 24.9% of the country variability in math performance benefit derived from student ICT skills.
Multilevel regression on student’s math performance of ICT skills moderated by national ICT environment (unstandardized coefficients).
N students = 248,720; N schools = 12,288; N countries = 43.
p < 0.001, **p < 0.01, *p < 0.05.

Augmenting effect of national ICT access in education.
Robustness checks
GDP per capita
By way of a sensitivity check, we proceeded to examine if similar conclusions would be drawn when analysing GDP per capita instead of measures of the country’s ICT promotive environment. We expected similar conclusions as the national ICT environment is closely tied to national wealth and development level (Skryabin et al., 2015), and GDP per capita correlates highly to both national ICT access in education (r = 0.639) and government promotion of ICT (r = 0.669) in our data. Indeed, our results (Appendix B), showed that GDP per capita also had significant positive influence on student math performance, and also positively moderated the relationship between student ICT skills and math performance. This also illustrates how effects of national ICT promotive environment are intertwined with that of GDP, but importantly have the advantage of providing more insights on the mechanisms at play.
Student ICT efficacy and information-seeking use
We additionally examined if similar conclusions would be drawn for other facets of ICT familiarity. We homed in on student ICT efficacy and student information-seeking usage, which previous research has shown to be beneficial for math performance (Loh, et al., 2023, Pagani et al., 2016). Both measures were derived from PISA 2018. Student ICT efficacy was measured using the unweighted mean across five items that reflected perceived self-efficacy regarding ICT, such as ‘I feel comfortable using digital devices that I am less familiar with’ which were recoded to range from 0 to 3 (α = 0.861). Student ICT usage for information-seeking was measured using the unweighted mean across two items that captured how often students used digital devices for ‘reading news on the internet’ and ‘obtaining practical information from the Internet’ outside of school. Responses were recoded to range from 0 to 4.
Our results (Appendix C) indicate that the results for student ICT efficacy corroborate our earlier findings with ICT skills; students with more ICT efficacy score better in math performance assessments, and students derive more benefit from their ICT efficacy in countries with greater national ICT access in education. Our results also indicate that students who more often use ICT for information-seeking score better in math performance assessments, though this impact was not subject to the national ICT environment. Possibly, student ICT efficacy is more competency-related, and therefore would function similarly to student ICT skills. In sum, the results relating to ICT efficacy align with our earlier findings on ICT skills. Taken together, the additional analyses illustrate that the math performance benefits students derive from their competency related ICT resources – ICT skills and efficacy, are augmented in countries with greater national ICT access in education.
Discussion and conclusion
In the present study, we set out to investigate the extent to which the math performance benefits students derive from their ICT skills depend on a country’s ICT promotive environment. It is a topic which has not yet received much empirical exploration, despite the existing insights into how student ICT skills and national ICT environment influence student educational performance. This area of work also has implications on national ICT policies moving forward (OECD, 2021). We elaborated on how students may reap more benefits from their ICT skills in countries which have a more robust ICT infrastructure in education – thereby providing students with more opportunities to use their ICT skills and express ICT skills as valuable, and in countries whose governments promote the use of ICT – therefore also expressing that ICT skills are highly valuable.
We first established that student ICT skills are beneficial to math performance. This is congruent with earlier studies (Loh, et al., 2023; Hu et al., 2018; Skryabin et al., 2015; Pagani et al., 2016). Next, we also found that students in countries with greater national ICT access in education had better math performance. Looking to the few studies that have examined the positive impact of the national ICT environment on math performance, our results concurred with those of Hu et al. (2018) and Skryabin et al. (2015). While these are not fairly matched comparisons, given the differences in country selections, student age, and conceptualization of the national ICT environment, the slightly varying results do highlight that the more pinpointed conceptualization of national ICT environment is preferred over more general measures of wealth and development, because it yields greater insight. By identifying key aspects of countries’ ICT promotive environments that matter for students, we were further able to examine if they influence how much students gain from their ICT skills. Our results illustrate that students’ ICT skills translate to math performance more strongly in countries where ICT opportunities are more plentiful in education: that is, within the context of our study, where the computer-to-student ratio is higher at schools. Taken together, our findings are consistent with the idea that student ICT skills serve as a learning and signalling resource, at least for math, and both functions are augmented in countries where ICT is more prominent and readily accessible in the educational environment. Although our study provides indications that such mechanisms matter, future work is needed to test the underlying theoretical mechanisms more directly. For example, information about (math) teachers’ beliefs on the importance of ICT skills for students’ future might provide a closer test of the processes going on in schools.
No study is without limitations. With the current data, we were unable to discern if the underlying mechanism leans more towards the provision of opportunities to put these skills to use, or towards students and teachers valuing ICT skills more in countries with a more developed national ICT promotive environment. In order to do so, future studies could dive into the school-environment to acquire more precise measures at the school-level that go beyond ICT-availability. Measures more specifically relating to the didactive quality of ICT teaching, dealing with both teacher and student usage of ICT in class, measures pertaining to student and teacher attitudes towards ICT, teacher’s own ICT skills and school leadership regarding ICT – which have been highlighted among the important determinants to ICT appropriation and integration in educational environments might yield more nuanced insights on the opportunity structures and valuation of ICT skills in schools (Comi et al., 2017; Voogt et al., 2013).
Additionally, the cross-national nature of PISA makes for a non-causal design. This poses limitations with respect to thorough testing of the mechanisms proposed. We have, however, taken steps to circumvent this. At the student-level, we extensively controlled for students’ past performance. At the country level, we utilized NRI 2016 data, which has information gathered quite some time before the 2018 iteration of PISA was implemented. And while we also used data aggregated from PISA 2018, specifically for our measure of national educational ICT access, this is likely a stable country characteristic as it reflects mainly access to ICT hardware and is not subject to shifting benchmarks of quality of access. Therefore, this limitation, while important, does not diminish our contribution of being one of the first studies to explicitly lay out and test substantive mechanisms through which different aspects of the national ICT environment influence the degree of math performance benefit that students derive from their ICT skills.
Notably, our ICT skills measure is based on self-reported information on student’s general ICT skills as this was what is available in PISA 2018. Collecting cross-national comparative information often comes at a price of having less detailed measures. Future research therefore could consider examining ICT skills specific to (math) learning in school. Next, our measure of government promotion of ICT is based on information provided by corporate executives, rather than by government officials or by a content-analysis of policies. While this presents a suboptimal notion of a government’s promotion of ICT, large-scale and comparable data from government officials on ICT policies, as far as we are aware, is hardly available. We therefore opted to utilize readily available and comparable information based on the perception of corporate executives, who are gatekeepers in the labour market, and therefore likely have a clear notion whether governments are pushing for ICT in their country.
As a final note, this study’s central finding that ICT in educational environments matters for the translation of student ICT skills to math performance, either through providing opportunity for utilization of ICT skills or through the saliency of ICT skills among teachers and students, serves as a meaningful springboard for future research. For instance, future research could investigate whether the influence of national ICT access in education is due to better opportunities to put skills to use or due to teachers’ more positive evaluation of ICT skills. Future research could also explore how the moderating impact of national ICT environment works differently between groups of students. Indeed, research is needed on the role ICT environments play in the reproduction or perpetuation of existing disparities in the educational benefits that students reap from their ICT skills and their ICT resources in general. Afterall, students have been found to benefit differently from their ICT skills based on their gender, ethnicity, immigration background, and family background (Loh, et al., 2023; Meggiolaro, 2018; Rafalow, 2020). Research in this realm would add greatly to the growing discussion on the cyclical reinforcement of digital and social inequalities (Helsper, 2012).
Supplemental Material
sj-docx-1-eer-10.1177_14749041231201197 – Supplemental material for Do students’ ICT skills pay off in math performance? Examining the moderating role of countries’ ICT promotive environment
Supplemental material, sj-docx-1-eer-10.1177_14749041231201197 for Do students’ ICT skills pay off in math performance? Examining the moderating role of countries’ ICT promotive environment by Renae Sze Ming Loh, Gerbert Kraaykamp and Margriet van Hek in European Educational Research Journal
Supplemental Material
sj-docx-2-eer-10.1177_14749041231201197 – Supplemental material for Do students’ ICT skills pay off in math performance? Examining the moderating role of countries’ ICT promotive environment
Supplemental material, sj-docx-2-eer-10.1177_14749041231201197 for Do students’ ICT skills pay off in math performance? Examining the moderating role of countries’ ICT promotive environment by Renae Sze Ming Loh, Gerbert Kraaykamp and Margriet van Hek in European Educational Research Journal
Supplemental Material
sj-docx-3-eer-10.1177_14749041231201197 – Supplemental material for Do students’ ICT skills pay off in math performance? Examining the moderating role of countries’ ICT promotive environment
Supplemental material, sj-docx-3-eer-10.1177_14749041231201197 for Do students’ ICT skills pay off in math performance? Examining the moderating role of countries’ ICT promotive environment by Renae Sze Ming Loh, Gerbert Kraaykamp and Margriet van Hek in European Educational Research Journal
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: We gratefully acknowledge funding from the Nationaal Regieorgaan Onderwijsonderzoek (NRO) Programmaraad voor Fundamenteel Onderwijsonderzoek (PROO) (NRO PROO grant code 40.5.18300.009.).
Supplemental material
Supplemental material for this article is available online.
Author biographies
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
