Brazil’s education system lags behind international standards, with two-fifths of students scoring below the minimum level of proficiency in mathematics, science, and reading. Thus, this study combined machine learning with traditional statistics to identify the most important predictors and to interpret their effects on proficiency in the PISA 2018 mathematics, science, and reading tests. Predictors encompassed a wide range of variables, sociodemographic characteristics, teaching and learning processes, and non-cognitive skills. The outcome of the present study was proficiency in mathematics, science, and reading. PISA proficiency levels were grouped into “low proficiency” and “proficient” categories, using a classification system commonly employed in PISA reports. Using random forest analysis, a machine learning method, I compared the importance of predictors for proficiency in mathematics, science, and reading. I then adjusted multilevel logistic regression analyses to investigate the relationship between the top predictors and the outcomes. Among the top predictors for the three outcomes identified, annual household income, parents’ highest occupational status, and early childhood education and care were positively associated with proficiency in mathematics, science, and reading, while grade repetition and additional instruction were negatively associated with these outcomes. These findings urge Brazilian policymakers and educators to prioritize initiatives that strengthen early childhood programs, minimize grade repetition, and promote effective learning strategies.
AfshartousD.de LeeuwJ. (2005). Prediction in multilevel models. Journal of Educational and Behavioral Statistics, 30(2), 109–139. A Quarterly Publication Sponsored by the American Educational Research Association and the American Statistical Association. https://doi.org/10.3102/10769986030002109
2.
AksuG.GüzellerC. O. (2016). Classification of PISA 2012 mathematical literacy scores using decision-tree method: Turkey sampling. Dialysis, Transplantation & Burn: Medical Journal of the Turkish Transplantation and Burn Foundation, 41(185), 101–121. https://doi.org/10.15390/eb.2016.4766
3.
AreepattamannilS.SantosI. M. (2019). Adolescent students’ perceived information and communication technology (ICT) competence and autonomy: Examining links to dispositions toward science in 42 countries. Computers in Human Behavior, 98(5), 50–58. https://doi.org/10.1016/j.chb.2019.04.005
4.
AronsonJ. Z.ZimmermanJ.CarlosL. (1999). Improving student achievement by extending school: Is it just a matter of time?WestEd.
BeckerF. (2007). Early Childhood Education in Brazil: The obstacles to a successful experience. Revista de La Sede Academica de Mexico de La Facultad Latinoamericana de Ciencias Sociales, 5(2), 515–537.
7.
BelskyJ.VandellD. L.BurchinalM.Clarke-StewartK. A.McCartneyK.OwenM. T.NICHD Early Child Care Research Network. (2007). Are there long-term effects of early child care?Child Development, 78(2), 681–701. https://doi.org/10.1111/j.1467-8624.2007.01021.x
BreimanL. (2001b). The Two Cultures (with comments and a rejoinder by the author). Statistical Science, 16(3), 199–231. https://doi.org/10.1214/ss/1009213726
CheungK.-C.SitP.-S.SohK.-C.IeongM.-K.MakS.-K. (2014). Predicting academic resilience with reading engagement and demographic variables: Comparing Shanghai, Hong Kong, Korea, and Singapore from the PISA perspective. The Asia-Pacific Education Researcher, 23(4), 895–909. https://doi.org/10.1007/s40299-013-0143-4
12.
CouronnéR.ProbstP.BoulesteixA.-L. (2018). Random forest versus logistic regression: A large-scale benchmark experiment. BMC Bioinformatics, 19(1), 270. https://doi.org/10.1186/s12859-018-2264-5
13.
CourtneyM.KarakusM.ErsozluZ.NurumovK. (2022). The influence of ICT use and related attitudes on students’ math and science performance: Multilevel analyses of the last decade’s PISA surveys. Large-Scale Assessments in Education, 10(1), 1–26. https://doi.org/10.1186/s40536-022-00128-6
FriedmanJ. H. (1997). Data mining and statistics What’s the connection? Computing science and statistics: Mining and modeling massive data sets in science, engineering, and business [Symposium]. In 29th Symposium on the Interface.
16.
GamazoA.Martínez-AbadF. (2020). An exploration of factors linked to academic performance in PISA 2018 through data mining techniques. Frontiers in Psychology, 11(1), 575167. https://doi.org/10.3389/fpsyg.2020.575167
17.
GamazoA.Martínez-AbadF.Olmos-MigueláñezS.Jose Rodriguez-CondeM. (2018). Assessment of factors related to school effectiveness in PISA 2015. A multilevel analysis. Revista de La Sociedad Española de Farmacia Hospitalaria, 379(3), 56–78. https://doi.org/10.4438/1988-592X-RE-2017-379-369
18.
Gómez-FernándezN.MediavillaM. (2021). Exploring the relationship between information and communication technologies (ICT) and academic performance: A multilevel analysis for Spain. Socio-Economic Planning Sciences, 77(5), 101009. https://doi.org/10.1016/j.seps.2021.101009
19.
GoosM.PipaJ.PeixotoF. (2021). Effectiveness of grade retention: A systematic review and meta-analysis. Educational Research Review, 34(3), 100401. https://doi.org/10.1016/j.edurev.2021.100401
20.
GordonM. S.CuiM. (2012). The effect of school-specific parenting processes on academic achievement in adolescence and young adulthood. Family Relations, 61(5), 728–741. https://doi.org/10.1111/j.1741-3729.2012.00733.x
21.
GottfredsonD. C.FinkC. M.GrahamN. (1994). Grade retention and problem behavior. American Educational Research Journal, 31(4), 761–784. https://doi.org/10.3102/00028312031004761
22.
GuoB.ZhangR.XuG.ShiC.YangL. (2015). Predicting students performance in educational data mining. In 2015 international symposium on educational Technology (ISET) (pp. 125–128). https://doi.org/10.1109/ISET.2015.33
23.
HerawatiN.NisaK.SetiawanE.NusyirwanN.TiryonoT. (2018). Regularized multiple regression methods to deal with severe multicollinearity. International Journal of Statistics and Applications, 8(4), 167–172. https://doi.org/10.5923/j.statistics.20180804.02
24.
HindmanM. (2015). Building better models: Prediction, replication, and machine learning in the social sciences. The Annals of the American Academy of Political and Social Science, 659(1), 48–62. https://doi.org/10.1177/0002716215570279
25.
IBGE. (2022). Synthesis of social indicators: An analysis of the living conditions of the Brazilian population. IBGE.
26.
ImmekusJ. C.JeongT.-S.YooJ. E. (2022). Machine learning procedures for predictor variable selection for schoolwork-related anxiety: Evidence from PISA 2015 mathematics, reading, and science assessments. Large-Scale Assessments in Education, 10(1), 30. https://doi.org/10.1186/s40536-022-00150-8
KarakolidisA.PitsiaV.EmvalotisA. (2016). Examining students’ achievement in mathematics: A multilevel analysis of the Programme for international student assessment (PISA) 2012 data for Greece. International Journal of Educational Research, 79(5), 106–115. https://doi.org/10.1016/j.ijer.2016.05.013
29.
KellM.KellP. (2014). Global testing: PISA, TIMSS and PIRLS. In KellM.KellP. (Eds.), Literacy and language in East Asia: Shifting meanings, values and approaches (pp. 33–49): Springer. https://doi.org/10.1007/978-981-4451-30-7_5
30.
KidronY.LindsayJ. (2014). The effects of increased learning time on student academic and nonacademic outcomes: Findings from a meta-analytic review. Institute of Education Sciences.
31.
KimY.SherradenM.ClancyM. (2013). Do mothers’ educational expectations differ by race and ethnicity, or socioeconomic status?Economics of Education Review, 33(2), 82–94. https://doi.org/10.1016/j.econedurev.2012.09.007
32.
KnightonT. (2006). Educational outcomes at age 19 associated with reading ability at age 15. Statistics Canada.
33.
KugerS.KliemeE.JudeN.KaplanD. (2016). Assessing contexts of learning: An international perspective. Springer International Publishing. https://doi.org/10.1007/978-3-319-45357-6
34.
LazarevicL. B.OrlicA. (2018). PISA 2012 mathematics literacy in Serbia: A multilevel analysis of students and schools. Psihologija, 51(4), 413–432. https://doi.org/10.2298/psi170817017l
35.
LépineA. (2015). School reputation and school choice in Brazil: A regression discontinuity design (No. 38). Department of Economics- FEA/USP.
36.
LezhninaO.KismihókG. (2022). Combining statistical and machine learning methods to explore German students’ attitudes towards ICT in PISA. International Journal of Research and Method in Education, 45(2), 180–199. https://doi.org/10.1080/1743727X.2021.1963226
37.
ManginoA. A.FinchW. H. (2021). Prediction with mixed effects models: A Monte Carlo simulation study. Educational and Psychological Measurement, 81(6), 1118–1142. https://doi.org/10.1177/0013164421992818
38.
Martínez-AbadF.GamazoA.Rodríguez-CondeM.-J. (2020). Educational Data Mining: Identification of factors associated with school effectiveness in PISA assessment. Studies In Educational Evaluation, 66(3), 100875. https://doi.org/10.1016/j.stueduc.2020.100875
39.
Martínez GarcíaJ. S.OinonenE.MerinoR.PerosaG. (2021). Education and inequality in Finland, Spain and Brazil. In Towards a comparative analysis of social inequalities between Europe and Latin America. https://doi.org/10.1007/978-3-030-48442-2_4
40.
MostafaJ.SilvaK. (2007). Brazil’s single registry experience: A tool for pro-poor social policies. International Labour Organization.
OECD. (2019c). PISA 2018 results (Volume II) where all students can succeed: Where all students can succeed. OECD Publishing. https://doi.org/10.1787/b5fd1b8f-en
OliveiraA. S. (2016). Progressão continuada e outros dispositivos escolares: êxito e fracasso escolar nos anos iniciais do ensino fundamental. [Universidade de São Paulo]. https://doi.org/10.11606/t.48.2014.tde-17122014-094918
48.
PedregosaF.VaroquauxG.GramfortA.MichelV.ThirionB.GriselO.BlondelM.MüllerA.NothmanJ.LouppeG.PrettenhoferP.WeissR.DubourgV.VanderplasJ.PassosA.CournapeauD.BrucherM.PerrotM.DuchesnayÉ. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12(85), 2825–2830. arXiv. http://arxiv.org/abs/1201.0490
49.
PengY.WangY.HuJ. (2023). Examining ICT attitudes, use and support in blended learning settings for students’ reading performance: Approaches of artificial intelligence and multilevel model. Computers & Education, 203(12), 104846. https://doi.org/10.1016/j.compedu.2023.104846
PholphirulP. (2017). Pre-primary education and long-term education performance: Evidence from Programme for international student assessment (PISA) Thailand. Journal of Early Childhood Research, 15(4), 410–432. https://doi.org/10.1177/1476718X15616834
52.
RomeroC.VenturaS. (2020). Educational data mining and learning analytics: An updated survey. Wiley Interdisciplinary Reviews. Data Mining and Knowledge Discovery, 10(3). https://doi.org/10.1002/widm.1355
53.
Royal Society. (2017). Machine learning: The power and Promise of computers that learn by example. Royal Society.
54.
RyanR. M.DeciE. L. (2017). Self-determination theory: Basic psychological needs in motivation, development, and wellness. Guilford Publications.
55.
RyoM.RilligM. C. (2017). Statistically reinforced machine learning for nonlinear patterns and variable interactions. Ecosphere, 8(11), Article e01976. https://doi.org/10.1002/ecs2.1976
56.
SchleicherA. (2007). Can competencies assessed by PISA be considered the fundamental school knowledge 15-year-olds should possess?Journal of Educational Change, 8(4), 349–357. https://doi.org/10.1007/s10833-007-9042-x
57.
SheH.-C.LinH.-S.HuangL.-Y. (2019). Reflections on and implications of the Programme for International Student Assessment 2015 (PISA 2015) performance of students in Taiwan: The role of epistemic beliefs about science in scientific literacy. Journal of Research in Science Teaching, 56(10), 1309–1340. https://doi.org/10.1002/tea.21553
58.
StroblC.BoulesteixA.-L.ZeileisA.HothornT. (2007). Bias in random forest variable importance measures: Illustrations, sources and a solution. BMC Bioinformatics, 8(1), 25. https://doi.org/10.1186/1471-2105-8-25
59.
StroblC.MalleyJ.TutzG. (2009). An introduction to recursive partitioning: Rationale, application, and characteristics of classification and regression trees, bagging, and random forests. Psychological Methods, 14(4), 323–348. https://doi.org/10.1037/a0016973
60.
TakayamaK. (2008). The politics of international league tables: PISA in Japan’s achievement crisis debate. Comparative Education Review, 44(4), 387–407. https://doi.org/10.1080/03050060802481413
61.
TouronJ.Lopez-GonzalezE.Lizasoain HernandezL.Garcia San PedroM. J.Navarro AsencioE. (2018). Spanish high and low achievers in science in PISA 2015: Impact analysis of some contextual variables. Revista de La Sociedad Española de Farmacia Hospitalaria, 380(2), 156–184. https://doi.org/10.4438/1988-592X-RE-2017-380-376
62.
Unesco. (1997). International standard classification of education: Isced 1997. Unesco.
63.
WagnerP.MerloJ. (2014). Discriminatory accuracy of a random effect in multilevel logistic regression. International Journal of Epidemiology, 44(5). https://doi.org/10.1093/IJE/DYV097.190
64.
WangX. S.PerryL. B.MalpiqueA.IdeT. (2023). Factors predicting mathematics achievement in PISA: A systematic review. Large-Scale Assessments in Education, 11(1), 24. https://doi.org/10.1186/s40536-023-00174-8
WrightM. N.KönigI. R. (2019). Splitting on categorical predictors in random forests. PeerJ, 7(1), Article e6339. https://doi.org/10.7717/peerj.6339
67.
YarkoniT.WestfallJ. (2017). Choosing prediction over explanation in psychology: Lessons from machine learning. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 12(6), 1100–1122. https://doi.org/10.1177/1745691617693393
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
