Restricted accessResearch articleFirst published online 2025-5
Are Students on Track? Comparing the Predictive Validity of Administrative and Survey Measures of Cognitive and Noncognitive Skills for Long-Term Outcomes
Education leaders need valid metrics to predict students’ long-term success. We use a unique data set with cognitive skills, self-regulation, behavior, course performance, and test scores for eighth-grade students from a Northeast school district. We link these data to students’ high school outcomes, college enrollment, persistence, and on-time degree completion. Survey-based cognitive and self-regulation measures predict high school and college outcomes. However, these relationships become small and lose statistical significance when test scores, grade point average, and an absences-suspensions index are included in the predictive models. For leaders hoping to identify the best on-track indicators for college completion, the information collected in student longitudinal data systems better predicts both short- and long-term educational outcomes than the survey-based self-regulation and cognitive measures.
AlmlundM.DuckworthA. L.HeckmanJ.KautzT. (2011). Personality psychology and economics. In HanushekE. A.MachinS.WoessmannL. (Eds.), Handbook of the economics of education (pp. 1–181). Elsevier.
2.
AngD. (2021). The effects of police violence on inner-city students. The Quarterly Journal of Economics, 136(1), 115–168, https://doi.org/10.1093/qje/qjaa027
3.
AustinW.FiglioD.GoldhaberD.HanushekE.KilbrideT.KoedelC.LeeJ. S.LouJ.ÖzekU.ParsonsE.RivkinS.SassT.StrunkK. (2020). Where are initially low-performing students the most likely to succeed? A multi-state analysis of academic mobility (CALDER Working Paper No. 227–0220). https://wes-austin.com/files/CALDER_academic_mobility.pdf
4.
BowenW. G.ChingosM. M.McPhersonM. S. (2009). Crossing the finish line: Completing college at America’s public universities. Princeton University Press.
5.
BrownL.SherbenouR. J.JohnsenS. K. (2010). Test of nonverbal intelligence—Fourth edition (Version A). The Psychological Corp.
CanbolatY. (2024). Early warning for whom? Regression discontinuity evidence from the effect of early warning system on student absence. Educational Evaluation and Policy Analysis. Advance online publication. https://doi.org/10.3102/01623737231221503
8.
CaseR.Midian KurlandD.GoldbergJ. (1982). Operational efficiency and the growth of short-term memory span. Journal of Experimental Child Psychology, 33, 386–404.
9.
ChettyR.FriedmanJ. N.HilgerN.SaezE.Whitmore SchanzenbachD.YaganD. (2011). How does your kindergarten classroom affect your earnings? Evidence from Project STAR. The Quarterly Journal of Economics, 126(4), 1593–1660.
10.
ChettyR.FriedmanJ. N.RockoffJ. E. (2014). Measuring the impacts of teachers I: Evaluating bias in teacher value-added estimates. American Economic Review, 104(9), 2593–2632.
11.
ConnellyB. S.OnesD. S. (2010). Another perspective on personality: Meta-analytic integration of observers’ accuracy and predictive validity. Psychological Bulletin, 136(6), 1092–1122.
12.
CowanN. (2005). Working-memory capacity limits in a theoretical context. In IzawaC.OhtaN. (Eds.), Human learning and memory: Advances in theory and applications. The 4th Tsukuba International Conference on Memory (pp. 155–175). Erlbaum.
13.
DeeT. S.WestM. R. (2011). The noncognitive returns to class size. Education Evaluation and Policy Analysis, 33(1), 23–46.
14.
DuckworthA. L.QuinnP. D. (2009). Development and validation of the Short Grit Scale (Grit-S). Journal of Personality Assessment, 91, 166–174.
15.
DuckworthA. L.PetersonC.MatthewsM. D.KellyD. R. (2007). Grit: Perseverance and passion for long-term goals. Journal of Personality and Social Psychology, 92, 1087–1101.
16.
DuckworthA. L.QuinnP. D.TsukayamaE. (2012). What No Child Left Behind leaves behind: The roles of IQ and self-control in predicting standardized achievement test scores and report card grades. Journal of Educational Psychology, 104(2), 439–451. https://doi.org/10.1037/a0026280
17.
DuckworthA. L.YeagerD. S. (2015). Measurement matters: Assessing personal qualities other than cognitive ability for educational purposes. Educational Researcher, 44(4), 237–251.
18.
DweckC. (2006). Mindset: The new psychology of success. Random House.
19.
EastonJ. Q.JohnsonE.SartainL. (2017). The predictive power of ninth-grade GPA. University of Chicago Consortium on School Research.
20.
FarringtonC. A.RoderickM.AllensworthE.NagaokaJ.KeyesT. S.JohnsonD. W.BeechumN. O. (2012). Teaching adolescents to become learners: The role of noncognitive factors in shaping school performance–A critical literature review. Consortium on Chicago School Research.
21.
FinnA. S.KraftM. A.WestM. R.LeonardJ. A.BishC. E.MartinR. E.SheridanM. A.GabrieliC. F. O.GabrieliJ. D. E. (2014). Cognitive skills, student achievement tests, and schools. Psychological Science, 25(3), 736–744. https://doi.org/10.1177/0956797613516008
22.
FryA. F.HaleS. (1996). Processing speed, working memory, and fluid intelligence: Evidence for a developmental cascade. Psychological Science, 7(4), 237–241. http://www.jstor.org/stable/40062952
23.
GallaB. M.ShulmanE. P.PlummerB. D.GardnerM.HuttS. J.GoyerJ. P.D’MelloS. K.FinnA. S.DuckworthA. L. (2019). Why high school grades are better predictors of on-time college graduation than are admissions test scores: The roles of self-regulation and cognitive ability. American Education Research Journal, 20(10), 1–39.
24.
GeiserS.SantelicesM. V. (2007). Validity of high-school grades in predicting student success beyond the freshman year: High-school record vs. standardized tests as indicators of four-year college outcomes. Center for Studies in Higher Education.
25.
GoldhaberD.WolffM.DalyT. (2020). Assessing the accuracy of elementary school test scores as predictors of students’ high school outcomes (CALDER Working Paper No. 235–0520). https://files.eric.ed.gov/fulltext/ED605735.pdf
26.
HeckmanJ. J.HumphriesE.VeramendiG. (2018). Returns to education: The causal effects of education on earnings, health, and smoking. Journal of Political Economy, 126(S1), S197–S246.
27.
HeckmanJ. J.MossoS. (2014). The economics of human development and social mobility. Annual Review of Economics, 6(1), 689–733.
28.
JacksonK. (2018). What do test scores miss? The importance of teacher effects on non-test score outcomes. Journal of Political Economy, 126, 2072–2107.
29.
JacobB.RothsteinJ. (2016). The measurement of student ability in modern assessment systems. Journal of Economic Perspectives, 30(3), 85–108.
30.
JohnO. P.SrivastavaS. (1999). The Big Five trait taxonomy: History, measurement, and theoretical perspectives. In PervinL. A.JohnO. P. (Eds.), Handbook of personality: Theory and research (2nd ed., pp. 102–138). The Guilford Press.
31.
KanopkaK.ClaroS.LoebS.WestM.FrickeH. (2024). Are changes in reported social-emotional skills just noise? The predictive power of longitudinal differences in self-reports. AERA Open, 10. https://doi.org/10.1177/23328584241233277
32.
KautzT. D.ZanoniW. (2014). Measuring and fostering noncognitive skills in adolescence: Evidence from Chicago Public Schools and the OneGoal Program [Unpublished manuscript]. Department of Economics, University of Chicago.
LlerasC. (2008). Do skills and behaviors in high school matter? The contribution of noncognitive factors in explaining differences in educational attainment and earnings. Social Science Research, 37(3), 888–902.
35.
LiuJ.KuhfeldM.LeeM. (2025). Noncognitive factors and student long-run success: Comparing the predictive validity of observable academic behaviors and social-emotional skills. Educational Policy, 39(1), 131–169. https://doi.org/10.1177/08959048231209262
36.
OhI. S.WangG.MountM. K. (2011). Validity of observer ratings of the five-factor model of personality traits: A meta-analysis. Journal of Applied Psychology, 96(4), 762–773.
37.
PoropatA. E. (2014). Other-rated personality and academic performance: Evidence and implication. Learning and Individual Differences, 34, 24–32.
38.
SackettP. R.BornemanM. J.ConnellyB. S. (2008). High stakes testing in higher education and employment: Appraising the evidence for validity and fairness. American Psychologist, 63(4), 215–227. https://doi.org/10.1037/0003-066X.63.4.215.
39.
SolandJ.ZamarroG.ChengA.HittC. (2019). Identifying naturally occurring direct assessments of social-emotional competencies: The promise and limitations of survey and assessment disengagement metadata. Educational Researcher, 48(7), 466–478.
WechslerD. (2003). Wechsler Intelligence Scale for Children—Fourth edition. The Psychological Corp.
42.
WestM. R.KraftM. A.FinnA. S.MartinR. E.DuckworthA. L.GabrieliC. F. O.GabrieliJ. D. E. (2016). Promise and paradox: Measuring students’ noncognitive skills and the impact of schooling. Education Evaluation and Policy Analysis, 38(1), 148–170.
43.
WestM. R.PierL.FrickeH.HoughH.LoebS.MeyerR. H.RiceA. B. (2020). Trends in student social-emotional learning: Evidence from the first large-scale panel student survey. Educational Evaluation and Policy Analysis, 42(2), 279–303.
44.
WillinghamW. W.PollackJ. M.LewisC. (2002). Grades and test scores: Accounting for observed differences. Journal of Educational Measurement, 39(1), 1–37.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
