The pandemic disrupted schools’ ability to provide students with essential academic and social-emotional resources. To date, research on pandemic-related learning loss has focused on standardized test scores and absenteeism, leaving the impact on student experiences underexplored. This study examines 6 years of student survey data to assess how student experience outcomes changed before, during, and after the pandemic. Using piecewise growth models, we find that school climate trajectories differed markedly from those of test scores and absenteeism, suggesting these measures capture distinct dimensions of school quality. Further, student engagement, in the highest poverty schools, declined an average of 0.21 points on a 1–5 Likert scale upon return to in-person (SE = 0.09, p < .05), even as students reported an average increase of 0.75 points in valuing of learning (SE = 0.16, p < .05). These findings underscore the limitations of existing accountability measures and offer lessons for addressing student engagement.
BackesB.CowanJ.DeArmondM.GoldhaberD.TheobaldR. (2025). Measuring the quality of the teacher workforce. Phi Delta Kappan, 107(1–2), 60–62.
2.
BetthäuserB. A.Bach-MortensenA. M.EngzellP. (2023). A systematic review and meta-analysis of the evidence on learning during the COVID-19 pandemic. Nature Human Behaviour, 7(3), 375–385.
CronbachL. J. (1988). Internal consistency of tests: Analyses old and new. Psychometrika, 53(1), 63–70.
6.
DatnowA.ParkV. (2014). Data-driven leadership. John Wiley & Sons.
7.
DatnowA.ParkV.WohlstetterP. (2007). Achieving with data: How high-performing school systems use data to improve instruction for elementary students. Center on Educational Governance, University of Southern California.
8.
DavisJ. R.WarnerN. (2018). Schools matter: The positive relationship between New York City high schools’ student academic progress and school climate. Urban Education, 53(8), 959–980
9.
DebnamK. J.EdwardsK.MaengJ. L.CornellD. (2022). Educational leaders’ perceptions and uses of school climate data. Journal of School Leadership, 32(4), 362–383.
10.
DeeT. S. (2024). Higher chronic absenteeism threatens academic recovery from the COVID-19 pandemic. Proceedings of the National Academy of Sciences, 121(3), Article e2312249121.
11.
DeeT. S.JacobB.SchwartzN. L. (2013). The effects of NCLB on school resources and practices. Educational Evaluation and Policy Analysis, 35(2), 252–279.
12.
DonnellyR.PatrinosH. A. (2022). Learning loss during Covid-19: An early systematic review. Prospects, 51(4), 601–609.
13.
DweckC. S.WaltonG. M.CohenG. L. (2014). Academic tenacity: Mindsets and skills that promote long-term learning. Bill & Melinda Gates Foundation.
14.
EarlL. M.KatzS. (2006). Leading schools in a data-rich world: Harnessing data for school improvement. Corwin Press.
15.
EngzellP.FreyA.VerhagenM. D. (2021). Learning loss due to school closures during the COVID-19 pandemic. Proceedings of the National Academy of Sciences, 118(17), e2022376118.
16.
ErwinE.BloomquistL. (2024). 50-State comparison: School accountability systems. Education Commission of the States.
17.
FergusonR. (2012). Learning analytics: Drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5–6), 304–317.
18.
GagnonD. J.SchneiderJ. (2019). Holistic school quality measurement and the future of accountability: Pilot-test results. Educational Policy, 33(5), 734–760.
GehlbachH.HoughH. J. (2018). Measuring social emotional learning through student surveys in the CORE Districts: A pragmatic approach to validity and reliability. Policy Analysis for California Education.
21.
GrossB.OpalkaA. (2020). Too many schools leave learning to chance during the pandemic. Center on Reinventing Public Education.
22.
HalloranC.JackR.OkunJ. C.OsterE. (2021). Pandemic schooling mode and student test scores: Evidence from US states (No. w29497). National Bureau of Economic Research.
23.
HarbatkinE.BurnsJ.CullumS. (2025). The role of school climate in school turnaround. Teachers College Record, 126(11–12), 126–170.
JacobB. A. (2005). Accountability, incentives and behavior: The impact of high-stakes testing in the Chicago Public Schools. Journal of Public Economics, 89(5–6), 761–796.
26.
JenningsJ. L.BearakJ. M. (2014). “Teaching to the test” in the NCLB era: How test predictability affects our understanding of student performance. Educational Researcher, 43(8), 381–389.
27.
JenningsJ.SohnH. (2014). Measure for measure: How proficiency-based accountability systems affect inequality in academic achievement. Sociology of Education, 87(2), 125–141.
28.
JonesS. E. (2022). Mental health, suicidality, and connectedness among high school students during the COVID-19 pandemic—Adolescent Behaviors and Experiences Survey, United States, January–June2021. MMWR Supplements, 71(3), 16–21.
29.
KaneM. T.WoolsS. W. (2020). Perspectives on the validity of classroom assessments. In BrookhartS. M.McMillanJ. H. (Eds.), Classroom assessment and educational measurement (pp. 11–26). Routledge. https://doi.org/10.4324/9780429507533
30.
KaputK. (2018). Briefing memo: Summaries of States’ ESSA school quality/student success measures. Education Evolving.
31.
KlevanS.Leung-GagnéM.NakajimaT. M. (2024). Using data to improve school climate: Insights from three California schools. Learning Policy Institute.
32.
KoretzD. (2017). The testing charade: Pretending to make schools better. University of Chicago Press.
33.
KrachmanS. B.ArnoldR.LaRoccaR. (2016). Expanding the definition of student success: A case study of the CORE Districts. Transforming Education.
34.
KuhfeldM.LewisK.PeltierT. (2023). Reading achievement declines during the COVID-19 pandemic: Evidence from 5 million US students in grades 3–8. Reading and Writing, 36(2), 245–261.
35.
KuhfeldM.SolandJ.LewisK. (2022). Test score patterns across three COVID-19-impacted school years. Educational Researcher, 51(7), 500–506.
36.
KuhfeldM.SolandJ.LewisK.RuzekE.JohnsonA. (2022). The COVID-19 school year: Learning and recovery across2020-2021. Aera Open, 8, 23328584221099306.
37.
KuhfeldM.TarasawaB.JohnsonA.RuzekE.LewisK. (2020). Learning during COVID-19: Initial Findings on Students’ Reading and Math Achievement and Growth (Brief). NWEA.
38.
LaddH.LoebS. (2013). The challenges of measuring school quality: Implications for educational equity. In AllenD.ReichR. (Eds.), Education, justice, and democracy (pp. 22–55). University of Chicago Press.
39.
LaneS. (2020). Test-based accountability systems: The importance of paying attention to consequences. ETS Research Report Series.
40.
LeeJ.LeeM. (2020). Is “whole child” education obsolete? Public school principals’ educational goal priorities in the era of accountability. Educational Administration Quarterly, 56(5), 856–884.
41.
LeeM.LouisK. S. (2019). Mapping a strong school culture and linking it to sustainable school improvement. Teaching and Teacher Education, 81, 84–96.
42.
LewisK.KuhfeldM. (2022). Progress toward Pandemic Recovery: Continued Signs of Rebounding Achievement at the Start of the 2022-23 School Year (Brief). Center for School and Student Progress at NWEA.
43.
MaasC. J.HoxJ. J. (2005). Sufficient sample sizes for multilevel modeling. Methodology, 1(3), 86–92.
44.
MaldonadoJ. E.De WitteK. (2022). The effect of school closures on standardised student test outcomes. British Educational Research Journal, 48(1), 49–94.
45.
MarionS. (2020). Using opportunity-to-learn data to support educational equity. Center for Assessment, National Center for the Improvement of Educational Assessment.
46.
McGuinnP. (2023). Assessment and evaluation. In BlokhuisJ. C. (Ed.), Legal foundations of education (pp. 141–170). Bloomsbury.
47.
McNeishD. M.StapletonL. M. (2016). The effect of small sample size on two-level model estimates: A review and illustration. Educational Psychology Review, 28(2), 295–314.
OakesJ.WelnerK. (2023). From schools of opportunity to systems of opportunity. In YorkA.WelnerK.KelleyL. M. (Eds.), Schools of opportunity (pp. 193–217). Teachers College Press.
PhillipsS. F.RowleyJ. F. (2016). The tripod school climate index: An invariant measure of school safety and relationships. Social Work Research, 40(1), 31–39.
52.
PhillipsS. F.RowleyJ. F.FergusonR. F. (2018). The Tripod school climate index: Evidence of score reliability and validity. Health Behavior and Policy Review, 5(4), 95–108.
53.
PolikoffM.ClayI.SilverD. (2023). Beyond Test Scores: Broader Academic Consequences of the COVID-19 Pandemic on American Students. Report from a Consensus Panel. Center on Reinventing Public Education.
54.
RoegmanR.Perkins-WilliamsR.BudzynM.Killian-TarrO.AllenD. (2022). How conceptions of equity inform principal data use: A purposeful examination of principals from four school districts. Educational Administration Quarterly, 58(2), 183–222. https://doi.org/10.1177/0013161X211056084
55.
RothsteinR.JacobsenR.WilderT. (2009). From accreditation to accountability. Phi Delta Kappan, 90(9), 624–629.
56.
SchneiderJ. (2017). Beyond test scores: A better way to measure school quality. Harvard University Press.
SchneiderJ.JacobsenR.WhiteR.GehlbachH. (2017). Building a better measure of school quality. Phi Delta Kappan, 98(7), 43–48.
59.
SchneiderJ.NoonanJ.WhiteR. S.GagnonD.CareyA. (2021). Adding “student voice” to the mix: Perception surveys and state accountability systems. AERA Open, 7, 2332858421990729.
60.
SchneiderJ.SaultzA. (2020). Authority and control: The tension at the heart of standards-based accountability. Harvard Educational Review, 90(3), 419–445.
61.
SeltzerM.SvartbergM. (1998). The use of piecewise growth models in evaluations of interventions (CSE-TR-477). National Center for Research on Evaluation, Standards, and Student Testing (CRESST), University of California, Los Angeles. https://files.eric.ed.gov/fulltext/ED426068.pdf
62.
TorpieC.KhananiN.QuarlesB.RaunerM. (2023). Equity in Measuring School Quality: Comparing the Robust and Equitable Measures to Inspire Quality Schools (REMIQS) framework and state accountability systems in Arizona, Kentucky, Massachusetts, Texas and Virginia. WestEd and KnowledgeWorks.
63.
WaymanJ. C.StringfieldS. (2006). Data use for school improvement: School practices and research perspectives. American Journal of Education, 112(4), 463–468. https://doi.org/10.1086/505055
64.
WyckoffJ. (2026). Puzzling over declining academic achievement. Journal of Policy Analysis and Management, 45(2), Article e70052.
65.
ZumpeE.DzormekuD.SchneiderJ. (2025). Understanding school principals’ perspectives on using data for improvement. Leadership and Policy in Schools, 1–17. Advance online publication. https://doi.org/10.1080/15700763.2025.2532074
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
