Impacts of Academic Recovery Interventions on Student Achievement in 2022 to 2023

Abstract

The COVID-19 pandemic caused large, persistent declines in student achievement. This paper examines 2022–2023 recovery efforts across eight districts, including tutoring, small-group instruction, after-school, extended year, double-dose, digital learning, and expert teacher interventions. The study includes 12 math programs and 15 literacy programs (three covered both subjects). Most served fewer students and provided lower dosage than planned. Using value-added models, we find positive, robust effects in just five subject-intervention pairs: one subject-specific tutoring program (+0.22 SD math, +0.22 SD reading), a small-group reading intervention (+0.33 SD), and assignment to “expert” teachers (+0.06 SD math, +0.11 SD reading). Results highlight the promise of intensive interventions while underscoring the difficulty of scaling them to meet recovery needs.

Keywords

achievement COVID-19 academic intervention tutoring K–12 education policy

Get full access to this article

View all access options for this article.

References

Abadie

Athey

Imbens

G. W.

Wooldridge

J. M.

(2023). When should you adjust standard errors for clustering?. The Quarterly Journal of Economics, 138(1), 1–35.

Arthur

A. M.

Davis

D. L.

(2016). A pilot study of the impact of double-dose robust vocabulary instruction on children’s vocabulary growth. Journal of Research on Educational Effectiveness, 9(2), 173–200.

Barry

S. S.

Sass

T. R.

(2022). The impact of a 2021 Summer School Program on student achievement. Georgia Policy Labs. https://doi.org/10.57709/FAJ9-8597

Bettinger

Fairlie

Kapuza

Kardanova

Loyalka

Zakharov

(2023). Diminishing marginal returns to computer-assisted learning. Journal of Policy Analysis and Management, 42(2), 552–570.

Bhatt

M. P.

Guryan

Khan

S. A.

LaForest-Tucker

Mishra

(2024). Can technology facilitate scale? Evidence from a randomized evaluation of high dosage tutoring (NBER Working Paper No. w32510). National Bureau of Economic Research.

Callen

Carbonari

M. V.

DeArmond

Dewey

Dizon-Ross

Goldhaber

Isaacs

Kane

T. J.

Kuhfeld

McDonald

McEachin

Morton

Muroga

Staiger

D. O.

(2023). Summer school as a learning loss recovery strategy after COVID-19: Evidence from Summer 2022 (Working Paper No. 291-0823). National Center for Analysis of Longitudinal Data in Education Research (CALDER).

Carbonari

M. V.

Davison

DeArmond

Dewey

Dizon-Ross

Goldhaber

Hashim

Kane

T. J.

McEachin

Morton

Muroga

Patterson

Staiger

D. O.

(2024). The challenges of implementing academic COVID recovery interventions: Evidence from the Road to Recovery Project (Working Paper No. 275-0624-2). National Center for Analysis of Longitudinal Data in Education Research (CALDER).

Checkoway

Gamse

Velez

Linkow

(2013). Evaluation of the Massachusetts expanded learning time (ELT) initiative: Final study findings. Society for Research on Educational Effectiveness.

Cortes

Kortecamp

Loeb

Robinson

(2025). A scalable approach to high-impact tutoring for young readers. Learning and Instruction, 95, 102021.

10.

Curriculum Associates. (2023). State of student learning in 2023. https://cdn.bfldr.com/LS6J0F7/at/x8v8wp2c6j4s4wttsw2nwphb/ca-state-of-student-learningtechnical-report-2023.pdf

11.

Curriculum Associates. (2024). State of student learning in 2024. https://cdn.bfldr.com/LS6J0F7/at/wxj37b648k5bkwvf3vrrgj8/2024-State-of-Student-Learning-Research-Awareness-Technical-Report.pdf

12.

DerSimonian

Laird

(1986). Meta-analysis in clinical trials. Controlled Clinical Trials, 7(3), 177–188. https://doi.org/10.1016/0197-2456(86)90046-2

13.

Dewey

Fahle

Kane

Reardon

Staiger

(2024). Federal pandemic relief and academic recovery. Center for Education Policy Research at Harvard University.

14.

Diliberti

M. K.

Schwartz

H. L.

(2022). Districts continue to struggle with staffing, political polarization, and unfinished instruction. RAND Corporation. https://www.rand.org/pubs/research_reports/RRA956-13.html

15.

Doty

Kane

T. J.

Patterson

Staiger

D. O.

(2022). What do changes in state test scores imply for later life outcomes? (NBER Working Paper No. 30701). National Bureau of Economic Research.

16.

Escueta

Quan

Nickow

A. J.

Oreopoulos

(2017). Education technology: An evidence-based review (NBER Working Paper No. 23744). National Bureau of Economic Research.

17.

Fahle

E. M.

Kane

T. J.

Patterson

Reardon

S. F.

Staiger

D. O.

Stuart

E. A.

(2023). School district and community factors associated with learning loss during the COVID-19 pandemic. Center for Education Policy Research at Harvard University.

18.

Fahle

E. M.

Kane

T. J.

Reardon

S. F.

Staiger

D. O.

(2024). The first year of pandemic recovery: A district-level analysis. Center for Education Policy Research at Harvard University.

19.

Goldhaber

Falken

(2024). ESSER and student achievement: Assessing the impacts of the largest one-time federal investment in K12 schools (CALDER Working Paper No. 301-0624). National Center for Analysis of Longitudinal Data in Education Research (CALDER).

20.

Goldhaber

Kane

T. J.

McEachin

Morton

Patterson

Staiger

D. O.

(2023). The educational consequences of remote and hybrid instruction during the pandemic. American Economic Review: Insights, 5(3), 377–392.

21.

Hahn

Todd

Van der Klaauw

(2001). Identification and estimation of treatment effects with a regression-discontinuity design. Econometrica, 69(1), 201–209.

22.

Hanushek

E. A.

Strauss

(2024). A global perspective on US learning losses. Hoover Institution. https://www.hoover.org/sites/default/files/research/docs/HanushekStrauss_WebreadyPDF_240229.pdf

23.

Hedges

L. V.

(1983). Combining independent estimators in research synthesis. British Journal of Mathematical and Statistical Psychology, 36, 123–131. https://doi.org/10.1111/j.2044-8317.1983.tb00768.x

24.

Hill

H. C.

Erickson

(2019). Using implementation fidelity to aid in interpreting program impacts: A brief review. Educational Researcher, 48(9), 590–598.

25.

Isaacs

Kuhfeld

Lewis

(2023). Technical appendix for: Education’s long COVID: 2022–23 Achievement data reveal stalled progress towards pandemic recovery. NWEA. https://www.nwea.org/uploads/Tech-appendix-July-2023-Final.pdf

26.

Jacob

B. A.

Lefgren

(2009). The effect of grade retention on high school completion. American Economic Journal: Applied Economics, 1(3), 33–58.

27.

Kim

J. S.

Quinn

D. M.

(2013). The effects of summer reading on low-income children’s literacy achievement from Kindergarten to Grade 8: A meta-analysis of classroom and home interventions. Review of Educational Research, 83(3), 386–431.

28.

Kohlmoos

Steinberg

M. P.

(2024). Contextualizing the impact of tutoring on student learning: Efficiency, cost effectiveness, and the known unknowns [Research report] Accelerate. https://accelerate.us/efficiency-and-cost-effectiveness

29.

Kraft

M. A.

(2015). How to make additional time matter: Integrating individualized tutorials into an extended day. Education Finance and Policy, 10(1), 81–116.

30.

Kraft

M. A.

(2020). Interpreting effect sizes of education interventions. Educational Researcher, 49(4), 241–253.

31.

Kraft

M. A.

Edwards

D. S.

Cannata

(2024). The scaling dynamics and causal effects of a district-operated tutoring program (EdWorkingPaper No. 24-1030). Annenberg Institute at Brown University. https://doi.org/10.26300/zcw7-4547

32.

Kraft

M. A.

Schueler

B. E.

Falken

(2024). What impacts should we expect from tutoring at scale? Exploring meta-analytic generalizability (EdWorkingPaper No. 24-1031). Annenberg Institute at Brown University. https://edworkingpapers.com/ai24-1031

33.

Kraft

M. A.

Lovison

V. S.

(2024). The effect of student-tutor ratios: Experimental evidence from a Pilot Online Math Tutoring Program (EdWorkingPaper No. 24-976.) Annenberg Institute at Brown University.

34.

Kraft

M. A.

Novicoff

(2024). Time in school: A conceptual framework, synthesis of the causal research, and empirical exploration. American Educational Research Journal, 61(4), 724–766.

35.

Kuhfeld

Diliberti

McEachin

Schweig

Mariano

L. T.

(2023). Typical learning for whom? Guidelines for selecting benchmarks to calculate months of learning (Research brief). NWEA.

36.

Lewis

Kuhfeld

(2022). Progress toward pandemic recovery: Continued signs of rebounding achievement at the start of the 2022–23 school year (Research brief). NWEA.

37.

Lewis

Kuhfeld

(2023). Education’s long COVID: 2022–23 Achievement data reveal stalled progress toward pandemic recovery (Research brief). NWEA.

38.

Lewis

Kuhfeld

(2024). Recovery still elusive: 2023–24 Student achievement highlights persistent achievement gaps and a long road ahead (Research brief). NWEA.

39.

Lynch

Mancenido

(2023). The impact of summer programs on student mathematics achievement: A meta-analysis. Review of Educational Research, 93(2), 275–315.

40.

Makori

Burch

Loeb

(2024). Scaling high-impact tutoring: School level perspectives on implementation challenges and strategies (EdWorkingPaper No. 24-932). Annenberg Institute at Brown University.

41.

McCombs

J. S.

Pane

J. F.

Augustine

C. H.

Schwartz

H. L.

Martorell

Zakaras

(2014). Ready for fall? Near-term effects of voluntary summer learning programs on low-income students’ learning opportunities and outcomes. RAND Corporation. https://doi.org/10.7249/RR815

42.

McCombs

J. S.

Whitaker

A. A.

Yoo

(2017). The value of out-of-school time programs. RAND Corporation. https://www.rand.org/pubs/perspectives/PE267.html

43.

McEachin

Atteberry

(2017). The impact of summer learning loss on measures of school performance. Education Finance and Policy, 12(4), 468–491.

44.

Nickow

Oreopoulos

Quan

(2024). The promise of tutoring for PreK–12 learning: A systematic review and meta-analysis of the experimental evidence. American Educational Research Journal, 61(1), 74–107.

45.

Nomi

(2015). “Double-dose” English as a strategy for improving adolescent literacy: Total effect and mediated effect through classroom peer ability change. Social Science Research, 52, 716–739.

46.

Nomi

Allensworth

E. M.

(2013). Sorting and supporting: Why double-dose algebra led to better test scores but more course failures. American Educational Research Journal, 50(4), 756–788.

47.

Opper

Özek

(2024). A global regression discontinuity design: Theory and application to grade retention policies (CESifo Working Paper No. 10972.) Center for Economic Studies.

48.

Oster

(2019). Unobservable selection and coefficient stability: Theory and evidence. Journal of Business and Economic Statistics, 37(2), 187–204.

49.

Özek

(2021). The effects of middle school remediation on postsecondary success: Regression discontinuity evidence from Florida. Journal of Public Economics, 203, 104518.

50.

Özek

Mariano

L. T.

(2023). Think again: Is grade retention bad for kids?. Thomas B. Fordham Institute.

51.

Pollard

Zandieh

Robinson

C. D.

Loeb

Waymack

(2024). Implementation of the OSSE high impact tutoring initiative: First year report school year 2022–2023 (Research report). National Student Support Accelerator. https://studentsupportaccelerator.org/briefs/implementation-osse-high-impact-tutoring-initiative

52.

Raudenbush

S. W.

(2009). Analyzing effect sizes: Random-effects models. In Cooper

Hedges

L. V.

Valentine

J. C.

(Eds.), The handbook of research synthesis and meta-analysis (pp. 295–315). Russell Sage Foundation.

53.

Ready

D. D.

McCormick

S. G.

Shmoys

R. J.

(2024). The effects of in-school virtual tutoring on student reading development: Evidence from a short-cycle randomized controlled trial (EdWorkingPaper No. 24-942). Annenberg Institute at Brown University.

54.

Reardon

S. F.

A. D.

Shear

B. R.

Fahle

E. M.

Kalogrides

Saliba

(2024). Stanford Education Data Archive (Version 5.0). The Educational Opportunity Project at Stanford University. https://purl.stanford.edu/cs829jn7849

55.

Robinson

C. D.

Bisht

Loeb

(2022). The inequity of opt-in educational resources and an intervention to increase equitable access (EdWorkingPaper No. 22-654). Annenberg Institute at Brown University.

56.

Sacerdote

(2012). When the saints go marching out: Long-term outcomes for student evacuees from Hurricanes Katrina and Rita. American Economic Journal: Applied Economics, 4(1), 109–135.

57.

The White House. (2024, January 17). FACT SHEET: Biden-Harris administration announces improving student achievement agenda in 2024. https://www.whitehouse.gov/briefing-room/statements-releases/2024/01/17/fact-sheet-biden-harris-administration-announces-improving-student-achievement-agenda-in-2024/

58.

Thum

Y. M.

Kuhfeld

(2020, April). NWEA 2020 MAP growth: Achievement status and growth norms—Tables for students and schools. NWEA. https://teach.mapnwea.org/impl/NormsTables.pdf

59.

U.S. Department of Education. (2021). Education in a pandemic: The disparate impacts of COVID-19 on America’s students. U.S. Department of Education, Office of Civil Rights.

60.

U.S. Department of Education. (2022). National Assessment of Educational Progress (NAEP) 2022 long-term trend assessment results: Reading and mathematics. Institute of Education Sciences, National Center for Education Statistics. https://www.nationsreportcard.gov/highlights/ltt/2022/

61.

U.S. Department of Education. (2023). Digest of education statistics: 2023. Institute of Education Sciences, National Center for Education Statistics.