Summer School as a Learning Loss Recovery Strategy After COVID-19: Evidence from Summer 2022

Abstract

To make up for pandemic-related learning losses, many U.S. public school districts have increased enrollment in their summer school programs. We assessed summer school as a strategy for COVID-19 learning recovery by tracking the academic progress of students who attended summer school in 2022 across eight districts serving 400,000 students. Using value-added models that control for students’ demographics and prior achievement, we estimated a positive effect of summer school on math test achievement (0.03 standard deviations) but not on reading tests. With only 13% of students participating, these districts’ summer programs closed ~2% of their total pandemic-related learning losses in math and none of their losses in reading.

Keywords

academic interventions achievement COVID-19 recovery pandemic recovery summer school

Get full access to this article

View all access options for this article.

References

Abadie

Athey

Imbens

G. W.

Wooldridge

J. M.

(2022). When should you adjust standard errors for clustering? Quarterly Journal of Economics, 138(1), 1–35. https://doi.org/10.1093/qje/qjac038

Abdulkadiroglu

Angrist

J. D.

Dynarski

Kane

Pathak

P. A.

(2011). Accountability and flexibility in public schools: Evidence from Boston's charters and pilots. Quarterly Journal of Economics, 126, 699–748. https://doi.org/10.1093/qje/qjr017

Augustine

C. H.

McCombs

J. S.

Pane

J. F.

Schwartz

H. L.

Schweig

McEachin

Siler-Evans

(2016). Learning from summer: Effects of voluntary summer learning programs on low-income urban youth (Report RR-1557-WF). RAND Corporation.

Bacher-Hicks

Chin

Kane

T. J.

Staiger

D. O.

(2019). An experimental evaluation of three teacher quality measures: Value-added, classroom observations, and student surveys. Economics of Education Review, 73, 101919. https://doi.org/10.1016/j.econedurev.2019.101919

Barry

S. S.

Sass

T. R.

(2022). The impact of a 2021 summer school program on student achievement. Georgia Policy Labs. https://doi.org/10.57709/FAJ9-8597

Bell

S. R.

Carrillo

(2007) Characteristics of effective summer learning programs in practice. New Directions for Youth Development, 114, 45–63. https://doi.org/10.1002/yd.212

Blazar

Heller

Kane

T. J.

Polikoff

Staiger

D. O.

Carrell

Goldhaber

Harris

D. N.

Hitch

Holden

K. L.

Kurlaendar

(2020). Curriculum reform in the Common Core era: Evaluating elementary math textbooks across six U.S. states. Journal of Policy Analysis and Management, 39(4), 966–1019. https://doi.org/10.1002/pam.22257

Boss

Railsback

(2002). Summer school programs: A look at the research, implications for practice, and program sampler. Northwest Regional Educational Laboratory.

Callen

Goldhaber

Kane

T. J.

McDonald

McEachin

Morton

(2024). Pandemic learning loss by student baseline achievement: Extent and sources of heterogeneity (Working Paper No. 292-0224). Calder Foundation

10.

Carbonari

M. V.

Davison

DeArmond

Dewey

Dizon-Ross

Goldhaber

Hashim

Kane

T. J.

McEachin

Morton

Muroga

Patterson

Staiger

D. O.

(2024a). The impact and implementation of academic interventions during COVID-19: Evidence from the road to recovery project. AERA Open. Advance online publication. https://doi.org/10.1177/23328584241281286

11.

Carbonari

M. V.

DeArmond

Dewey

Dizon-Ross

Goldhaber

Kane

T. J.

McDonald

McEachin

Morton

Muroga

Salazar

Staiger

D. O.

(2024b). Impacts of academic recovery interventions on student achievement in 2022-23 (Working Paper No. 303-0724). Calder Foundation.

12.

Cascio

E. U.

Staiger

D. O.

(2012). Knowledge, tests, and fadeout in educational interventions (NBER Working Paper No. 18038). National Bureau of Economic Research. https://www.nber.org/papers/w18038

13.

Chetty

Friedman

J. N.

Rockoff

J. E.

, (2014). Measuring the impacts of teachers: I. Evaluating bias in teacher value-added estimates. American Economic Review, 104(9), 2593–2632. https://doi.org/10.1257/aer.104.9.2593

14.

Cooper

(2001). Summer school: Research-based recommendations for policymakers. SERVE Policy Brief. http://files.eric.ed.gov/fulltext/ED456557.pdf

15.

Cooper

Nye

Charlton

Lindsay

Greathouse

(1996). The effects of summer vacation on achievement test scores: A narrative and meta-analytic review. Review of Educational Research, 66(3), 227–268. https://doi.org/10.3102/00346543066003227

16.

Curriculum Associates. (2020). Understanding student needs: Early results from fall assessments. https://www.curriculumassociates.com/-/media/mainsite/files/i-ready/iready-diagnostic-results-understanding-student-needs-paper-2020.pdf

17.

Dee

T. S.

(2024). Higher chronic absenteeism threatens academic recovery from the COVID-19 pandemic. Proceedings of the National Academy of Sciences, 121(3), e2312249121. https://doi.org/10.1073/pnas.2312249121

18.

Deming

D. J.

(2014). Using school choice lotteries to test measures of school effectiveness (NBER Working Paper No. 19803). National Bureau of Economic Research.

19.

DerSimonian

Laird

(1986). Meta-analysis in clinical trials. Controlled Clinical Trials, 7(3), 177–188. https://doi.org/10.1016/0197-2456(86)90046-2

20.

Diliberti

M. K.

Schwartz

H. L.

(2024). School was in session this summer, but less than half of eligible students enrolled: Selected findings from the fifth American school district panel survey. RAND Corporation. https://doi.org/10.7249/RRA956-22

21.

Diliberti

M. K.

Schwartz

H. L.

(2022). Districts continue to struggle with staffing, political polarization, and unfinished instruction: Selected findings from the fifth American School District Panel Survey. RAND Corporation. https://doi.org/10.7249/RRA956-13

22.

DiMarco

Jordan

P. W.

(2022). How local educators plan to spend billions in federal COVID aid. FutureEd Explainer. https://www.future-ed.org/local-covid-relief-spending/

23.

Dorn

Hancock

Sarakatsannis

Viruleg

(2021). COVID-19 and education: The lingering effects of unfinished learning (Vol. 27). McKinsey & Company.

24.

Gambi

De Witte

(2024). The resilience in test scores and the mitigating effect of summer schools one year after the COVID-19 school closures. Labour Economics. Advance online publication. https://doi.org/10.1016/j.labeco.2024.102549

25.

Goldhaber

Kane

T. J.

McEachin

Morton

Patterson

Staiger

D. O.

(2023). The educational consequences of remote and hybrid instruction during the pandemic. American Economic Review: Insights, 5(3), 377–392. https://doi.org/10.1257/aeri.20220180

26.

Harter

(2023, June 26). This summer LAUSD tackles learning loss in a fun-filled environment. Los Angeles Daily News. https://www.dailynews.com/2023/06/26/this-summer-lausd-tackles-learning-loss-in-a-fun-filled-environment/

27.

Hedges

L. V.

(1983). Combining independent estimators in research synthesis. British Journal of Mathematical and Statistical Psychology, 36, 123–131. https://doi.org/10.1111/j.2044-8317.1983.tb00768.x

28.

Henry

Purtell

K. M.

Bastian

K. C.

Fortner

C. K.

Thompson

C. L.

Campbell

S. L.

Patterson

K. M.

(2014). The effects of teacher entry portals on student achievement. Journal of Teacher Education, 65(1), 7–23. https://doi.org/10.1177/0022487113503871

29.

Jackson

C. K.

Rockoff

J. E.

Staiger

D. O.

(2014). Teacher effects and teacher-related policies. Annual Review of Economics, 6(1), 801–825. https://doi.org/10.1146/annurev-economics-080213-040845

30.

Kilbride

Hopkins

Strunk

K. O.

Imberman

(2021). K–8 student achievement and achievement gaps on Michigan's 2020–21 benchmark and summative assessments [Research brief.] Education Policy Innovation Collaborative.

31.

Kim

J. S.

Quinn

D. M.

(2013). The effects of summer reading on low-income children's literacy achievement from kindergarten to grade 8: A meta-analysis of classroom and home interventions. Review of Educational Research, 83(3), 386–431. https://doi.org/10.3102/0034654313483906

32.

Kraft

M. A.

Novicoff

(2024). Time in school: A conceptual framework, synthesis of the causal research, and empirical exploration. American Educational Research Journal, 61(4), 724–766. https://doi.org/10.3102/00028312241251857

33.

Kraft

M. A.

Edwards

D. S.

Cannata

(2024a). The scaling dynamics and causal effects of a district-operated tutoring program (EdWorkingPaper No. 24-1030). https://doi.org/10.26300/zcw7-4547

34.

Kraft

M. A.

Schueler

B. E.

Falken

(2024b). What impacts should we expect from tutoring at scale? Exploring meta-analytic generalizability (EdWorkingPaper No. 24 -1031). https://doi.org/10.26300/zygj-m525

35.

Lewis

Kuhfeld

(2023). Education’s long COVID: 2022–23 achievement data reveal stalled progress toward pandemic recovery [Research brief.] NWEA.

36.

Lewis

Kuhfeld

(2022). Progress toward pandemic recovery: Continued signs of rebounding achievement at the start of the 2022–23 school year [Research brief.] NWEA.

37.

Lewis

Kuhfeld

Ruzek

McEachin

(2021). Learning during COVID-19: Reading and math achievement in the 2020–21 school year [Research brief.] NWEA.

38.

Lipsey

M. W.

Puzio

Yun

Hebert

M. A.

Steinka-Fry

Cole

M. W.

Roberts

Anthony

K. S.

Busick

M. D.

(2012), Translating the statistical representation of the effects of education intervention into more readily interpretable forms (NCSER 2013–3000). National Center for Special Education Research, Institute of Education Sciences, U.S. Department of Education. https://ies.ed.gov/ncser/pubs/20133000/pdf/20133000.pdf

39.

Lynch

Mancenido

(2023). The impact of summer programs on student mathematics achievement: A meta-analysis. Review of Educational Research, 93(2), 275–315. https://doi.org/10.3102/00346543221105543

40.

MacGillis

(2023, June 19). What can we do about pandemic-related learning loss? The New Yorker. https://www.newyorker.com/magazine/2023/06/26/what-can-we-do-about-pandemic-related-learning-loss

41.

McCombs

J. S.

Pane

J. F.

Augustine

C. H.

Schwartz

H. L.

Martorell

Zakaras

(2014). Ready for fall? Near-term effects of voluntary summer learning programs on low-income students’ learning opportunities and outcomes. RAND Corporation. https://doi.org/10.7249/RR815

42.

McEachin

Augustine

C. H.

McCombs

(2018). Effective summer programming: What educators and policymakers should know. American Educator, 42(1), 10–11.

43.

Nickow

Oreopoulos

Quan

(2024). The promise of tutoring for pre-K–12 learning: A systematic review and meta-analysis of the experimental evidence. American Educational Research Journal, 61(1), 74–107. https://doi.org/10.3102/00028312231208687

44.

Plecki

M. L.

Elfers

A. M.

Nakamura

(2012). Using evidence for teacher education program improvement and accountability: An illustrative case of the role of value-added measures. Journal of Teacher Education, 63(5), 318–334. https://doi.org/10.1177/0022487112447110

45.

Raudenbush

S. W.

(2009). Analyzing effect sizes: Random-effects models. In Cooper

Hedges

L. V.

Valentine

J. C.

(Eds.), The handbook of research synthesis and meta-analysis (pp. 295–315). Sage.

46.

Reardon

S. F.

Fahle

E. M.

A. D.

Shear

B. R.

Kalogrides

Saliba

Kane

T. J.

(2023). Stanford education data archive (Version SEDA 2022 2.0). Stanford University Libraries. http://purl.stanford.edu/dt080zr0625

47.

Riehl

Welch

(2022). Accountability, test prep incentives, and the design of math and English exams. Journal of Policy Analysis and Management, 42(1), 60–96. https://doi.org/10.1002/pam.22417

48.

Robinson

C. D.

Bisht

Loeb

(2025). The inequity of opt-in educational resources and an intervention to increase equitable access. Educational Researcher. Advance online publication. https://doi.org/10.3102/0013189X251331518

49.

Roche

(2023, June 20). 75+ ESSER III spending categories. Burbio School Tracker.https://about.burbio.com/weekly-updates/burbio-school-tracker-6/20-75-esser-iii-spending-categories

50.

Schwartz

J. L.

McCombs

J. S.

Augustine

C. H.

Leschitz

J. T.

(2018). Getting to work on summer learning: Recommended practices for success (2nd ed). RAND Corporation.

51.

Steiner

E. D.

Doan

Woo

Gittens

A. D.

Lawrence

R. A.

Berdie

Wolfe

R. L.

Greer

Schwartz

H. L.

(2022). Restoring teacher and principal well-being is an essential step for rebuilding schools: Findings from the State of the American Teachers and State of the American Principal Surveys. RAND Corporation. https://www.rand.org/pubs/research_reports/RRA1108-4.html

52.

Thum

Y. M.

Kuhfeld

(2020). NWEA 2020 MAP Growth achievement status and growth norms for students and schools. NWEA.

53.

U.S. Department of Education. (2023). National Assessment of Educational Progress (NAEP) long-term trend assessment results: Reading and mathematics. Institute of Education Sciences, National Center for Education Statistics. https://www.nationsreportcard.gov/highlights/ltt/2023/

54.

U.S. Department of Education. (2022). National Assessment of Educational Progress (NAEP) 2022 long-term trend assessment results: Reading and mathematics. Institute of Education Sciences, National Center for Education Statistics. https://www.nationsreportcard.gov/highlights/ltt/2022/

55.

Zhao

Ding

(2022). To adjust or not to adjust? Estimating the average treatment effect in randomized experiments with missing covariates. Journal of the American Statistical Association, 119(545), 450–460. https://doi.org/10.1080/01621459.2022.2123814