We present results from a meta-analysis of 37 contemporary experimental and quasi-experimental studies of summer programs in mathematics for children in grades pre-K–12, examining what resources and characteristics predict stronger student achievement. Children who participated in summer programs that included mathematics activities experienced significantly better mathematics achievement outcomes compared to their control group counterparts. We find an average weighted impact estimate of +0.10 standard deviations on mathematics achievement outcomes. We find similar effects for programs conducted in higher- and lower-poverty settings. We undertook a secondary analysis exploring the effect of summer programs on noncognitive outcomes and found positive mean impacts. The results indicate that summer programs are a promising tool to strengthen children’s mathematical proficiency outside of school time.
AlexanderK. L.EntwisleD. R.OlsonL. S. (2007). Lasting consequences of the summer learning gap. American Sociological Review, 72(2), 167–180.
4.
AlexanderK.PitcockS.BoulayM. C. (Eds.). (2016). The summer slide: What we know and can do about summer learning loss. Teachers College Press.
5.
AngristJ. D. (2004). American education research changes tack. Oxford Review of Economic Policy, 20(2), 198–212.
6.
AtteberryA.McEachinA. (2021). School’s out: The role of summers in understanding achievement disparities. American Educational Research Journal, 58(2), 239–282.
7.
AugustineC. H.McCombsJ. S.PaneJ. F.SchwartzH. L.SchweigJ.McEachinA.Siler-EvansK. (2016). Learning from summer: Effects of voluntary summer learning programs on low-income urban youth. RR-1557-WF. RAND Corporation.
BorensteinM.CooperH.HedgesL. V.ValentineJ. C. (2009). The handbook of research synthesis and meta-analysis. Russell Sage Foundation.
10.
BormanG. D.BensonJ.OvermanL. T. (2005). Families, schools, and summer learning. The Elementary School Journal, 106(2), 131–150.
11.
BormanG. D.D’AgostinoJ. V. (1996). Title I and student achievement: A meta-analysis of federal evaluation results. Educational Evaluation and Policy Analysis, 18(4), 309–326.
12.
BurkamD. T.ReadyD. D.LeeV. E.LoGerfoL. F. (2004). Social-class differences in summer learning between kindergarten and first grade: Model specification and estimation. Sociology of Education, 77(1), 1–31.
13.
CaiJ. (2005). US and Chinese teachers’ constructing, knowing, and evaluating representations to teach mathematics. Mathematical Thinking and Learning, 7(2), 135–169.
14.
CarterD. F. (2006). Key issues in the persistence of underrepresented minority students. New Directions for Institutional Research, 130, 33–46.
15.
CeciS. J.PapiernoP. B. (2005). The rhetoric and reality of gap closing: When the “have-nots” gain but the “haves” gain even more. American Psychologist, 60(2), 149.
16.
ChettyR.FriedmanJ. N.RockoffJ. E. (2014). Measuring the impacts of teachers II: Teacher value-added and student outcomes in adulthood. American Economic Review, 104(9), 2633–2679.
17.
ChinT.PhillipsM. (2004). Social reproduction and child-rearing practices: Social class, children’s agency, and the summer activity gap. Sociology of Education, 77(3), 185–210.
18.
ChmielewskiA. K.ReardonS. F. (2016). Patterns of cross-national variation in the association between income and academic achievement. Aera Open, 2(3), 2332858416649593.
19.
CooperH.CharltonK.ValentineJ. C.MuhlenbruckL.BormanG. D. (2000). Making the most of summer school: A meta-analytic and narrative review. Monographs of the Society for Research in Child Development, 65(1), i–127.
20.
CooperH.NyeB.CharltonK.LindsayJ.GreathouseS. (1996). The effects of summer vacation on achievement test scores: A narrative and meta-analytic review. Review of Educational Research, 66(3), 227–268.
21.
Dabla-NorrisM. E.KochharM. K.SuphaphiphatM. N.RickaM. F.TsountaM. E. (2015). Causes and consequences of income inequality: A global perspective. International Monetary Fund.
22.
Darling-HammondL.SchachnerA.EdgertonA. K. (2020). Restarting and reinventing school: Learning in the time of COVID and beyond. Learning Policy Institute.
23.
DaviesS.McKerrowM.AuriniJ.MassaghianR. (2019, August13). Effects of summer programs on early numeracy. [Paper presentation]. American Sociological Association Annual Meeting. New York, NY, United States.
24.
DemingD. J. (2017). The growing importance of social skills in the labor market. The Quarterly Journal of Economics, 132(4), 1593–1640.
25.
DemingD. J.NorayK. (2020). Earnings dynamics, changing job skills, and STEM careers. The Quarterly Journal of Economics, 135(4), 1965–2005.
26.
DowneyD. B.von HippelP. T.BrohB. A. (2004). Are schools the great equalizer? Cognitive inequality during the summer months and the school year. American Sociological Review, 69(5), 613–635.
27.
DowneyD. B.WorkmanJ.von HippelP. T. (2019). Socioeconomic, ethnic, racial, and gender gaps in children’s social/behavioral skills: Do they grow faster in school or out?Sociological Science, 6, 446–466.
28.
DuckworthA. L.YeagerD. S. (2015). Measurement matters: Assessing personal qualities other than cognitive ability for educational purposes. Educational Researcher, 44(4), 237–251.
29.
DumontH.ReadyD. D. (2020). Do schools reduce or exacerbate inequality? How the associations between student achievement and achievement growth influence our understanding of the role of schooling. American Educational Research Journal, 57(2), 728–774.
30.
DurlakJ. A.WeissbergR. P.PachanM. (2010). A meta-analysis of after-school programs that seek to promote personal and social skills in children and adolescents. American Journal of Community Psychology, 45(3), 294–309.
31.
DuvalS.TweedieR. (2000). Trim and fill: A simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56(2), 455–463.
32.
ErlwangerS. H. (1973). Benny’s conception of rules and answers in IPI Mathematics. Journal of Children’s Mathematical Behavior, 1(2), 7–26.
33.
FarringtonC. A.RoderickM.AllensworthE.NagaokaJ.KeyesT. S.JohnsonD. W.BeechumN. O. (2012). Teaching adolescents to become learners: The role of noncognitive factors in shaping school performance: A critical literature review. Consortium on Chicago School Research.
34.
FisherZ.TiptonE. (2015). robumeta: An R-package for robust variance estimation in meta-analysis. arXiv preprint arXiv:1503.02220.
35.
FryerR. G.Jr. (2017). The production of human capital in developed countries: Evidence from 196 randomized field experiments. In DufloE.BanerjeeA. (Eds.), Handbook of field experiments (Vol. 2, pp. 95–322). North-Holland.
36.
FuchsD.FuchsL.ThompsonA.Al OtaibaS.YenL.YangN.BraunM.O’ConnorR. (2001). Is reading important in reading-readiness programs? A randomized field trial with teachers as program implementers. Journal of Educational Psychology, 93(2), 251–267.
37.
GarrettR.CitkowiczM.WilliamsR. (2019). How responsive is a teacher’s classroom practice to intervention? A meta-analysis of randomized field studies. Review of Research in Education, 43(1), 106–137.
38.
GershensonS. (2013). Do summer time-use gaps vary by socioeconomic status?American Educational Research Journal, 50(6), 1219–1248.
39.
GlenbergA.WillfordJ.GibsonB.GoldbergA.ZhuX. (2012). Improving reading to improve math. Scientific Studies of Reading, 16(4), 316–340.
40.
Gilbert-MacmillanK.LeitzS. J. (1986). Cooperative small groups: A method for teaching problem solving. The Arithmetic Teacher, 33(7), 9–11.
41.
GorardS.SiddiquiN.SeeB. H. (2015). How effective is a summer school for catch-up attainment in English and maths?International Journal of Educational Research, 73, 1–11.
42.
GottfriedM. A. (2017). Linking getting to school with going to school. Educational Evaluation and Policy Analysis, 39(4), 571–592.
HartK. C.GrazianoP. A.KentK. M.KuriyanA.GarciaA.RodriguezM.PelhamW. EJr. (2016). Early intervention for children with behavior problems in summer settings. Journal of Early Intervention, 38(2), 92–117.
45.
HeckmanJ. J.KautzT. (2014). Fostering and measuring skills: Interventions that improve character and cognition. In HeckmanJ. J.HumphriesJ. E.KautzT. (Eds.), The myth of achievement tests: The GED and the role of character in American life (pp. 341–430). University of Chicago Press.
46.
HeynsB. (1978). Summer learning and the effects of schooling. Academic Press.
47.
HiebertJ.WearneD. (1996). Instruction, understanding, and skill in multidigit addition and subtraction. Cognition and Instruction, 14(3), 251–283.
IrwinV.ZhangJ.WangX.HeinS.WangK.RobertsA.YorkC.BarmerA.Bullock MannF.DiligR.ParkerS. (2021). Report on the condition of education 2021. National Center for Education Statistics, U.S. Department of Education.
50.
JacobB. A.LefgrenL. (2004). Remedial education and student achievement: A regression-discontinuity analysis. Review of Economics and Statistics, 86(1), 226–244.
51.
KendallW. (2009). The effect of a summer academy on math achievement. [Unpublished doctoral dissertation.] Aurora University.
52.
KimJ.GilbertJ.YuQ.GaleC. (2021). Measures matter: A meta-analysis of the effects of educational apps on preschool to grade 3 children’s literacy and math skills. AERA Open, 7.
53.
KimJ. S.QuinnD. M. (2013). The effects of summer reading on low-income children’s literacy achievement from kindergarten to grade 8: A meta-analysis of classroom and home interventions. Review of Educational Research, 83(3), 386–431.
KornrichS.FurstenbergF. (2013). Investing in children: Changes in parental spending on children, 1972–2007. Demography, 50(1), 1–23.
56.
KraftM. A. (2020). Interpreting effect sizes of education interventions. Educational Researcher, 49(4), 241–253.
57.
KruegerA. B. (2003). Economic considerations and class size. The Economic Journal, 113(485), F34–F63.
58.
KuhfeldM.SolandJ.TarasawaB.JohnsonA.RuzekE.LiuJ. (2020). Projecting the potential impact of COVID-19 school closures on academic achievement. Educational Researcher, 49(8), 549–565.
59.
LauerP. A.AkibaM.WilkersonS. B.ApthorpH. S.SnowD.Martin-GlennM. L. (2006). Out-of-school-time programs: A meta-analysis of effects for at-risk students. Review of Educational Research, 76(2), 275–313.
60.
LipseyM. W.PuzioK.YunC.HebertM. A.Steinka-FryK.ColeM. W.RobertsM.AnthonyK.BusickM. D. (2012). Translating the statistical representation of the effects of education interventions into more readily interpretable forms. National Center for Special Education Research. https://ies.ed.gov/ncser/pubs/20133000/
61.
Lortie-ForguesH.InglisM. (2019). Rigorous large-scale RCTs are often uninformative: Should we be concerned?Educational Researcher, 48(3), 158–166.
62.
LynchK.HillH. C.GonzalezK. E.PollardC. (2019). Strengthening the research base that informs STEM instructional improvement efforts: A meta-analysis. Educational Evaluation and Policy Analysis, 41(3), 260–293.
63.
LynchK.KimJ. S. (2017). Effects of a summer mathematics intervention for low-income children: A randomized experiment. Educational Evaluation and Policy Analysis, 39(1), 31–53.
64.
Mac IverM. A.Mac IverD. J. (2015). The Baltimore City Schools Middle School STEM Summer Program with VEX Robotics. Baltimore Education Research Consortium. https://files.eric.ed.gov/fulltext/ED570654.pdf
65.
ManganelloJ. A.TaylorC. A. (2009). Television exposure as a risk factor for aggressive behavior among 3-year-old children. Archives of Pediatrics & Adolescent Medicine, 163(11), 1037–1045.
66.
MarianoL. T.MartorellP. (2013). The academic effects of summer instruction and retention in New York City. Educational Evaluation and Policy Analysis, 35(1), 96–117.
67.
MarshJ. A.GershwinD.KirbyS. N.XiaN. (2009). Retaining students in grade: Lessons learned regarding policy design and implementation. RAND Corporation.
MatsudairaJ. D. (2008). Mandatory summer school and student achievement. Journal of Econometrics, 142(2), 829–850.
70.
McCombsJ. S.AugustineC. H.PaneJ. F.SchweigJ. (2020). Every summer counts: A longitudinal analysis of outcomes from the National Summer Learning Project. RAND Corporation. https://www.rand.org/pubs/research_reports/RR3201.html
71.
McCombsJ. S.AugustineC. H.SchwartzH. L.BodillyS. J.McInnisB. I.LichterD. S.CrossA. B. (2011). Making summer count: How summer programs can boost children’s learning. RAND Corporation.
72.
McCombsJ. S.AugustineC. H.UnluF.Ziol-GuestK. M.NaftelS.GomezC. J.MarshT.AkinniranyeG.ToddI. (2019). Investing in successful summer programs: A review of evidence under the Every Student Succeeds Act. RAND Corporation.
73.
McCombsJ. S.PaneJ. F.AugustineC. H.SchwartzH. L.MartorellP.ZakarasL. (2014). Ready for fall? Near-term effects of voluntary summer learning programs on low-income students’ learning opportunities and outcomes. RAND Corporation.
74.
McFarlandJ.HussarB.de BreyC.SnyderT.WangX.Wilkinson-FlickerS.GebrekristosS.ZhangJ.RathbunA.BarmerA.Bullock MannF.HinzS. (2017). The condition of education 2017. National Center for Education Statistics.
75.
McKownC. (2017). Social and emotional learning: A policy vision for the future [Policy brief]. The Future of Children. Princeton University.
76.
MesaV. (2004). Characterizing practices associated with functions in middle school textbooks: An empirical approach. Educational Studies in Mathematics, 56(2), 255–286.
77.
MeyerD. K.TurnerJ. C.SpencerC. A. (1997). Challenge in a mathematics classroom: Students’ motivation and strategies in project-based learning. The Elementary School Journal, 97(5), 501–521.
78.
MoherD.LiberatiA.TetzlaffJ.AltmanD. G., & The PRISMA Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLOS Medicine, 6(7), e1000097. https://doi.org/10.1371/journal.pmed.1000097
79.
National Academies of Sciences, Engineering, and Medicine. (2019). Shaping summertime experiences. The National Academies Press. https://doi.org/10.17226/25546
80.
National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics.
81.
National Governors Association Center for Best Practices, Council of Chief State School Officers. (2010). Common Core State Standards. Author.
82.
National Research Council. (2011). Successful K-12 STEM education: Identifying effective approaches in science, technology, engineering, and mathematics. The National Academies Press.
83.
OlsenR.SeftorN.SilvaT.MyersD.DesRochesD.YoungJ. (2007). Upward Bound Math-Science. U.S. Department of Education.
84.
OwensA. (2016). Inequality in children’s contexts: Income segregation of households with and without children. American Sociological Review, 81(3), 549–574.
85.
PatallE. A.CooperH.AllenA. B. (2010). Extending the school day or school year: A systematic review of research. Review of Educational Research, 80(3), 401–436.
86.
PierL.HoughH. J.ChristianM.BookmanN.WilkenfeldB.MillerR. (2021). COVID-19 and the educational equity crisis. Policy Analysis for California Education.
87.
PigottT. (2012). Advances in meta-analysis. Springer Science & Business Media.
88.
QuinnD. (2015). Black–White summer learning gaps: Interpreting the variability of estimates across representations. Educational Evaluation and Policy Analysis, 37(1), 50–69.
89.
Rey-LópezJ. P.Vicente-RodríguezG.BioscaM.MorenoL. A. (2008). Sedentary behaviour and obesity development in children and adolescents. Nutrition, Metabolism and Cardiovascular Diseases, 18(3), 242–251.
90.
RitterG.BarnettJ.DennyG.AlbinG. (2009). The effectiveness of volunteer tutoring programs for elementary and middle school students: A meta-analysis. Review of Educational Research, 79(1), 3–38.
91.
RoderickM.EngelM.NagaokaJ. (2003). Ending social promotion: Results from Summer Bridge. Charting Reform in Chicago Series. Consortium on Chicago School Research.
92.
ScherL. (2018). Examining the long-term effects of the Horizons National Student Enrichment Program on student academic outcomes. Concentric Research & Evaluation.
93.
SchwartzA.RothbartM. (2020). Let them eat lunch: The impact of universal free meals on student performance. Journal of Policy Analysis & Management, 39(2), 376–410.
94.
SomersM. A.WelbeckR.GrossmanJ. B.GoodenS. (2015). An analysis of the effects of an academic summer program for middle school students. MDRC.
95.
StataCorp. 2021. Stata 17 base reference manual. Stata Press.
96.
SteinmayrR.SpinathB. (2009). The importance of motivation as a predictor of school achievement. Learning and Individual Differences, 19(1), 80–90.
97.
StrongeJ. H.WardT. J.GrantL. W. (2011). What makes good teachers good? A cross-case analysis of the connection between teacher effectiveness and student achievement. Journal of Teacher Education, 62(4), 339–355.
98.
Tanner-SmithE. E.TiptonE. (2014). Robust variance estimation with dependent effect sizes: Practical considerations including a software tutorial in Stata and SPSS. Research Synthesis Methods, 5(1), 13–30.
99.
Tanner-SmithE. E.TiptonE.PolaninJ. R. (2016). Handling complex meta-analytic data structures using robust variance estimates: A tutorial in R. Journal of Developmental and Life-Course Criminology, 2, 85–112.
100.
TerzianM.MooreK. A.HamiltonK. (2009). Approaches for economically disadvantaged children and youth: A white paper for the Wallace Foundation. Child Trends, 23, 1–42.
101.
TiptonE.PustejovskyJ. E. (2015) Small-sample adjustments for tests of moderators and model fit using robust variance estimation in meta-regression. Journal of Educational and Behavioral Statistics, 40(6), 604–634.
102.
TornroosJ. (2005). Mathematics textbooks, opportunity to learn and student achievement. Studies in Educational Evaluation, 31(4), 315–327.
103.
von HippelP. T. (2019a). Is summer learning loss real? How I lost faith in one of education research’s classic results. Education Next, 19(4), 8–15.
von HippelP. T.HamrockC. (2019). Do test score gaps grow before, during, or between the school years? Measurement artifacts and what we can know in spite of them. Sociological Science, 6(3), 43.
106.
WestM.KraftM.FinnA.MartinR.DuckworthA.GabrieliC.GabrieliJ. D. (2016). Promise and paradox: Measuring students’ non-cognitive skills and the impact of schooling. Educational Evaluation and Policy Analysis, 38(1), 148–170.
107.
What Works Clearinghouse. (n.d.). Questions and answers from demystifying the What Works Clearinghouse: A webinar for developers and researchers. https://ies.ed.gov/ncee/wwc/Document/246
108.
What Works Clearinghouse. (2020). What Works Clearinghouse standards handbook, version 4.1. Institute of Education Sciences, U.S. Department of Education.
109.
WoodworthJ. L.RaymondM. E.ChirbasK.GonzalezM.NegassiY.SnowW.Van DongeC. (2015). Online charter school study 2015. Stanford University CREO.
110.
YeagerD. S.WaltonG. M. (2011). Social-psychological interventions in education: They’re not magic. Review of Educational Research, 81(2), 267–301.
111.
YoonK. S.DuncanT.LeeW.-Y.ScarlossB.ShapleyK. (2007). Reviewing the evidence on how teacher professional development affects student achievement. U.S. Department of Education.
112.
ZakariaE.ChinL. C.DaudM. Y. (2010). The effects of cooperative learning on students’ mathematics achievement and attitude towards mathematics. Journal of Social Sciences, 6(2), 272–275.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
