The public narrative surrounding efforts to improve low-performing K–12 schools in the United States has been notably gloomy. But what is known empirically about whether school improvement works, which policies are most effective, which contexts respond best to intervention, and how long it takes? We meta-analyze 141 estimates from 67 studies of post–No Child Left Behind Act turnaround policies. On average, policies had moderate positive effects on math and no effect on English Language Arts achievement on high-stakes exams. We find positive impacts on low-stakes exams and no evidence of harm on nontest outcomes. Extended learning time and teacher replacements predict greater effects. Contexts serving majority-Latina/o populations saw the largest improvements. We cannot rule out publication bias entirely but find no differences between peer-reviewed versus nonpeer-reviewed estimates.
*Indicates studies that contributed estimates to the meta-analysis
2.
*AbdulkadiroğluA.AngristJ.HullP.PathakP. (2016). Charters without lotteries: Testing takeovers in New Orleans and Boston. American Economic Review, 106(7), 1878–1920. https://doi.org/10.1257/aer.20150479
3.
*AhnT.VigdorJ. (2014). The Impact of No Child Left Behind’s Accountability Sanctions on School performance: Regression discontinuity evidence from North Carolina (NBER Working Paper 20511). https://doi.org/10.3386/w20511
4.
BaileyM.DynarskiS. (2011). Inequality in postsecondary education. In DuncanG. J.MurnaneR. J. (Eds.), Whither opportunity: Rising inequality, schools, and children’s life chances (pp. 117–132). Russell Sage Foundation.
BaroneC. (2017). What ESSA says: Continuities and departures. In HessF.EdenM. (Eds.), The Every Student Succeeds Act (pp. 59–74). Harvard Education Press.
*BifulcoR.DuncombeW.YingerJ. (2005). Does whole-school reform boost student performance? The case of New York City. Journal of Policy Analysis and Management, 24(1), 47–72. https://doi.org/10.1002/pam.2006
*BonillaS.DeeT. (2020). The effects of school reform under NCLB waivers: Evidence from Focus Schools in Kentucky. Education Finance and Policy, 15(1), 75–103. https://doi.org/10.1162/edfp_a_00275
11.
BorensteinM.HedgesL.HigginsH.RothsteinH. (2009). Introduction to meta-analysis. John Wiley.
12.
BorensteinM.HigginsJ. P. T.HedgesL. V.RothsteinH. R. (2017). Basics of meta-analysis: I2 is not an absolute measure of heterogeneity. Research Synthesis Methods, 8(1), 5–18. https://doi.org/10.1002/jrsm.1230
13.
BormanG. D.HewesG. M.OvermanL. T.BrownS. (2003). Comprehensive school reform and achievement. Review of Educational Research, 73(2), 125–230. https://doi.org/10.3102/00346543073002125
CalkinsA.GuentherW.BelfioreG.LashD. (2007). The turnaround challenge: Why America’s best opportunity to dramatically improve student achievement lies in our worst-performing schools. Mass Insight Education & Research Institute. https://files.eric.ed.gov/fulltext/ED538298.pdf
18.
*CarlsonD.BormanG. D.RobinsonM. (2011). A multistate district-level cluster randomized trial of the impact of data-driven reform on reading and mathematics achievement. Educational Evaluation and Policy Analysis, 33(3), 378–398. https://doi.org/10.3102/0162373711412765
19.
*CarlsonD.LavertuS. (2016). Charter school closure and student achievement: Evidence from Ohio. Journal of Urban Economics, 95(September), 31–48. https://doi.org/10.1016/j.jue.2016.07.001
20.
*CarlsonD.LavertuS. (2018). School Improvement Grants in Ohio: Effects on student achievement and school administration. Educational Evaluation and Policy Analysis, 40(3), 287–315. https://doi.org/10.3102/0162373718760218
21.
CarnoyM.ElmoreR. F.SiskinL. S. (Eds.). (2003). The new accountability: High schools and high-stakes testing. Routledge Falmer.
ChettyR.FriedmanJ. N.HilgerN.SaezE.SchanzenbachD. W.YaganD. (2011). How does your kindergarten classroom affect your earnings? Evidence from Project STAR. Quarterly Journal of Economics, 126(4), 1593–1660. https://doi.org/10.1093/qje/qjr041
24.
*ChiangH. (2009). How accountability pressure on failing schools affects student achievement. Journal of Public Economics, 93(9–10), 1045–1057. https://doi.org/10.1016/j.jpubeco.2009.06.002
25.
*ChinM.KaneT. J.KozakowskiW.SchuelerB. E.StaigerD. O. (2018). School district reform in Newark: Within- and between-school changes in achievement growth. Industrial and Labor Relations Review, 72(2), 323–354. https://doi.org/10.1177/0019793918774318
26.
CohenD. K.HillH. C. (2008). Learning policy: When state education reform works. Yale University Press.
27.
CooperH. (2010). Research synthesis and meta-analysis: A step-by-step approach. Sage.
28.
CoralloC.McDonaldD. (2001). What works with low-performing schools: A review of research literature on low-performing schools. Office of Educational Research and Improvement. https://eric.ed.gov/?id=ED462737
*DeeT. (2012). School turnarounds: Evidence from the 2009 stimulus. National Bureau of Economic Research WP 17990. https://doi.org/10.3386/w17990
31.
*DeeT.Dizon-RossE. (2019). School performance, accountability, and waiver reforms. Educational Evaluation and Policy Analysis. https://doi.org/10.3102/0162373719849944
DesimoneL. (2002). How can comprehensive school reform models be successfully implemented?Review of Educational Research, 72(3), 433–479. https://doi.org/10.3102/00346543072003433
*DoughertyS.WeinerJ. (2017). The Rhode to turnaround: The impact of waivers to No Child Left Behind on school performance. Educational Policy, 33(4), 555–586. https://doi.org/10.1177/0895904817719520
37.
*DragosetL.ThomasJ.HerrmannD.DekeJ.James-BurdumyG.GraczewskiC.BoyleA.UptonR.TanenbaumC.GiffinJ. (2017). School Improvement Grants: Implementation and effectiveness (NCEE 2017–4013). https://ies.ed.gov/ncee/pubs/20174013/
38.
DuncanC. J.MurnaneR. J. (2011). “Introduction: The American Dream, then and now.” In DuncanG.MurnaneR. (Eds.), Whither opportunity: Rising inequality, schools, and children’s life chances (pp. 3–26). Russell Sage Foundation.
39.
DuvalS.TweedieR. (2000). Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56(2), 455–463. https://doi.org/10.1111/j.0006-341x.2000.00455.x
40.
EggerM.SmithG. D.SchneiderM.MinderC. (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ, 315(7109), 629–634. https://doi.org/10.1136/bmj.315.7109.629
41.
*EngbergJ.GillB.ZamarroG.ZimmerR. (2012). Closing schools in a shrinking district: Do student outcomes depend on which schools are closed?Journal of Urban Economics, 71(2), 189–203. https://doi.org/10.1016/j.jue.2011.10.001
*FiglioD.RouseC. (2006). Do accountability and voucher threats improve low-performing schools?Journal of Public Economics, 90(1–2), 239–255. https://doi.org/10.1016/j.pubeco.2005.08.005
*FryerR. (2014). Injecting charter school best practices into traditional public schools. Quarterly Journal of Economics, 129(3), 1355–1407. https://doi.org/10.1093/qje/qju011
*GandhiA. G.SlamaR.ParkS. J.RussoP.WinnerK.BzuraR.JonesW.WilliamsonS. (2018). Focusing on the whole student: An evaluation of Massachusetts’s Wraparound Zone Initiative. Journal of Research on Educational Effectiveness, 11(2), 240–266. https://doi.org/10.1080/19345747.2017.1413691
48.
*GigliottiP. (2019). Leveraging managerial autonomy to turn around low-performing schools: Evidence from the Innovation Schools Program in Denver public schools (Working Paper). Department of Public Administration and Policy, State University of New York (SUNY). https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3419472
*GordonM. F.delaTorreM.CowhyJ. R.MooreP. T.SartainL.KnightD. (2018). School closings in Chicago: Staff and student experiences and academic outcomes. University of Chicago Consortium on School Research. https://files.eric.ed.gov/fulltext/ED589712.pdf
GrossB.BookerT. K.GoldhaberD. (2009). Boosting student achievement: The effect of comprehensive school reform on student achievement. Educational Evaluation and Policy Analysis, 31(2), 111–126. https://doi.org/10.3102/0162373709333886
54.
*GuthrieJ. E.HenryG. T. (2016, November). When the LATE Ain’t ATE: Comparing alternative methods for evaluating reform impacts in low-achieving schools [Paper presentation]. Annual meeting of the Association for Public Policy Analysis and Management, Washington, DC.
55.
*HallbergK.WilliamsR.SwanlundA.EnoJ. (2018). Short comparative interrupted time series using aggregate school-level data in education research. Educational Researcher, 47(5), 295–306. https://doi.org/10.3102/0013189X18769302
56.
*HallgrenK.GonzalezN.KellyK.DemersA.GillB. (2019). Year 2 Report of the Atlanta public schools turnaround strategy. Mathematica Policy Research. www.atlantapublicschools.us/domain/11375
HanushekE. A.PetersonP. E.TalpeyL. M.WoessmannL. (2019). The unwavering SES achievement gap: Trends in U.S. student performance (NBER Working Paper No. 25648). National Bureau of Economic Research. https://doi.org/10.3386/w25648
*HarrisD.LarsenM. (2018). The Effects of the New Orleans post-Katrina market-based school reforms on student achievement, high school graduation, and college outcomes. Education Research Alliance for New Orleans. https://ldr.lafayette.edu/concern/publications/df65v838h
61.
HashimS.KaneT.Kelley-KempleT.LaskiM.StaigerD. (2020). Have income-based achievement gaps widened or narrowed? (NBER Working Paper No. 27714). National Bureau of Economic Research. https://doi.org/10.3386/w27714
62.
HedgesL. V. (2007). Effect sizes in cluster-randomized designs. Journal of Educational and Behavioral Statistics, 32(4), 341–370. https://doi.org/10.3102/1076998606298043
63.
HedgesL. V.TiptonE.JohnsonM. C. (2010). Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods, 1(1), 39–65. https://doi.org/10.1002/jrsm.5
64.
*HeisselJ.LaddH. (2018). School turnaround in North Carolina: A regression discontinuity analysis. Economics of Education Review, 62(February), 302–320. https://doi.org/10.1016/j.econedurev.2017.08.001
65.
*HemeltS. W. (2011). Performance effects of failure to make Adequate Yearly Progress (AYP): Evidence from a regression discontinuity framework. Economics of Education Review, 30(4), 702–723. https://doi.org/10.1016/j.econedurev.2011.02.009
66.
*HemeltS. W.JacobB. (2020). How does an accountability program that targets achievement gaps affect student performance?Education Finance and Policy, 15(1), 45–74. https://doi.org/10.1162/edfp_a_00276
*HenryG. T.HarbatkinE. (2019). The next generation of state reforms to improve their lowest performing schools: An evaluation of North Carolina’s school transformation initiative (Annenberg EdWorkingPaper: 19-103). Annenberg Institute at Brown University. https://doi.org/10.26300/evs5-nc27
HochbeinC. (2012). Relegation and reversion: Longitudinal analysis of school turnaround and decline. JESPAR: Journal of Education for Students Placed at Risk, 17(1–2), 92–107. https://doi.org/10.1080/10824669.2012.636728
73.
HochschildJ. L.ScovronickN. (2004). The American dream and the public schools. Oxford University Press.
74.
*HolbeinJ. B.LaddH. F. (2017). Accountability pressure: Regression discontinuity estimates of how No Child Left Behind influenced student behavior. Economics of Education Review, 58(June), 55–67. https://doi.org/10.1016/j.econedurev.2017.03.005
75.
HonigM. I.HatchT. C. (2004). Crafting coherence: How schools strategically manage multiple, external demands. Educational Researcher, 33(8), 16–30. https://doi.org/10.3102/0013189X033008016
76.
HurlburtS.Le FlochK. C.TherriaultS. B.ColeS. (2011). Baseline analyses of SIG applications and SIG-eligible and SIG-awarded schools (NCEE 2011-4019). U.S. Department of Education. https://ies.ed.gov/ncee/pubs/20114019/pdf/20114019.pdf
77.
JohnsonS. M.MariettaG.HigginsM. C.MappK. L.GrossmanA. (2015). Achieving coherence in district improvement: Managing the relationship between the central office and schools. Harvard Education Press.
*KirshnerB.GaertnerM.PozzoboniK. (2010). Tracing transitions: The effect of high school closure on displaced students. Educational Evaluation and Policy Analysis, 32(3), 407–429. https://doi.org/10.3102/0162373710376823
*LarsenM. F. (2018). Does closing schools close doors? The effect of high school closings on achievement and attainment. Economics of Education Review, 76(June), 101980 https://doi.org/10.1016/j.econedurev.2020.101980.
LeeJ. (2008). Is test-driven external accountability effective? Synthesizing the evidence from cross-state causal-comparative and correlational studies. Review of Educational Research, 78(3), 608–644. https://doi.org/10.3102/0034654308324427
84.
*LiCalsiC.CitkowiczM.FriedmanL. B.BrownM. (2015). Evaluation of Massachusetts Office of District and School Turnaround Assistance to Commissioner’s Districts and Schools: Impact of School Redesign Grants. American Institutes for Research. https://files.eric.ed.gov/fulltext/ED583091.pdf
85.
LittellJ. H.CorcoranJ.PillaiV. (2008). Systematic reviews and meta-analysis. Oxford University Press.
86.
MalenB.CroningerR.MunceyD.Redmond-JonesD. (2002). Reconstituting schools: “Testing” the “theory of action.”Educational Evaluation and Policy Analysis, 24(2), 113–132. https://doi.org/10.3102/01623737024002113
87.
MalenB.RiceJ. K. (2016). School reconstitution as a turnaround strategy: An analysis of the evidence. In MathisW. J.TrujilloT. (Eds.), Learning from the federal market-based reforms: Lessons for the Every Student Succeeds Act (pp. 99–127). Information Age.
McGuinnP. (2006). No Child Left Behind and the transformation of federal education policy 1965-2005. University Press of Kansas.
90.
McGuinnP. (2012). Stimulating reform: Race to the Top, competitive grants, and the Obama Education Agenda. Educational Policy, 26(1), 136–159. https://doi.org/10.1177/0895904811425911
91.
MeyersC.VanGronigenB. (2018). So many educational service providers, so little evidence. American Journal of Education, 125(1), 109–139. https://doi.org/10.1086/699823
92.
MurphyJ.BleibergJ. (2019). School turnaround politics and practices in the U.S.: Learning from failed school reform. Springer. https://doi.org/10.1007/978-3-030-01434-6
93.
*OpperI. M.JohnstonW. R.EngbergJ.XenakisL. (2019). Assessing the short-term impact of the New York City Renewal Schools Program. RAND Corporation. https://www.rand.org/pubs/working_papers/WR1303
94.
*OrlandM.HoffmanA. E.VaughnS. (2010). Evaluation of the comprehensive school reform program implementation and outcomes. Office of Planning, Evaluation and Policy Development. https://www.wested.org/online_pubs/csr-5-year-report.pdf
95.
*OsekU.HansenM.GonzalezT. (2012). A leg up or boot out? Student achievement and mobility under school restructuring. Center for Analysis of Longitudinal Data in Education Research. https://files.eric.ed.gov/fulltext/ED587152.pdf
PetersJ. L.SuttonA. J.JonesD. R.AbramsK. R.RushtonL.MorenoS. G. (2010). Assessing publication bias in meta-analyses in the presence of between-study heterogeneity. Statistics in Society, 173(3), 575–591. https://doi.org/10.1111/j.1467-985X.2009.00629.x
PeurachD.NeumerskiC. (2015). Mixing metaphors: Building infrastructure for large scale school turnaround. Journal of Educational Change, 16(4), 379–429. https://doi.org/10.1007/s10833-015-9259-z
100.
PhamL.HenryG.KhoA.ZimmerR. (2020). Sustainability and maturation of school turnaround: A multiyear evaluation of Tennessee’s Achievement School District and local Innovation Zones. AERA Open, 6(2). https://doi.org/10.1177/2332858420922841
101.
*PlayerD.KatzV. (2016). Assessing school turnaround: Evidence from Ohio. Elementary School Journal, 116(4), 675–698. https://doi.org/10.1086/686467
102.
PustejovskyJ.RodgersM. (2019). Testing for funnel plot asymmetry of standardized mean differences. Research Synthesis Methods, 10(1), 57–71. https://doi.org/10.1002/jrsm.1332
ReddingC.NguyenT. (2020). The relationship between school turnaround and student outcomes: A meta-analysis. Educational Evaluation and Policy Analysis, 42(4), 493–519. https://doi.org/10.3102/0162373720949513
105.
*RiceJ.BojorquezJ. C.DiazM.WendtS.NakamotoJ. (2014). Evaluation of Michigan’s School Improvement Grants: Outcomes after three years. Office of Education Improvement and Innovation. Michigan Department of Education
106.
RiceJ. K.CroningerR. G. (2005). Resource generation, reallocation, or depletion: An analysis of the impact of reconstitution on school capacity. Leadership and Policy in Schools, 4(2), 73–103. https://doi.org/10.1080/15700760590965569
107.
RiceJ. K.MalenB. (2003). The human costs of education reform: The case of school reconstitution. Educational Administration Quarterly, 39(5), 635–666. https://doi.org/10.1177/0013161X03257298
108.
*RockoffJ.TurnerL. J. (2010). Short-run impacts of accountability on school quality. American Economic Journal: Economic Policy, 2(4), 119–147. https://doi.org/10.1257/pol.2.4.119
109.
RodgersM. A.PustejovskyJ. E. (2020). Evaluating meta-analytic methods to detect selective reporting in the presence of dependent effect sizes. Psychological Methods, 26(2), 141–160. https://doi.org/10.1037%2Fmet0000300
110.
*RouseC. E.HannawayJ.GoldhaberD.FiglioD. (2013). Feeling the Florida heat? How low-performing schools respond to voucher and accountability pressure. American Economic Journal: Economic Policy, 5(2), 251–281. https://doi.org/10.1257/pol.5.2.251
111.
*SawG.SchneiderB.FrankK.ChenI.-C.KeeslerV.MartineauJ. (2017). The impact of being labeled as a persistently lowest achieving school: Regression discontinuity evidence on consequential school labeling. American Journal of Education, 123(4), 585–613. https://www.journals.uchicago.edu/doi/abs/10.1086/692665
112.
SchuelerB.BleibergJ. (2022). Evaluating education governance: Does state takeover of school districts affect student achievement?Journal of Policy Analysis and Management, 41(1), 162–192. https://doi.org/10.1002/pam.22338
113.
*SchuelerB.GoodmanJ.DemingD. (2017). Can states take over and turn around school districts? Evidence from Lawrence, Massachusetts. Educational Evaluation and Policy Analysis, 39(2), 311–332. https://doi.org/10.3102/0162373716685824
114.
ScottC. (2009). Improving low-performing schools: Lessons from five years of studying school restructuring under No Child Left Behind. Center on Education Policy. https://files.eric.ed.gov/fulltext/ED507412.pdf
115.
*SlavinR.CheungA.HolmesG.MaddenN.ChamberlainA. (2013). Effects of a data-driven district reform model on state assessment outcomes. American Educational Research Journal, 50(2), 371–396. https://doi.org/10.3102/0002831212466909
*SpringerM. (2008). The influence of an NCLB accountability plan on the distribution of student test score gains. Economics of Education Review, 27(5), 556–563. https://doi.org/10.1016/j.econedurev.2007.06.004
*SteinbergM. P.MacDonaldJ. M. (2019). The effects of closing urban schools on students’ academic and behavioral outcomes: Evidence from Philadelphia. Economics of Education Review, 69(April), 25–60. https://doi.org/10.1016/j.econedurev.2018.12.005
120.
*StrunkK. O.MarshJ. A.HashimA. K.Bush-MecenasS.WeinsteinT. (2016). The impact of turnaround reform on student outcomes: Evidence and insights from the Los Angeles Unified School District. Education Finance and Policy, 11(3), 251–282. https://doi.org/10.1162/EDFP_a_00188
121.
*StrunkK. O.McEachinA. (2014). More than sanctions: Closing achievement gaps through California’s use of intensive technical assistance. Educational Evaluation and Policy Analysis, 36(3), 281–306. https://doi.org/10.3102/0162373713510967
122.
*StrunkK. O.McEachinA.WestoverT. N. (2014). The use and efficacy of capacity-building assistance for low-performing districts: The case of California’s District Assistance and Intervention Team. Journal of Policy Analysis and Management, 33(3), 719–751. https://doi.org/10.1002/pam.21658
123.
StuitD. A. (2010). Are bad schools immortal? The scarcity of turnarounds and shutdowns in both charters and district sectors. Thomas B. Fordham Institute. https://files.eric.ed.gov/fulltext/ED513905.pdf
124.
SunM.LiuJ.ZhuJ.LeClairZ. (2019). Using a text-as-data approach to understand reform processes: A deep exploration of school improvement strategies. Educational Evaluation and Policy Analysis, 41(4), 510–536. https://doi.org/10.3102/0162373719869318
*SunM.PennerE. K.LoebS. (2017). Resource- and approach-driven multi-dimensional change: Three-year effects of School Improvement Grants. American Education Research Journal, 54(4), 607–643. https://doi.org/10.3102/0002831217695790
127.
Tanner-SmithE. E.TiptonE. (2014). Robust variance estimation with dependent effect sizes. Research Synthesis Methods, 5(1), 13–30. https://doi.org/10.1002/jrsm.1091
128.
Tanner-SmithE. E.TiptonE.PolaninJ. R. (2016). Handling complex meta-analytic data structures using robust variance estimates: A tutorial in R. Journal of Developmental and Life-Course Criminology, 2, 85–112. https://doi.org/10.1007/s40865-016-0026-5
129.
TiptonE. (2015). Small sample adjustments for robust variance estimation with meta-regression. Psychological Methods, 20(3), 375–393. https://doi.org/10.1037/met0000011
130.
TiptonE.PustejovskyJ. E.AhmadiH. (2018). A history of meta-regression: Technical, conceptual, and practical developments between 1974 and 2018. Research Synthesis Methods, 2(10), 161–179. https://doi.org/10.1002/jrsm.1338
131.
TiptonE.PustejovskyJ. E.AhmadiH. (2019). Current practices in meta-regression in psychology, education, and medicine. Research Synthesis Methods, 10(2), 180–194. https://doi.org/10.1002/jrsm.1339
132.
TrujilloT.ReneeM. (2015). Irrational exuberance for market-based reform: How federal turnaround policies thwart democratic schooling. Teachers College Record, 117(6). http://www.tcrecord.org/Content.asp?ContentId=17880
*WintersM. A.CowenJ. M. (2012). Grading New York: Accountability and student proficiency in America’s largest school district. Educational Evaluation and Policy Analysis, 34(3), 313–327. https://doi.org/10.3102/0162373712440039
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
