Summer reading programs are a form of extended school year services for students in special education. However, previous studies have not reported including high percentages of participants in special education, nor have studies sufficiently controlled for selection bias. This study combined propensity score weighting with partially clustered models to examine the effects of a summer reading program on the growth in reading skills of K–4 students, roughly 50% to 75% of whom were in special education. Results suggest that students in most grades improved on some but not all skills. However, fewer improvements were apparent when participating students were compared with peers via propensity score analyses. In addition, Grade 3 students in the control group outperformed their peers who attended summer school.
Get full access to this article
View all access options for this article.
References
1.
AlexanderK. L.EntwisleD. R.OlsonL. S. (2007). Lasting consequences of the summer learning gap. American Sociological Review, 72, 167–180. doi:10.1177/000312240707200202
Al OtaibaS.AllorJ.OrtizM.GreulichL.WanzekJ.TorgesenJ. (2016). Tier 3 primary grade reading interventions: Can we distinguish necessary from sufficient? In JimersonS. R.BurnsM. K.VanDerHeydenA. M. (Eds.), Handbook of response to intervention (pp. 389–404): Boston, MA: Springer. doi:10.1007/978-1-4899-7568-3_23
4.
ArpinoB.MealliF. (2011). The specification of the propensity score in multilevel studies. Computational Statistics and Data Analysis, 55, 1770–1780 doi:10.1016/j.csda.2010.11.008.
5.
BatesD.MaechlerM.BolkerB.WalkerS. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. doi:10.18637/jss.v067.i01.
6.
Battle v. Commonwealth of Pennsylvania, 629 F.2d 269 (3rd Cir. 1980).
7.
BeachK. D.McIntyreE.PhilippakosZ. A.MrazM.PilonietaP.VintinnerJ. P. (2018). Effects of a summer reading intervention on reading skills for low-income black and Hispanic students in elementary school. Reading & Writing Quarterly, 34, 263–280. doi: 10.1080/10573569.2018.1446859
8.
BellS. R.CarrilloN. (2007). Characteristics of effective summer learning programs in practice. New Directions for Youth Development, 2007(114), 45–63. doi:10.1002/yd.212
9.
BormanG. D.DowlingN. M. (2006). Longitudinal achievement effects of multiyear summer school: Evidence from the Teach Baltimore randomized field trial. Educational Evaluation and Policy Analysis, 28, 25–48. doi:10.3102/01623737028001025
10.
BormanG. D.GoetzM. E.DowlingN. M. (2009). Halting the summer achievement slide: A randomized field trial of the KindergARTen summer camp. Journal of Education for Students Placed at Risk, 14, 133–147. doi:10.1080/10824660802427652
11.
Center for Literacy. (2007). A comprehensive literacy model: A guidebook for small group interventions. Little Rock, AR: University of Arkansas at Little Rock.
12.
ChallJ. S. (1983). Stages of reading development. New York, NY: McGraw-Hill
13.
ChristT. J.ArañasY. A.KemberJ. M.KissA. J.McCarthy-TrentmanA.MonaghenB. D.. . . WhiteM. J. (2014). Formative assessment system for teachers technical manual: earlyReading, CBMReading, aReading, aMath, and earlyMath. Minneapolis, MN: Formative Assessment System for Teachers.
14.
ChristodoulouJ. A.CyrA.MurtaghJ.ChangP.LinJ.GuarinoA. J.. . . GabrieliJ. D. E. (2017). Impact of intensive summer reading intervention for children with reading disabilities and difficulties in early elementary school. Journal of Learning Disabilities, 50, 115–127. doi:10.1177/0022219415617163
15.
D’AgostinoJ. V.LoseM. K.KellyR. H. (2017). Examining the sustained effects of Reading Recovery. Journal of Education for Students Placed at Risk, 22, 116–127. doi:10.1080/10824669.2017.1286591
16.
DentonC. A.SolariE. J.CiancioD. J.HechtS. A.SwankP. R. (2010). A pilot study of a kindergarten summer school reading program in high-poverty urban schools. Elementary School Journal, 110, 423–439. doi:10.1086/651190
17.
DyniaJ. M.PiastaS. B.JusticeL. M. & Columbus Metropolitan Library. (2016). Impact of library-based summer reading clubs on primary-grade children’s literacy activities and achievement. Library Quarterly: Information, Community, Policy, 85, 386–405. doi: 10.1086/682733
18.
EastmanL. A. (1897). The child, the school, and the library. Library Journal, 21(4), 134–139.
19.
Every Student Succeeds Act. (2015). Pub. L. No. 114-95, 20 U.S.C. §6301 et seq.
20.
FairchildR.SminkJ.StewartA. B. (2009). It’s time for summer: An analysis of recent policy and funding opportunities. New York, NY: Wallace Foundation.
21.
FoormanB.PetscherY.SchatschneiderC., & (2015). Reading Assessment for Prescriptive Instructional Data (RAPID). Concord, MA: Lexia Learning.
22.
FoormanB.PetscherY.SchatschneiderC., & Lexia Learning. (2016a). The technical manual for Lexia RAPID Assessment Version 2.0: Grades K–2. Concord, MA: Lexia Learning.
23.
FoormanB.PetscherY.SchatschneiderC. & Lexia Learning. (2016b). The technical manual for Lexia RAPID Assessment Version 2.0: Grades 3–12. Concord, MA: Lexia Learning.
24.
ForbesS.HutchisonA.MissallK. (2015). A look at summer reading programs across Iowa. Cedar Falls, IA: University of Northern Iowa.
25.
GoossensN. A. M. C.CampG.VerkoeijenP. P. J. L.TabbersH. K.ZwaanR. A. (2012). Spreading the words: A spacing effect in vocabulary learning. Journal of Cognitive Psychology, 24, 965–971. doi: 10.1080/20445911. 2012.722617
26.
HarnB.ParisiD.StoolmillerM. (2013). Balancing fidelity with flexibility and fit: What do we really know about fidelity of implementation in schools?Exceptional Children, 79, 181–193. doi:10.1177/001440291307900204
27.
HarrisL. A. (1968). Reports on reading and the disadvantaged: Elementary level (Report ED 015 350). Bloomington, IN: Indiana University. Retrieved from https://files.eric.ed.gov/fulltext/ED013350.pdf
28.
HedgesL. V.CitkowiczM. (2015). Estimating effect size when there is clustering in one treatment group. Behavior Research Methods, 47, 1295–1308. doi:10.3758/s13428-014-0538-z
29.
Individuals with Disabilities Education Act Regulations, 34 C.F.R. § 300 et seq. (2012).
30.
JenkinsJ. R.SchillerE.BlackorbyJ.ThayerS. K.TillyW. D. (2013). Responsiveness to intervention in reading: Architecture and practices. Learning Disability Quarterly, 36, 36–46. doi:10.1177/0731948712464963
31.
JohnstonJ.RileyJ.RyanC.Kelly-VanceL. (2015). Evaluation of a summer reading program to reduce summer setback. Reading & Writing Quarterly, 31, 334–350. doi:10.1080/10573569.2013.857978
32.
KampsD.AbbottM.GreenwoodC.WillsH.VeerkampM.KaufmanJ. (2008). Effects of small-group reading instruction and curriculum differences for students most at risk in kindergarten. Journal of Learning Disabilities, 41, 101–114. doi:10.1177/0022219407313412
33.
KimJ. S.BurkhauserM. A.QuinnD. M.GuryanJ.KingstonH. C.AlemanK. (2017). Effectiveness of structured teacher adaptations to an evidence-based summer literacy program. Reading Research Quarterly, 52, 443–467. doi:10.1002/rrq.178
34.
KimJ. S.GuryanJ.WhiteT. G.QuinnD. M.CapotostoL.KingstonH. C. (2016). Delayed effects of a low-cost and large-scale summer reading intervention on elementary school children’s reading comprehension. Journal of Research on Educational Effectiveness, 9, 1–22. doi:10.1080/19345747.2016.1164780
35.
KimJ. S.QuinnD. M. (2013). The effects of summer reading on low-income children’s literacy achievement from kindergarten to Grade 8: A meta-analysis of classroom and home interventions. Review of Educational Research, 83, 386–431. doi:10.3102/0034654313483906
36.
LeeO.PenfieldR.Maerten-RiveraJ. (2009). Effects of fidelity of implementation on science achievement gains among English language learners. Journal of Research in Science Teaching, 46, 836–859. doi:10.1002/8tea.20335
37.
LeiteW. L. (2016). Practical propensity score methods using R. Thousand Oaks, CA: Sage.
38.
LeiteW. L.JimenezF.KayaY.StapletonL. M.MacInnesJ. W.SandbachR. (2015). An evaluation of weighting methods based on propensity scores to reduce selection bias in multilevel observational studies. Multivariate Behavioral Research, 50, 1532–7906. doi:10.1080/00273171.2014.991018
39.
LiF.ZaslavaskyA. M.LandumM. B. (2013). Propensity score weighting with multilevel data. Statistics in Medicine, 32, 3373–3387. doi:10.1002/sim.5786
40.
LoganJ. R.HartS. A.CuttingL.Deater-DeckardK.SchatschneiderC.PetrillS. (2013). Reading development in young children: Genetic and environmental influences. Child Development, 84, 2131–2144. doi:10.1111/cdev.12104
41.
LohrS.SchochetP. Z.SandersE. (2014). Partially nested randomized controlled trials in education research: A guide to design and analysis (Publication No. 2014-2000) Washington, DC: National Center for Education Research.
42.
MatsudairaJ. D. (2008). Mandatory summer school and student achievement. Journal of Econometrics, 142, 829–850. doi:10.1016/j.jeconom.2007.05.015
43.
McCombsJ. S.AugustinC. H.SchwartzH. L.BodillyS. J.McInnisB.LichterD. S.CrossA. B. (2011). Making summer count: How summer programs can boost children’s learning. New York, NY: Wallace Foundation.
McGraw-Hill Education. (2017a). Wonders: The power of connection. New York, NY: Author.
46.
McGraw-Hill Education. (2017b). WonderWorks: Intervention. New York, NY: Author.
47.
McMurrayB.KapnoulaE.GaskellM. G. (2016). Learning and integration of new word-forms: Consolidation, pruning and the emergence of automaticity. In GaskellM. G.MirkovićJ. (Eds.), Speech perception and spoken word recognition (pp. 116–142). London, UK: Taylor & Francis.
48.
MorganP. L.FriscoM. L.FarkasG.HibelJ. (2010). A propensity score matching analysis of the effects of special education services. The Journal of Special Education, 43, 236–254. doi:10.1177/0022466908323007
49.
O’DonnellC. L. (2008). Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K–12 curriculum intervention research. Review of Educational Research, 78, 33–84. doi:10.3102/0034654307313793
50.
OslundE. L.Hagan-BurkeS.SimmonsD. C.ClemensN. H.SimmonsL. E.TaylorA. B.. . . CoyneM. D. (2017). Predictive validity of curriculum-embedded measures on outcomes of kindergarteners identified as at risk for reading difficulty. Journal of Learning Disabilities, 50, 712–723. doi:10.1177/0022219416664866
51.
PattonK. L. S.ReschlyA. L. (2013). Using curriculum-based measurement to examine summer learning loss. Psychology in the Schools, 50, 738–753. doi:10.1002/pits.21704
52.
PinheiroJ.BatesD.DebRoyS.SarkarD., & R Core Team (2017). _nlme: Linear and nonlinear mixed effects models. R Package Version 3.1-131. Retrieved from https://CRAN.R-project.org/package=nlme
53.
R Core Team. (2017). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.
54.
Rambo-HernandezK. E.McCoachD. B. (2015). High-achieving and average students’ reading growth: Contrasting school and summer trajectories. Journal of Educational Research, 108, 112–129. doi:10.1080/00220671.2013.850398
55.
ReedD. K.CookK.AloeA. M. (in press). A benefit-cost analysis of summer reading programs implemented under state guidelines. Educational Policy. doi:10/1177/0895904818802112
56.
ReedD. K.SchmitzS.AloeA. M.FolsomJ. S. (2016). Report of the 2016 Intensive Summer Reading Program (ISRP) study. Iowa City, IA: Iowa Reading Research Center.
57.
RetelsdorfJ.BeckerM.KöllerO.MöllerJ. (2012). Reading development in a tracked school system: A longitudinal study over 3 years using propensity score matching. British Journal of Educational Psychology, 82, 647–671. doi:10.1111/j.2044–8279.2011.02051.x
58.
RosenbaumP. R. (2010). Design of observational studies (2nd ed.). New York, NY: Springer.
59.
RosenbaumP. R.RubinD. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41–55. doi:10.1017/cbo9780511810725.016
60.
ScarboroughH. S. (2001). Connecting early language and literacy to later reading (dis)abilities: Evidence, theory, and practice. In NeumanS.DickinsonD. (Eds.), Handbook for research in early literacy (pp. 97–110). New York, NY: Guilford Press.
61.
SchacterJ.JoB. (2005). Learning when school is not in session: A reading summer day-camp intervention to improve the achievement of exiting first-grade students who are economically disadvantaged. Journal of Research in Reading, 28, 158–169. doi:10.1111/j.1467-9817.2005.00260.x
62.
SeabrookR.BrownG. A.SolityJ. E. (2005). Distributed and massed practice: From laboratory to classroom. Applied Cognitive Psychology, 19, 107–122. doi:10.1002/acp.1066
63.
ShadishW. R.CookT. D.CampbellD. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin.
64.
SteinM. L. (2017). Supporting the summer reading of urban youth: An evaluation of the Baltimore SummerReads program. Education and Urban Society, 49, 29–52. doi: 10.1177/001312451663 0595
65.
SteinerP. M.CookT. D.LiW.ClarkM. H. (2015). Bias reduction in quasi-experiments with little selection theory but many covariates. Journal of Research on Educational Effectiveness, 8, 552–576. doi:10.1080/19345747.2014.978058
66.
TranL.SanchezT.ArellanoB.SwansonH. L. (2011). A meta-analysis of the RTI literature for children at risk for reading disabilities. Journal of Learning Disabilities, 44, 283–295. doi:10.1177/0022219410378447
67.
What Works Clearinghouse. (2017). What Works Clearinghouse procedures and standards handbook (Version 4.0). Washington, DC: Institute of Educational Science.
68.
WhiteT. G.KimJ. S.KingstonH. C.FosterL. (2013). Replicating the effects of a teacher-scaffolded voluntary summer reading program: The role of poverty. Reading Research Quarterly, 49, 5–30. doi:10.1002/rrq.62
69.
WilkinsC.GerstenR.DeckerL.GrundenL.BrasielS.BrunnertK.JayanthiM. (2012). Does a summer reading program based on Lexiles affect reading comprehension? (Publication No. 2012-4006). Washington, DC: National Center for Education Evaluation and Regional Assistance.
70.
ZvochK.StevensJ. J. (2015). Identification of summer school effects by comparing the in- and out-of-school growth rates of struggling early readers. The Elementary School Journal, 115, 432–456. doi:10.1086/680229
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
