Restricted accessResearch articleFirst published online 2023-5
Small-Group,Emergent Literacy Intervention Under Two Implementation Models: Intent-to-Treat and Dosage Effects for Preschoolers at Risk for Reading Difficulties
Preschool-age children identified as at risk for later reading difficulties can benefit from supplemental, small-group emergent literacy intervention. As such interventions become commercially available and marketed to preschool programs, it is important to understand their impacts when implemented by intended end users under routine conditions. In this study, we examined the effects of the Nemours BrightStart! (NBS!) intervention on children’s emergent literacy skills when implemented by teachers and community aides in authentic preschool classrooms. We randomly assigned 98 classrooms to one of three conditions (NBS! teacher-implemented, NBS! community aide-implemented, or control). Children enrolled in these classrooms who met eligibility criteria and were identified as at risk via an early literacy screener (n = 281) completed pretest and posttest emergent literacy assessments; those assigned to NBS! conditions received intervention from their classroom teacher or a community aide affiliated with a local kindergarten-readiness initiative. Intent-to-treat analyses showed no significant impacts of NBS! on any outcome, and an instrumental variable, as-treated approach showed one significant intervention effect on letter writing. Consequently, we did not replicate results of prior highly controlled efficacy trials. Findings have implications for revising the NBS! theory of change, conducting dosage and as-treated analyses, and moving research-based interventions toward scale-up.
Al OtaibaS.SchatschneiderC.SilvermanE. (2005). Tutor-assisted intensive learning strategies in kindergarten: How much is enough?Exceptionality, 13(4), 195–208. https://doi.org/10.1207/s15327035ex1304_2
2.
AngristJ. D.ImbensG. W.RubinD. B. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91(434), 444–455. http://www.jstor.org/stable/2291629
BailetL. L.RepperK.MurphyS.PiastaS.Zettler-GreeleyC. (2013). Emergent literacy intervention for prekindergarteners at risk for reading failure: Years 2 and 3 of a multiyear study. Journal of Learning Disabilities, 46(2), 133–153. https://doi.org/10.1177/0022219411407925
5.
BailetL. L.RepperK. K.PiastaS. B.MurphyS. P. (2009). Emergent literacy intervention for prekindergarteners at risk for reading failure. Journal of Learning Disabilities, 42(4), 336–355. https://doi.org/10.1177/0022219409335218
6.
BlesesD.HøjenA.JusticeL. M.DaleP. S.DybdalL.PiastaS. B.Markussen-BrownJ.ClausenM.HaghishE. F. (2018). The effectiveness of a large-scale language and preliteracy intervention: The SPELL randomized controlled trial in Denmark. Child Development, 89, 342–363. https://doi.org/10.1111/cdev.12859
7.
BowlesR. P.JusticeL. M.KhanK. S.PiastaS. B.SkibbeL. E.FosterT. D. (2020). Development of the Narrative Assessment Protocol-2: A tool for examining young children’s narrative skill. Language, Speech, and Hearing Services in Schools, 51(2), 390–404. https://doi.org/10.1044/2019_LSHSS-19-00038
8.
ByrneB.Fielding-BarnsleyR. (1991). Evaluation of a program to teach phonemic awareness to young children. Journal of Educational Psychology, 83(4), 451–455. https://doi.org/10.1037/0022-0663.83.4.451
9.
ByrneB.Fielding-BarnsleyR. (1995). Evaluation of a program to teach phonemic awareness to young children: A 2- and 3-year follow-up and a new preschool trial. Journal of Educational Psychology, 87(3), 488–503. https://doi.org/10.1037/0022-0663.87.3.488
10.
ChatterjiM. (2006). Reading achievement gaps, correlates, and moderators of early reading achievement: Evidence from the Early Childhood Longitudinal Study (ECLS) kindergarten to first grade sample. Journal of Educational Psychology, 98(3), 489–507. https://doi.org/10.1037/0022-0663.98.3.489
11.
ClementsD. H.SaramaJ.WolfeC. B.SpitlerM. E. (2015). Sustainability of a scale-up intervention in early mathematics: A longitudinal evaluation of implementation fidelity. Early Education and Development, 26(3), 427–449. https://doi.org/10.1080/10409289.2015.968242
12.
de BoerH.DonkerA. S.van der WerfM. P. C. (2014). Effects of the attributes of educational interventions on students’ academic performance: A meta-analysis. Review of Educational Research, 84(4), 509–545. https://doi.org/10.3102/0034654314540006
13.
DickinsonD. K. (2011). Teachers’ language practices and academic outcomes of preschool children. Science, 333(6045), 964–967. https://doi.org/10.1126/science.1204526
14.
DickinsonD. K.McCabeA.Clark-ChiarelliN. (2004). Preschool-based prevention of reading disability. In StoneC. A.SilllimanE. R.EhrenB. J.ApelK. (Eds.), Handbook of language and literacy (pp. 209–227). Guilford Press.
15.
DurlakJ. A.DuPreE. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41(3–4), 327–350. https://doi.org/10.1007/s10464-008-9165-0
16.
EhriL. C.NunesS. R.WillowsD. M.SchusterB. V.Yaghoub-ZadehZ.ShanahanT. (2001). Phonemic awareness instruction helps children learn to read: Evidence from the National Reading Panel’s meta-analysis. Reading Research Quarterly, 36(3), 250–287. https://doi.org/10.1598/RRQ.36.3.2
17.
FarleyK. S.PiastaS.DogucuM.O’ConnellA. (2017). Assessing and predicting small-group literacy instruction in early childhood classrooms. Early Education and Development, 28(4), 488–505. https://doi.org/10.1080/10409289.2016.1250549
18.
FerrerE.ShaywitzB. A.HolahanJ. M.MarchioneK. E.MichaelsR.ShaywitzS. E. (2015). Achievement gap in reading is present as early as first grade and persists through adolescence. The Journal of Pediatrics, 167(5), 1121–1125.e1122. https://doi.org/10.1016/j.jpeds.2015.07.045
19.
FieldingL.KerrN.RosierP. (2007). Annual growth for all students: Catch up growth for those who are behind. The New Foundation Press.
20.
FitzgeraldT.RobillardL.O’GradyA. (2018). Exploring the impact of a volunteer shared reading programme on preschool-aged children. Early Child Development and Care, 188(6), 851–861. https://doi.org/10.1080/03004430.2016.1240679
21.
GerdeH. K.BinghamG. E.PendergastM. L. (2015). Reliability and validity of the Writing Resources and Interactions in Teaching Environments (WRITE) for preschool classrooms. Early Childhood Research Quarterly, 31, 34–46. https://doi.org/10.1016/j.ecresq.2014.12.008
22.
GettingerM.StoiberK. (2008). Applying a response-to-intervention model for early literacy development in low-income children. Topics in Early Childhood Special Education, 27(4), 198–213. https://doi.org/10.1177/0271121407311238
23.
GoldsteinH.OlszewskiA.HaringC.GreenwoodC. R.McCuneL.CartaJ.AtwaterJ.GuerreroG.SchneiderN.McCarthyT.KelleyE. S. (2017). Efficacy of a supplemental phonemic awareness curriculum to instruct preschoolers with delays in early literacy development. Journal of Speech, Language, and Hearing Research, 60(1), 89–103. https://doi.org/10.1044/2016_JSLHR-L-15-0451
24.
GonzalezJ. E.Pollard-DurodolaS.SimmonsD. C.TaylorA. B.DavisM. J.KimM.SimmonsL. (2011). Developing low-income preschoolers’ social studies and science vocabulary knowledge through content-focused shared book reading. Journal of Research on Educational Effectiveness, 4(1), 25–52. https://doi.org/10.1080/19345747.2010.487927
25.
GottfredsonD.CookT.GardnerF. M.Gorman-SmithD.HoweG.SandlerI.ZafftK. (2015). Standards of evidence for efficacy, effectiveness, and scale-up research in prevention science: Next generation. Prevention Science, 16(7), 898–926. https://doi.org/10.1007/s11121-015-0555-x
GreenwoodC. R.CartaJ. J.AtwaterJ.GoldsteinH.KaminskiR.McConnellS. (2013). Is a response to intervention (RTI) approach to preschool language and early literacy instruction needed?Topics in Early Childhood Special Education, 33(1), 48–64. https://doi.org/10.1177/0271121412455438
28.
HaleyA.HulmeC.Bowyer-CraneC.SnowlingM. J.FrickeS. (2017). Oral language skills intervention in pre-school—A cautionary tale. International Journal of Language and Communication Disorders, 52(1), 71–79. https://doi.org/10.1111/1460-6984.12257
29.
HamreB. K.ParteeA.MulcahyC. (2017). Enhancing the impact of professional development in the context of preschool expansion. AERA Open, 3(4), 2332858417733686. https://doi.org/10.1177/2332858417733686
30.
HarnB. A.Forbes-SpearC.FritzR.BergT. (2012). Quality of intervention delivery and receipt observation tool. University of Oregon.
HjetlandH. N.BrinchmannE. I.SchererR.HulmeC.Melby-LervågM. (2020). Preschool pathways to reading comprehension: A systematic meta-analytic review. Educational Research Review, 30, Article 100323. https://doi.org/10.1016/j.edurev.2020.100323
33.
JiangY.GranjaM. R.KoballH. (2017). Basic facts about low-income children: Children under 6 years, 2015. National Center for Children in Poverty.
34.
JusticeL. M.ChowS.-M.CapelliniC.FlaniganK.ColtonS. (2003). Emergent literacy intervention for vulnerable preschoolers: Relative effects of two approaches. American Journal of Speech-Language Pathology, 12(3), 320–332. https://doi.org/10.1044/1058-0360(2003/078)
35.
KelleyE. S.GoldsteinH.SpencerT. D.ShermanA. (2015). Effects of automated tier 2 storybook intervention on vocabulary and comprehension learning in preschool children with limited oral language skills. Early Childhood Research Quarterly, 31, 47–61. https://doi.org/10.1016/j.ecresq.2014.12.004
36.
KimJ. S. (2019). Making every study count: Learning from replication failure to improve intervention research. Educational Researcher, 48(9), 599–607. https://doi.org/10.3102/0013189x19891428
37.
KoutsoftasA. D.HarmonM. T.GrayS. (2009). The effect of tier 2 intervention for phonemic awareness in a response-to-intervention model in low-income preschool classrooms. Language, Speech, and Hearing Services in Schools, 40(2), 116–130. https://doi.org/10.1044/0161-1461(2008/07-0101)
38.
KruseL. G.SpencerT. D.OlszewskiA.GoldsteinH. (2015). Small groups, big gains: Efficacy of a tier 2 phonological awareness intervention with preschoolers with early literacy deficits. American Journal of Speech-Language Pathology, 24(2), 189–205. https://doi.org/10.1044/2015_ajslp-14-0035
39.
KuhfeldM.SolandJ.PittsC.BurchinalM. (2020). Trends in children’s academic skills at school entry: 2010 to 2017. Educational Researcher, 49(6), 403–414. https://doi.org/10.3102/0013189x20931078
40.
LinY.-C.MagnusonK. A. (2018). Classroom quality and children’s academic skills in child care centers: Understanding the role of teacher qualifications. Early Childhood Research Quarterly, 42, 215–227. https://doi.org/10.1016/j.ecresq.2017.10.003
41.
LipseyM. W.PuzioK.YunC.HebertM. A.Steinka-FryK.ColeM. W.RobertsM.AnthonyK. S.BusickM. D. (2012, November). Translating the statistical representation of the effects of education interventions into more readily interpretable forms. https://ies.ed.gov/ncser/pubs/20133000/
42.
LoniganC. J.AnthonyJ. L.PhillipsB. M.PurpuraD. J.WilsonS. B.McQueenJ. D. (2009). The nature of preschool phonological processing abilities and their relations to vocabulary, general cognitive abilities, and print knowledge. Journal of Educational Psychology, 101(2), 345–358. https://doi.org/10.1037/a0013837
43.
LoniganC. J.PhillipsB. M. (2016). Response to instruction in preschool: Results of two randomized studies with children at significant risk of reading difficulties. Journal of Educational Psychology, 108(1), 114–129. https://doi.org/10.1037/edu0000054
44.
LoniganC. J.PurpuraD. J.WilsonS. B.WalkerP. M.Clancy-MenchettiJ. (2013). Evaluating the components of an emergent literacy intervention for preschool children at risk for reading difficulties. Journal of Experimental Child Psychology, 114(1), 111–130. https://doi.org/10.1016/j.jecp.2012.08.010
45.
LoniganC. J.WagnerR. K.TorgesenJ. K. (2007). Test of preschool early literacy. Pro-Ed.
46.
MarkovitzC. E.HernandezM. W.HedbergE.SilbergelittB. (2015). Outcome evaluation of the Minnesota Reading Corps PreK program. NORC at the University of Chicago.
47.
McConnellS. R.BradfieldT.Wackerle-HollmanA. K.RodriguezM. (2012). Individual growth and development indicators of early literacy (2nd ed.). Regents of the University of Minnesota.
48.
National Early Literacy Panel. (2008). Developing early literacy. National Institute for Literacy. www.nifl.gov
49.
NelsonJ. R.SandersE. A.GonzalezJ. (2009). The efficacy of supplemental early literacy instruction by community-based tutors for preschoolers enrolled in Head Start. Journal of Research on Educational Effectiveness, 3(1), 1–25. https://doi.org/10.1080/19345740903381031
PhillipsB. M.LoniganC. J.WyattM. A. (2009). Predictive validity of the Get Ready to Read! Screener: Concurrent and long-term relations with reading-related skills. Journal of Learning Disabilities, 42(2), 133–147. https://doi.org/10.1177/0022219408326209
53.
PhillipsB. M.TabuldaG.IngroleS. A.BurrisP. W.SedgwickT. K.ChenS. (2016). Literate language intervention with high-need prekindergarten children: A randomized trial. Journal of Speech, Language, and Hearing Research, 59(6), 1409–1420. https://doi.org/10.1044/2016_JSLHR-L-15-0155
54.
PiastaS. B. (2016). Current understandings of what works to support the development of emergent literacy in early childhood classrooms. Child Development Perspectives, 10(4), 234–239. https://doi.org/10.1111/cdep.12188
55.
PiastaS. B.LoganJ. A. R.ThomasL. J. G.Zettler-GreeleyC. M.BailetL. L.LewisK. (2021). Implementation of a small-group emergent literacy intervention by preschool teachers and community aides. Early Childhood Research Quarterly, 54, 31–43. https://doi.org/10.1016/j.ecresq.2020.08.002
56.
PiastaS. B.PhillipsB. M.WilliamsJ. M.BowlesR. P.AnthonyJ. L. (2016). Measuring young children’s alphabet knowledge: Development and validation of brief letter-sound knowledge assessments. The Elementary School Journal, 116(4), 523–548. https://doi.org/10.1086/686222
57.
PiastaS. B.WagnerR. K. (2010). Developing emergent literacy skills: A meta-analysis of alphabet learning and instruction. Reading Research Quarterly, 45(1), 8–38. https://doi.org/10.1598/RRQ.45.1.2
58.
PolaninJ. R.Tanner-SmithE. E.HennessyE. A. (2016). Estimating the difference between published and unpublished effect sizes: A meta-review. Review of Educational Research, 86(1), 207–236. https://doi.org/10.3102/0034654315582067
59.
Pollard-DurodolaS. D.GonzalezJ. E.SimmonsD. C.KwokO.TaylorA. B.DavisM. J.MinjungK. I. M.SimmonsL. (2011). The effects of an intensive shared book-reading intervention for preschool children at risk for vocabulary delay. Exceptional Children, 77(2), 161–183. https://doi.org/10.1177/001440291107700202
60.
RhodesH.HustonA. (2012). Building the workforce our youngest children deserve (Vol. 26). Society for Research in Child Development.
61.
RitterG. W.BarnettJ. H.DennyG. S.AlbinG. R. (2009). The effectiveness of volunteer tutoring programs for elementary and middle school students: A meta-analysis. Review of Educational Research, 79(1), 3–38. https://doi.org/10.3102/0034654308325690
62.
RitterG. W.DennyG.AlbinG.BarnettJ.BlankenshipV. (2006). The effectiveness of volunteer tutoring programs: A systematic review. Campbell Systematic Reviews, 2(1), 1–63. https://doi.org/10.4073/csr.2006.7
63.
RogdeK.HagenÅ. M.Melby-LervågM.LervågA. (2019). The effect of linguistic comprehension instruction on generalized language and reading comprehension skills: A systematic review. Campbell Systematic Reviews, 15(4), Article e1059. https://doi.org/10.1002/cl2.1059
64.
SagarinB. J.WestS. G.RatnikovA.HomanW. K.RitchieT. D.HansenE. J. (2014). Treatment noncompliance in randomized experiments: Statistical approaches and design issues. Psychological Methods, 19(3), 317–333. https://doi.org/10.1037/met0000013
65.
SavageR.CarlessS. (2005). Learning support assistants can deliver effective reading interventions for “at-risk” children. Educational Research, 47(1), 45–61. https://doi.org/10.1080/0013188042000337550
66.
SavageR.CarlessS.StuartM. (2003). The effects of rime- and phoneme-based teaching delivered by learning support assistants. Journal of Research in Reading, 26(3), 211–233. https://doi.org/10.1111/1467-9817.00199
67.
SchneiderW.RothE.EnnemoserM. (2000). Training phonological skills and letter knowledge in children at risk for dyslexia: A comparison of three kindergarten intervention programs. Journal of Educational Psychology, 92(2), 284–295. https://doi.org/10.1037/0022-0663.92.2.284
68.
SpencerE. J.GoldsteinH.ShermanA.NoeS.TabbahR.ZiolkowskiR.SchneiderN. (2012). Effects of an automated vocabulary and comprehension intervention: An early efficacy study. Journal of Early Intervention, 34(4), 195–221. https://doi.org/10.1177/1053815112471990
69.
SpencerT. D.PetersenD. B.AdamsJ. L. (2015). Tier 2 language intervention for diverse preschoolers: An early-stage randomized control group study following an analysis of response to intervention. American Journal of Speech-Language Pathology, 24(4), 619–636. https://doi.org/10.1044/2015_AJSLP-14-0101
70.
SuggateS. P. (2016). A meta-analysis of the long-term effects of phonemic awareness, phonics, fluency, and reading comprehension interventions. Journal of Learning Disabilities, 49(1), 77–96. https://doi.org/10.1177/0022219414528540
71.
ThomasL. J. G.GerdeH. K.PiastaS. B.LoganJ. A. R.BailetL. L.Zettler-GreeleyC. M. (2020). The early writing skills of children identified as at-risk for literacy difficulties. Early Childhood Research Quarterly, 51, 392–402. https://doi.org/10.1016/j.ecresq.2020.01.003
72.
TortorelliL. S.BowlesR. P.SkibbeL. E. (2017). Easy as AcHGzrjq: The quick letter name knowledge assessment. The Reading Teacher, 71(2), 145–156. https://doi.org/10.1002/trtr.1608
73.
WasikB. A. (2008). When fewer is more: Small groups in early childhood classrooms. Early Childhood Education Journal, 35(6), 515–521. https://doi.org/10.1007/s10643-008-0245-4
74.
WhitehurstG. J.LoniganC. J. (2010). Get read to read!—Revised edition. Pearson Assessments.
75.
WolfR.MorrisonJ.InnsA.SlavinR.RismanK. (2020). Average effect sizes in developer-commissioned and independent evaluations. Journal of Research on Educational Effectiveness, 13(2), 428–447. https://doi.org/10.1080/19345747.2020.1726537
WyattT.Chapman-DeSousaB. (2017). Teaching as interaction: Challenges in transitioning teachers’ instruction to small groups. Early Childhood Education Journal, 45(1), 61–70. https://doi.org/10.1007/s10643-015-0758-6
78.
Zettler-GreeleyC. M.BailetL. L.MurphyS.DeLuccaT.Branum-MartinL. (2018). Efficacy of the Nemours BrightStart! Early literacy program: Treatment outcomes from a randomized trial with at-risk prekindergartners. Early Education and Development, 29, 873–892. https://doi.org/10.1080/10409289.2018.1475202
79.
ZhangC.HurJ.DiamondK.PowellD. (2015). Classroom writing environments and children’s early writing skills: An observational study in Head Start classrooms. Early Childhood Education Journal, 43(4), 307–315. https://doi.org/10.1007/s10643-014-0655-4
80.
ZuckerT. A.SolariE. J.LandryS. H.SwankP. R. (2013). Effects of a brief tiered language intervention for prekindergartners at risk. Early Education and Development, 24(3), 366–392. https://doi.org/10.1080/10409289.2012.664763
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
