Learning disabilities (LD) may affect a range of academic skills but are most often observed in reading. Researchers and policymakers increasingly recommend addressing reading difficulties encountered by students with LD using evidence-based practices, or interventions validated through multiple, high-quality research studies. A valuable tool in identifying evidence-based practices is the meta-analysis, which entails statistically aggregating the results obtained through primary studies. Specific methods used in meta-analyses have the potential to influence their findings, with ramifications for research and practice. This review assessed the methodological features of the systematic reviews and analytic procedures featured in meta-analyses of reading intervention studies that included students with LD written between 2000 and 2020. Identified articles (N = 23) suggest that meta-analyses have become more prevalent and transparent over time, notwithstanding issues related to publication bias and the opacity of coding procedures. A discussion of implications follows a description of results.
*AustinC. R.WanzekJ.ScammaccaN. K.VaughnS.GeselS. A.DoneganR. E.EngelmannM. L. (2019). The relationship between study quality and the effects of supplemental reading interventions: A meta-analysis. Exceptional Children, 853(3), 347–366. https://doi.org/10.1177/0014402918796164
2.
BeckerT. E. (2005). Potential problems in the statistical control of variables in organizational research: A qualitative analysis with recommendations. Organizational Research Methods, 8(3), 274–289. https://doi.org/10.1177/1094428105278021
3.
BerkeleyS.ScanlonD.BaileyT. R.SuttonJ. C.SaccoD. M. (2020). A snapshot of RTI implementation a decade later: New picture, same story. Journal of Learning Disabilities, 53(5), 332–342. https://doi.org/10.1177/0022219420915867
4.
*BerkeleyS.ScruggsT. E.MastropieriM. A. (2010). Reading comprehension instruction for students with learning disabilities, 1995–2006: A meta-analysis. Remedial and Special Education, 31, 423–436. https://doi.org/10.1177/0741932509355988
5.
*BlokH.OostdamR.OtterM. E.OvermaatM. (2002). Computer-assisted instruction in support of beginning reading instruction: A review. Review of Educational Research, 72(1), 101–130. https://doi.org/10.3102/00346543072001101
6.
BoothA. (2010). How much searching is enough? Comprehensive versus optimal retrieval for technology assessments. International Journal of Technology Assessment in Health Care, 26, 431–435. https://doi.org/10.1017/S0266462310000966
7.
BorensteinM.HedgesL. V.HigginsJ. P.RothsteinH. R. (2011). Introduction to meta-analysis. John Wiley & Sons.
8.
BownM. J.SuttonA. J. (2010). Quality control in systematic reviews and meta-analyses. European Journal of Vascular and Endovascular Surgery, 40, 669–677. https://doi.org/10.1016/j.ejvs.2010.07.011
CheungM. W. L. (2019). A guide to conducting a meta-analysis with non-independent effect sizes. Neuropsychology Review, 29(4), 387–396. https://doi.org/10.1007/s11065-019-09415-6
11.
ChitiyoA.KingS. A.KrizonM. D.AblakwaC. N.MarkelzA. M. (2020). A methodological review of research syntheses involving reading interventions for students with emotional and behavioral disorders. Behavioral Disorders, 46, 214–225. https://doi.org/10.1177/0198742920919095
12.
CookB. G.CookS. C. (2013). Unraveling evidence-based practices in special education. The Journal of Special Education, 47(2), 71–82. https://doi.org/10.1177/0022466911420877
13.
CookB. G.LloydJ. W.MellorD.NosekB. A.TherrienW. J. (2018). Promoting open science to increase the trustworthiness of evidence in special education. Exceptional Children, 85(1), 104–118. https://doi.org/10.1177/0014402918793138
14.
CooperW. W.SeifordL. M.ToneK. (2000). Data envelopment analysis: Handbook on data envelopment analysis (2nd ed.). Springer.
DeeksJ. J.HigginsJ. P.AltmanD. G., Cochrane Statistical Methods Group. (2019). Analyzing data and undertaking meta-analyses. In HigginsJ. P. T.ThomasJ.ChandlerJ.CumpstonM.LiT.PageM. J.WelchV. A. (Eds.), Cochrane handbook for systematic reviews of interventions (pp. 241–284). John Wiley & Sons. https://doi.org/10.1002/9781119536604.ch10
17.
DelaneyA.TamásP. A. (2018). Searching for evidence or approval? A commentary on database search in systematic reviews and alternative information retrieval methodologies. Research Synthesis Methods, 9(1), 124–131. https://doi.org/10.1002/jrsm.1282
18.
DuvalS. (2005). The trim and fill method. In RothsteinH. R.SuttonA. J.BorensteinM. (Eds.), Publication bias in meta-analysis: Prevention, assessment and adjustments (pp. 127–144). John Wiley & Sons.
19.
*EdmondsM. S.VaughnS.WexlerJ.ReutebuchC.CableA.TackettK. K.SchnakenbergJ. W. (2009). A synthesis of reading interventions and effects on reading comprehension outcomes for older struggling readers. Review of Educational Research, 79(1), 262–300. https://doi.org/10.3102/0034654308325998
20.
*ElbaumB.VaughnS.HughesM. T.MoodyS. W. (2000). How effective are one-to-one tutoring programs in reading for elementary students at risk for reading failure? A meta-analysis of the intervention research. Journal of Educational Psychology, 92(4), 605–619. https://doi.org/10.1037/0022-0663.92.4.605
21.
*EllemanA. M.LindoE. J.MorphyP.ComptonD. L. (2009). The impact of vocabulary instruction on passage-level comprehension of school-age children: A meta-analysis. Journal of Research on Educational Effectiveness, 2(1), 1–44. https://doi.org/10.1080/19345740802539200
22.
FergusonC. J.BrannickM. T. (2012). Publication bias in psychological science: Prevalence, methods for identifying and controlling, and implications for the use of meta-analyses. Psychological Methods, 17(1), 120–128. https://doi.org/10.1037/a0024445
23.
*FildermanM. J.TosteJ. R.DidionL. A.PengP.ClemensN. H. (2018). Data-based decision making in reading interventions: A synthesis and meta-analysis of the effects for struggling readers. The Journal of Special Education, 52(3), 174–187. https://doi.org/10.1177/0022466918790001
24.
GerstenR.FuchsL. S.ComptonD.CoyneM.GreenwoodC.InnocentiM. S. (2005). Quality indicators for group experimental and quasi-experimental research in special education. Exceptional Children, 71(2), 149–164. https://doi.org/10.1177/001440290507100202
25.
*HanI. (2009). Evidence-based reading instruction for English language learners in preschool through sixth grades: A meta-analysis of group design studies [Doctoral dissertation]. University of Minnesota.
26.
HedgesL. V.TiptonE.JohnsonM. C. (2010). Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods, 1(1), 39–65. https://doi.org/10.1002/jrsm.5
27.
HigginsJ. P.ThompsonS. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21(11), 1539–1558. https://doi.org/10.1002/sim.1186
28.
Huedo-MedinaT. B.Sánchez-MecaJ.Marín-MartínezF.BotellaJ. (2006). Assessing heterogeneity in meta-analysis: Q statistic or I² index?Psychological Methods, 11(2), 193–206. https://doi.org/10.1037/1082-989X.11.2.193
29.
*KimD.AnY.ShinH. G.LeeJ.ParkS. (2020). A meta-analysis of single-subject reading intervention studies for struggling readers: Using Improvement Rate Difference (IRD). Heliyon, 6(11), Article e05024. https://doi.org/10.1016/j.heliyon.2020.e05024
30.
KingS. A.KostewiczD.EndersO.BurchT.ChitiyoA.TaylorJ.DeMariaS.ReidM. (2020). Search and selection procedures of literature reviews in behavior analysis. Perspectives on Behavior Science, 43(4), 725–760. https://doi.org/10.1007/s40614-020-00265-9
31.
KingS.DavidsonK.ChitiyoA.AppleD. (2020). Evaluating article search and selection procedures in special education literature reviews. Remedial and Special Education, 41(1), 3–17. https://doi.org/10.1177/0741932518813142
32.
KostewiczD.KingS.DatchukS.BrennanK.CaseyS. (2016). Data collection and measurement assessment in behavioral research: 1958–2013. Behavior Analysis: Research and Practice, 16(1), 19–33. https://doi.org/10.1037/bar0000031
33.
LedfordJ. R.GastD. L. (Eds.) (2018). Single case research methodology: Applications in special education and behavioral sciences. Routledge.
34.
*LeeJ.YoonS. Y. (2017). The effects of repeated reading on reading fluency for students with reading disabilities: A meta-analysis. Journal of Learning Disabilities, 50(2), 213–224. https://doi.org/10.1177/0022219415605194
35.
LiberatiA.AltmanD. G.TetzlaffJ.MulrowC.GøtzscheP. C.IoannidisJ. P.DevereauxP. J.KleijnenJ.MoherD. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Journal of Clinical Epidemiology, 62(10), e1–e34. https://doi.org/10.1016/j.jclinepi.2009.06.006
MagginD. M.LaneK. L.PustejovskyJ. E. (2017). Introduction to the special issue on single-case systematic reviews and meta-analyses. Remedial and Special Education, 38(6), 323–330. https://doi.org/10.1177/0741932517717043
38.
MoeyaertM.MagginD.VerkuilenJ. (2016). Reliability, validity, and usability of data extraction programs for single-case research designs. Behavior Modification, 40(6), 874–900. https://doi.org/10.1177/0145445516645763
39.
*MorganP. L.SideridisG. D. (2006). Contrasting the effectiveness of fluency interventions for students with or at risk for learning disabilities: A multilevel random coefficient modeling meta-analysis. Learning Disabilities Research & Practice, 21, 191–210. https://doi.org/10.1111/j.1540-5826.2006.00218.x
40.
*MorganP. L.SideridisG.HuaY. (2012). Initial and over-time effects of fluency interventions for students with or at risk for disabilities. The Journal of Special Education, 46, 94–116. https://doi.org/10.1177/0022466910398016
41.
MostertM. P. (1996). Reporting meta-analyses in learning disabilities. Learning Disabilities Research and Practice, 11(1), 2–14.
42.
MostertM. P. (2001). Facilitated communication since 1995: A review of published studies. Journal of Autism and Developmental Disorders, 31(3), 287–313. http://doiorg/101023/a:1010795219886.
43.
NoortgateW. V.Lopez-LopezJ. A.Marin-MartinezF.Sanchez-MecaJ. (2013). Three-level meta-analysis of dependent effect sizes. Behavioral Research Methods, 45, 576–594. http://doi.org/10.3758/s13428-012-0261-6
44.
OlkinI.GleserL. (2009). Stochastically dependent effect sizes. In CooperH.HedgesL. V.ValentineJ. C. (Eds.), The handbook of research synthesis and meta-analysis.
45.
OrwinR. G. (1983). A fail-safe N for effect size in meta-analysis. Journal of Educational Statistics, 8(2), 157–159. https://doi.org/10.2307/1164923
46.
PigottT. D.PolaninJ. R. (2020). Methodological guidance paper: High-quality meta-analysis in a systematic review. Review of Educational Research, 90(1), 24–46. https://doi.org/10.3102/0034654319877153
47.
*PyleN.VasquezA. C.Lignugaris/KraftB.GillamS. L.ReutzelD. R.OlszewskiA.SeguraH.HartzheimD.LaingW.PyleD. (2017). Effects of expository text structure interventions on comprehension: A meta-analysis. Reading Research Quarterly, 52(4), 469–501. https://doi.org/10.1002/rrq.179
48.
RaoG.Lopez-JimenezF.BoydJ.D’AmicoF.DurantN. H.HlatkyM. A.HowardG.KirleyK.MasiC.Powell-WileyT. M.SolomonidesA. E.SolomonidesA. E.WestC. P.WesselJ. (2017). Methodological standards for meta-analyses and qualitative systematic reviews of cardiac prevention and treatment studies: A scientific statement from the American Heart Association. Circulation, 136, e172–e194. https://doi.org/10.1161/CIR.0000000000000523
49.
RoitschJ.GumpertM.SpringleA.RaymerA. M. (2020). Writing Instruction for Students with Learning Disabilities: Quality Appraisal of Systematic Reviews and Meta-Analyses. Reading & Writing Quarterly, 1–13. https://doi.org/10.1080/10573569.2019.1708221
50.
*ScammaccaN.RobertsG.StuebingK. K. (2014). Meta-analysis with complex research designs: Dealing with dependence from multiple measures and multiple group comparisons. Review of Educational Research, 84, 328–364. https://doi.org/10.3102/0034654313500826
51.
ScammaccaN.RobertsG.VaughnS.EdmondsM.WexlerJ.ReutebuchC. K.TorgesenJ. K. (2007). Interventions for adolescent struggling readers: A meta-analysis with implications for practice. RMC Research Corporation, Center on Instruction. https://files.eric.ed.gov/fulltext/ED521837.pdf
52.
*ScammaccaN. K.RobertsG.VaughnS.StuebingK. K. (2015). A meta-analysis of interventions for struggling readers in grades 4–12: 1980–2011. Journal of Learning Disabilities, 48(4), 369–390. http://doi.org/10.1177/0022219413504995
53.
ShadishW. R.LecyJ. D. (2015). The meta-analytic big bang. Research Synthesis Methods, 6(3), 246–264. https://doi.org/10.1002/jrsm.1132
54.
ShadishW. R.HedgesL. V.HornerR. H.OdomS. L. (2015). The role of between-case effect size in conducting, interpreting, and summarizing single-case research (NCER 2015-002). National Center for Education Research. https://ies.ed.gov/ncser/pubs/2015002/pdf/2015002.pdf
55.
ShadishW. R.KyseE. N.RindskopfD. M. (2013). Analyzing data from single-case designs using multilevel models: New applications and some agenda items for future research. Psychological Methods, 18(3), 385–405. https://doi.org/10.1037/a0032964
56.
ShiL.LinL. (2019). The trim-and-fill method for publication bias: Practical guidelines and recommendations based on a large database of meta-analyses. Medicine, 98(23), Article e15987. https://doi.org/10.1097/MD.0000000000015987
57.
*StephensD. (2014). Effective reading interventions for Spanish-speaking English learners with reading disabilities, English learners who struggle with reading, or both: A meta-analysis of second through fifth grades [Doctoral dissertation]. University of San Francisco).
58.
SterneJ. A.BeckerB. J.EggerM. (2005). The funnel plot. In RothsteinH. R.SuttonA. J.BorensteinM. (Eds.), Publication bias in meta-analysis: Prevention, assessment and adjustments (pp. 75–98). John Wiley & Sons.
59.
SterneJ. A.EggerM. (2005). Regression methods to detect publication and other bias in meta-analysis. In RothsteinH. R.SuttonA. J.BorensteinM. (Eds.), Publication bias in meta-analysis: Prevention, assessment and adjustments (pp. 99–110). John Wiley & Sons.
60.
*StevensE. A.ParkS.VaughnS. (2019). A review of summarizing and main idea interventions for struggling readers in grades 3 through 12: 1978–2016. Remedial and Special Education, 40(3), 131–149. https://doi.org/10.1177/0741932517749940
61.
*SuggateS. P. (2010). Why what we teach depends on when: Grade and reading intervention modality moderate effect size. Developmental Psychology, 46(6), 1556–1579. https://doi.org/10.1037/a0020612
62.
SuttonA. J.SongF.GilbodyS. M.AbramsK. R. (2000). Modelling publication bias in meta-analysis: A review. Statistical Methods in Medical Research, 9(5), 421–445. https://doi.org/10.1177/096228020000900503
SwansonE.HairrellA.KentS.CiulloS.WanzekJ. A.VaughnS. (2014). A synthesis and meta-analysis of reading interventions using social studies content for students with learning disabilities. Journal of Learning Disabilities, 47(2), 178–195. https://doi.org/10.1177/0022219412451131
65.
*SwansonE.VaughnS.WanzekJ.PetscherY.HeckertJ.CavanaughC.KraftG.TackettK. (2011). A synthesis of read-aloud interventions on early reading outcomes among preschool through third graders at risk for reading difficulties. Journal of Learning Disabilities, 44(3), 258–275. https://doi.org/10.1177/0022219410378444
66.
SwansonH. L. (2001). Research on interventions for adolescents with learning disabilities: A meta-analysis of outcomes related to higher-order processing. The Elementary School Journal, 101(3), 331–348. https://www.jstor.org/stable/1002250
67.
SwansonH. L.Sachse-LeeC. (2000). A meta-analysis of single-subject-design intervention research for students with LD. Journal of Learning Disabilities, 33(2), 114–136. https://doi.org/10.1177/002221940003300201
68.
TalbottE.MagginD. M.Van AckerE. Y.KummS. (2018). Quality indicators for reviews of research in special education. Exceptionality, 26(4), 245–265. https://doi.org/10.1080/09362835.2017.1283625
69.
TherrienW. J.ZamanM.BandaD. R. (2011). How can meta-analyses guide practice? A review of the learning disability research base. Remedial and Special Education, 32(3), 206–218. https://doi.org/10.1177/0741932510361266
70.
TiptonE. (2015). Small sample adjustments for robust variance estimation with meta-regression. Psychological Methods, 20(3), 375–393. https://doi.org/10.1037/met0000011
71.
U.S. Department of Education, Office of Special Education and Rehabilitative Services, .Office of Special Education Programs (2021). 43rd annual report to Congress on the implementation of the Individuals with Disabilities Education Act. https://sites.ed.gov/idea/files/43rd-arc-for-idea.pdf
72.
*WanzekJ.StevensE. A.WilliamsK. J.ScammaccaN.VaughnS.SargentK. (2018). Current evidence on the effects of intensive early reading interventions. Journal of Learning Disabilities, 51(6), 612–624. https://doi.org/10.1177/0022219418775110
73.
*WanzekJ.VaughnS.ScammaccaN. K.MetzK.MurrayC. S.RobertsG.DanielsonL. (2013). Extensive reading interventions for students with reading difficulties after grade 3. Review of Educational Research, 83(2), 163–195. https://doi.org/10.3102/0034654313477212
74.
WebsterK. E.HewettT. E. (2018). Meta-analysis of meta-analyses of anterior cruciate ligament injury reduction training programs. Journal of Orthopaedic Research, 36(10), 2696–2708. https://doi.org/10.1002/jor.24043
75.
What Works Clearinghouse. (2020). What Works Clearinghouse standards handbook (Version 4.1). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. https://ies.ed.gov/ncee/wwc/handbooks
76.
WoleryM.BusickM.ReichowB.BartonE. E. (2010). Comparison of overlap methods for quantitatively synthesizing single-subject data. The Journal of Special Education, 44(1), 18–28. https://doi.org/10.1177/0022466908328009
77.
*WolfA. W. (2006). Using technology with learning-disabled readers: A meta-analysis [Doctoral dissertation, Nova Southeastern University]
78.
*WoodS. G.MoxleyJ. H.TigheE. L.WagnerR. K. (2018). Does use of text-to-speech and related read-aloud tools improve reading comprehension for students with reading disabilities? A meta-analysis. Journal of Learning Disabilities, 51(1), 73–84. https://doi.org/10.1177/0022219416688170
