The Relationship Between Study Quality and the Effects of Supplemental Reading Interventions: A Meta-Analysis

Abstract

Empirical studies investigating supplemental reading interventions for students with or at risk for reading disabilities in the early elementary grades have demonstrated a range of effect sizes. Identifying the findings from high-quality research can provide greater certainty of findings related to the effectiveness of supplemental reading interventions. This meta-analysis investigated how four variables of study quality (study design, statistical treatment, Type I error, and fidelity of implementation) were related to effect sizes from standardized measures of foundational reading skills and language and comprehension. The results from 88 studies indicated that year of publication was a significant predictor of effect sizes for both standardized measures of foundational reading skills and language and comprehension, with more recent studies demonstrating smaller effect sizes. Results also demonstrated that with the exception of research design predicting effect sizes on foundational reading skills measures, study quality was not related to the effects of supplemental reading interventions. Implications for research and practice are discussed.

Keywords

study quality reading intervention meta-analysis

Get full access to this article

View all access options for this article.

References

Borenstein

Hedges

L. V.

Higgins

J. P. T.

Rothstein

H. R.

(2013). Comprehensive meta analysis (Version 3.3070). [Computer software]. Englewood, NJ: Biostat.

Carney

K. J.

Stiefel

G. S.

(2008). Long-term results of a problem-solving approach to response to intervention: Discussion and implications. Learning Disabilities: A Contemporary Journal, 6(2), 61–75.

Council for Exceptional Children. (2015). What every special educator must know: Professional ethics and standards. Arlington, VA: Author.

Duval

Tweedie

(2000). Trim and fill: A simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56, 455–463.

Every Student Succeeds Act of 2015, Pub. L. No. 114-95 § 114 Stat. 1177 (2015–2016).

Fisher

Tipton

(2015). robumeta: An R-package for robust variance estimation in meta-analysis. Retrieved from arXiv:1503.02220

Fuchs

L. S.

Fuchs

Compton

D. L.

Wehby

Schumacher

R. F.

Gersten

Jordan

N. C.

(2015). Inclusion versus specialized intervention for very-low-performing students: What does access mean in an era of academic challenge? Exceptional Children, 81, 134–157. doi:10.1177/0014402914551743

Goldstein

Lackey

K. C.

Schneider

N. J. B.

(2014). A new framework for systematic reviews: Application to social skills interventions for preschoolers with autism. Exceptional Children, 80, 262–286. doi:10.1177/0014402914522423

Gwet

(2001). Handbook of inter-rater reliability. Gaithersburg, MD: STATAXIS.

10.

Harn

Parisi

Stoolmiller

(2013). Balancing fidelity with flexibility and fit: What do we really know about fidelity of implementation in schools? Exceptional Children, 79, 181–193. doi:10.1177/001440291307900204

11.

Hedges

L. V.

Tipton

Johnson

M. C.

(2010). Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods, 1, 39–65.

12.

Lemons

C. J.

Fuchs

Gilbert

J. K.

Fuchs

L. S.

(2014). Evidence-based practices in a changing world: Reconsidering the counterfactual in education research. Educational Researcher, 43, 242–252. doi:10.3102/0013189X14539189

13.

Mathes

P. G.

Denton

C. A.

Fletcher

J. M.

Anthony

J. L.

Francis

D. J.

Schatschneider

(2005). The effects of theoretically different instruction and student characteristics on the skills of struggling readers. Reading Research Quarterly, 40, 148–182. doi:10.1598/RRQ.40.2.2

14.

National Center for Education Statistics. (2011). NAEP: The nation’s report card: An overview of NAEP. Washington, DC: Author.

15.

National Center for Education Statistics. (2013). NAEP: The nation’s report card: An overview of NAEP. Washington, DC: Author.

16.

National Center for Education Statistics. (2015). NAEP: The nation’s report card: An overview of NAEP. Washington, DC: Author.

17.

National Early Literacy Panel. (2008). Executive summary: Developing early literacy. Report of the National Early Literacy Panel. Washington, DC: National Institute for Literacy. Retrieved from http://lincs.ed.gov/publications/pdf/NELPSummary.pdf.

18.

National Reading Panel. (2000). Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction. Reports of the subgroups. Bethesda, MD: National Institute of Child Health and Human Development.

19.

O’Connor

R. E.

Fulmer

Harty

K. R.

Bell

K. M.

(2005). Layers of reading intervention in kindergarten through third grade: Changes in teaching and student outcomes. Journal of Learning Disabilities, 38, 440–455. doi:10.1177/00222194050380050701

20.

Pustejovsky

J. E.

(2015). clubSandwich: Cluster-robust (sandwich) variance estimators with small sample corrections. R package Version 0.0.0.9000. Retrieved from https://github.com/jepusto/clubSandwich

21.

Scammacca

N. K.

Roberts

G. J.

Cho

Williams

K. J.

Roberts

Vaughn

S. R.

Carroll

(2016). A century of progress: Reading interventions for students in Grades 4–12, 1914–2014. Review of Educational Research, 86, 756–800. doi:10.3102/0034654316652942

22.

Scammacca

Roberts

Vaughn

Stuebing

(2015). A meta-analysis of interventions for struggling readers in grades 4–12: 1980–2011. Journal of Learning Disabilities, 48, 369–390. doi:10.1177/0022219413504995

23.

Swanson

H. L.

Hoskyn

(2000). Experimental intervention research for students with learning disabilities: A comprehensive meta-analysis of group design studies. Advances in Learning and Behavioral Disabilities, 14, 1–154. doi:10.2307/1170599

24.

Swanson

H. L.

Hoskyn

Lee

(1999). Interventions for students with learning disabilities: A meta-analysis of treatment outcomes. New York, NY: Guilford.

25.

Tipton

(2015). Small sample adjustments for robust variance estimation with meta-regression. Psychological Methods, 20, 375–393. doi:10.1037/met000001

26.

Tipton

Pustejovsky

J. E.

(2015). Small-sample adjustments for tests of moderators and model fit using robust variance estimation in meta-regression. Journal of Educational and Behavioral Statistics, 40, 604–634. doi:10.3102/1076998615606099

27.

Torgesen

J. K.

Houston

D. D.

Rissman

L. M.

Decker

S. M.

Roberts

Vaughn

. . . Lesaux

(2017). Academic literacy instruction for adolescents: A guidance document from the Center on Instruction. Portsmouth, NH: Center on Instruction.

28.

Tran

Sanchez

Arellano

Swanson

H. L.

(2011). A meta-analysis of the RTI literature for children at risk for reading disabilities. Journal of Learning Disabilities, 44, 283–295. doi:10.1177/0022219410378447

29.

Vaughn

Wanzek

Murray

C. S.

Scammacca

Linan-Thompson

Woodruff

A. L.

(2009). Response to early reading intervention examining higher and lower responders. Exceptional Children, 75, 165–183. doi:10.1177/001440290907500203

30.

Wanzek

Stevens

E. A.

Williams

K. J.

Scammacca

Vaughn

Sargent

(2018). Current evidence on the effects of intensive early reading interventions. Journal of Learning Disabilities. doi:10.1177/0022219418775110

31.

Wanzek

Vaughn

(2007). Research-based implications from extensive early reading interventions. School Psychology Review, 36, 541–561.

32.

Wanzek

Vaughn

Scammacca

Gatlin

Walker

M. A.

Capin

(2016). Meta-analyses of the effects of Tier 2 type reading interventions in Grades K–3. Educational Psychology Review, 28, 551–576.

33.

Willingham

D. T.

(2007). Ask the cognitive scientist: The usefulness of brief instruction in reading comprehension strategies. American Educator, 30, 39–45.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.14 MB

0.03 MB