The purpose of this study was to compare the results of a regression discontinuity design (RDD) with those of an experimental design of a tiered vocabulary intervention for children at risk for reading disability to determine RDD’s feasibility as a research methodology for this type of study. Researchers reanalyzed an archival dataset of a previous study that determined the effectiveness of the intervention through a randomized experiment. Results were comparable for the receptive outcome scores in direction, magnitude, and significance, although the magnitude of the treatment effect for the RDD was slightly higher. Results were also comparable for contextual outcome scores with respect to direction and magnitude. We present possible reasons for the different results between studies, limitations, and future research.
AikenL. S.WestS. G.SchwalmD. E.CarrollJ.HsuingS. (1998). Comparison of a randomized and two quasi-experimental designs in a single outcome evaluation: Efficacy of a university-level remedial writing program. Evaluation Review, 22, 207–244. doi:10.1177/0193841X9802200203
2.
BerkR.BarnesG.AhlmanL.KurtzE. (2010). When second best is good enough: A comparison between a true experiment and a regression discontinuity quasi-experiment. Journal of Experimental Criminology, 6, 191–208. doi:10.1007/s11292-010-9095-3
BuddelmeyerH.SkoufiasE. (2004). An evaluation of the performance of regression discontinuity design on PROGRESA (World Bank Policy Research Working Paper No. 3386, IZA Discussion Paper No. 827). Retrieved from http://ssrn.com/abstract=434600
5.
CainG. C. (1975). Regression and selection models to improve nonexperimental comparisons. In BennetC. A.LumsdaineC. A. (Eds.), Evaluation and experiment (pp. 297–317). New York, NY: Academic Press.
6.
CappelleriJ. C.TrochimW. M. K. (2003). Cutoff designs. In ChowS. (Ed.), Encyclopedia of biopharmaceutical statistics (Vol. 2, pp. 263–269). New York, NY: Informa HealthCare.
7.
CookT. D. (2008). Waiting for life to arrive: A history of the regression-discontinuity design in psychology, statistics and economics. Journal of Econometrics, 142, 636–654.
8.
CoyneM. D.McCoachB.KappS. (2007). Vocabulary intervention for kindergarten students: Comparing extended instruction to embedded instruction and incidental exposure. Learning Disabilities Quarterly, 30, 74–88.
FuchsD.FuchsL. S. (2006). Introduction to response to intervention: What, why, and how valid is it?Reading Research Quarterly, 41, 93–99. doi:10.1598/RRQ.41.1.4
11.
ImbensG. W.LemieuxT. (2008). Regression discontinuity designs: A guide to practice. Journal of Econometrics, 142, 615–635.
12.
JacksonR.McCoyA.PistorinoC.WilkinsonA.BurghardtJ.ClarkM.. . . SchmidtS. R. (2007). National evaluation of early reading first: Final report to congress (Publication No. NCEE 2007-4007). Washington, DC: U.S. Department of Education, National Center for Education Evaluation and Regional Assistance.
13.
JacobB. A.LefgrenL. (2004). Remedial education and student achievement: A regression-discontinuity analysis. Review of Economics and Statistics, 86, 226–244.
14.
LesikS. A. (2007). Do developmental mathematics programs have a causal impact on student retention? An application of discrete-time survival and regression-discontinuity analysis. Research in Higher Education, 48, 583–608.
15.
LudwigJ.MillerD. L. (2007). Does Head Start improve children’s life chances? Evidence from a regression discontinuity design. Quarterly Journal of Economics, 122, 159–208. doi:10.1162/qjec.122.1.159
16.
PullenP. C.TuckwillerE. D.KonoldT. R.MaynardK. L.CoyneM. D. (2010). A tiered intervention model for early vocabulary instruction: The effects of tiered instruction for young students at risk for reading disability. Learning Disabilities Research & Practice, 25, 110–123.
17.
RumrillP. D.CookB. G.WileyA. L. (2011). Research in special education: Designs, methods, and applications (2nd ed.). Springfield, IL: Charles C. Thomas.
18.
SchochetP.CookT.DekeJ.ImbensG.LockwoodJ. R.PorterJ.SmithJ. (2010). Standards for regression discontinuity designs, Version 1.0. Retrieved from http://ies.ed.gov/ncee/wwc/pdf/wwc_rd.pdf
19.
ShadishW. R.GalindoR.WongV. C.SteinerP. M.CookT. D. (2011). A randomized experiment comparing random and cutoff-based assignment. Psychological Methods, 16, 179–191. doi:10.1037/a0023345
20.
ThistlethwaiteD. L.CampbellD. T. (1960). Regression-discontinuity analysis: An alternative to the ex-post facto experiment. Journal of Educational Psychology, 51, 309–317.
21.
TrochimW. M. K. (1984). Research design for program evaluation. Beverly Hills, CA: SAGE.
22.
TrochimW. M. K. (2001). Research methods knowledge base (2nd ed.). Cincinnati, OH: Atomic Dog.
TuckwillerE. D.PullenP. C.CoyneM. D. (2010). The use of the regression discontinuity design in tiered intervention research: A pilot study exploring vocabulary instruction for at-risk kindergarteners. Learning Disabilities Research & Practice, 25, 137–150.
25.
VaughnS.WanzekJ.MurrayC. S.ScammaccaN.Linan-ThompsonS.WoodruffA. L. (2009). Response to early reading intervention: Examining higher and lower responders. Exceptional Children, 75, 165–183.
26.
WaltersP. B.LareauA.RanisS. H. (2009). Education research on trial: Policy reform and the call for scientific rigor. New York, NY: Routledge.
27.
WongV. C.CookT. D.BarnettW. S.JungK. (2008). An effectiveness-based evaluation of five state prekindergarten programs. Journal of Policy Analysis and Management, 27, 122–154.
