This paper demonstrates the application, analysis, and interpretation of a Hybrid Regression Discontinuity (HRD) design. HRD is a combination of a Regression Discontinuity (RD) design and a Randomized Control Trial (RCT). The HRD was implemented in conjunction with a summer literacy program administered by a school district in the U.S. Pacific Northwest. Students scoring below a lower cutoff of 30 words correctly read per minute (WCPM) in the spring of first grade comprised the RD treatment group, students scoring above an upper cutoff of 50 WCPM comprised the RD control group, and those within the interval between cutoffs were randomly assigned to the RCT treatment or control condition. Across multiple parametric and nonparametric analyses, treatment effects were observed at the upper cutscore and within the randomization interval. Implications for evaluating functional form, statistical power, and generalizing the RD treatment effect beyond the immediate region surrounding the assignment cutscore are discussed.
AngristJ. D.RokkanenM. (2015). Wanna get away? Regression discontinuity estimation of exam school effects away from the cutoff. Journal of the American Statistical Association, 110(512), 1331–1344.https://doi.org/10.1080/01621459.2015.1012259
2.
AngristJ. D.AutorD.HudsonS.PallaisA. (2017). Leveling up: Early results from a randomized evaluation of post-secondary aid. Cambridge, MA: National Bureau of Economic Research. Working Paper No. 20800.
3.
AngristJ. D.ImbensG. W.RubinD. B. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91(434), 444–455.https://doi.org/10.1080/01621459.1996.10476902
4.
Baker, D. L., Biancarosa, G., Park, B. J., Bousselot, T., Smith, J. L., Baker, S. K., …Tindal, G. (2015). Validity of CBM measures of oral reading fluency and Reading comprehension on high-stakes reading assessments in grades 7 and 8. Reading and Writing, 28(1), 57–104. https://doi.org/10.1007/s11145-014-9505-4
BettsJ.PickartM.HeistadD. (2009). An investigation of the psychometric evidence of CBM-R passage equivalence: Utility of reliability statistics and equating alternate forms. Journal of School Psychology, 47(1), 1–18.https://doi.org/10.1016/j.jsp.2008.09.001
7.
BlackD.GaldoJ.SmithJ. A. (2005). Evaluating the regression discontinuity design using experimental data. Unpublished manuscript.
BoruchR. F. (1975). Coupling randomized experiments and approximations to experiments in social program evaluation. Sociological Methods & Research, 4(1), 31–53. https://doi.org/10.1177/004912417500400103
10.
CalonicoS.CattaneoM. D.TitiunikR. (2014). Robust data-driven inference in the regression-discontinuity design. Stata Journal, 14(4), 909–946. https://doi.org/10.1177/1536867X1401400413
11.
CalonicoS.CattaneoM. D.TitiunikR. (2015). Rdrobust: An R package for robust nonparametric inference in regression-discontinuity designs. R Journal, 7(1), 38–51.
12.
CappelleriJ. C.TrochimW. M. K. (1994). An illustrative statistical analysis of cut-off based randomized clinical trials. Journal of Clinical Epidemiology, 47(3), 261–270.https://doi.org/10.1016/0895-4356(94)90007-8
13.
CappelleriJ. C.TrochimW. M. K. (1995). Ethical and scientific features of cutoff-based designs. Medical Decision Making, 15(4), 387–394.https://doi.org/10.1177/0272989X9501500409
14.
CattaneoM. D.FrandsenB. R.TitiunikR. (2015). Randomization inference in the regression discontinuity design: An application to party advantages in the U.S. Senate. Journal of Causal Inference, 3(1), 1–24. https://doi.org/10.1515/jci-2013-0010
15.
CattaneoM. D.IdroboN.TitiunikR. (2020).A practical introduction to regression discontinuity designs: Foundations. Cambridge, UK: Cambridge University Press. https://doi.org/10.1017/9781108684606
16.
CattaneoM. D.TitiunikR.Vazquez-BareG.KeeleL. (2016). Interpreting regression discontinuity designs with multiple cutoffs. The Journal of Politics, 78(4), 1229–1248. https://doi.org/10.1086/686802
17.
Children’s Educational Services. (1987). Test of oral reading fluency (TORF). Eden Prairie: MN: Author.
18.
CookT. D. (2008). Waiting for life to arrive: A history of the regression-discontinuity design in psychology, statistics and economics. Journal of Econometrics, 142(2), 636–654. https://doi.org/10.1016/j.jeconom.2007.05.002
19.
CookT. D.CampbellD. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston, MA: Houghton Mifflin.
20.
CummingsK. D.ParkY.Bauer SchaperH. A. (2013). Form effects on DIBELS next oral reading fluency progress-monitoring passages. Assessment for Effective Intervention, 38(2), 91–104. https://doi.org/10.1177/1534508412447010
21.
DekeJ.DragosetL. (2012). Statistical power for regression discontinuity designs in education: Empirical estimates of design effects relative to randomized controlled trials. Working Paper. Mathematica Policy Research, Inc.
22.
FrancisD. J.SantiK. L.BarrC.FletcherJ. M.VariscoA.FoormanB. R. (2008). Form effects on the estimation of students’ oral reading fluency using DIBELS. Journal of School Psychology, 46(3), 315–342. https://doi.org/10.1016/j.jsp.2007.06.003
23.
FuchsL. S.FuchsD.HospM. K.JenkinsJ. R. (2001). Oral reading fluency as an indicator of reading competence: A theoretical, empirical, and historical analysis. Scientific Studies of Reading, 5(3), 239–256. https://doi.org/10.1207/S1532799XSSR0503_3
24.
GelmanA.ImbensG. (2019). Why high-order polynomials should not be used in regression discontinuity designs. Journal of Business & Economic Statistics, 37(3), 447–456. https://doi.org/10.1080/07350015.2017.1366909
25.
GoldbergerA. S. (1972). Selection bias in evaluating treatment effects: Some formal illustrations. University of Wisconsin--Madison. Unpublished manuscript.
26.
GoodR. H.SimmonsD. C.Kame’enuiE. J. (2001). The importance and decision-making utility of a continuum of fluency-based indicators of foundational reading skills for third-grade high-stakes outcomes. Scientific Studies of Reading, 5(3), 257–288. https://doi.org/10.1207/S1532799XSSR0503_4
27.
HasbrouckJ.TindalG. (2006). Oral reading fluency norms: A valuable assessment tool for reading teachers. The Reading Teacher, 59(7), 636–644.https://doi.org/10.1598/RT.59.7.3
28.
HongM. R. (2019). Embedding a randomized experiment within a regression discontinuity design: A meta-analytic approach. Doctoral dissertation, Northwestern University.
ImbensG. W.RubinD. B. (1997). Bayesian Inference for causal effects in randomized experiments with noncompliance. The Annals of Statistics, 25(1), 305–327. http://www.jstor.org/stable/2242722https://doi.org/10.1214/aos/1034276631
31.
ImbensG. W.RubinD. B. (2015). Causal inference in statistics, social, and biomedical sciences. Cambridge, UK: Cambridge University Press.
32.
JohnsonP.NeymanJ. (1936). Tests of certain linear hypotheses and their applications to some educational problems. Statistical Research Memoirs, 1, 57–93.
KlugerD. M.OwenA. B. (2023). Kernel regression analysis of tie-breaker designs. Electronic Journal of Statistics, 17(1), 243–290. https://doi.org/10.1214/23-EJS2102
35.
KrantzC. (2022). Modeling the outcomes of a longitudinal tie-breaker regression discontinuity design to assess an in-home training program for families at risk of child abuse and neglect. Doctoral dissertation, University of Oklahoma.
36.
LeeD.LemieuxT. (2010). Regression discontinuity designs in economics. Journal of Economic Literature, 48(2), 281–355.https://doi.org/10.1257/jel.48.2.281
37.
LinW. (2013). Agnostic notes on regression adjustments to experimental data: Reexamining freedman's critique. The Annals of Applied Statistics, 7(1), 295–318. https://www.jstor.org/stable/23566512
38.
LouieJ.RhoadsC.MarkJ. (2016). Challenges to using the regression discontinuity design in educational evaluations: Lessons from the transition to algebra study. American Journal of Evaluation, 37(3), 381–407. https://doi.org/10.1177/1098214015621787
39.
MandellM. B. (2008). Having one’s cake and eating it, too: Combining true experiments with regression discontinuity designs. Evaluation Review, 32(5), 415–434. https://doi.org/10.1177/0193841X08316672
40.
MatteiA.MealliF. (2017). Regression discontinuity designs as local randomized experiments. Observational Studies, 3(2), 156–173. https://doi.org/10.1353/obs.2017.0004
41.
McCraryJ. (2008). Manipulation of the running variable in the regression discontinuity design: A density test. Journal of Econometrics, 142(2), 698–714.https://doi.org/10.1016/j.jeconom.2007.05.005
42.
MossB. G.YeatonW. H.LloydJ. E. (2014). Evaluating the effectiveness of developmental mathematics by embedding a randomized experiment within a regression discontinuity design. Educational Evaluation and Policy Analysis, 36(2), 170–185. https://doi.org/10.3102/0162373713504988
43.
National Early Literacy Panel. (2008). Developing early literacy. Washington, DC: National Institute for Literacy.
44.
National Reading Panel. (2000). Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction. Rockville, MD: National Institutes of Health. NIH Pub. No. 00-4754.
45.
OwenA. B.VarianH. (2020). Optimizing the tie-breaker regression discontinuity design. Electronic Journal of Statistics, 14(2), 4004–4027.https://doi.org/10.1214/20-EJS1765
46.
PetscherY.KimY. S. (2011). The utility and accuracy of oral reading fluency score types in predicting Reading comprehension. Journal of School Psychology, 49(1), 107–129. https://doi.org/10.1016/j.jsp.2010.09.004
47.
PreacherK. J.CurranP. J.BauerD. J. (2006). Computational tools for probing interaction effects in multiple linear regression, multilevel modeling, and latent curve analysis. Journal of Educational and Behavioral Statistics, 31(4), 437–448. https://doi.org/10.3102/10769986031004437
48.
R Core Team. (2016). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org
49.
ReschlyA. L.BuschT. W.BettsJ.DenoS. L.LongJ. D. (2009). Curriculum-based measurement oral reading as an indicator of reading achievement: A meta-analysis of the correlational evidence. Journal of School Psychology, 47(6), 427–469.https://doi.org/10.1016/j.jsp.2009.07.001
50.
SagarinB. J.WestS. G.RatnikovA.HomanW. K.RitchieT. D.HansenE. J. (2014). Treatment noncompliance in randomized experiments: Statistical approaches and design issues. Psychological Methods, 19(3), 317–333. https://doi.org/10.1037/met0000013
51.
SchochetP. (2009). Statistical power for regression discontinuity design in educational evaluations. Journal of Educational and Behavioral Statistics, 34(2), 238–266.https://doi.org/10.3102/1076998609332748
52.
ShadishW. R.CookT. D.CampbellD. T. (2002). Experimental and non-experimental designs for generalized causal inference. Belmont, CA: Wadsworth.
53.
SlocumT. A.StreetE. M.GilbertsG. (1995). A review of research and theory on the relation between oral reading rate and reading comprehension. Journal of Behavioral Education, 5, 377–398. https://doi.org/10.1007/BF02114539
54.
StanovichK. E. (1986). Matthew effects in reading: Some consequences of individual differences in the acquisition of literacy. Reading Research Quarterly, 21(4), 360–406. https://www.jstor.org/stable/747612https://doi.org/10.1598/RRQ.21.4.1
55.
TangY.CookT. D. (2018). Statistical power for the comparative regression discontinuity design with a pretest no-treatment control function: Theory and evidence from the national head start impact study. Evaluation Review, 42(1), 71–110. https://doi.org/10.1177/0193841X18776117
56.
TangY.CookT. D.Kisbu-SakaryaY. (2018). Statistical power for the comparative regression discontinuity design with a nonequivalent comparison group. Psychological Methods, 23(1), 150. https://doi.org/10.1037/met0000118
57.
TangY.CookT. D.Kisbu-SakaryaY.HockH.ChiangH. (2017). The comparative regression discontinuity (CRD) design: An overview and demonstration of its performance relative to basic RD and the randomized experiment. Regression Discontinuity Designs, 38, 237–279. https://doi.org/10.1108/S0731-905320170000038011
58.
ThistlethwaiteD.CampbellD. (1960). Regression-discontinuity analysis: An alternative to the ex post facto experiment. Journal of Educational Psychology, 51(6), 309–317. https://psycnet.apa.org/doi/10.1037/h0044319
59.
TrochimW. M. K. (1984). Research design for program evaluation. Beverly Hills, CA: Sage.
60.
TrochimW. M. K.CappelleriJ. C. (1992). Cutoff assignment strategies for enhancing randomized clinical trials. Controlled Clinical Trials, 13(3), 190–212. https://doi.org/10.1016/0197-2456(92)90003-I
61.
WingC.Bello-GomezR. A. (2018). Regression discontinuity and beyond: Options for studying external validity in an internally valid design. American Journal of Evaluation, 39(1), 91–108. https://doi.org/10.1177/1098214017736155
62.
WingC.CookT. D. (2013). Strengthening the regression discontinuity design using additional design elements: A within study comparison. Journal of Policy Analysis and Management, 32(4), 853–877. https://doi.org/10.1002/pam.21721
63.
YoonH.ZvochK.SmolkowskiK.ClarkeB. (2025). Multiple-cutoff regression discontinuity designs in educational evaluation. International Journal of Evaluation and Research in Education, 14(3), 1597–1607. https://doi.org/10.11591/ijere.v14i3.31081
64.
ZvochK.StevensJ. J. (2015). Identification of summer school effects by comparing the in- and out-of-school growth rates of struggling early readers. Elementary School Journal, 115(3), 433–456. https://doi.org/10.1086/680229
