Fuzzy Regression Discontinuity Designs With Multiple Control Groups Under One-Sided Noncompliance: Evaluating Extended Time Accommodations

Abstract

Since its conception by Thistlethwaite and Campbell, a regression discontinuity (RD) design has continuously evolved and progressed in various aspects. This paper explores a novel theoretical advancement in fuzzy RD designs by combining them with multiple control groups under one-sided noncompliance. Within this approach, we estimate the average treatment effect on the treated at the cutoff from the fuzzy RD design and construct its bounds derived from multiple control groups. Using the bounds as a sensitivity check, we examine whether the underlying causal or statistical assumptions for the fuzzy RD design are warranted. We verify the effectiveness of our approach via a simulation study and demonstrate its application by studying the effect of extended time accommodations using data from the National Assessment of Educational Progress.

Keywords

regression discontinuity quasi-experimental designs causal inference sensitivity analysis bounds multiple control groups testing accommodations NAEP process data

Get full access to this article

View all access options for this article.

References

Angrist

J. D.

Pischke

J.-S.

(2009). Mostly harmless econometrics: An empiricist’s companion. Princeton University Press.

Bloom

H. S.

(1984). Accounting for no-shows in experimental evaluation designs. Evaluation Review, 8(2), 225–246. https://doi.org/10.1177/0193841x8400800205

Cattaneo

M. D.

Idrobo

Titiunik

(2024). A practical introduction to regression discontinuity designs: Extensions. Cambridge University Press.

Cattaneo

M. D.

Keele

Titiunik

(2023). Covariate adjustment in regression discontinuity designs. In Zubizarreta

J. R.

Stuart

E. A.

Small

D. S.

Rosenbaum

P. R.

(Eds.), Handbook of matching and weighting adjustments for causal inference (pp. 153–168). Chapman; Hall/CRC.

Cattaneo

M. D.

Titiunik

(2022). Regression discontinuity designs. Annual Review of Economics, 14(1), 821–851. https://doi.org/10.1146/annurev-economics-051520-021409

Grembi

Nannicini

Troiano

(2016). Do fiscal rules matter? American Economic Journal: Applied Economics, 8(3), 1–30. https://doi.org/10.1257/app.20150076

Hahn

Todd

van der Klaauw

(2001). Identification and estimation of treatment effects with a regression-discontinuity design. Econometrica, 69(1), 201–209. https://doi.org/10.1111/1468-0262.00183

Hernan

M. A.

Robins

J. M.

(2020). Causal inference: What if. Chapman & Hall/CRC. https://cdn1.sph.harvard.edu/wp-content/uploads/sites/1268/2021/03/ciwhatif_hernanrobins_30mar21.pdf

D. E.

Imai

King

Stuart

E. A.

(2011). MatchIt: Nonparametric preprocessing for parametric causal inference. Journal of Statistical Software, 42(8), 1–28. https://doi.org/10.18637/jss.v042.i08

10.

Imbens

G. W.

Lemieux

(2008). Regression discontinuity designs: A guide to practice. Journal of Econometrics, 142(2), 615–635. https://doi.org/10.1016/j.jeconom.2007.05.001

11.

Imbens

G. W.

Rubin

D. B.

(2015). Causal inference in statistics, social, and biomedical sciences. Cambridge University Press. https://doi.org/10.1017/cbo9781139025751

12.

Kang

Jiang

Zhao

Small

(2023). Ivmodel: Statistical inference and sensitivity analysis for instrumental variables model [R package version 1.9.1]. https://CRAN.R-project.org/package=ivmodel

13.

Lee

D. S.

Card

(2008). Regression discontinuity inference with specification error. Journal of Econometrics, 142(2), 655–674. https://doi.org/10.1016/j.jeconom.2007.05.003

14.

Lee

D. S.

Lemieux

(2010). Regression discontinuity designs in economics. Journal of Economic Literature, 48(2), 281–355. https://doi.org/10.1257/jel.48.2.281

15.

Neyman

J. S.

(1923). On the application of probability theory to agricultural experiments: Essay on principles. Section 9 (with discussion). Statistical Science, 4, 465–480.

16.

Rosenbaum

P. R.

(1987). The role of a second control group in an observational study. Statistical Science, 2(3), 292–306.

17.

Rosenbaum

P. R.

(2002). Observational studies. Springer.

18.

Rosenbaum

P. R.

(2020). Design of observational studies. Springer International Publishing.

19.

Rosenbaum

P. R.

Rubin

D. B.

(1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41–55. https://doi.org/10.1093/biomet/70.1.41

20.

Rubin

D. B.

(1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5), 688–701. https://doi.org/10.1037/h0037350

21.

Rubin

D. B.

(1986). Comment: Which ifs have causal answers. Journal of the American Statistical Association, 81(396), 961–962. https://doi.org/10.2307/2289065

22.

Steiner

P. M.

Cook

(2013). Matching and propensity scores. In Little

(Ed.), The oxford handbook of quantitative methods (pp. 236–258). Oxford University Press.

23.

Stuart

E. A.

(2010). Matching methods for causal inference: A review and a look forward. Statistical Science, 25(1), 1. https://doi.org/https://doi.org/10.1214/09-STS313

24.

Suk

(2024). Regression discontinuity designs in education: A practitioner’s guide. Asia Pacific Education Review, 25, 629–645. https://doi.org/10.1007/s12564-024-09956-3

25.

Suk

Steiner

P. M.

Kim

J.-S.

Kang

(2022). Regression discontinuity designs with an ordinal running variable: Evaluating the effects of extended time accommodations for English-language learners. Journal of Educational and Behavioral Statistics, 47(4), 459–484. https://doi.org/10.3102/10769986221090275

26.

Thistlethwaite

D. L.

Campbell

D. T.

(1960). Regression-discontinuity analysis: An alternative to the ex post facto experiment. Journal of Educational Psychology, 51(6), 309–317. https://doi.org/10.1037/h0044319

27.

Wing

Cook

T. D.

(2013). Strengthening the regression discontinuity design using additional design elements: A within-study comparison. Journal of Policy Analysis and Management, 32(4), 853–877. https://doi.org/10.1002/pam.21721

28.

Witmer

S. E.

Lovett

B. J.

Buzick

H. M.

(2022). Extended time accommodations on the 2017 NAEP grade 8 mathematics test: Eligibility, use, and benefit. Journal of Psychoeducational Assessment, 41(1), 073428292211304. https://doi.org/10.1177/07342829221130457

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.03 MB