In paired experiments, participants are grouped into pairs with similar characteristics, and one observation from each pair is randomly assigned to treatment. The resulting treatment and control groups should be well-balanced; however, there may still be small chance imbalances. Building on work for completely randomized experiments, we propose a design-based method to adjust for covariate imbalances in paired experiments. We leave out each pair and impute its potential outcomes using any prediction algorithm such as lasso or random forests. This method addresses a unique trade-off that exists for paired experiments. By addressing this trade-off, the method has the potential to improve precision over existing methods.
AronowP. M.MiddletonJ. A. (2013). A class of unbiased estimators of the average treatment effect in randomized experiments. Journal of Causal Inference, 1(1), 135–154.
2.
BalzerL. B.LaanM. J.PetersenM. L., & the SEARCH Collaboration. (2016). Adaptive pre-specification in randomized trials with and without pair-matching. Statistics in Medicine, 35(25), 4528–4545.
3.
BalzerL. B.PetersenM. L.LaanM. J., & the SEARCH Collaboration. (2016). Targeted estimation and inference for the sample average treatment effect in trials with and without pair-matching. Statistics in Medicine, 35(21), 3717–3732.
4.
ChernozhukovV.ChetverikovD.DemirerM.DufloE.HansenC.NeweyW.RobinsJ. (2018). Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21(1), C1–C68.
5.
DixonP. (2016). Should blocks be fixed or random?[Conference session].28th Annual Conference on Applied Statistics in Agriculture, Manhattan, KS.
6.
FogartyC. B. (2018). Regression-assisted inference for the average treatment effect in paired experiments. Biometrika, 105(4), 994–1000.
7.
FreedmanD. A. (2008). On regression adjustments to experimental data. Advances in Applied Mathematics, 40(2), 180–193.
8.
ImaiK. (2008). Variance identification and efficiency analysis in randomized experiments under the matched-pair design. Statistics in Medicine, 27(24), 4857–4873.
9.
ImaiK.KingG.NallC. (2009). Rejoinder: Matched pairs and the future of clusterrandomized experiments. Statistical Science, 24(1), 65–72.
10.
ImbensG. W. (2010). Better LATE than nothing: Some comments on Deaton (2009) and Heckman and Urzua (2009). Journal of Economic literature, 48(2), 399–423.
11.
ImbensG. W.RubinD. B. (2015). Causal inference in statistics, social, and biomedical sciences. Cambridge University Press.
12.
LiX.DingP. (2020). Rerandomization and regression adjustment. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 82(1), 241–268.
13.
LinW. (2013). Agnostic notes on regression adjustments to experimental data: Reexamining Freedman’s critique. The Annals of Applied Statistics, 7(1), 295–318.
14.
MooreK. L.van der LaanM. J. (2009). Covariate adjustment in randomized trials with binary outcomes: Targeted maximum likelihood estimation. Statistics in Medicine, 28(1), 39–64.
15.
PaneJ. F.GriffinB. A.McCaffreyD. F.KaramR. (2014). Effectiveness of Cognitive Tutor Algebra I at scale. Educational Evaluation and Policy Analysis, 36(2), 127–144.
16.
PashleyN. E.MiratrixL. W. (2017). Insights on variance estimation for blocked and matched pairs designs. ArXiv preprint ArXiv:1710.10342. https://arxiv.org/abs/1710.10342.
17.
PinheiroJ. C.BatesD. M. (2000). Linear mixed-effects models: Basic concepts and examples. In J. Chambers, W. Eddy, W. Haerdle, S. Sheather, & L. Tierney (Eds.), Mixed-effects models in S and S-Plus. Statistics and Computing (pp. 3–56). Springer.
18.
RaudenbushS. W.BrykA. S. (2002). Hierarchical linear models: Applications and data analysis methods (Vol. 1). Sage.
19.
RobinsJ. M. (2000). Robust estimation in sequentially ignorable missing data and causal inference models. Proceedings of the American Statistical Association Section on Bayesian Statistical Science, 1999, 6–10.
20.
RobinsJ. M.RotnitzkyA.ZhaoL. P. (1994). Estimation of regression coefficients when some regressors are not always observed. Journal of the American Statistical Association, 89(427), 846–866.
RubinD. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5), 688–701.
23.
ScharfsteinD. O.RotnitzkyA.RobinsJ. M. (1999). Rejoinder. Journal of the American Statistical Association, 94(448), 1135–1146.
24.
SmallD. S.Ten HaveT. R.RosenbaumP. R. (2008). Randomization inference in a group–randomized trial of treatments for depression: Covariate adjustment, noncompliance, and quantile effects. Journal of the American Statistical Association, 103(481), 271–279.
25.
SpiessJ. (2018). Optimal estimation when researcher and social preferences are misaligned. Technical Report Job Market Paper.
26.
Splawa-NeymanJ.DabrowskaD. M.SpeedT. P. (1990). On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Statistical Science, 5(4), 465–472.
27.
TsiatisA. A.DavidianM.ZhangM.LuX. (2008). Covariate adjustment for two-sample treatment comparisons in randomized clinical trials: A principled yet flexible approach. Statistics in Medicine, 27(23), 4658–4677.
28.
WagerS.DuW.TaylorJ.TibshiraniR. J. (2016). High-dimensional regression adjustments in randomized experiments. Proceedings of the National Academy of Sciences, 113(45), 12673–12678.
29.
WoltmanH.FeldstainA.MacKayJ. C.RocchiM. (2012). An introduction to hierarchical linear modeling. Tutorials in Quantitative Methods for Psychology, 8(1), 52–69.
30.
WuE.Gagnon-BartschJ. A. (2018). The LOOP estimator: Adjusting for covariates in randomized experiments. Evaluation Review, 42(4), 458–488.
31.
WuZ.FrangakisC. E.LouisT. A.ScharfsteinD. O. (2014). Estimation of treatment effects in matched-pair cluster randomized trials by calibrating covariate imbalance between clusters. Biometrics, 70(4), 1014–1022.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
