Model assessment is a standard component of statistical analysis, but it has received relatively little attention within the dynamic treatment regime literature. In this paper, we focus on the dynamic-weighted ordinary least squares approach to optimal dynamic treatment regime estimation, introducing how its double-robustness property may be leveraged for model assessment, and how quasilikelihood may be used for model selection. These ideas are demonstrated through simulation studies, as well as through application to data from the sequenced treatment alternatives to relieve depression study.
WallaceMPMoodieEEM. Doubly-robust dynamic treatment regimen estimation via weighted least squares. Biometrics2015; 71: 636–644.
2.
RobinsJMOptimal structural nested models for optimal sequential decisions. In: LinDHeagertyP (eds). Proceedings of the second Seattle symposium on biostatistics, New York: Springer-Verlag, 2004, pp. 189–326.
3.
KangJDYSchaferJL. Demystifying double robustness: a comparison of alternative strategies for estimating a population mean from incomplete data. Stat Sci2007; 22: 523–539.
4.
RichBMoodieEEMStephensDA, et al.Model checking with residuals for G-estimation of optimal dynamic treatment regimes. Int J Biostat2010; 6: Article 12.
5.
LaberEBLinnKAStefanskiLA. Interactive model building for Q-learning. Biometrika2014; 101: 831–847.
6.
ErtefaieAShortreedSChakrabortyB. Q-learning residual analysis: application to the effectiveness of sequences of antipsychotic medications for patients with schizophrenia. Stat Med2016; 35: 2221–2234.
7.
WallaceMPMoodieEEMStephensDA. Model assessment in dynamic treatment regimen estimation via double robustness. Biometrics2016; 72: 855–864.
8.
Watkins CJCH. Learning from delayed rewards. PhD Dissertation, Cambridge University Press, Cambridge, England, 1989.
9.
SuttonRSAndrewGB. Reinforcement learning: an introduction, MIT press: Cambridge, 1998.
10.
MoodieEDeanNSunY. Q-learning: flexible learning about useful utilities. Stat Biosci2013; 6: 223–243.
11.
RubinD. Discussion of randomized analysis of experimental data: the Fisher randomization test by D. Basu. Journal of the American Statistical Association1980; 75: 591–593.
12.
RobinsJCausal inference from complex longitudinal data. In: BerkaneM (ed). Latent variable modeling and applications to causality: lecture notes in statistics, New York: Springer-Verlag, 1997, pp. 69–117.
13.
WedderburnRWM. Quasi-likelihood Functions, generalized linear models, and the Gauss-Newton method. Biometrika1974; 61: 439–447.
14.
PanW. Akaike’s information criterion in generalized estimating equations. Biometrics2001; 57: 120–125.
15.
TaguriMMatsuyamaYOhashiY. Model selection criterion for causal parameters in structural mean models based on a quasi-likelihood. Biometrics2014; 70: 724–733.
16.
Wallace MP, Moodie EEM and Stephens DA. Generalized G-estimation and Model Selection (submitted).
17.
MoodieEEM. A note on the variance of doubly-robust G-estimates. Biometrika2009; 96: 998–1004.
18.
FavaMRushATrivediM, et al.Background and rationale for the sequenced treatment alternatives to relieve depression (STAR*D) study. Psychiatric Clinics of North America2003; 26: 457–494.
19.
RushAFavaMWisniewskiS, et al.Sequenced treatment alternatives to relieve depression (STAR*D): rationale design. Control Clin Trials2004; 25: 119–142.
20.
ChakrabortyBLaberEBZhaoY. Inference for optimal dynamic treatment rgimes using an adaptive m-out-of-n bootstrap scheme. Biometrics2013; 69: 714–723.
21.
Chakraborty B, Ghosh P, Moodie EEM, et al. Estimating optimal shared-parameter dynamic regimens with application to a multistage depression clinical trial. Biometrics 2016; 72: 865–876.
