This article presents a latent class model for multilevel data to identify latent subgroups and estimate heterogeneous treatment effects. Unlike sequential approaches that partition data first and then estimate average treatment effects (ATEs) within classes, we employ a Bayesian procedure to jointly estimate mixing probability, selection, and outcome models so that misclassification does not obstruct estimation of treatment effects. Simulation demonstrates that the proposed method finds the correct number of latent classes, estimates class-specific treatment effects well, and provides proper posterior standard deviations and credible intervals of ATEs. We apply this method to Trends in International Mathematics and Science Study data to investigate the effects of private science lessons on achievement scores and then find two latent classes, one with zero ATE and the other with positive ATE.
AtheyS.ImbensG. (2016). Recursive partitioning for heterogeneous causal effects. Proceedings of the National Academy of Sciences, 113(27), 7353–7360. https://doi.org/10.1073/pnas.1510489113
2.
BishopC. M. (2006). Pattern recognition and machine learning. Springer.
3.
CloggC. C. (1995). Latent class models. In ArmingerG.CloggC. C.SobelM. E. (Eds.), Handbook of statistical modeling for the social and behavioral sciences (pp. 311–359). Springer.
4.
CrumpR. K.HotzV. J.ImbensG. W.MitnikO. A. (2009). Dealing with limited overlap in estimation of average treatment effects. Biometrika, 96(1), 187–199. https://doi.org/10.1093/biomet/asn055
5.
FuentesA.LüdtkeO.RobitzschA. (2021). Causal inference with multilevel data: A comparison of different propensity score weighting approaches. Multivariate Behavioral Research, 1–24. https://doi.org/10.1080/00273171.2021.1925521
6.
GelmanA.HullmanJ.WlezienC.MorrisG. E. (2020). Information, incentives, and goals in election forecasts. Judgment and Decision Making, 15(5), 863–880.
7.
GelmanA.LeeD.GuoJ. (2015). Stan: A probabilistic programming language for Bayesian inference and optimization. Journal of Educational and Behavioral Statistics, 40(5), 530–543. https://doi.org/10.3102/1076998615606113
8.
HeinenT. (1996). Latent class and discrete latent trait models: Similarities and differences. Sage Publications.
9.
HillJ. L. (2011). Bayesian nonparametric modeling for causal inference. Journal of Computational and Graphical Statistics, 20(1), 217–240. https://doi.org/10.1198/jcgs.2010.08162
10.
HollandP. W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81(396), 945–960. https://doi.org/10.2307/2289064
11.
HongG.RaudenbushS. W. (2006). Evaluating kindergarten retention policy: A case study of causal inference for multilevel observational data. Journal of the American Statistical Association, 101(475), 901–910. https://doi.org/10.1198/016214506000000447
12.
HongG.RaudenbushS. W. (2013). Heterogeneous agents, social interactions, and causal inference. In MorganS. L. (Ed.), Handbook of causal analysis for social research (pp. 331–352). Springer. https://doi.org/10.1007/978-94-007-6094-3_16
13.
ImaiK.RatkovicM. (2013). Estimating treatment effect heterogeneity in randomized program evaluation. The Annals of Applied Statistics, 7(1), 443–470. https://doi.org/10.1214/12-aoas593
14.
ImaiK.StraussA. (2011). Estimation of heterogeneous treatment effects from randomized experiments, with application to the optimal planning of the get-out-the-vote campaign. Political Analysis, 19(1), 1–19. https://doi.org/10.1093/pan/mpq035
15.
ImbensG. WRubinD. B. (2015). Causal inference in statistics, social, and biomedical sciences. Cambridge University Press. https://doi.org/10.1017/CBO9781139025751
16.
KimH. J.LuB.NehusE. J.KimM.-O. (2019). Estimating heterogeneous treatment effects for latent subgroups in observational studies. Statistics in Medicine, 38(3), 339–353. https://doi.org/10.1002/sim.7970
17.
KimJ.-S.LimW.-C.SteinerP. M. (2017). Causal inference with observational multilevel data: Investigating selection and outcome heterogeneity. In van der ArkL. A.WibergM.CulpepperS. A.DouglasJ. A.WangW.-C. (Eds.), Quantitative psychology (pp. 287–308). Springer. https://doi.org/10.1007/978-3-319-56294-0_26
18.
KimJ.-S.SteinerP. M. (2015). Multilevel propensity score methods for estimating causal effects: A latent class modeling strategy. In van der ArkL. A.BoltD. M.WangW.-C.DouglasJ. A.ChowS.-M. (Eds.), Quantitative psychology research (pp. 293–306). Springer. https://doi.org/10.1007/978-3-319-19977-1_21
19.
KimJ.-S.SteinerP. M.LimW.-C. (2015). Mixture modeling methods for causal inference with multilevel data. In HarringJ. R.StapletonL. M.BeretvasS. N. (Eds.), Advances in multilevel modeling for educational research: Addressing practical issues found in real-world applications (pp. 335–359). IAP.
20.
KünzelS. R.WalterS. J.SekhonJ. S. (2019). Causaltoolbox—Estimator stability for heterogeneous treatment effects. Observational Studies, 5(2), 105–117. https://doi.org/10.1353/obs.2019.0005
LindsayB.CloggC. C.GregoJ. (1991). Semiparametric estimation in the Rasch model and related exponential response models, including a simple latent class model for item analysis. Journal of the American Statistical Association, 86(413), 96–107. https://doi.org/10.1080/01621459.1991.10475008
Luque-FernandezM. A.SchomakerM.RachetB.SchnitzerM. E. (2018). Targeted maximum likelihood estimation for a binary treatment: A tutorial. Statistics in Medicine, 37(16), 2530–2546. https://doi.org/10.1002/sim.7628
26.
MartinM. O.MullisI. V. S.HooperM. (2016). Methods and Procedures in TIMSS 2015. TIMSS & PIRLS International Study Center, Boston College.
MoerkerkeB.LoeysT.VansteelandtS. (2015). Structural equation modeling versus marginal structural modeling for assessing mediation in the presence of posttreatment confounding. Psychological Methods, 20(2), 204–220. https://doi.org/10.1037/a0036368
29.
NeymanJ.IwaszkiewiczK. (1935). Statistical problems in agricultural experimentation. Supplement to the Journal of the Royal Statistical Society, 2(2), 107–180. https://doi.org/10.2307/2983637
ParkD. K.GelmanA.BafumiJ. (2006). State-level opinions from national surveys: Poststratification using multilevel logistic regression. In CohenJ. E. (Ed.), Public opinion in state politics (pp. 209–228). Stanford University Press. https://doi.org/10.11126/stanford/9780804753005.003.0011
RosenbaumP. R.RubinD. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41–55. https://doi.org/10.1093/biomet/70.1.41
34.
RubinD. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5), 688. https://doi.org/10.1037/h0037350
35.
RubinD. B. (1978). Bayesian inference for causal effects: The role of randomization. The Annals of Statistics, 6(1): 34–58. https://doi.org/10.1214/aos/1176344064
36.
RubinD. B. (1986). Statistics and causal inference: Comment: Which ifs have causal answers. Journal of the American Statistical Association, 81(396), 961–962. https://doi.org/10.2307/2289065
37.
ScharfsteinD. O.RotnitzkyA.RobinsJ. M. (1999). Adjusting for nonignorable drop-out using semiparametric nonresponse models. Journal of the American Statistical Association, 94(448), 1096–1120. https://doi.org/10.2307/2669923
38.
ShahnZ.MadiganD. (2017). Latent class mixture models of treatment effect heterogeneity. Bayesian Analysis, 12(3), 831–854. https://doi.org/10.1214/16-ba1022
39.
SpiltJ. L.KootJ. M.van LierP. A. (2013). For whom does it work? Subgroup differences in the effects of a school-based universal prevention program. Prevention Science, 14(5), 479–488. https://doi.org/10.1007/s11121-012-0329-7
40.
SukY.KimJ.-S.KangH. (2021). Hybridizing machine learning methods and finite mixture models for estimating heterogeneous treatment effects in latent classes. Journal of Educational and Behavioral Statistics, 46(3), 323–347. https://doi.org/10.3102/1076998620951983
41.
VehtariA.GelmanA.GabryJ. (2017). Practical Bayesian model evaluation using leave-one-out cross-validation and waic. Statistics and Computing, 27(5), 1413–1432. doi: https://doi.org/10.1007/s11222-016-9696-4
42.
VehtariA.GelmanA.SimpsonD.CarpenterB.BürknerP.-C. (2021). Rank-normalization, folding, and localization: An improved b R for assessing convergence of mcmc (with discussion). Bayesian Analysis, 16(2), 667–718. https://doi.org/10.1214/20-BA1221
43.
VehtariA.SimpsonD. P.YaoY.GelmanA. (2019). Limitations of “limitations of Bayesian leave-one-out cross-validation for model selection.”Computational Brain & Behavior, 2(1), 22–27. https://doi.org/10.1007/s42113-018-0020-6
44.
VermuntJ. K.MagidsonJ. (2002). Latent class cluster analysis. In HagenaarsJ. A.McCutcheonA. L. (Eds.), Applied latent class analysis (pp. 89–106). Cambridge University Press. https://doi.org/10.1017/cbo9780511499531.004
45.
VriezeS. I. (2012). Model selection and psychological theory: A discussion of the differences between the Akaike information criterion (AIC) and the Bayesian information criterion (BIC). Psychological Methods, 17(2), 228. https://doi.org/10.1037/a0027127
46.
WagerS.AtheyS. (2018). Estimation and inference of heterogeneous treatment effects using random forests. Journal of the American Statistical Association, 113(523), 1228–1242. https://doi.org/10.1080/01621459.2017.1319839
47.
WendlingT.JungK.CallahanA.SchulerA.ShahN.GallegoB. (2018). Comparing methods for estimation of heterogeneous treatment effects using observational data from health care databases. Statistics in Medicine, 37(23), 3309–3324. https://doi.org/10.1002/sim.7820
48.
WhiteA.MurphyT. B. (2014). BayesLCA: An R package for Bayesian latent class analysis. Journal of Statistical Software, 61(13), 1–28. https://doi.org/10.18637/jss.v061.i13
49.
WillkeR. J.ZhengZ.SubediP.AlthinR.MullinsC. D. (2012). From concepts, theory, and evidence of heterogeneity of treatment effects to methodological approaches: A primer. BMC Medical Research Methodology, 12(1), 185. https://doi.org/10.1186/1471-2288-12-185
ZeldowB.ReIII, V. LRoyJ. (2019). A semiparametric modeling approach using Bayesian additive regression trees with an application to evaluate heterogeneous treatment effects. The Annals of Applied Statistics, 13(3), 1989–2010. https://doi.org/10.1214/19-aoas1266
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
