A review and empirical comparison of causal inference methods for clustered observational data with application to the evaluation of the effectiveness of medical devices

Abstract

Observational studies are commonplace in medicine. A frequent concern is confounding bias due to differences in patient characteristics across treatment groups, but other important issues include dependency among observations nested within clusters (e.g. patients clustered within physicians or surgeons) and confounding due to cluster characteristics (e.g. physician or surgeon experience or training). Given the frequency with which these issues arise in medical research, as well as their relative complexity, methods for the analysis of clustered observational data are reviewed. We argue for estimating causal treatment effects using marginal models that either match or weight observations using a suitable distance metric (e.g. the propensity score). Simulation results demonstrated that methods incorporating clustering into calculation of the variance were generally more accurate than those that did not. Moreover, methods that account for cluster confounding when estimating the treatment effect were least biased and most accurate. Throughout the paper we illustrate the proposed methods in a medical device setting that compares the effectiveness of femoral heads used in total hip replacements. Whenever possible the clustered aspect of the data should be considered in the design of the study when constructing the distance measure or in the matching process, as well as in the analysis when estimating the variance of the treatment effect.