Bayesian variable selection in hierarchical difference-in-differences models

Abstract

A popular method for estimating a causal treatment effect with observational data is the difference-in-differences model. In this work, we consider an extension of the classical difference-in-differences setting to the hierarchical context in which data cannot be matched at the most granular level. Our motivating example is an application to assess the impact of primary care redesign policy on diabetes outcomes in Minnesota, in which the policy is administered at the clinic level and individual outcomes are not matched from pre- to post-intervention. We propose a Bayesian hierarchical difference-in-differences model, which estimates the policy effect by regressing the treatment on a latent variable representing the mean change in group-level outcome. We present theoretical and empirical results showing a hierarchical difference-in-differences model that fails to adjust for a particular class of confounding variables, biases the policy effect estimate. Using a structured Bayesian spike-and-slab model that leverages the temporal structure of the difference-in-differences context, we propose and implement variable selection approaches that target sets of confounding variables leading to unbiased and efficient estimation of the policy effect. We evaluate the methods’ properties through simulation, and we use them to assess the impact of primary care redesign of clinics in Minnesota on the management of diabetes outcomes from 2008 to 2017.