The Choice of a Noninformative Prior on Between-Study Variance Strongly Affects Predictions of Future Treatment Effect

Abstract

Purpose. Bayesian random-effects meta-analyses require the analyst to specify the prior distribution for between-study variance of the treatment effect. We assessed the sensitivity of prediction and other outputs of the meta-analysis to the choice of this prior. Methods. We reanalyzed 7 published meta-analyses (5–14 trials) with rare (event rates <5%), moderate (15%–50%), and frequent binary outcomes (>50%). We examined 10 noninformative priors: inverse gamma on between-study variance (τ ²), 2 uniforms on each of the between-study standard deviation (τ) and τ ², uniform shrinkage on τ ², DuMouchel shrinkage on τ, half-normal on τ ², and half-normal priors on τ with large and small variances. For each analysis, we calculated the posterior distributions for τ, the population treatment effect in current studies, and the predicted treatment effect in a future study. We assessed goodness of fit using total residual deviance, the deviance information criterion, and predictive deviance (by cross-validations). Results. According to total residual deviance, the best-fitting priors were uniform on τ ². According to predictive deviance, half-normal on τ ² and the shrinkage priors were optimal. Across analyses with the 10 priors, there were no important differences in the posteriors for the population treatment effect, but there were substantial differences in the posteriors for τ and predictions. The priors that fitted best according to predictive deviance resulted in less uncertainty around predictions of future treatment effect. Conclusions. In this sample of Bayesian meta-analyses with binary outcomes, the choice of noninformative prior for between-study variance affected model fit and the predictions of future treatment effect. When the predictive distribution is of interest, we highly recommend examination of multiple prior distributions for between-study variance, especially the half-normal on τ ² and the shrinkage priors.