Separation of Safety Effects of Multiple Improvements by Alternate Empirical Bayes Methods

Abstract

Empirical Bayes (EB) before–after evaluations provide the average safety effectiveness estimate for safety improvement treatments that have been implemented on a group of sites. When multiple improvements have been installed at each site, a single effectiveness estimate is provided for the combination of improvements. To obtain individual safety effectiveness estimates for the improvements by the current EB method, separate studies of each improvement must be conducted. Alternative EB methods may be applied, however, when sites that have the combined improvements installed and sites that have all but the desired improvement to be estimated are both available. This paper examines the safety evaluation methodology, safety effect estimators, and statistical test of significance for these alternative EB methods. The recent study of the safety edge treatment is used to illustrate these methods. Recommendations include the conduct of individual EB analyses for study sites with and without the desired treatment and the estimation of the effect directly from the results. It was found that a statistical test of significance could be conducted by generating a confidence interval from the estimated variance of the estimates. Areas for further research were also identified.