Is There One Measure-of-Fit That Fits All? A Taxonomy and Review of Measures-of-Fit for Risk-Equalization Models

Abstract

This study provides a taxonomy of measures-of-fit that have been used for evaluating risk-equalization models since 2000 and discusses important properties of these measures, including variations in analytic method. It is important to consider the properties of measures-of-fit and variations in analytic method, because they influence the outcomes of evaluations that eventually serve as a basis for policymaking. Analysis of 81 eligible studies resulted in the identification of 71 unique measures that were divided into 3 categories based on treatment of the prediction error: measured based on squared errors, untransformed errors, and absolute errors. We conclude that no single measure-of-fit is best across situations. The choice of a measure depends on preferences about the treatment of the prediction error and the analytic method. If the objective is measuring financial incentives for risk selection, the only adequate evaluation method is to assess the predictive performance for non-random groups.