Specification issues in measurement of quality of medical care using risk adjusted outcomes

Abstract

Governments and private organizations have recently begun publishing "report cards" that compare quality of hospitals, physicians and health care plans. Often these reports include quality measures based on risk adjusted health outcomes of the patients treated by each health care provider. The use of these measures is, however, controversial due to concerns about their accuracy. To-date, concerns focused on the risk adjustment methodology and small sample sizes. We raise an additional issue related to the definition of quality measures as either the difference between observed and predicted outcome rates or the ratio between these rates. A theoretical analysis of the properties of the two measures is presented. Monte Carlo simulations quantify the effects identified in the theoretical analysis. We show that the two risk adjusted outcome measures of quality may lead to different conclusions about relative quality among providers. Which measure should be used depends on the underlying relationship between patient risks and quality of care in determining health outcomes. For the case replicating the HCFA hospital mortality statistics, the percent of true outliers identified by the incorrect measure ranges from 64% 78%. than health care, where performance is measured based on outcomes.