Using Collectively-Derived Standards to Evaluate Individual Performance: A Cautionary Note on Clinical Practice Guidelines

Abstract

The purpose of this work is to demonstrate the problem of evaluating an individual physician's performance relative to practice guidelines which have typically been derived from group consensus or some measure of central tendency. It is argued that when evaluated against a set of criteria derived at the macro-level, an individual physician's performance may justifiably vary due to the patient characteristics or the evolving process of care. It is also argued that it is not necessarily true that costs are reduced when practice variation is reduced. The results indicate that there are cost reduction in areas not targeted by the guidelines, suggesting a possible ‘spillover effect’ due to the increased vigilance in monitoring provider performance. The results also provide some evidence of increased costs following a reduction in variation. Caution should be exercised when evaluating individual physician performance relative to guidelines established at the aggregate level. Acceptable individual physician performance should be judged within the upper and lower boundaries of the implicit distribution of physicians' performances from which the established guidelines generated.