Setting Standards on Performance Assessments of Physicians' Clinical Skills Using Contrasting Groups and Receiver Operating Characteristic Curves

Abstract

During recent years, the use of performance assessmentsfor high-stakes evaluation ofphysicians' clinical skills has increased. As a result, considerable attention has been given to the identification of appropriate procedures for setting standards on performance assessments. Several articles have described application of the contrasting groups procedure. It appears that the procedure is not well understood. Relatively little has been written about the requirements for using this procedure appropriately. This article provides a discussion of the theoretical frameworkfor setting standards using contrasting groups. The discussion is then extended to include the circumstance in which the contrasting groups data are interpreted using receiver operating characteristic curves. Examples from the research literature involving variations on the contrasting groups method for setting standards on performance assessments of clinical skills are discussed. The article concludes with guidelines for evaluators interested in using the contrasting groups procedure.