Applications of Signal Detection Theory in Monitoring Performance and Medical Diagnosis

Abstract

Signal detection theory (SDT) provides a means for distinguishing between accuracy and decision strategies in evaluating observer performance in decision-making tasks. This is useful in determining which general human engineering principles, for example, design, or selection and training, are likely to be effective in a particular situation. Two illustrative areas of application are discussed: (1) Performance decrements in monitoring (vigilance) tasks result either from a decrement in observer sensitivity or from changes in decision criteria. A taxonomic approach is outlined which predicts which of these performance trends will occur in a particular monitoring situation. This enables an identification of those cases where sensitivity decrements occur, for which design changes may be needed and where training techniques are unlikely to be helpful in enhancing performance. (2) Poor reader performance in medical diagnosis may result from inappropriate reporting criteria, especially when disease prevalence and decision goals vary. A study is reported in which staff radiologists, but not radiology residents, were found to vary their reporting criteria appropriately in response to a change in disease prevalence. Staff radiologists were superior to the residents in disease detection and classification, but the residents were not significantly worse if both first and second choice diagnoses were analyzed. ROC measures of diagnostic accuracy were independent of disease prevalence and reporting criteria. The results have implications for the identification of training criteria and the assessment of system accuracy in diagnostic radiology. These two studies illustrate the general usefulness of SDT in identifying target areas for the application of human engineering principles.