Testing an Underlying Assumption on a ROC Curve Based on Rating Data

Abstract

The receiver operating characteristic (ROC) curve is widely accepted as the measurement that best represents the accuracy of a diagnostic system that consists of a diagnostic imaging modality and a reader who is interpreting results from the modality. However, there is one implicit assumption in a ROC curve: that the cutoff points to declare a positive test for decision variables of a diseased patient and a nondiseased patient are the same. If this assumption is violated, then a ROC curve does not exist. The author develops a statistical procedure to test this assumption. To demonstrate, he applies the method to a ROC data set and shows that the assumption of the equal cutoff points could be rejected. Key words: ROC curve; likelihood ratio test; cutoff points. (Med Decis Making 1995;15:276-282)

Get full access to this article

View all access options for this article.

References

Hanley JA

Receiver operating characteristic (ROC) methodology: the state of the art

Radiology. 1989;29:307-35.

Metz C. ROC methodology in radiologic imaging. Invest Radiol. 1986;21:720-33.

Hanley JA The robustness of the "binormal" assumptions used in fitting ROC curves . Med Decis Making. 1988;8:197-203.

Swets JA Form of empirical ROCs in discrimination and diagnostic tasks: implications for theory and measurement of performance. Psychol Bull. 1986;2:181-98.

Dorfman D. , Alf E. Maximum likelihood estimation of parameters of signal detection theory and determination of confidence intervals. Rating method data. J Math Psychol. 1969;6:487-96.

Mardia KV , Kent JT , Bibby JM Multivariate Analysis. New York: Academic Press, 1980.

Metz C. , et al. ROCFIT: A Program for ROC Analysis of Single Categorical, Rating-scale Data. Department of Radiology, University of Chicago, Chicago, IL, 1989 .

Ogilvie JC , Creelman CD Maximum likelihood estimation of ROC curve parameters. J Math Psychol. 1968;5:377-91.

Thisted RA Elements of Statistical Computing. New York: Chapman and Hall, 1988.

10.

McCullagh P. Regression models for ordinal data. J R Statist Soc, series B. 1980;42:109-42.

11.

Swets JA Measuring the accuracy of diagnostic systems. Science. 1988;2:1285-93.

12.

Tosteson Ana , Begg CB A general regression methodology for ROC curve estimation. Med Decis Making. 1987;8:204-15.