Determining a Range of False-positive Rates for Which ROC Curves Differ

Abstract

Many indices have been proposed for summarizing the information contained in the ROC curve. When comparing two ROC curves, though, there are times when global summary measures are either not optimal or not appropriate. The author presents a method for directly comparing true-positive rates for two diagnostic, screening or prognostic tools, determining over what range of false-positive values the tests differ. The method is applicable for in dependent or dependent samples. An example concerning gallium citrate imaging is pre sented, as well as an example using a prognostic index for severity of illness in the ICU. The range of false-positive rates for which the ROC curves differ is determined for each example.

Keywords

Key words: ROC curves true-positive rates binormal data. (Med Decis Making 1990;10:283-287)

Get full access to this article

View all access options for this article.

References

Metz CE Basic principles of ROC analysis. Semin Nucl Med. 1978;8:283-98.

Swets JA ROC analysis applied to the evaluation of medical imaging techniques. Invest Radiol. 1979;14:109-21.

Swets JA Indices of discrimination or diagnostic accuracy. Their ROCs and implied models . Psychol Bull. 1986;6:100-17.

Swets JA , Pickett RM Evaluation of diagnostic systems: methods from signal detection theory. New York: Academic Press, 1982.

McClish DK Analyzing a portion of the ROC curve. Med Decis Making. 1989;9:190-5.

McNeil BJ , Hanley JA Statistical approaches to the analysis of receiver operating characteristic (ROC) curves. Med Decis Making. 1984;4:137-50.

Hanley JA The robustness of the "binormal" assumptions used in fitting ROC curves. Med Decis Making. 1988;8:197-203.

Swets JA Forms of empirical ROCs in discrimination and diagnostic tasks: implications for theory and measurement of performance. Psychol Bull. 1986;99:181-98.

Dorfman DD , Alf E. Maximum likelihood estimation of parameters of signal detection theory and determination of confidence intervals—rating method data. J Math Psychol. 1969;6:487-96.

10.

Metz CE FORTRAN programs ROCFIT, CORROC and ROCPWR. Available from CE Metz, Department of Radiology, University of Chicago, Chicago, IL.

11.

Metz CE , Wang PL , Kronman HB A new approach for testing the significance of differences between ROC curves from correlated data. In: Deconink F , ed. Information processing in medical imaging. The Hague: Nojhoff, 1984;432-45.

12.

Metz CE , Kronman HB Statistical significance tests for binormal ROC curves. J Math Psychol. 1980;22:219-43.

13.

Knaus WA , Zimmerman JE , Wagner DP , et al. APACHE. Acute physiology and chronic health evaluation. A physiologically based classification system. Crit Care Med. 1981;9:591-7.

14.

Henning R. , McClish DK , Daly B. , et al. Clinical characteristics and resource utilization of medical and surgical ICU patients: implications for organization of intensive care . Crit Care Med. 1987;15:264-9.

15.

Johnson PO , Neyman J. Tests of certain linear hypotheses and their applications to some educational problems. Stat Res Mem. 1939;1:57-9.

16.

Potthoff RF On the Johnson-Neyman technique and some extensions thereof. Psychometrika. 1964;29:241,256.