Sage Journals: Discover world-class research

Abstract

Receiver operating-characteristic curve is a popular graphical method frequently used in order to study the diagnostic capacity of continuous (bio)markers. In spite of the existence of a huge number of papers devoted to both theoretical and practical aspects of this topic, the construction of confidence bands has had little impact in the specialized literature. As far as the authors know, in the CRAN there are only three R packages providing receiver operating-characteristic curve confidence regions: plotROC, pROC and fbroc. This work tries to fill this gap studying and proposing a new nonparametric method to build confidence bands for both the standard receiver operating-characteristic curve and its generalization for nonmonotone relationships. The behavior of the proposed procedure is studied via Monte Carlo simulations and the methodology is applied on two real-world biomedical problems. In addition, an R function to compute the proposed and some of the previously existing methodologies is provided as online supplementary material.

Keywords

Confidence bands receiver operating-characteristic curve bootstrap method sensitivity specificity

Get full access to this article

View all access options for this article.

References

Green

Swets

. Signal detection theory and psychophysics, New York: Wiley, 1966.

Pepe

. The statistical evaluation of medical tests for classification and prediction, Oxford: Oxford University Press, 2003.

Zhou

Obuchowski

McClish

. Statistical methods in diagnostic medicine, New York: Wiley, 2002.

Krzanowski

Hand

. ROC curves for continuous data, Boca Raton, FL: Chapman & Hall/CRC Press, 2009.

Broemeling

. Bayesian biostatistics and diagnostic medicine, Boca Raton, FL: Chapman & Hall/CRC Press, 2007.

Broemeling

. Advanced bayesian methods for medical test accuracy, Boca Raton, FL: Chapman & Hall/CRC Press, 2011.

Gonçalvez

Subtil

Rosário Oliveira

, et al. ROC curve estimation: an overview. REVSTAT Stat J 2014; 12: 1–20.

Zou

Hall

Shapiro

. Smooth non-parametric receiver operating characteristic (ROC) curves for continuous diagnostic tests. Stat Med 1997; 16: 2143–2156.

Cheam

ASM

McNicholas

. Modelling receiver operating characteristic curves using Gaussian mixtures. Comput Stat Data Anal 2016; 93: 192–208.

10.

Hanley

. The use of the binormal model for parametric ROC analysis of quantitative diagnostic tests. Stat Med 1996; 15: 1575–1585.

11.

Hsieh

Turnbull

. Nonparametric and semiparametric estimation of the receiver operating characteristic curve. Ann Stat 1996; 24: 25–40.

12.

Hanley

NcNeil

. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982; 148: 29–36.

13.

Jensen

Müller

H-H

Schäfer

. Regional confidence bands for ROC curves. Stat Med 2000; 19: 493–509.

14.

Horváth

Zhou

. Confidence bands for ROC curves. J Stat Plann Infer 2008; 138: 1894–1904.

15.

Hall

. Confidence bands for receiver operating characteristic curves. Med Decis Making 1993; 13: 191–198.

16.

Working

Hotelling

. Application of the theory of error to the interpretation of trends. J Am Stat Soc 1929; 24: 73–85.

17.

Demidenko

. Confidence intervals and bands for the binormal ROC curve revisited. J Appl Stat 2012; 39: 67–79.

18.

Macskassy S, Provost F and Rosset S. ROC confidence bands: an empirical evaluation. In: Proceedings of the 22nd international conference on machine learning, 2005, pp. 537–544.

19.

Martínez-Camblor

Corral

Rey

, et al. ROC curve generalization for non-monotone relationships. Stat Methods Med Res 2017; 26: 113–123.

20.

Martínez-Camblor

de Uña-Álvarez

. Studying the bandwidth in k-sample smooth tests. Comput Stat 2013; 28: 875–892.

21.

Martínez-Camblor

. Nonparametric cutoff point estimation for diagnostic decisions with weighted errors. Rev Colomb Estad 2011; 34: 133–146.

22.

Hall

DiCiccio

Romano

. On smoothing and the bootstrap. Ann Stat 1989; 17: 692–704.

23.

Nadaraya

. Some new estimates for distribution functions. Theory Prob Appl 1964; 9: 497–500.

24.

Wand

Jones

. Kernel smoothing, New York: Chapman & Hall, 1996.

25.

Martínez-Camblor

Carleos

Corral

. General nonparametric ROC curves comparison. J Kor Stat Soc 2013; 42: 71–81.

26.

Vidal Castañeira

López-Vázquez

Alonso-Arias

, et al. A predictive model of treatment outcome in patients with chronic HCV infection using IL28B and PD-1 genotyping. J Hepatol 2012; 56: 1230–1238.

27.

López-Ratón M. Optimal cutoff points for classification in diagnostic studies: new contributions and software developments. PhD dissertation, Universidade de Santiago de Compostela, Spain, 2015.

28.

Martínez-Camblor

Bayón

Pérez-Fernández

. Cumulative/dynamic ROC curve estimation. J Stat Comput Simulat 2016; 86: 3582–3594.

29.

Mossman

. Three-way ROCs. Med Decis Making 1999; 19: 78–89.

30.

Nakas

. Developments in ROC surface analysis and assessment of diagnostic markers in three-class classification problems. REVSTAT Stat J 2014; 12: 43–65.

31.

Robin

Turck

Hainard

, et al. pROC: an open source package for R and S+ to analyze and compare ROC curves. BMC Bioinformat 2011; 12: 1–8.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.59 MB

0.02 MB

0.26 MB

Efficient nonparametric confidence bands for receiver operating-characteristic curves

Abstract

Keywords

Get full access to this article

References

Supplementary Material