Receiver operating characteristic estimation and threshold selection criteria in three-class classification problems for clustered data

Abstract

Statistical evaluation of diagnostic tests, and, more generally, of biomarkers, is a constantly developing field, in which complexity of the assessment increases with the complexity of the design under which data are collected. One particularly prevalent type of data is clustered data, where individual units are naturally nested into clusters. In these cases, Bias can arise from omission, in the evaluation process, of cluster-level effects and/or individual covariates. Focusing on the three-class case and for continuous-valued diagnostic tests, we investigate how to exploit the clustered structure of data within a linear-mixed model approach, both when the assumption of normality holds and when it does not. We provide a method for the estimation of covariate-specific receiver operating characteristic surfaces and discuss methods for the choice of optimal thresholds, proposing three possible estimators. A proof of consistency and asymptotic normality of the proposed threshold estimators is given. All considered methods are evaluated by extensive simulation experiments. As an application, we study the use of the Lysosomal Associated Membrane Protein Family Member 5 gene expression as a biomarker to distinguish among three types of glutamatergic neurons.

Keywords

Receiver operating characteristic analysis clustered data covariate adjustment linear-mixed models Box–Cox transformation

Get full access to this article

View all access options for this article.

References

Pepe

. The statistical evaluation of medical tests for classification and prediction. New York, USA: Oxford University Press, 2003.

Zhou

McClish

Obuchowski

. Statistical methods in diagnostic medicine. Inc., Hoboken, New Jersey, USA: John Wiley & Sons, 2011.

To Duc

Chiogna

Adimari

. Bias–corrected methods for estimating the receiver operating characteristic surface of continuous diagnostic tests. Electron J Stat 2016; 10: 3063–3113.

Brinkman

Gottardo

. Automated gating of flow cytometry data via robust model-based clustering. Cytometry A 2008; 73: 321–332.

Ecker

Geschwind

Kriegstein

, et al. The brain initiative cell census consortium: lessons learned toward generating a comprehensive brain cell atlas. Neuron 2017; 96: 542–557.

Nakas

Yiannoutsos

. Ordered multiple-class ROC analysis with continuous measurements. Stat Med 2004; 23: 3437–3449.

Nakas

Alonzo

Yiannoutsos

. Accuracy and cut-off point selection in three-class classification problems using a generalization of the youden index. Stat Med 2010; 29: 2946–2955.

Attwood

Tian

Xiong

. Diagnostic thresholds with three ordinal groups. J Biopharm Stat 2014; 24: 608–633.

Hua

Tian

. A comprehensive and comparative review of optimal cut-points selection methods for diseases with multiple ordinal stages. J Biopharm Stat 2020; 30: 46–68.

10.

Xiong

Luo

Chen

et al. Estimating diagnostic accuracy for clustered ordinal diagnostic groups in the three-class case – application to the early diagnosis of Alzheimer disease. Stat Methods Med Res 2018; 27: 701–714.

11.

McCulloch

Searle

. Generalized, linear, and mixed models. Inc., New York, USA: John Wiley & Sons, 2001.

12.

Gurka

Edwards

Muller

et al. Extending the Box–Cox transformation to the linear mixed model. J R Stat Soc A Stat 2006; 169: 273–288.

13.

Tasic

Yao

Graybuck

et al. Shared and distinct transcriptomic cell types across neocortical areas. Nature 2018; 563: 72–78.

14.

Liang

Zeger

. Longitudinal data analysis using generalized linear models. Biometrika 1986; 73: 13–22.

15.

Kauermann

Carroll

. A note on the efficiency of sandwich covariance matrix estimation. J Am Stat Assoc 2001; 96: 1387–1396.

16.

Mancl

DeRouen

. A covariance estimator for GEE with improved small-sample properties. Biometrics 2001; 57: 126–134.

17.

Nakas

Alonzo

. ROC graphs for assessing the ability of a diagnostic marker to detect three disease classes with an umbrella ordering. Biometrics 2007; 63: 603–609.

18.

Luo

Xiong

. Youden index and associated cut-points for three ordinal diagnostic groups. Commun Stat-Theor Simul C 2013; 42: 1213–1234.

19.

Lipsitz

Ibrahim

Molenberghs

. Using a Box–Cox transformation in the analysis of longitudinal data with incomplete responses. J R Stat Soc C-Appl 2000; 49: 287–296.

20.

Box

Cox

. An analysis of transformations (with discussion). J R Stat Soc B 1964; 26: 211–252.

21.

Gurka

Edwards

. Estimating variance components and random effects using the Box–Cox transformation in the linear mixed model. Commun Stat-Theor M 2011; 40: 515–531.

22.

Nakas

. Developments in ROC surface analysis and assessment of diagnostic markers in three-class classification problems. REVSTAT – Stat J 2014; 12: 43–65.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.05 MB