Sage Journals: Discover world-class research

Abstract

Classifications that involve subclasses are common in many applied fields. “Compound multi-class classification” refers to the settings which involve three or more main classes and at least one of the main classes has multiple subclasses. In this paper, we propose an accuracy metric proper for “compound $M$ -class classification,” namely “hypervolume under compound $R O C$ manifold $(H U M_{C, M})$ .” The proposed $H U M_{C, M}$ evaluates the overall accuracy of a biomarker measured on continuous scale correctly identifying $M$ main classes without requiring specification of an ordering in terms of marker values for subclasses relative to each other within each main class. The probabilistic interpretation of $H U M_{C, M}$ is analytically derived. A network-based computing algorithm which enables efficient computation of the empirical estimate of $H U M_{C, M}$ is developed. Non-parametric bootstrap percentile confidence intervals of $H U M_{C, M}$ are assessed through extensive simulation studies. Lastly, a real data example is included to illustrate the usage of our proposed method.

Keywords

Biomarker evaluation diagnostic studies hypervolume under receiver operating characteristic manifold network-based algorithm alzheimer’s disease

Get full access to this article

View all access options for this article.

References

Nakas

Yiannoutsos

. Ordered multiple-class roc analysis with continuous measurements. Stat Med 2004; 23: 3437–3449.

Xiong

van Belle

Miller

, et al. Measuring and estimating diagnostic accuracy when there are three ordinal diagnostic groups. Stat Med 2006; 25: 1251–1273.

Kang

Tian

. Estimation of the volume under the roc surface with three ordinal diagnostic categories. Comput Stat Data Anal 2013; 62: 39–51.

Nakas

Dalrymple-Alford

Anderson

, et al. Generalization of youden index for multiple-class classification problems applied to the assessment of externally validated cognition in parkinson disease screening. Stat Med 2013; 32: 995–1003.

Fine

. Roc analysis with multiple classes and multiple tests: Methodology and its application in microarray studies. Biostatistics 2008; 9: 566–576.

Dong

Attwood

Hutson

, et al. A new diagnostic accuracy measure and cut-point selection criterion. Stat Methods Med Res 2017; 26: 2832–2852.

Feng

Tian

. Flexible diagnostic measures and new cut-point selection methods under multiple ordered classes. Pharm Stat 2022; 21: 220–240.

Nan

Tian

. A new accuracy metric under three classes when subclasses are involved and its confidence interval estimation. Stat Med 2023; 42: 5207–5228.

Bapat

Beg

. Order statistics for nonidentically distributed variables and permanents. Sankhya: Indian J Stat Ser A 1989; 79–93.

10.

Balakrishnan

. Permanents, order statistics, outliers, and robustness. Rev Mat Complut 2007; 20: 7–107.

11.

Feng

Liu

Kuan

, et al. A network approach to compute hypervolume under receiver operating characteristic manifold for multi-class biomarkers. Stat Med 2023; 42: 834–859.

12.

Cummings

Apostolova

Rabinovici

, et al. Lecanemab: Appropriate use recommendations. J Prev Alzheimers Dis 2023; 10: 362–377.

13.

Wechsler

. Wechsler memory scale-revised. Psychol Corpor 1987.

14.

Yan

Tian

Liu

. Combining large number of weak biomarkers based on auc. Stat Med 2015; 34: 3811–3830.

15.

Fine

Pencina

. Multi-category diagnostic accuracy based on logistic regression. Stat Theory Relat Fields 2017; 1: 143–158.

16.

Wang

Attwood

Tian

. Receiver operating characteristic analysis under tree orderings of disease classes. Stat Med 2016; 35: 1907–1926.

17.

Feng

Tian

. Issues and solutions in biomarker evaluation when subclasses are involved under binary classification. Stat Methods Med Res 2021; 30: 87–98.

18.

Kolaczyk

Csárdi

. Statistical analysis of network data with R. New York: Springer, 2014.

Assessing accuracy for multi-class classification when subclasses are involved

Abstract

Keywords

Get full access to this article

References