We review in this article several classification methods, especially for high-dimensional and low-sample size data. We discuss several desirable properties for classifiers in such settings, including predictability, consistency, generality, stability, robustness and sparsity. Specifically, a good classifier should have a small prediction error (predictability); converge to the Bayes-rule classifier asymptotically (consistency); be stable when adding/removing an observation (generality); be stable for different data sets of the same kind (stochastic stability); be stable when there are a small number of contaminated observations (robustness); and have a small number of variables in the classifier (interpretability or sparsity). Several simulation examples and real applications are used to illustrate the usefulness of the existing popular classifiers and compare their performance.
DomingosPPazzaniM. On the optimality of the simple Bayesian classifier under zero-one loss. Mach Learn1997; 29: 103–130.
8.
BurgesCJC. A Tutorial on Support Vector Machines for Pattern Recognition. Data Min Know Discov1998; 2: 121–167.
9.
VapnikVN. The nature of statistical learning theory, New York: Springer, 1995.
10.
VapnikVN. Statistical learning theory, New York: John Wiley and Sons, 1998.
11.
LoboMVandenbergheLBoydSLebretH. Applications of Second-Order Cone Programming. Linear Algebra App1998; 284: 193–228.
12.
JungSMarronJS. PCA consistency in High dimension, low sample size context. Ann Sta2009; 37: 4104–4130.
13.
AhnJMarronJSMullerKMChiY. The high dimension, low sample size geometric representation holds under mild conditions. Biometrika2007; 94: 760–766.
14.
BaikJSilversteinJ. Eigenvalues of large sample covariance matrices of spiked population models. J Multivar Anal2006; 97: 1382–1408.
15.
PaulD. Asymptotics of sample eigenstruture for a large dimensional spiked covariance model. Stat Sin2007; 17: 1617–1642.
16.
BickelPLevinaE. Some theory of Fisher's linear discriminant function, “naive Bayes”, and some alternatives when there are many more variables than observations. Bernoulli2004; 10: 989–1010.
17.
McCullaghPNelderJA. Generalized linear models, 2nd ed. London: Chapman and Hall, 1989.
18.
LinY. Support Vector Machines and the Bayes Rule in Classification. Data Mining Knowledge Discovery2002; 6: 259–275.
19.
BartlettPJordanMMcAuliffeJ. Convexity, classification, and risk bounds. J Am Stat Asso2006; 101: 138–156.
20.
QiaoXZhangHLiuYToddMMarronJ. Weighted Distance Weighted Discrimination and its asymptotic properties. J Am Stat Assoc2010; 105: 401–414.
21.
HallPMarronJSNeemanA. Geometric representation of high dimension, low sample size data. J R Stat Soc B Stat Meth2005; 67: 427–444.
22.
TibshiraniRHastieTNarasimhanBChuG. Diagnosis of multiple cancer types by shrunken centroids of gene expression. PNAS2002; 99: 6567–6572.
23.
WangSZhuJ. Improved centroids estimation for the nearest shrunken centroid classifier. Bioinformatics2007; 23: 972–979.
24.
GuoYHastieTTibshiraniR. Regularized linear discriminant analysis and its application in microarrays. Biostatistics2007; 8: 86–100.
25.
QiaoZ. Variable selection in multivariate data analysis using regularization. PhD thesis, University of Pennsylvania, 2006.
26.
QiaoZZhouLHuangJ. Sparse linear discriminant analysis with applications to high dimensional low sample size data. IAENG Int J Appl Math2008; 39: 48–60.
27.
WuMZhangLWangZChristianiDLinX. Sparse linear discriminant analysis for simultaneous testing for the significance of a gene set/pathway and gene selection. Bioinformatics2009; 25: 1145–1145.
28.
CaiTLiuW. A Direct Estimation Approach to Sparse Linear Discriminant Analysis2011. Arxiv preprint arXiv:1107.3442.
29.
ShaoJWangYDengXWangS. Sparse linear discriminant analysis by thresholding for high dimensional data. Ann Stat2011; 39: 1241–1265.
30.
SchimekM. Penalized logistic regression in gene expression analysis, In: Proceedings to 2003 Semiparametric Conference, 13–20 August 2003. Berlin, Germany, 2003.
31.
ZhuJHastieT. Classification of gene microarrays by penalized logistic regression. Biostatistics2004; 5: 427–443.
LeeSLeeHAbbeelPNgY. Efficient L1 regularized logistic regression. In: Proceedings of the Twenty-First National Conference on Artificial Intelligence (AAAI), 16–20 July 2006, 2006; Boston MA.
34.
KohKKimSJBoydS. An Interior-Point Method for Large-Scale l1-Regularized Logistic Regression. J Mach Learn Res2007, pp. 1519–1555.
35.
FanJLiR. Variable selection via nonconcave penalized likelihood and its oracle properties. J Am Stat Assoc2001; 96: 1348–1360.
36.
ZhuJRossetSHastieRT. 1-norm support vector machines. Advances in Neural Information Processing Systems2004; 16: 49–49.
37.
ZhangHHAhnJLinXParkC. Gene Selection Using Support Vector Machines With Nonconvex Penalty. Bioinformatics2006; 22: 88–95.
