Because of high dimensionality, correlation among covariates, and noise contained in data, dimension reduction (DR) techniques are often employed to the application of machine learning algorithms. Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), and their kernel variants (KPCA, KLDA) are among the most popular DR methods. Recently, Supervised Kernel Principal Component Analysis (SKPCA) has been shown as another successful alternative. In this paper, brief reviews of these popular techniques are presented first. We then conduct a comparative performance study based on three simulated datasets, after which the performance of the techniques are evaluated through application to a pattern recognition problem in face image analysis. The gender classification problem is considered on MORPH-II and FG-NET, two popular longitudinal face aging databases. Several feature extraction methods are used, including biologically-inspired features (BIF), local binary patterns (LBP), histogram of oriented gradients (HOG), and the Active Appearance Model (AAM). After applications of DR methods, a linear support vector machine (SVM) is deployed with gender classification accuracy rates exceeding 95% on MORPH-II, competitive with benchmark results. A parallel computational approach is also proposed, attaining faster processing speeds and similar recognition rates on MORPH-II. Our computational approach can be applied to practical gender classification systems and generalized to other face analysis tasks, such as race classification and age prediction.
AntipovG.BaccoucheM.BerraniS.A. and DugelayJ.-L., Effective training of convolutional neural networks for face-based gender and age prediction, Pattern Recognition72 (2017), 15–26.
3.
AntipovG.BerraniS.-A. and DugelayJ.-L., Minimalistic cnn-based ensemble model for gender prediction from face images, Pattern Recognition Letters70 (2016), 59–65.
4.
AshtianiH. and GhodsiA., A dimension-independent generalization bound for kernel supervised principal component analysis, In Feature Extraction: Modern Questions and Challenges, (2015), 19–29.
5.
BarshanE.GhodsiA.AzimifarZ. and JahromiM.Z., Supervised principal component analysis: Visualization, classification and regression on subspaces and submanifolds, Pattern Recognition44(7) (2011), 1357–1371.
6.
BarzilayO. and BrailovskyV.L., On domain knowledge and feature selection using a support vector machine, Pattern Recognition Letters20(5) (1999), 475–484.
7.
BaudatG. and AnouarF., Generalized discriminant analysis using a kernel approach, Neural Computation12(10) (2000), 2385–2404.
8.
BelhumeurP.N.HespanhaJ.P. and KriegmanD.J., Eigenfaces vs. fisherfaces: Recognition using class specific linear projection, IEEE Transactions on Pattern Analysis and Machine Intelligence19(7) (1997), 711–720.
9.
BurtonA.M.BruceV. and DenchN., What’s the difference between men and women? evidence from facial measurement, Perception22(2) (1993), 153–176.
10.
ByunH. and LeeS.-W., Applications of support vector machines for pattern recognition: A survey, In Pattern Recognition with Support Vector Machines, Springer, (2002), 213–236.
11.
CootesT.F.EdwardsG.J. and TaylorC.J., Active appearance models, In European Conference on Computer Vision, Springer, (1998), 484–498.
12.
CottrellG.W. and MetcalfeJ., Empath: Face, emotion, and gender recognition using holons, In Advances in Neural Information Processing Systems, (1991), 564–571.
13.
DimitriadouE.HornikK.LeischF.MeyerD. and WeingesselA., Misc functions of the department of statistics (e1071), tu wien, R package Version, (2005), 1–5.
14.
EdwardsG.J.TaylorC.J. and CootesT.F., Interpreting face images using active appearance models, In Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition, IEEE, (1998), 300–305.
15.
FewzeeP. and KarrayF., Dimensionality reduction for emotional speech recognition, In Privacy, Security, Risk and Trust (PASSAT), 2012 International Conference on and 2012 International Confernece on Social Computing (SocialCom), IEEE, (2012), 532–537.
16.
FisherR.A., The use of multiple measurements in taxonomic problems, Annals of Human Genetics7(2) (1936), 179–188.
17.
FodorI.K., A survey of dimension reduction techniques, Technical report, Lawrence Livermore National Lab., CA (US), (2002).
18.
FuY.GuoG. and HuangT.S., Age synthesis and estimation via faces: A survey, IEEE Transactions on Pattern Analysis and Machine Intelligence32(11) (2010), 1955–1976.
19.
GolombB.A.LawrenceD.T. and SejnowskiT.J., Sexnet: A neural network identifies sex from human faces, In NIPS, volume 1, page 2, 1990.
20.
GrettonA.BousquetO.SmolaA. and SchölkopfB., Measuring statistical dependence with hilbert-schmidt norms, In International Conference on Algorithmic Learning Theory Springer, (2005), 63–77.
21.
GuoG.DyerC.R.FuY. and HuangT.S., Is gender recognition affected by age? In Computer Vision Workshops (ICCV Workshops), 2009 IEEE 12th International Conference on, IEEE, (2009), 2032–2039.
22.
GuoG. and MuG., Human age estimation: What is the influence across race and gender? In Computer Vision and Pattern Recognition Workshops (CVPRW), 2010 IEEE Computer Society Conference on, IEEE, (2010), 71–78.
23.
GuoG. and MuG., A study of large-scale ethnicity estimation with gender and age variations, In Computer Vision and Pattern Recognition Workshops (CVPRW), 2010 IEEE Computer Society Conference on, IEEE, (2010), 79–86.
24.
GuoG. and MuG., Simultaneous dimensionality reduction and human age estimation via kernel partial least squares regression, In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, IEEE, (2011), 657–664.
25.
GuoG. and MuG., A framework for joint estimation of age, gender and ethnicity on a large database, Image and Vision Computing32(10) (2014), 761–770.
26.
HanH.OttoC.LiuX. and JainA.K., Demographic estimation from face images: Human vs. machine performance, IEEE Transactions on Pattern Analysis & Machine Intelligence (6) (2015), 1148–1161.
27.
HintonG.E. and SalakhutdinovR.R., Reducing the dimensionality of data with neural networks, Science313(5786) (2006), 504–507.
28.
HotellingH., Analysis of a complex of statistical variables into principal components, Journal of Educational Psychology24(6) (1933), 417.
29.
IzenmanA.J., Modern multivariate statistical techniques, Regression, Classification and Manifold Learning, (2008).
30.
JapkowiczN. and StephenS., The class imbalance problem: A systematic study, Intelligent Data Analysis6(5) (2002), 429–449.
31.
JohanssonS. and JohanssonJ., Interactive dimensionality reduction through user-defined combinations of quality metrics, IEEE Transactions on Visualization and Computer Graphics15(6) (2009), 993–1000.
32.
KargM.JenkeR.SeiberlW.KühnlenzK.SchwirtzA. and BussM., A comparison of pca, kpca and lda for feature extraction to recognize affect in gait kinematics, In Affective Computing and Intelligent Interaction and Workshops, 2009. ACII 2009. 3rd International Conference on, IEEE, (2009), 1–6.
33.
KimK.I.JungK. and KimH.J., Face recognition using kernel principal component analysis, IEEE Signal Processing Letters9(2) (2002), 40–42.
34.
KlingT., Morph-ii: Feature vector documentation: Nsf-reu site at unc wilmington, http://libres.uncg.edu/ir/uncw/f/wangy2018-1.pdf, (2017).
35.
LeeJ.A. and VerleysenM., Nonlinear Dimensionality Reduction, Springer Science & Business Media, 2007.
36.
LiM. and YuanB., 2d-lda: A statistical linear discriminant analysis for image matrix, Pattern Recognition Letters26(5) (2005), 527–532.
37.
LianH.C. and LuB.L., Multi-view gender classification using local binary patterns and support vector machines, In International Symposium on Neural Networks, Springer, (2006), 202–209.
38.
LuJ.PlataniotisK.N. and VenetsanopoulosA.N., Face recognition using kernel direct discriminant analysis algorithms, IEEE Transactions on Neural Networks14(1) (2003), 117–126.
39.
MaY. and FuY., Manifold Learning Theory and Applications, CRC press, 2011.
40.
MäkinenE. and RaisamoR., An experimental comparison of gender classification methods, Pattern Recognition Letters29(10) (2008), 1544–1556.
41.
MartínezA.M. and KakA.C., Pca versus lda, IEEE Transactions on Pattern Analysis and Machine Intelligence23(2) (2001), 228–233.
42.
MikaS.RatschG.WestonJ.ScholkopfB. and MullersK.-R., Fisher discriminant analysis with kernels, In Neural Networks for Signal Processing IX, 1999. Proceedings of the 1999 IEEE Signal Processing Society Workshop., IEEE, (1999), 41–48.
43.
MotaiY., Kernel association for classification and prediction: A survey, IEEE Transactions on Neural Networks and Learning Systems26(2) (2015), 208–223.
44.
NgC.B.TayY.H. and GoiB.M., Vision-based human gender recognition: A survey, arXiv preprint arXiv:12041611, (2012).
45.
Nhan DuongC.LuuK., QuachG. and BuiT.D., Beyond principal components: Deep boltzmann machines for face modeling, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2015), 4786–4794.
46.
PanisG.LanitisA.TsapatsoulisN. and CootesT.F., Overview of research on facial ageing using the fg-net ageing database, IET Biometrics5(2) (2016), 37–46.
47.
PearsonK., Liii. on lines and planes of closest fit to systems of points in space, The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science2(11) (1901), 559–572.
48.
PlattJ. et al., Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods, Advances in Large Margin Classifiers10(3) (1999), 61–74.
49.
PoggioB.BrunelliR. and PoggioT., Hyberbf networks for gender classification, (1992).
50.
QiuY.MeiJ. and QiuM.Y., Package ‘rarpack’, 2016.
51.
RaoC.R., The utilization of multiple measurements in problems of biological classification, Journal of the Royal Statistical Society. Series B (Methodological)10(2) (1948), 159–203.
52.
RicanekK. and TesafayeT., Morph: A longitudinal image database of normal adult age-progression, In Automatic Face and Gesture Recognition, 2006. FGR 2006. 7th International Conference on, IEEE, (2006), 341–345.
53.
SamadaniA.-A.GhodsiA. and KulićD., Discriminative functional analysis of human movements, Pattern Recognition Letters34(15) (2013), 1829–1839.
54.
SchölkopfB.HerbrichR. and SmolaA.J., A generalized representer theorem, In International Conference on Computational Learning Theory, Springer, (2001), 416–426.
55.
SchölkopfB.SimardP.SmolaA.J. and VapnikV., Prior knowledge in support vector kernels, In Advances in Neural Information Processing Systems, (1998), 640–646.
56.
SchölkopfB.SmolaA. and MüllerK.R., Kernel principal component analysis, In International Conference on Artificial Neural Networks, Springer, (1997), 583–588.
57.
SchölkopfB.SmolaA. and MüllerK.R., Nonlinear component analysis as a kernel eigenvalue problem, Neural Computation10(5) (1998), 1299–1319.
58.
ShanC., Learning local binary patterns for gender classification on real-world face images, Pattern Recognition Letters33(4) (2012), 431–437.
59.
Shawe-TaylorJ. and CristianiniN., Kernel Methods for Pattern Analysis, Cambridge university press, (2004).
60.
SorzanoC.O.S.VargasJ. and MontanoA.P., A survey of dimensionality reduction techniques, arXiv preprint arXiv:14032877, (2014).
61.
SteinwartI. and ChristmannA., Support Vector Machines, Springer Science & Business Media, (2008).
62.
SunY.ZhangM.SunZ. and TanT., Demographic analysis from biometric data: Achievements, challenges, and new frontiers, IEEE Transactions on Pattern Analysis and Machine Intelligence40(2) (2018), 332–351.
63.
TurkM. and PentlandA., Eigenfaces for recognition, Journal of Cognitive Neuroscience3(1) (1991), 71–86.
64.
WahbaG., Spline Models for Observational Data, 59 Siam, (1990).
65.
WangY.ChenC.WatkinsV. and RicanekK., Modified supervised kernel pca for gender classification, In International Conference on Intelligent Science and Big Data Engineering, Springer, (2015), 60–71.
66.
WangY.RicanekK.ChenC. and ChangY., Gender classification from infants to seniors, In 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS), IEEE, (2010), 1–6.
67.
WiskottL.FellousJ.M.KrügerN. and Von der MalsburgC., Face recognition and gender determination, (1995).
68.
WuH.BowersD.M.HuynhT.T. and SouvenirR., Biomedical video denoising using supervised manifold learning, In Biomedical Imaging (ISBI), 2013 IEEE 10th International Symposium on, IEEE, (2013), 1244–1247.
69.
XiaB.SunH. and LuB.-L., Multi-view gender classification based on local gabor binary mapping pattern and support vector machines, In Neural Networks, 2008. IJCNN 2008.(IEEE World Congress on Computational Intelligence). IEEE International Joint Conference on, IEEE, (2008), 3388–3395.
70.
XieY.LuuK. and SavvidesM., A robust approach to facial ethnicity classification on large scale face databases, In Biometrics: Theory, Applications and Systems (BTAS), 2012 IEEE Fifth International Conference on, IEEE, (2012), 143–149.
71.
YangH.F.LinB.Y.ChangK.Y. and ChenC.S., Automatic age estimation from face images via deep ranking, Networks35(8) (2013), 1872–1886.
72.
YangJ.JinZ.YangJ.Y.ZhangD. and FrangiA.F., Essence of kernel fisher discriminant: Kpca plus lda, Pattern Recognition37(10) (2004), 2097–2100.
73.
YangJ.PengW.WardM.O. and RundensteinerE.A., Interactive hierarchical dimension ordering, spacing and filtering for exploration of high dimensional datasets, In IEEE Symposium on Information Visualization 2003 (IEEE Cat. No.03TH8714), (2003), 105–112.
74.
YangJ. and YangJ.Y., Why can lda be performed in pca transformed space? Pattern Recognition36(2) (2003), 563–566.
75.
YangJ.ZhangD.FrangiA.F. and YangJ.Y., Two-dimensional pca: a new approach to appearance-based face representation and recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence26(1) (2004), 131–137.
76.
YangZ. and AiH., Demographic classification with local binary patterns, In International Conference on Biometrics, Springer, (2007), 464–473.
77.
YeF.ShiZ. and ShiZ., A comparative study of pca, lda and kernel lda for image classification, In Ubiquitous Virtual Reality, 2009. ISUVR’09. International Symposium on, IEEE, (2009), 51–54.
78.
YiD.LeiZ. and LiS.Z., Age estimation by multi-scale convolutional network, In Asian Conference on Computer Vision, Springer, (2014), 144–158.
79.
YinJ.LiuZ.JinZ. and YangW., Kernel sparse representation based classification, Neurocomputing77(1) (2012), 120–128.
80.
YipB.BinghamG.KempfertK.FabishJ.KlingT.ChenC. and WangY., Preliminary studies on a large face database, arXiv preprint arXiv:181106446, (2018).
81.
YipB.TownerR.KlingT.ChenC. and WangY., Image pre-processing using opencv library on morph-ii face database, arXiv preprint arXiv:181106934, (2018).
82.
ZhaoW.KrishnaswamyA.ChellappaR.SwetsD.L. and WengJ., Discriminant analysis of principal components for face recognition, In Face Recognition, Springer, (1998), 73–85.
