Robust sparse principal component analysis by DC programming algorithm

Abstract

The classical principal component analysis (PCA) is not sparse enough since it is based on the L₂-norm that is also prone to be adversely affected by the presence of outliers and noises. In order to address the problem, a sparse robust PCA framework is proposed based on the min of zero-norm regularization and the max of L_p-norm (0 < p ≤ 2) PCA. Furthermore, we developed a continuous optimization method, DC (difference of convex functions) programming algorithm (DCA), to solve the proposed problem. The resulting algorithm (called DC-LpZSPCA) is convergent linearly. In addition, when choosing different p values, the model can keep robust and is applicable to different data types. Numerical simulations are simulated in artificial data sets and Yale face data sets. Experiment results show that the proposed method can maintain good sparsity and anti-outlier ability.

Keywords

Principal component analysis sparseness robustness zero-norm DC programming face reconstruction

Get full access to this article

View all access options for this article.

References

Jolliffe

I.T.

, Principal Component Analysis, Springer-Verlag, New York, NY, 1986.

Jolliffe

I.T.

and Cadima

, Principal component analysis: A review and recent developments, Philos Trans Roy Soc A, Math, Phys Eng Sci 374(2065) (2016), 20150202.

Seghouane

A.K.

, Shokouhi

and Koch

, Sparse Principal Component Analysis with Preserved Sparsity Pattern, IEEE Transactions on Image Processing 28(7) (2019), 3274–3285.

and Chen

S.B.

, compressive principal component analysis, Front Comput Sci 14(4) (2020), 144303.

, Zhu

and Lu

, Principal component analysis based on block-norm minimization, Appl Intell 49(6) (2019), 2169–2177.

Menon

T.V.

and Kalyani

, Structured and Unstructured Outlier Identification for Robust PCA: A Fast Parameter Free Algorithm, IEEE Transactions on Signal Processing 67(9) (2019), 2439–2452.

and Kanade

, Robust L1-norm factorization in the presence of outliers and missing data by alternative convex programming, IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2005), 739–746.

Brooks

J.P.

, Dul

J.H.

and Boone

E.L.

, A pure L1-norm principal component analysis, Computational Statistics and Data Analysis (2013), 83–98.

Ding

, Zhou

, He

X.F.

and Zha

H.Y.

, R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization, Proc 23rd Int Conf Machine Learning (2006), 281–288.

10.

Kwak

, Principal component analysis based on L1-norm maximization, IEEE Transactions on Pattern Analysis and Machine Intelligence (2008), 1672–1680.

11.

Kwak

, Principal component analysis by Lp-norm maximization, IEEE Transactions on Cybernetics (2014), 594–609.

12.

, Jiusun

and Lei

, Structured Joint Sparse Principal Component Analysis for Fault Detection and Isolation, IEEE Transactions on Industrial Informatics 15(5) (2019), 2721–2731.

13.

Journee

, Nesterov

, Richtarik

and Sepulchre

, Generalized power method for sparse principal component analysis, Journal of Machine Learning Research 11(2008070) (2010), 517–553.

14.

Zou

, Hastie

and Tibshirani

, Sparse principal component analysis, Journal of Computational and Graphical Statistics (2006), 265–286.

15.

Sigg

C.D.

and Buhmann

J.M.

, Expectation maximization for sparse and non-negative PCA, ACM, Helsinki, Finland, (2008), 960–967.

16.

Meng

D.Y.

, Zhao

and Xu

Z.B.

, Improve robustness of sparse PCA by L1-norm maximization, Pattern Recognition 45(1) (2012), 487–497.

17.

C.H.

, Chen

and Shao

, Robust sparse Lp-norm Principal Component Analysis, Acta Automatica Sinica 43(1) (2017), 142–151.

18.

Yang

and Qian

, A sparse logistic regression framework by difference of convex functions programming, Applied Intelligence 45 (2016), 241–254.

19.

Nguyen

and Baets

, An approach to supervised distance metric learning based on difference of convex functions programming, Pattern Recognition 81 (2018), 562–574.

20.

Yang

and Siyun

, A sparse extreme learning machine framework by continuous optimization algorithms and its application in pattern recognition, Engineering Applications of Artificial Intelligence 53 (2016), 176–189.

21.

Yang

, Ren

, Wang

, et al., A Robust Regression Framework with Laplace Kernel-Induced Loss, Neural Computation 29(11) (2017), 3014–3039.