Tensor data are becoming more and more common in machine learning. Compared with vector data, the curse of dimensionality of tensor data is more serious. The motivation of this paper is to combine Hilbert-Schmidt Independence Criterion (HSIC) and tensor algebra to create a new dimensionality reduction algorithm for tensor data. There are three contributions in this paper. (1) An HSIC-based algorithm is proposed in which the dimension-reduced tensor is determined by maximizing HSIC between the dimension-reduced and high-dimensional tensors. (2) A tensor algebra-based algorithm is proposed, in which the high-dimensional tensor are projected onto a subspace and the projection coordinate is set to be the dimension-reduced tensor. The subspace is determined by minimizing the distance between the high-dimensional tensor data and their projection in the subspace. (3) By combining the above two algorithms, a new dimensionality reduction algorithm, called PDMHSIC, is proposed, in which the dimensionality reduction must satisfy two criteria at the same time: HSIC maximization and subspace projection distance minimization. The proposed algorithm is a new attempt to combine HSIC with other algorithms to create new algorithms and has achieved better experimental results on 8 commonly-used datasets than the other 7 well-known algorithms.
KoldaT.G. and BaderB.W., Tensor decompositions and applications, SIAM Rev51(3) (2009), 455–500.
2.
SchölkopfB., SmolaA. and MüllerK.-R., Nonlinear component analysis as a kernel eigenvalue problem, Neural Comput10(5) (1998), 1299–1319.
3.
MikaS., RatschG., WestonJ., SchölkopfB. and MullersK.R., Fisher discriminant analysis with kernels, in Proc. IEEE Signal Process. Soc. Workshop Neural Netw. Signal Process., (1999), 41–48.
4.
KreyszigE., Introductory Functional Analysis With Applications1. New York, NY, USA: Wiley, (1978).
5.
HuM., ChenY. and KwokJ.T.-Y., Building sparse multiplekernel SVM classifiers, IEEE Trans Neural Netw20(5) (2009), 827–839.
6.
Shawe-TaylorJ. and CristianiniN., Kernel Methods for Pattern Analysis, Cambridge, U.K.: Cambridge Univ. Press, (2004).
7.
HeinM. and BousquetO., Kernels, associated structures and generalizations, Max-Planck-Inst Biologische Kybernetik Tübingen Germany Tech Rep127 (2004).
8.
MoodA.M., Introduction to the Theory of Statistics, New York, NY, USA: McGraw-Hill, (1950).
9.
BerrettiS., del BimboA. and PalaP., 3D face recognition using is geodesic stripes, IEEE Trans Pattern Anal Mach Intell32(12) (2010), 2162–2177.
10.
BessaoudiM., BelahceneM., OuamaneA.et al., A novel approach based on high order tensor and multi-scale locals features for 3D face recognition[C]//, 2018 4th International Conference on Advanced Technologies for Signal and Image Processing (ATSIP). IEEE2018, 1–5.
11.
YaoC., HanJ., NieF.et al., Local regression and global information-embedded dimension reduction[J], IEEE Transactions on Neural Networks and Learning Systems29(10) (2018), 4882–4893.
12.
ZhangJ., YuJ. and TaoD., Local deep-feature alignment for unsupervised dimension reduction[J], IEEE Transactions on Image Processing27(5) (2018), 2420–2432.
13.
VoulodimosA., DoulamisN., DoulamisA.et al., Deep learning for computer vision: A brief review[J], Computational intelligence and neuroscience2018, 2018.
14.
LeCunY., BengioY. and HintonG., Deep learning, Nature521 (2015), 436–444.
15.
LuiY.M., Human gesture recognition on product manifolds, J Mach Learn Res13(1) (2012), 3297–3321.
16.
BessaM.A., BostanabadR., LiuZ.et al., A framework for datadriven analysis of materials under uncertainty: Countering the curse of dimensionality[J], Computer Methods in Applied Mechanics and Engineering320 (2017), 633–667.
17.
MakantasisK.et al., Common mode patterns for supervised tensor subspace learning, ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, (2019).
18.
HuW., GaoJ., XingJ.et al., Semi-supervised tensor-based graph embedding learning and its application to visual discriminant tracking[J], IEEE Transactions on Pattern Analysis and Machine Intelligence39(1) (2016), 172–188.
19.
ChenY.-L., HsuC.-T. and LiaoH.-Y.M., Simultaneous tensor decomposition and completion using factor priors, IEEE Trans Pattern Anal Mach Intell36(3) (2014), 577–591.
20.
DengY.J., LiH.C., PanL.et al., Modified tensor locality preserving projection for dimensionality reduction of hyperspectral images[J], IEEE Geoscience and Remote Sensing Letters15(2) (2018), 277–281.
21.
AnJ., LeiJ., SongY.et al., Tensor Based Multiscale Low Rank Decomposition for Hyperspectral Images Dimensionality Reduction[J], Remote Sensing11(12) (2019), 1485.
22.
DengY.J., LiH.C., FuK.et al., Tensor low-rank discriminant embedding for hyperspectral image dimensionality reduction[J], IEEE Transactions on Geoscience and Remote Sensing56(12) (2018), 7183–7194.
23.
PanL., LiH.C., DengY.J.et al., Hyperspectral dimensionality reduction by tensor sparse and low-rank graph-based discriminant analysis[J], Remote Sensing9(5) (2017), 452.
24.
KimY.-D. and ChoiS., Nonnegative Tucker Decomposition[C]// 2007 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2007), (2007), 18–23. Minneapolis, Minnesota, USA. IEEE, 2007.
25.
De La TorreF. and BlackM.J., A framework for robust subspace learning[J], International Journal of Computer Vision54(1–3) (2003), 117–142.
26.
GrettonA., BousquetO., SmolaA.et al., Measuring statistical dependence with Hilbert-Schmidt norms[C]//, International conference on algorithmic learning theory. Springer, Berlin, Heidelberg2005, 63–77.
27.
CaiD., HeX., HanJ.et al., Graph regularized nonnegative matrix factorization for data representation[J], IEEE Transactions on Pattern Analysis and Machine Intelligence33(8) (2010), 1548–1560.
28.
ZhangZ. and ZhaoK., Low-rank matrix approximation with manifold regularization[J], IEEE Transactions on Pattern Analysis and Machine Intelligence35(7) (2012), 1717–1729.
29.
SunY., GaoJ., HongX.et al., Heterogeneous tensor decomposition for clustering via manifold optimization[J], IEEE Transactions on Pattern Analysis and Machine Intelligence38(3) (2015), 476–489.
30.
LiX., NgM.K., CongG.et al., MR-NTD: Manifold regularization nonnegative tucker decomposition for tensor data dimension reduction and representation[J], IEEE Transactions on Neural Networks and Learning Systems28(8) (2016), 1787–1800.
31.
ZhangJ., LiX., JingP.et al., Low-Rank Regularized Heterogeneous Tensor Decomposition for Subspace Clustering[J], IEEE Signal Processing Letters25(3) (2018), 333–337.
32.
JiangB., DingC., TangJ.et al., Image representation and learning with graph-laplacian tucker tensor decomposition[J], IEEE Transactions on Cybernetics49(4) (2018), 1417–1426.
33.
ZareA., OzdemirA., IwenM.A.et al., Extension of PCA to higher order data structures: An introduction to tensors, tensor decompositions, and tensor PCA[J], Proceedings of the IEEE106(8) (2018), 1341–1358.
34.
AnJ., ZhangX. and JiaoL.C., Dimensionality reduction based on group-based tensor model for hyperspectral image classification[J], IEEE Geoscience and Remote Sensing Letters13(10) (2016), 1497–1501.
35.
ZhangS., MaZ. and GanW., Dimensionality Reduction for Tensor Data Based on Local Decision Margin Maximization[J], IEEE Transactions on Image Processing30 (2020), 234–248.
36.
QiuY., ZhouG., ZhangY.et al., Graph Regularized Nonnegative Tucker Decomposition for Tensor Data Representation[C]//, ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE2019, 8613–8617.
37.
LuoD., DingC. and HuangH., Are tensor decomposition solutions unique: On the global convergence HOSVD and parafac algorithms, in Proc. PAKDD, Shenzhen, China, (2011), 148–159.
38.
MaZ., ZhanZ., OuyangX.et al., Nonlinear Dimensionality Reduction Based on HSIC Maximization[J], IEEE Access2018, 1–1.
39.
SunJ.G., Eigenvalues of Rayleigh quotient matrices[J], Numerische Mathematik59(1) (1991), 603–614.
40.
ZhangG., MaZ. and HuangH., Dimensionality reduction of tensor based on subspace learning and local homeomorphism[J], Journal of Intelligent & Fuzzy Systems2020(Preprint), 1–15.
41.
PengX., MaZ. and XuH., Maximum Discriminant Difference Criterion for Dimensionality Reduction of Tensor Data[J], IEEE Access8 (2020), 193593–193607.
42.
ZhangS., MaZ. and GanW., Dimensionality Reduction for Tensor Data Based on Local Decision Margin Maximization[J], IEEE Transactions on Image Processing30 (2020), 234–248.
