Novel Convolutional Restricted Boltzmann Machine manifold learning inspired dynamic user clustering hybrid precoding for millimeter-wave massive multiple-input multiple-output systems

Abstract

Millimeter-wave massive multiple-input multiple-output is a key technology in 5G communication system. In particular, the hybrid precoding method has the advantages of being power efficient and less expensive than the full-digital precoding method, so it has attracted more and more attention. The effectiveness of this method in simple systems has been well verified, but its performance is still unknown due to many problems in real communication such as interference from other users and base stations, and users are constantly on the move. In this article, we propose a dynamic user clustering hybrid precoding method in the high-dimensional millimeter-wave multiple-input multiple-output system, which uses low-dimensional manifolds to avoid complicated calculations when there are many antennas. We model each user set as a novel Convolutional Restricted Boltzmann Machine manifold, and the problem is transformed into cluster-oriented multi-manifold learning. The novel Convolutional Restricted Boltzmann Machine manifold learning seeks to learn embedded low-dimensional manifolds through manifold learning in the face of user mobility in clusters. Through proper user clustering, the hybrid precoding is investigated for the sum-rate maximization problem by manifold quasi-conjugate gradient methods. This algorithm avoids the traditional method of processing high-dimensional channel parameters, achieves a high signal-to-noise ratio, and reduces computational complexity. The simulation result table shows that this method can get almost the best summation rate and higher spectral efficiency compared with the traditional method.

Introduction

Millimeter-wave (mmWave) massive MIMO (multiple-input multiple-output) technology gradually becomes a key technology in 5G communication due to its rich spectrum resources.^1–3 Due to the high carrier frequency, mmWave signal suffers from high propagation loss so that large-scale antenna arrays are leveraged for path compensation. ⁴ However, in massive MIMO system, the number of antennas at the transmitter and receiver is very large, ⁵ configuring a radio frequency (RF) chain for each antenna in the traditional all-digital solution requires a lot of hardware cost and causes the loss of power. In response to this problem, a hybrid scheme has emerged, considering the reduction of hardware requirements in spectrum efficiency (SE) and energy efficiency (EE).^6–8 However, the hybrid precoding scheme in wideband channels is currently a difficult problem to solve.

How to obtain the optimal precoding matrix is the key issue of hybrid precoding. For the case of large-scale antennas in mmWave communication, large-scale matrix calculations are usually required. ⁸ The difficulty of hybrid precoding is to reduce the complexity of the above situation. ⁹ Some advanced beam-space-based hybrid precoding algorithms have been studied.^10,11 Previous investigations^12–16 make full use of the sparsity of the beam space channel according to the sparse signal processing scheme. In the literature,^12,13 the problem is transformed into finding the optimal precoder with hybrid structure, and an algorithm based on the basis tracking method is proposed. A hybrid precoding scheme designed is proposed according to the Orthogonal Match Pursuit (OMP) algorithm in the literature,^14,15 which can make full use of channel sparsity. In the multi-user scenario, the low-complexity multi-user hybrid precoding of the mmWave system is studied. ¹⁶ A Kronecker decomposition hybrid beamforming (KDHB) method for multi-cell multi-user massive MIMO system based on sparse propagation path is proposed. ¹⁷

However, the resolution of the beam space is not infinite. Due to the existence of power leakage, the sparse channel is non-ideal, and there are many possible non-zero terms. Some papers consider hybrid precoding of interfering mmWave channels.^18,19 Dealing with interference is very challenging, because the number of antennas is large, and the high-complexity precoding matrix is difficult to implement. ²⁰ To address the high interference problem, a closed-form broadband hybrid precoding scheme was proposed in the literature.^21–24 An analytical framework of hybrid beamforming (AFHB) in multi-cell mmWave systems was proposed. ²⁵ A combination of analog and digital beamforming is adopted. The former is based on a phase shifter, and the latter is based on a regularized zero-forcing method.

Recently, scholars have proposed the manifold learning in mmWave massive MIMO systems. Yu et al. ²⁶ proposed a manifold optimization (MO)-based hybrid precoding algorithm with lower complexity. To replace the range of the constant envelope with a circular manifold, Chen ²⁷ proposed a Riemannian conjugate gradient manifold algorithm. In Mai et al., ²⁸ a Riemann vector perturbation manifold for a multi-user massive MIMO system was studied, in which the RF-baseband hybrid precoding was jointly arranged. A Riemannian trust-region Newton manifold (RTRNM) showed an improved method of beamforming in multi-cluster scenarios. ²⁹ The optimization beamforming is utilized to mitigate inter-cell interference by dividing multi-users into multi-clusters with spatial correlation. However, multi-user high-dimensional channels are not mapped into low-dimensional subspaces to achieve dimensionality reduction. Learning a form of a double digital beamforming schemes optimizes the network resource allocation in massive MIMO networks. ³⁰ Moe Thet et al. ³¹ analyze by greedy algorithm how fast-moving users in static and time-varying user clustering are executed according to the system sum-rate. The manifold learning algorithm is used to reduce the multi-user high-dimensional channels. It reduces the computational complexity while mitigating inter-cell interference-based fully digital beamforming. It focuses on the local linear spatial structure between user channels, and ignores the global spatial characteristics. And it is not possible to quickly analyze the global and local correlations between user channels in the case of moving users.

The traditional user precoding methods are not applicable to multiple users, although they optimize precoding using channel sparsity. In the multi-user scenario, the traditional method suffers from high precoding complexity, high channel dimensionality, and does not consider user mobility. Therefore, it is necessary to propose new algorithms that take into account these important problems in practical communication.

In this article, we propose a low-complexity hybrid precoding algorithm for dynamic user clustering in mmWave mass MIMO systems. Specifically, a large-scale number of antennas embedded in a low-dimensional subspace. The mmWave channel measurement results show that the mmWave has a diffuse scattering phenomenon on the surface of the rough scatterer, and the scattering range will increase as the wavelength decreases. ³² For scenarios where users are dense, when there is not enough space between users, diffuse scattering may cause adjacent users to receive signals of the same path. Therefore, it causes serious inter-user interference. Our goal is to design a mixed precoding matrix, so they manage intra-cell and inter-cell interference requires a lower channel knowledge, and can be used to achieve low-complexity mixed analog/digital architecture, that is, compared with a small number of RF chains, the number of antennas. In order to solve the set classification problem, a manifold discriminant analysis (MDA) ³³ is proposed. Set by each user is modeled as a manifold, we will issue expressed as clustering for multi-manifold learning. The manifold discriminative learning seeks to learn the embedding low-dimensional manifolds, wherein manifolds with different user cluster label better separation of high-dimensional partial space of each flow channel in the shape of the correlation is enhanced. Learning by discriminant manifold, the majority of high-dimensional mapping of the channel to a low-dimensional manifold, it is possible to fully utilize the potential of the high-dimensional channel spatial correlation. By transforming the non-linear problems of high-dimensional channels into global non-linearities and local non-linearities, the purpose of dimensionality reduction is achieved. In low-dimensional manifolds, the intra-cluster channels become more clustered and the separability of embedded features is enhanced. Facing the situation that users will move in different clusters, we introduce the novel Convolutional Restricted Boltzmann Machine (NCRBM) framework into our stream shape learning. By continuously updating the stream shape discriminations in clusters, we get the best state of stream shape learning for users in clusters. Through proper user clustering, the hybrid precoding is investigated for the sum-rate maximization problem by manifold quasi-conjugate gradient methods. ³⁴ To enhance the spectral efficiency of the system, the design of each cluster analog RF precoder should balance the optimizing self-transmission and the interference. The digital precoding matrix is obtained by Karush Kuhn Tucker (KKT).^35–37 Compared with the traditional method, the proposed method does not require the solution of large-scale channel parameters, and can achieve a high signal-to-noise ratio (SNR) while reducing the computational complexity. The results show that the algorithm can obtain close to the optimal sum-rate and quite high spectral efficiency.

The rest of this article is as follows. Section “System model and channel model” introduces the system model and channel model. Section “User clustering hybrid precoding scheme” introduces the algorithm for dimensionality reduction and the hybrid precoding algorithm in multi-user high-dimensional channel scenarios. Section “Simulation results” presents the simulation results. Section “Conclusion” part summarizes this article.

Notations

Upper and lower-case boldface letters represent the matrices and the vectors, respectively. $(·{)}^H$ , $(·{)}^{-1}$ , $(·{)}^T$ , $(·{)}^{*}$ , $\mathrm{tr}(·)$ , and $||·|{|}_F$ are the Hermitian transpose, inverse, transpose, complex conjugate, trace, and the Frobenius norm of a matrix, respectively. $E(·)$ is the expectation. $\mathrm{diag}(·)$ denotes the diagonal matrix. $\left|G\right|$ is the cardinality of the set $G$ .⊗ indicates the Kronecker product. $\mathrm{CN}(0,{\sigma }^2)$ represents the zero-mean complex Gaussian distribution with zero-mean and the variance ${\sigma }^2$ . $\mathrm{span}(Y)$ denotes the subspace spanned by the column vectors of $Y$ . $\nabla (·)$ indicates the gradient. Finally, $I_N$ denotes the $N\times N$ identity matrix.

System model and channel model

System model

We consider a hybrid mmWave massive MIMO system model consisting of B cells. We assume that a base station (BS) equipped with $N_t$ antenna and $N_{\mathrm{RF}}$ RF chains $(N_t\ge N_{\mathrm{RF}}\ge K)$ serves $K$ single-antenna users, as shown in Figure 1. To manage the interference and improve the data rate for users, the users are partitioned into $L$ clusters $G_1,\dots ,G_L$ with $g_i=\left|G_i\right|$ , $\sum _{i=1}^L{g}_i=K$ , and $G_i\cap G_{i\prime }=\varnothing$ , $\forall i\ne i\prime$ . $G_i$ is $i\mathrm{th}$ cluster, where $i=1,\dots ,L$ . The sets $\{G_1,\dots ,G_L\}$ are all user clusters.

Figure 1.

Hybrid mmWave massive MIMO system model.

Let $u_{b,i,k}$ , $k=1,\dots ,g_i$ denote the $k\mathrm{th}$ user of ${\mathcal{G}}_i$ in the $b\mathrm{th}$ cell $(b=1,2,\dots ,B)$ . Hybrid precoding consists of two parts: baseband domain digital precoding and radio frequency domain analog precoding. In the downlink system, the transmitted symbol first passes through the digital precoder, and the generated signal is fed back to the radio frequency chain. The output of the RF chain is analog precoded and then sent to the antenna element. The transmitted signal vector $x_{b,i,k}$ at the base station is first precoded with a digital precoding $W_{b,i,k}$ . The resulting signals are fed to analog precoding ${F_{b,}}_{i,k}$ . The received signal $y_{b,i,k}$ of user $u_{b,i,k}$ can be given by

$$\begin{array}{c}\multicolumn{1}{c}{y_{b,i,k}=h_{b,i,k}^H{F_{b,}}_{i,k}{W}_{b,i,k}{x}_{b,i,k}}\\ +\sum _{k\prime =1,k\prime \ne k}^{\left|G_i\right|}{h}_{b,i,k}^H{F_{b,}}_{i,k\prime }{W}_{b,i,k\prime }{x}_{b,i,k\prime }\hfill \\ +\sum _{i\prime =1,i\prime \ne i}^L{h}_{b,i\prime }^H{F_{b,}}_{i\prime }{W}_{b,i\prime }{x}_{b,i\prime }\hfill \\ +\sum _{b\prime =1,b\prime \ne b}^B{h}_{b\prime ,i\prime }^H{F_{b\prime ,}}_{i\prime }{W}_{b\prime ,i\prime }{x}_{b\prime ,i\prime }+n_{b,i,k}\hfill \end{array}$$

(1)

where $h_{b,i,k}\in C^{N_t}$ is the channel vector between the BS and user $u_{b,i,k}$ . $x_{b,i,k}\in C^{N_t}$ represents the transmit signal of user $u_{b,i,k}$ . $n_{b,i,k}~\mathrm{CN}(0,{\sigma }^2)$ is the spatially white additive Gaussian noise. ${F_{b,}}_{i,k}\in C^{N_t\times n_{\mathrm{RF},i}}$ is the analog precoding matrix that adaptively steers an $n_{\mathrm{RF},i}-\mathrm{dimensional}$ RF beamspace for the coverage of $G_i$ with $n_{\mathrm{RF},i}\ge g_i$ .

$W_{b,i,k}\in C^{n_{\mathrm{RF},i}}$ is the digital precoding matrix. $C$ is the set of complex numbers. $h_{b,i,k}^H{F_{b,}}_{i,k}{W}_{b,i,k}{x}_{b,i,k}$ are the desired signal. $\sum _{k\prime =1,k\prime \ne k}^{\left|G_i\right|}{h}_{b,i,k}^H{F_{b,}}_{G_{i,k\prime }}{W}_{b,i,k\prime }{x}_{b,i,k\prime }$ are the intra-cluster interference. $\sum _{i\prime =1,i\prime \ne i}^L{h}_{b,i\prime }^H{F_{b,}}_{G_{i\prime }}{W}_{b,i\prime }{x}_{b,i\prime }$ are the inter-cluster interference. $\sum _{b\prime =1,b\prime \ne b}^B{h}_{b\prime ,i\prime }^H{F_{b\prime ,}}_{G_{i\prime }}{W}_{b\prime ,i\prime }{x}_{b\prime ,i\prime }$ are the inter-cell interference. The result of the hybrid method is more accurate than the statistical method, and the method can get faster and more generalized results, but it cannot provide enough accuracy in modeling the intra-cluster angle, which is necessary for beamforming and inter-cluster interference optimization.^28,38

Channel model

In order to take advantage of the unique spatial selectivity or scattering characteristics of the mmWave massive MIMO channel, this article adopts the Saleh–Valenzuela (SV) model, ³⁵ where the channel matrix of the user in cluster can be expressed as

$$h_{i,k}=\sum _{l=1}^{N_l}{\alpha }_{i,l}{\mathit{a}}_r\left({\theta }_{r,i,l}\right){\mathit{a}}_t\left({\theta }_{t,i,l}\right)$$

(2)

where $N_l$ indicates the number of paths, ${\alpha }_l$ is the complex gain in the $l\mathrm{th}$ path, ${\mathit{a}}_r({\theta }_{r,i,l})$ and ${\mathit{a}}_t({\theta }_{t,i,l})$ are the user and the base station array response vectors, respectively, where ${\theta }_{r,i,l}$ indicates the angle of arrival (AoA) at the user, and ${\theta }_{t,i,l}$ is the angle of departure (AoD) at the base station. For a simple N-element uniform linear antenna array, the response vector is

$$\mathit{a}\left(\theta \right)=\frac{1}{\sqrt{N}}{[1,e^{j(2\pi /\lambda )d_{\mathrm{ULA}}\sin \left(\theta \right)},\dots ,e^{j(N-1)j(2\pi /\lambda )d_{\mathrm{ULA}}\sin \left(\theta \right)}]}^T$$

(3)

where $\lambda$ is the wavelength, and $d_{\mathrm{ULA}}$ denotes the spacing between antennas. Due to the limited space during scattering of during mmWave propagation, the mmWave massive MIMO channel $h_{i,k}$ is low-rank. Therefore, we can use a limited number of RF chains to obtain near-optimal throughput.

User clustering hybrid precoding scheme

Our goal is to design a hybrid precoding matrix. Therefore, we must first deal with intra-cluster, inter-cluster, and inter-cell interference with less known channel knowledge, and second, we need to use little RF chains to complete the hybrid analog/digital architecture, avoiding the high complexity of traditional methods. Next, we propose a hybrid precoding method based on manifold learning to achieve the above goals.

NCRBM manifold learning for user clusters

With the increase of antennas and users in the mmWave massive MIMO system, inter-cell and intra-cell directional interference will occur during signal transmission. The high-dimensional channel matrix requires high-complexity hybrid analog/digital architectures. By modeling each user set as a manifold, we formulate the problem as clustering-oriented manifold discriminative learning.

The undirected similarity graph of multi-users is represented by the graph embedding method. To represent each user set as a manifold, the user channel characteristic graphs $\{(h_{i,k},m_{k,j}){\}}_{i=1}^L$ are constructed, as shown in Figure 2.

Figure 2.

User cluster undirected characteristic graph.

$\mu {\prime }_{i,k}^{\left(0\right)}$ represents the intra-cluster channel weight function between users $k$ and user $j$ . $m_{\zeta ,k,j}$ represents the inter-cluster channel weight function between users $k$ and $j$ . The sets of the cluster channel weight functions are $M=\{m_{k,j}:k,j\in (1,\dots ,K)\}$ .

The weight function $m_{\xi ,k,j}$ of the intra-cluster is defined as follows

$$\{\begin{array}{cc}0<m_{\xi ,k,j}\le 1,\hfill & k,j\mathrm{in}\mathrm{the}\mathrm{intra}-\mathrm{cluster}\hfill \\ \multicolumn{1}{c}{0,}& \mathrm{otherwise}\hfill \end{array}$$

(4)

The weight function $m_{\zeta ,k,j}$ of the inter-cluster is defined as follows

$$\{\begin{array}{cc}0<m_{\zeta ,k,j}\le 1,\hfill & k,j\mathrm{in}\mathrm{the}\mathrm{inter}-\mathrm{cluster}\hfill \\ \multicolumn{1}{c}{0,}& \mathrm{otherwise}\hfill \end{array}$$

(5)

The weight functions of the intra-cluster show that when users $k$ and $j$ are the same cluster, the weight is larger; when users $k$ and $j$ are the different cluster, the weight is 0. The weight functions of the inter-cluster show that when users $k$ and $j$ are different cluster, the weight is larger; when users $k$ and $j$ are the same cluster, the weight is 0. The manifold discriminative learning seek to learn the embedding low-dimensional manifolds, where manifolds with labels of different user groups can be separated more conveniently, and the local spatial correlation of the high-dimensional channels within each manifold is enhanced. Some existing manifold learning algorithms, such as Locally Linear Embedding (LLE), ³⁹ cannot retain the complete global non-linear channel structure of user clusters.

We propose to perform the manifold discriminative learning for global dimensionality reduction. The high-dimensional channels are mapped in the low-dimensional manifolds, as shown in Figure 3. In order to reveal the potential non-linear manifold structure of high-dimensional channels, intra-cluster graph and inter-cluster graph are constructed using the label information of user characteristics. In addition, it can make the low-dimensional channels more clustered, and enhance the separability of embedded low-dimensional channels. The radio frequency eigen-beamformer is considered to be the best solution for user group transmission. The channel eigenvector learning corresponding to the maximum eigenvalue is taken as the spatial direction. In theory, the main direction learned is the beamforming. Multi-users of the same cluster have highly correlated transmission paths. We seek to learn a generic mapping $A$ that is defined as

$$h{\prime }_k=A^T{h}_k$$

(6)

where $A$ is projection matrix, $h{\prime }_k$ is the $k\mathrm{th}$ user low-dimensional mapping of the high-dimensional channels $h_k$ . The original high-dimensional channels $h_k$ can be transformed into a low-dimensional channel $h{\prime }_k$ . The relative spatial relationship of neighboring users in high-dimensional channels remains unchanged in low-dimensional manifolds. In order to maintain the manifold structure of the high-dimensional channels, the optimization problem is the projection direction of manifold, that is, $\forall h_k,h_j(k\ne j)$ of the intra-cluster, the target function of the intra-cluster can be obtained as

$$\underset{A}{max}\sum _{k,j}{({h\prime }_k-{h\prime }_j)}^2{m}_{\xi ,k,j}$$

(7)

Figure 3.

Schematic diagram of dimension reduction.

The projection can maximize the use of all users in the cluster of the intra-cluster as equation (8), where $S_{\xi ,\mathrm{local}}=H(D_{\xi }-M_{\xi })H^{\mathrm{T}}$ is the local manifold structure of the intra-cluster, and $D_{\xi }$ is the diagonal matrix and $D_{\xi }=\sum _{k\ne j}{M}_{\xi }(k,j)$

$$\begin{array}{c}\underset{A}{max}\sum _{k,j}{({h\prime }_k-{h\prime }_j)}^2{m}_{\xi ,k,j}\hfill \\ =\underset{A}{max}\frac{1}{2}\sum _{k,j}{(A^T{h}_k-A^T{h}_j)}^2{m}_{\xi ,k,j}\hfill \\ =\underset{A}{max}{A}^{\mathrm{T}}H{D}_{\xi }{H}^{\mathrm{T}}A-A^{\mathrm{T}}H{M}_{\xi }{H}^{\mathrm{T}}A\hfill \\ =\underset{A}{max}{A}^{\mathrm{T}}{S}_{\xi ,\mathrm{local}}A\hfill \end{array}$$

(8)

According to the SV model, $R_{i,k}=E[h_{i,k}{h}_{i,k}^{\mathrm{H}}]$ is the covariance matrix of the $k\mathrm{th}$ user in the $i\mathrm{th}$ cluster. The transmission covariance matrix of users in a cluster is the same, so, $R_{i,k}$ , that is

$$R_{i,k}=U_{i,k}{\Lambda }_{i,k}{U}_{i,k}^H$$

(9)

where $U_{i,k}\in C^{N_t\times r_i}$ is a matrix of eigenvectors corresponding to $r_i(r_i=N_{\mathrm{t}})$ non-zero eigenvalues of $R_{i,k}$ . ${\Lambda }_{i,k}$ is the diagonal matrix whose elements are the non-zero eigenvalues of $R_{i,k}\in C^{N_t\times N_t}$ corresponding to the non-zero eigenvalues, satisfying ${\Lambda }_k\in C^{r\times r}$ . Since users in the same user cluster have similar spatial correlations, they have similar local scattering, $R_i=R_{i,k},\forall k\in G_i$ . Measure of similarity between the user and the similarity criterion is a function of the distance function coefficients. Since $\mathrm{span}(U)=U{U}^T$ , $\forall U_{i,k},U_{i\prime ,k\prime }$ , the similarity measurement function between any two users based on the distance of subspace projection matrix can be expressed as

$$\begin{array}{c}{d}_{\mathrm{pm}}(U_k{U}_k^T,V_i{V}_i^T)\hfill \\ =\frac{1}{\sqrt{2}}\Vert U_k{U}_k^T-V_i{V}_i^T{\Vert }_F^2\hfill \\ =\frac{1}{\sqrt{2}}\mathrm{tr}\left({\psi }_{k,i}{\psi }_{k,i}^T\right)\hfill \end{array}$$

(10)

where $U_k$ is the eigenvectors matrix of $R_k$ in any cluster, that is, $R_k=U_k{\Lambda }_k{U}_k^H$ , and $V_i$ is the eigenvectors matrix of the $i\mathrm{th}$ cluster center $R_i$ . ${\psi }_{k,i}=U_k{U}_k^T-V_i{V}_i^T$ is the symmetric positive semi-definite matrix that needs to be learned. The global manifold structure $S_{\xi ,\mathrm{global}}$ of intra-cluster is measured as

$$S_{\xi ,\mathrm{global}}=\sum _{i=1}^L\sum _{k\in G_i}\frac{1}{\sqrt{2}}\frac{1}{g_i}\mathrm{tr}\left({\psi }_{k,i}{\psi }_{k,i}^T\right)$$

(11)

To effectively utilize the global characteristics and local manifold structure of intra-cluster channels, we can get the intra-cluster dispersion ${\eta }_{\xi }$ by combining equations (9) and (11) ³³

$${\eta }_{\xi }=\upsilon S_{\xi ,\mathrm{global}}+(1-\upsilon )S_{\xi ,\mathrm{local}}$$

(12)

where $\upsilon$ is the constants.

The weight functions $m_{\xi ,k,j}$ of the intra-cluster can be obtained as

$$m_{\xi ,k,j}=\exp \left(\frac{-d_{k,j}}{s\prime }\right)$$

(13)

where $s\prime$ is the constants, and $d_{k,j}$ is the similarity measurement function between user $k$ and user $j$ .

In order to maintain the manifold structure of the inter-cluster user channels, the optimization problem is the projection direction of manifold, that is, $\forall h_k,h_j(k\ne j)$ of the inter-cluster, the objective function of the inter-cluster can be obtained as

$$\underset{A}{max}\sum _{k,j}{({h\prime }_k-{h\prime }_j)}^2{m}_{\zeta ,k,j}$$

(14)

Therefore, the projection can maximize the use of all users of the inter-cluster, that is

$$\begin{array}{c}\underset{A}{max}\sum _{k,j}{({h\prime }_k-{h\prime }_j)}^2{m}_{\zeta ,k,j}\hfill \\ =\frac{1}{2}\underset{A}{max}\sum _{k,j}{(A^T{h}_k-A^T{h}_j)}^2{m}_{\zeta ,k,j}\hfill \\ =\underset{A}{max}{A}^{\mathrm{T}}H{D}_{\zeta }{H}^{\mathrm{T}}A-A^{\mathrm{T}}H{M}_{\zeta }{H}^{\mathrm{T}}A\hfill \\ =\underset{A}{max}{A}^{\mathrm{T}}{S}_{\zeta ,\mathrm{local}}A\hfill \end{array}$$

(15)

where $S_{\zeta ,\mathrm{local}}=H(D_{\zeta }-M_{\zeta })H^{\mathrm{T}}$ is the local manifold structure of the inter-cluster, and $D_{\zeta }$ is the diagonal matrix and $D_{\zeta }=\sum _{k\ne j}{M}_{\zeta }(k,j)$ . The global inter-cluster $S_{\zeta ,\mathrm{global}}$ is measured as

$$S_{\zeta ,\mathrm{global}}=\sum _{i=1}^L\sum _{k\notin G_i}\frac{1}{\sqrt{2}}\frac{1}{K-g_{i\prime }}\mathrm{tr}\left({\psi }_{k,i}{\psi }_{k,i}^T\right)$$

(16)

To effectively utilize the global characteristics and local manifold structure of inter-cluster channels, we can get the inter-cluster dispersion ${\eta }_{\zeta }$ by combining equations (15) and (16)

$${\eta }_{\zeta }=\wp S_{\zeta ,\mathrm{global}}+(1-\wp )S_{\zeta ,\mathrm{local}}$$

(17)

where ℘ is the constants.

The weight functions $m_{\xi ,k,j}$ of the inter-cluster can be obtained as

$$m_{\zeta ,k,j}=\exp \left(\frac{-s″}{d_{k,j}}\right)$$

(18)

where $s″$ is the constants.

After getting the intra-cluster dispersion ${\eta }_{\xi }$ and the inter-cluster dispersion ${\eta }_{\zeta }$ , the mobility of the user causes these two values to change continuously. To cope with this problem, we introduce the NCRBM model ⁴⁰ to acquire optimal value. The NCRBM is a multi-layer network constructed by Convolutional Restricted Boltzmann Machines (CRBMs) stacked on top of each other. NCRBM is an extension of the standard Restricted Boltzmann Machine (RBM) that has inherited all its properties but with faster operation. NCRBM optimize the intra-cluster dispersion value over time. In $i\mathrm{th}(i=1,...,L)$ cluster of users, let us denote the intra-cluster dispersion of $\mu$ time interval by $\{{\eta }_{\xi }^i{\}}_{i=1}^L$ , where ${\eta }_{\xi }^i=\{{\eta }_{\xi }^{t_0},{\eta }_{\xi }^{t_0+\mathrm{\Delta }t},...,{\eta }_{\xi }^{t_0+\mu \mathrm{\Delta }t}\}$ is a set of intra-cluster dispersions belonging to $i\mathrm{th}$ cluster. And $t_0$ is the start time, and $\mathrm{\Delta }t$ is the time interval. In addition, we define ${\tilde{\eta }}_{\xi }$ as the optimal intra-cluster dispersion after optimization. The optimal value is computed by averaging across manifold structure feature over all the intra-cluster dispersions belonging to set ${\eta }_{\xi }^i$ . An optimal value is a mean representation of a specific cluster in manifold structure feature space and must be calculated for each cluster separately. The energy of the joint configuration $({\eta }_{\xi }^i,{\tilde{\eta }}_{\xi })$ of the input intra-cluster dispersion and optimized units for an NCRBM with real-valued input units can be defined as follows

$$\begin{array}{l}E\left({\eta }_{\xi }^i,{\tilde{\eta }}_{\xi }^i\right)=\frac{1}{2}\sum _{u,v=1}^{\mu }{\left({\eta }_{\xi }^i\right)}_{u,v}^2\\ -\sum _{q=1}^Q\sum _{m,n=1}^{\mu }{\left({\tilde{\eta }}_{\xi }^i\right)}_{m,n}^q{\left(w^q\ast {\eta }_{\xi }^i\right)}_{m,n}\\ -\sum _{q=1}^Q{b}_q\sum _{m,n=1}^{\mu }{\left({\tilde{\eta }}_{\xi }^i\right)}_{m,n}^q-c\sum _{u,v=1}^{\mu }{\left({\eta }_{\xi }^i\right)}_{u,v}\end{array}$$

(19)

where * stands for the convolution; $w^q$ , $({\tilde{\eta }}_{\xi }^i{)}_{m,n}^q$ , and $({\eta }_{\xi }^i{)}_{u,v}$ denote the manifold structure feature detectors horizontally and vertically in filter q, the optimized unit on location $(m,n)$ , and the input intra-cluster dispersion unit on location $(u,v)$ , respectively. Also, $b_q$ is the shared bias among all units in feature map q, and $c$ is the bias value for input intra-cluster dispersion units.

The network assigns a probability to every possible pair of input and optimized unit through this energy function as follows

$$P({\eta }_{\xi }^i,{\tilde{\eta }}_{\xi }^i)=\frac{1}{Z}\exp (-E({\eta }_{\xi }^i,{\tilde{\eta }}_{\xi }^i))$$

(20)

where Z is the partition function.

The optimization of the model parameters can be performed by minimizing the following objective function using the Contrastive Divergence

$$\underset{w,b,c}{min}\{-\sum _{i=t_0}^{t_0+\mu \mathrm{\Delta }t}\log (\sum P({\eta }_{\xi }^i,{\tilde{\eta }}_{\xi }^i))+{\lambda }_{\mathrm{sparsity}}\sum _{q=1}^Q{|p-\frac{1}{{\mu }^2}\sum _{m,n=1}^{\mu }P({\left({\tilde{\eta }}_{\xi }^i\right)}_{m,n}^q=1|{\eta }_{\xi }^i)|}^2\}$$

(21)

The second term in equation (21) is the sparsity regularization proposed to prevent the model from being overcomplete. For each intra-cluster dispersion, ${\eta }_{\xi }^i$ , we have ${\tilde{\eta }}_{\xi }^i$

$${\tilde{\eta }}_{\xi }^i=\frac{1}{Q}\sum _{q=1}^{Q}P\left({\left({\tilde{\eta }}_{\xi }^i\right)}^q=1|{\eta }_{\xi }^i\right)$$

(22)

where $Q$ is the number of manifold structure features. Then, the optimal value for each cluster can be computed as follows

$${\tilde{\eta }}_{\xi }=\frac{1}{\mu }\sum _{i=t_0}^{t_0+\mu \mathrm{\Delta }t}{\tilde{\eta }}_{\xi }^i$$

(23)

NCRBM objective function is comprised of two main parts, for example, generative and optimized parts. Generative objective (the first two terms in equation (24)) is the same as sparsity regularized CRBM, while this way of optimization does not guarantee the best intra-cluster dispersion value. Optimization of the generative part is performed by minimizing Contrastive Divergence. The second two terms in equation (24) correspond to the optimized function, which can be optimized by following a gradient-based process. Minimizing the following objective function can get the optimal intra-cluster dispersion

$$\begin{array}{c}\underset{w,b,c}{min}\{-\sum _{i=t_0}^{t_0+\mu \mathrm{\Delta }t}\log (\sum P({\eta }_{\xi }^i,{\tilde{\eta }}_{\xi }^i))+{\lambda }_{\mathrm{sparsity}}\sum _{q=1}^Q{|p-\frac{1}{{\mu }^2}\sum _{m,n=1}^{\mu }P({\left({\tilde{\eta }}_{\xi }^i\right)}_{m,n}^q=1|{\eta }_{\xi }^i)|}^2\hfill \\ +{\lambda }_{\mathrm{dis}}\frac{1}{Q}\sum _{r=1}^L\sum _{q=1}^Q{e}_i\Vert P({\left({\tilde{\eta }}_{\xi }^i\right)}_{m,n}^q=1|{\eta }_{\xi }^i)-M_r{\Vert }_F^2-{\beta }_{\mathrm{dis}}\sum _{u,v=1}^L\Vert M_u-M_v{\Vert }_F^2\}\hfill \end{array}$$

(24)

$$e_l=\{\begin{array}{c}1\mathrm{if}{\tilde{\eta }}_{\xi }^i\in {\eta }_{\xi }^i\hfill \\ \multicolumn{1}{c}{0\mathrm{otherwise}}\end{array}$$

(25)

where L is the number of clusters. In equation (24), the third term tries to move the features of each intra-cluster dispersion in cluster r to its manifold structure $M_r$ , while the last term tries to maximize the distance between class-maps $M_u$ and $M_v$ for a better separation between clusters. The weighting parameters ${\lambda }_{\mathrm{dis}}$ and ${\beta }_{\mathrm{dis}}$ are used to adjust the amount of contribution of the optimized terms in the overall process.

The gradient of the optimized part can be computed exactly at each iteration. For each cluster, we summed the gradient contributions brought by the two components. Based on the definition in equation (23), update of intra-cluster dispersion in each cluster can be performed after optimizations. The limitation of this approach is that too long-time intervals can cause the obtained intra-cluster dispersion to be inaccurate. So, it needs to be run several times, which makes the overall computing time of the algorithm increase.

For inter-cluster dispersion, we use a similar approach to obtain the optimized inter-cluster values ${\tilde{\eta }}_{\zeta }$ . In this way, if the users in a cluster move within a certain time, we use the NCRBM model to obtain optimal intra-cluster dispersion and inter-cluster dispersion while maintaining the manifold structure feature within a certain time. However, the overhead of manifold learning increases due to the constant updating of the dispersion values for intra- and inter-cluster users. This is mainly due to the fact that the introduced NCRBM model requires some iterations.

The discriminative function $J(A)$ is transformed as

$$J\left(A\right)=\underset{A}{max}\frac{A^T{\tilde{\eta }}_{\xi }A}{A^T{\tilde{\eta }}_{\zeta }A}$$

(26)

$$\begin{array}{cc}\multicolumn{1}{c}{J\left(A\right)}& =\underset{A}{max}\frac{A^{\mathrm{T}}(\tilde{\upsilon }{\tilde{S}}_{\xi ,\mathrm{whole}}+(1-\tilde{\upsilon }){\tilde{S}}_{\xi ,\mathrm{local}})A}{A^{\mathrm{T}}(\tilde{\wp }{\tilde{S}}_{\zeta ,\mathrm{whole}}+(1-\tilde{\wp }){\tilde{S}}_{\zeta ,\mathrm{local}})A}\hfill \\ \multicolumn{1}{c}{s.t.{h\prime }_{i,k}}& =A^T{h}_{i,k}\hfill \end{array}$$

(27)

According to equation (27), the low-dimensional mapping of the $k\mathrm{th}$ user channel matrix $h{\prime }_{i,k}$ is determined by the projection matrix $A$ . The Lagrange multipliers are introduced to transform the original optimization problem into a Lagrangian function problem to find the optimal projection matrix $A$ with intra-cluster and inter-cluster dispersion values. ⁴¹ By solving the generalized eigenvalues of the discriminative function, we can obtain the projection matrix $A=[A_1,\dots ,A_n]$ . $n$ is the dimensionality reduction of user channel matrix. After user clustering, the channel correlation of users in the same cluster is enhanced.

Then, according to the intra-cluster graph and inter-cluster graph constructed using the label information of user characteristics, the user clusters can be divided more accurately with lower complexity. Based on the maximum and minimum distances and the weighted likelihood similarity criterion, an optimized spatial fuzzy c-means clustering algorithm is proposed. The algorithm is an iterative optimization that minimizes the cost function defined as follows

$$J\left({\mu }_{i,k}\right)=\sum _{k=1}^K\sum _{i=1}^L{\mu }_{i,k}^{\Im }{d}_{i,k}$$

(28)

where $d_{i,k}=(1/\sqrt{2})\mathrm{tr}({\psi }_{k,i}{\psi }_{k,i}^T)$ is the similarity measurement function between the $k\mathrm{th}$ user and the $i\mathrm{th}$ cluster center. ${\mu }_{i,k}$ represents the membership function of user $u_{i,k}$ in the $i\mathrm{th}$ cluster, and ℑ is a constant. The parameter ℑ controls the ambiguity of the result partition; in this article, we set $\Im =2$ . When assigning high membership values to users $u_{i,k}$ close to the cluster center and assigning low membership values to users $u_{i,k}$ far away from the cluster center, the cost function $J({\mu }_{i,k})$ is the smallest. The membership function represents the probability that a user $u_{i,k}$ belongs to a specific cluster. Member functions and cluster center are updated as follows

$${\mu }_{i,k}=\frac{1}{\sum _{i\prime =1}^L{\left(\frac{d_{i,k}}{d_{i,i\prime }}\right)}^{\frac{1}{(\Im -1)}}}$$

(29)

and

$$V_{i,k}{V}_{i,k}^T=\frac{\sum _{k=1}^K{\mu }_{i,k}^{\Im }{U}_{i,k}{U}_{i,k}^T}{\sum _{k=1}^K{\mu }_{i,k}^{\Im }}$$

(30)

where $d_{i,i\prime }=(1/\sqrt{2})\mathrm{tr}({\psi }_{i,i\prime }{\psi }_{i,i\prime }^T)$ is the similarity measurement function between the $i\mathrm{th}$ cluster center and the $i\prime \mathrm{th}$ cluster center. In summary, to represent each user set as a manifold, the process of clustering-oriented manifold discriminative learning is as follows:

• Step 1: construct the user channel characteristic graphs $\{(h_{i,k},m_{k,j}){\}}_{i=1}^L$ .

• Step 2: find the two farthest distances $U_i$ and $U_{i\prime }$ , and their center as the first user group, that is, $V_1^{(0)}=U_i,V_2^{(0)}=U_{i\prime }$ . The number of the user clusters is $i=2$ .

• Step 3: from the Euclidean distance criterion $d_{\mathrm{pm}}(U_k{U}_k^T,V_i{V}_i^T)=(1/\sqrt{2})\mathrm{tr}({\psi }_{k,i}{\psi }_{k,i}^T)$ , all users are gathered in the $i$ user cluster.

• Step 4: among the $i$ user groups that have completed clustering, find the weakest similarity point in each user group to obtain the $i$ user clusters. Calculate the sum distance $d_{i,k}$ between the user $k(k=1,2,...,K)$ , the membership functions ${\mu }_{i,k}^{(0)}$ , and the center point $V_i^{(0)}(i=1,2,...,L)$ of each user cluster in turn.

• Step 5: calculate the spatial membership function and update the center point $V_i^{(0)}(i=1,2,...,L)$ of each user cluster with

$$V_{i,k}{V}_{i,k}^T=\frac{\sum _{k=1}^K{\left({\mu \prime }_{i,k}\right)}^{\Im }{U}_{i,k}{U}_{i,k}^T}{\sum _{k=1}^K{\left({\mu \prime }_{i,k}\right)}^{\Im }}$$

Then, the maximum value among $d_{i,k}$ is found. $V_{i+1}^{(0)}=\arg \underset{k}{max}{d}_{i,k}$ . All users entering $(i+1)$ are re-divided into different user clusters.

• Step 6: if $i=i+1\ge L$ is true, perform Step 5; otherwise repeat Step 3.

• Step 7: $\Vert {(U_k{\Sigma }_k^{1/2})}^H{V}_i{\Vert }_F^2$ is calculated, each user is arranged to the user clusters with the largest similarity coefficient.

• Step 8: output cluster result, and the number of users in each cluster.

• Step 9: calculate the $m_{\xi ,k,j}$ and $m_{\zeta ,k,j}$ according to equations (13) and (18); construct intra-cluster graph and inter-cluster graph using the label information of user characteristics.

• Step 10: calculate the $S_{\zeta ,\mathrm{whole}}$ , $S_{\xi ,\mathrm{local}}$ , $S_{\zeta ,\mathrm{local}}$ , and $S_{\zeta ,\mathrm{whole}}$ according to equations (9), (11), (15), and (16).

• Step 11: calculate the ${\eta }_{\zeta }$ and ${\eta }_{\xi }$ according to equations (12) and (17).

• Step 12: calculate the optimized ${\tilde{\eta }}_{\xi }$ and ${\tilde{\eta }}_{\zeta }$ according to equation (23).

• Step 13: optimize the discriminative function $J(A)$ according to equation (27).

• Step 14: according to the obtained projection matrix, get the projection in low-dimensional subspace $h^{\prime }$ .

Manifold discriminative learning for hybrid precoding

On the basis of manifold discriminative learning for global dimensionality reduction and user clustering, we investigate the sum-rate maximization problem for hybrid precoding. In order to design the precoding matrix $F{\prime }_{G_i}W{\prime }_{G_i}$ , the sum-rate maximization of mixed precoding g is studied, such that they manage intra-cluster interference and inter-cluster interference. To improve the spectral efficiency of the systems, the design of each cluster analog precoding should balance optimizing self-transmission and interference. By representing each user set as a manifold, the received signal of the cluster can be represented as

$$\begin{array}{c}\multicolumn{1}{c}{{y\prime }_{G_i}=H{\prime }_{G_i}^H{F\prime }_{G_i}{W\prime }_{G_i}{x}_{G_i}}\\ +\sum _{k\prime =1,k\prime \ne k}^{\left|G_i\right|}H{\prime }_{G_i,k}^H{F\prime }_{G_i,k\prime }{W\prime }_{G_i,k\prime }{x}_{G_i,k\prime }\hfill \\ +\sum _{i\prime =1,i\prime \ne i}^{L}H{\prime }_{G_{i\prime }}^H{F\prime }_{G_{i\prime }}{W\prime }_{G_{i\prime }}{x}_{G_{i\prime }}+n_{G_i}\hfill \end{array}$$

(31)

where $y{\prime }_{G_i}=[y{\prime }_{G_i,1}^T,\dots ,y{\prime }_{G_i,g_i}^T{]}^T$ represents the received signal, $H{\prime }_{G_i}=[H{\prime }_{G_i,1},\dots ,H{\prime }_{G_i,g_i}]$ represents the channel matrix for the $i\mathrm{th}$ cluster, $F{\prime }_{G_i}=[F{\prime }_{G_i,1},\dots ,F{\prime }_{G_i,g_i}]$ , and $W{\prime }_{G_i}=\mathrm{diag}(W{\prime }_{G_i,1},\dots ,W{\prime }_{G_i,g_i})$ . $H{\prime }_{G_i}^{H}F{\prime }_{G_i}W{\prime }_{G_i}{x}_{G_i}$ is the desired signal, $\sum _{k\prime =1,k\prime \ne k}^{\left|G_i\right|}{H}_{G_i,k}{\prime }^H{F\prime }_{G_{i,k\prime }}{W\prime }_{G_i,k\prime }{x}_{G_i,k\prime }$ is the intra-cluster interference, $\sum _{i\prime =1,i\prime \ne i}^{L}H{\prime }_{G_{i\prime }}^H{F\prime }_{G_{i\prime }}{W\prime }_{G_{i\prime }}{x}_{G_{i\prime }}$ is the inter-cluster interference after the low-dimensional mapping. In order to adapt to special scenarios and requirements, the hybrid precoding matrix can be determined by per-cluster processing (PCP). The goal of PCP is to balance the performance and complexity by effectively separating the clusters in the RF beam domain.

In PCP mode, the analog precoding matrix $F{\prime }_{G_i}$ of each cluster is calculated according to manifold quasi-conjugate gradient algorithm, while the digital precoding matrix $W{\prime }_{G_i}$ is calculated by each user cluster according to their equivalent channel matrix. Let $H{\prime }_{\mathrm{eq}}=H{\prime }^{H}F\prime$ denote the equivalent channel matrix after analog precoding, and it is an approximate block diagonal matrix, which can be expressed as

$$H{\prime }_{\mathrm{eq}}=\left[\begin{array}{cccc}H{\prime }_{G_1}^H{F\prime }_{G_1}\hfill & H{\prime }_{G_1}^H{F\prime }_{G_2}\hfill & \dots \hfill & H{\prime }_{G_1}^H{F\prime }_{G_L}\hfill \\ \multicolumn{1}{c}{H{\prime }_{G_2}^H{F\prime }_{G_1}}& H{\prime }_{G_2}^H{F\prime }_{G_2}\hfill & \dots \hfill & H{\prime }_{G_2}^H{F\prime }_{G_L}\hfill \\ \multicolumn{1}{c}{\dots }& \dots \hfill & \dots \hfill & \dots \hfill \\ \multicolumn{1}{c}{H{\prime }_{G_L}^H{F\prime }_{G_1}}& H{\prime }_{G_L}^H{F\prime }_{G_2}\hfill & \dots \hfill & H{\prime }_{G_L}^H{F\prime }_{G_L}\hfill \end{array}\right]$$

(32)

where $H{\prime }_{e{q}_{G_i}}=H{\prime }_{G_i}^{H}F{\prime }_{G_i}$ represents the diagonal elements of the matrix in equation (32), and off-diagonal elements of the matrix $H{\prime }_{G_i}^{H}F{\prime }_{G_{i\prime }}(i\ne i\prime )$ represents the interference channel matrix between user clusters. After analog precoding, the inter-cluster interference is eliminated, that is, $H{\prime }_{G_i}^{H}F{\prime }_{G_j}\approx 0$ . $H{\prime }_{\mathrm{eq}}$ can be expressed as

$$H{\prime }_{\mathrm{eq}}=\left[\begin{array}{cccc}H{\prime }_{G_1}^H{F\prime }_{G_1}\hfill & 0\hfill & \dots \hfill & 0\hfill \\ \multicolumn{1}{c}{0}& H{\prime }_{G_2}^H{F\prime }_{G_2}\hfill & \dots \hfill & 0\hfill \\ \multicolumn{1}{c}{\dots }& \dots \hfill & \dots \hfill & \dots \hfill \\ \multicolumn{1}{c}{0}& 0\hfill & \dots \hfill & H{\prime }_{G_L}^H{F\prime }_{G_L}\hfill \end{array}\right]$$

(33)

The digital precoding matrix $W\prime$ is a block diagonal matrix as follows

$$W\prime =\mathrm{diag}({W\prime }_{G_1},\dots ,{W\prime }_{G_L})$$

(34)

With scalar equalization ${\beta }_{G_i}^{-1}$ , the signal estimate ${\hat{x}}_{G_i}$ for $G_i$ can be expressed as

$${\hat{x}}_{G_i}={\beta }_{G_i}^{-1}(H{\prime }_{G_i}^H{F\prime }_{G_i}{W\prime }_{G_i}{x}_{G_i}+\sum _{k\prime =1,k\prime \ne k}^{\left|G_i\right|}H{\prime }_{G_i,k}^H{F\prime }_{G_{i,k\prime }}{W\prime }_{G_i,k\prime }{x}_{G_i,k\prime }+\sum _{i\prime =1,i\prime \ne i}^{L}H{\prime }_{G_{i\prime }}^H{F\prime }_{G_{i\prime }}{W\prime }_{G_{i\prime }}{x}_{G_{i\prime }}+n_{G_i})$$

(35)

where ${\beta }_{G_i}$ is a scaling equalization that is jointly optimized with the hybrid precoding. The conditional mean square error (MSE) for $G_i$ is

$$\begin{array}{c}\epsilon ({F\prime }_{G_i},{W\prime }_{G_i},{\beta }_{G_i})=E\left[{\Vert x_{G_i}-{\hat{x}}_{G_i}\Vert }^2\right]\hfill \\ =E\left[{\Vert x_{G_i}-{\beta }_{G_i}^{-1}\left(H{\prime }_{G_i}^H{F\prime }_{G_i}{W\prime }_{G_i}{x}_{G_i}\right)\Vert }^2\right]\hfill \\ +E[\sum _{k\prime =1,k\prime \ne k}^{\left|G_i\right|}\Vert {\beta }_{G_i}^{-1}H{\prime }_{G_i,k}^H{F\prime }_{G_{i,k\prime }}{W\prime }_{G_i,k\prime }{x}_{G_i,k\prime }\Vert ]\hfill \\ +[\sum _{i\prime =1,i\prime \ne i,}^L{\Vert {\beta }_{G_i}^{-1}H{\prime }_{G_{i\prime }}^H{F\prime }_{G_{i\prime }}{W\prime }_{G_{i\prime }}{x}_{G_{i\prime }}\Vert }^2+{\beta }_{G_i}^{-2}{n}_{G_i}]\hfill \end{array}$$

(36)

The conditional MSE in equation (31) is simplified as

$$\epsilon ({F\prime }_{G_i},{W\prime }_{G_i},{\beta }_{G_i})={\epsilon }_{G_i}^{\left(1\right)}+{\epsilon }_{G_i}^{\left(2\right)}$$

(37)

where

$${\epsilon }_{G_i}^{\left(1\right)}=E\left[{\Vert x_{G_i}-{\beta }_{G_i}^{-1}\left(H{\prime }_{G_i}^H{F\prime }_{G_i}{W\prime }_{G_i}{x}_{G_i}\right)\Vert }^2\right]$$

(38)

$$\begin{array}{c}\multicolumn{1}{c}{{\epsilon }_{G_i}^{\left(2\right)}=E[\sum _{k\prime =1,k\prime \ne k}^{\left|G_i\right|}\Vert {\beta }_{G_i}^{-1}H{\prime }_{G_i,k}^H{F\prime }_{G_{i,k\prime }}{W\prime }_{G_i,k\prime }{x}_{G_i,k\prime }\Vert ]}\\ +E[\sum _{i\prime =1,i\prime \ne i,}^L{\Vert {\beta }_{G_i}^{-1}H{\prime }_{G_{i\prime }}^H{F\prime }_{G_{i\prime }}{W\prime }_{G_{i\prime }}{x}_{G_{i\prime }}\Vert }^2+{\beta }_{G_i}^{-2}{n}_{G_i}]\hfill \end{array}$$

(39)

Therefore, the hybrid precoding based on interference leakage is jointly optimized with $F{\prime }_{G_i}$ , $W{\prime }_{G_i}$ , and ${\beta }_{G_i}$ . According to the literature, ¹⁹ $W{\prime }_{G_i}$ can be transformed to into $W{\prime }_{G_i}={\beta }_{G_i}{W″}_{G_i}$ , where ${W″}_{G_i}$ is an unnormalized digital precoding matrix, which can be obtained by KKT conditions as

$$\begin{array}{c}{W″}_{G_i}={(H{\prime }_{e{q}_{G_i}}^H{H\prime }_{e{q}_{G_i}}+{\gamma }_{G_i}^{-1}{I}_{G_i})}^{-1}H{\prime }_{e{q}_{G_i}}^H\hfill \\ ={(F{\prime }_{G_i}^H{H\prime }_{G_i}H{\prime }_{G_i}^H{F\prime }_{G_i}+{\gamma }_{G_i}^{-1}{I}_{G_i})}^{-1}F{\prime }_{G_i}^H{H\prime }_{G_i}\hfill \end{array}$$

(40)

where ${\gamma }_{G_i}^{-1}$ is the regularization factor, which depends on noise variance and base station transmit power. $I_{G_i}$ is

$$\begin{array}{c}\multicolumn{1}{c}{I_{G_i}=\sum _{k\prime =1,k\prime \ne k}^{\left|G_i\right|}{H}_{G_i,k}^H{F}_{G_{i,k\prime }}{W}_{G_i,k\prime }{x}_{G_i,k\prime }}\\ +\sum _{i\prime =1,i\prime \ne i,}^{L}H{\prime }_{G_{i\prime }}^H{F\prime }_{G_{i\prime }}{W\prime }_{G_{i\prime }}{x}_{G_{i\prime }}+n_{G_i}\hfill \end{array}$$

(41)

The optimal value given in Ayach et al. ¹² is ${\gamma }_{G_i}^{-1}=P_{\mathrm{tol}}/K{\sigma }^2$ . $P_{\mathrm{tol}}$ is the total power of the transmitted signal. The optimal scaling factor ${\beta }_{G_i}$ can be obtained from the base station transmission power with $\mathrm{tr}(F\prime W\prime W{\prime }^{H}F{\prime }^H)\le P_{\mathrm{tol}}$ as

$${\beta }_{G_i}=\sqrt{\frac{P_{\mathrm{tol}}}{\sum _{i=1}^L\mathrm{tr}\left({F\prime }_{G_i}{W\prime }_{G_i}W{\prime }_{G_i}^{H}F{\prime }_{G_i}^H\right)}}$$

(42)

Accordingly, equation (38) can be expressed as

$$\begin{array}{c}\multicolumn{1}{c}{{\epsilon }_{G_i}^{\left(1\right)}=E\left[{\Vert x_{G_i}-{\beta }_{G_i}^{-1}\left(H{\prime }_{G_i}^H{F\prime }_{G_i}{W\prime }_{G_i}{x}_{G_i}\right)\Vert }^2\right]}\\ =E\left\{\mathrm{tr}\left[{(x_{G_i}-{\beta }_{G_i}^{-1}\left(H{\prime }_{G_i}^H{F\prime }_{G_i}{W\prime }_{G_i}{x}_{G_i}\right))}^H(x_{G_i}-{\beta }_{G_i}^{-1}\left(H{\prime }_{G_i}^H{F\prime }_{G_i}{W\prime }_{G_i}{x}_{G_i}\right))\right]\right\}\hfill \\ =E\left\{\mathrm{tr}\left[(x_{G_i}^H-{\beta }_{G_i}^{-1}{\left(H{\prime }_{G_i}^H{F\prime }_{G_i}{W\prime }_{G_i}{x}_{{G_i}_i}\right)}^H)(x_{G_i}-{\beta }_{G_i}^{-1}\left(H{\prime }_{G_i}^H{F\prime }_{G_i}{W\prime }_{G_i}{x}_{G_i}\right))\right]\right\}\hfill \\ =E\left\{\mathrm{tr}\left(x_{{\mathcal{G}}_i}^H{x}_{{\mathcal{G}}_i}\right)\right\}-E\left\{\mathrm{tr}\left[{\beta }_{{\mathcal{G}}_i}^{-1}{x}_{{\mathcal{G}}_i}^H\left({\tilde{H}}_{{\mathcal{G}}_i}^H{\tilde{F}}_{{\mathcal{G}}_i}{\tilde{W}}_{{\mathcal{G}}_i}{x}_{{\mathcal{G}}_i}\right)\right]\right\}-\hfill \\ E\left\{\mathrm{tr}\left[{\beta }_{G_i}^{-1}{\left(H{\prime }_{G_i}^H{F\prime }_{G_i}{W\prime }_{G_i}{x}_{G_i}\right)}^H{x}_{G_i}\right]\right\}+E\left\{\mathrm{tr}\left[{\beta }_{G_i}^{-2}{\left(H{\prime }_{G_i}^H{F\prime }_{G_i}{W\prime }_{G_i}{x}_{G_i}\right)}^H\left(H{\prime }_{G_i}^H{F\prime }_{G_i}{W\prime }_{G_i}{x}_{G_i}\right)\right]\right\}\hfill \end{array}$$

(43)

After simple mathematical derivation, equation (39) can be expressed as

$$\begin{array}{c}\multicolumn{1}{c}{{\epsilon }_{G_i}^{\left(2\right)}=E[\sum _{k\prime =1,k\prime \ne k}^{\left|G_i\right|}\Vert {\beta }_{G_i}^{-1}H{\prime }_{G_i,k}^H{F\prime }_{{G_i}_{,k\prime }}{W\prime }_{G_i,k\prime }{x}_{G_i,k\prime }\Vert ]+\sum _{i\prime =1,i\prime \ne i}^{L}E[\Vert {\beta }_{G_i}^{-1}H{\prime }_{G_i}^H{F\prime }_{G_{i\prime }}{W\prime }_{G_{i\prime }}{x}_{G_{i\prime }}{\Vert }_F^2+n_{G_i}]}\\ =\sum _{k\prime =1,k\prime \ne k}^{\left|G_i\right|}{\beta }_{G_i}^{-2}\mathrm{tr}\left[H{\prime }_{G_i,k}^H{F\prime }_{G_{i,k\prime }}{W\prime }_{G_i,k\prime }E\left(x_{G_i,k\prime }{x}_{G_i,k\prime }^H\right){{W\prime }^H}_{G_i,k\prime }{{F\prime }^H}_{G_{i,k\prime }}H{\prime }_{G_i,k}\right]\hfill \\ +\sum _{i\prime =1,i\prime \ne i}^L{\beta }_{G_i}^{-2}\mathrm{tr}\left[H{\prime }_{G_{i\prime }}^H{F\prime }_{G_{i\prime }}{W\prime }_{G_{i\prime }}E\left(x_{G_{i\prime }}{x}_{G_{i\prime }}^H\right)W{\prime }_{G_{i\prime }}^{H}F{\prime }_{G_{i\prime }}^H{H\prime }_{G_{i\prime }}\right]+{\beta }_{G_i}^{-2}{g}_{G_i}{\sigma }^2\hfill \\ =\sum _{k\prime =1,k\prime \ne k}^{\left|G_i\right|}{\beta }_{G_i}^{-2}{g}_i\mathrm{tr}\left(H{\prime }_{G_i,k}^H{F\prime }_{G_{i,k\prime }}{W\prime }_{G_i,k\prime }{{W\prime }^H}_{G_i,k\prime }{{F\prime }^H}_{G_{i,k\prime }}H{\prime }_{G_i,k}\right)\hfill \\ +\sum _{i\prime =1,i\prime \ne i}^L{\beta }_{G_i}^{-2}{g}_{i\prime }\mathrm{tr}\left(H{\prime }_{G_{i\prime }}^H{F\prime }_{G_{i\prime }}{W\prime }_{G_{i\prime }}W{\prime }_{G_{i\prime }}^{H}F{\prime }_{G_{i\prime }}^H{H\prime }_{G_{i\prime }}\right)+{\beta }_{G_i}^{-2}{g}_{G_i}{\sigma }_n^2\hfill \end{array}$$

(44)

From the above analysis, it is essentially to find a radio frequency precoding matrix $F{\prime }_{G_i}$ so that the objective function obtains a minimum value of ${\epsilon }_{G_i}^{(1)}$ and ${\epsilon }_{G_i}^{(2)}$ . However, the above function is not a convex optimization, but this problem is equivalent to the unconstrained optimization problem, which can be solved using the manifold optimization method. ⁴⁰

To obtain a minimum value, we discover both geometrical and discriminant embedding space ${\mathrm{\Psi }}_L$ for each cluster L in the radio frequency precoding matrix $F{\prime }_{G_i}$ . We solve the following objective function with the choice of symmetric weight $B_{\varphi ,f_1{f}_2}^L$ , where $f_1,f_2\in G_i$ are the users in the cluster L

$$min\sum _{f_1{f}_2}{\Vert {\mathrm{\Psi }}_L^T{F\prime }_{f_1}-{\mathrm{\Psi }}_L^T{F\prime }_{f_2}\Vert }^2{B}_{\varphi ,f_1{f}_2}^L$$

(45)

$$max\sum _{f_1{f}_2}{\Vert {\mathrm{\Psi }}_L^T{F\prime }_{f_1}-{\mathrm{\Psi }}_L^T{F\prime }_{f_2}\Vert }^2{B}_{\phi ,f_1{f}_2}^L$$

(46)

For the Lth cluster, $B_{\phi ,f_1{f}_2}^L$ incurs a penalty if the users are close to each other but in different clusters, whereas $B_{\varphi ,f_1{f}_2}^L$ encourages users $f_1$ and $f_2$ map closer if users $f_1$ and $f_2$ are in the same cluster. We define

$$B_{\phi ,f_1{f}_2}^L=\{\begin{array}{c}\exp \left(\frac{-\Vert {F\prime }_{f_1}-{F\prime }_{f_2}\Vert }{t}\right),\mathrm{if}{F\prime }_{f_1}\in N_{\phi }\left({F\prime }_{f_2}\right)\mathrm{or}{F\prime }_{f_2}\in N_{\phi }\left({F\prime }_{f_1}\right)\hfill \\ \multicolumn{1}{c}{0,\mathrm{otherwise}}\end{array}$$

(47)

$$B_{\varphi ,f_1{f}_2}^L=\{\begin{array}{c}\exp \left(\frac{-\Vert {F\prime }_{f_1}-{F\prime }_{f_2}\Vert }{t}\right),\mathrm{if}{F\prime }_{f_1}\in N_{\varphi }\left({F\prime }_{f_2}\right)\mathrm{or}{F\prime }_{f_2}\in N_{\varphi }\left({F\prime }_{f_1}\right)\hfill \\ \multicolumn{1}{c}{0,\mathrm{otherwise}}\end{array}$$

(48)

where $N_{\varphi }(F{\prime }_{f_1})$ contains the $G_i^{\prime }-\mathrm{nearest}$ users sharing the precoding with $F{\prime }_{f_1}$ , whereas $N_{\phi }(F{\prime }_{f_1})$ contains the $G_i^{\prime }-\mathrm{nearest}$ neighbors having different precoding, and t is the parameter of kernel function that follows a Gaussian distribution approximately. ⁴²

Following some simple algebraic steps, equation (45) can be reduced to

$$\begin{array}{c}\multicolumn{1}{c}{S_{\varphi }=\frac{1}{2}\sum _{f_1{f}_2}{\Vert {\mathrm{\Psi }}_L^T{F\prime }_{f_1}-{\mathrm{\Psi }}_L^T{F\prime }_{f_2}\Vert }^2{B}_{\varphi ,f_1{f}_2}^L}\\ =\sum _{f_1}{\mathrm{\Psi }}_L^T{F\prime }_{f_1}{D}_{\varphi ,f_1{f}_1}^L{\left({F\prime }_{f_1}\right)}^T{\mathrm{\Psi }}_L\hfill \\ -\sum _{f_1{f}_2}{\mathrm{\Psi }}_L^T{F\prime }_{f_1}{B}_{\varphi ,f_1{f}_2}^L{\left({F\prime }_{f_1}\right)}^T{\mathrm{\Psi }}_L\hfill \\ ={\mathrm{\Psi }}_L^T\tilde{F}\prime (D_{\varphi }^L-B_{\varphi }^L){(\tilde{F}\prime )}^T{\mathrm{\Psi }}_L\hfill \\ ={\mathrm{\Psi }}_L^T\tilde{F}\prime {\mathrm{\Omega }}_{\varphi }{(\tilde{F}\prime )}^T{\mathrm{\Psi }}_L\hfill \end{array}$$

(49)

where $D_{\varphi }^L$ a diagonal matrix with $D_{\varphi }^L(f_1,f_1)=\sum _{f_2}{B}_{\varphi ,f_1{f}_2}(f_1,f_2)$ , ${\mathrm{\Omega }}_{\varphi }=(D_{\varphi }^L-B_{\varphi }^L)$ is the Laplacian matrix of the $B_{\varphi ,f_1{f}_2}^L$ , and $\tilde{F}\prime =\{1,2,\dots ,G_i\}$ .

Similarly, equation (46) can be simplified as

$$\begin{array}{c}\multicolumn{1}{c}{S_{\varphi }=\frac{1}{2}\sum _{f_1{f}_2}{\Vert {\mathrm{\Psi }}_L^T{F\prime }_{f_1}-{\mathrm{\Psi }}_L^T{F\prime }_{f_2}\Vert }^2{B}_{\phi ,f_1{f}_2}^L}\\ ={\mathrm{\Psi }}_L^T\tilde{F}\prime (D_{\phi }^L-B_{\phi }^L){(\tilde{F}\prime )}^T{\mathrm{\Psi }}_L\hfill \\ ={\mathrm{\Psi }}_L^T\tilde{F}\prime {\mathrm{\Omega }}_{\phi }{(\tilde{F}\prime )}^T{\mathrm{\Psi }}_L\hfill \end{array}$$

(50)

Thus, the discriminant embedding space ${\mathrm{\Psi }}_L$ can be derived by maximizing the following objective function

$$J\left({\mathrm{\Psi }}_L\right)=\frac{\mathrm{tr}\left(S_{\varphi }\right)}{\mathrm{tr}\left(S_{\phi }\right)}$$

(51)

This is equivalent to find the largest $\vartheta$ eigenvalue of the following generalized eigenvalue problem

$$\tilde{F}\prime {\mathrm{\Omega }}_{\varphi }{(\tilde{F}\prime )}^{T}v=\alpha \tilde{F}\prime {\mathrm{\Omega }}_{\phi }{(\tilde{F}\prime )}^{T}v$$

(52)

Let the largest $\vartheta$ eigenvalues $\{{\alpha }_1,{\alpha }_2,\dots ,{\alpha }_{\vartheta }\}$ that correspond to the largest $\vartheta$ eigenvectors $\{v_1,v_2,\dots ,v_{\vartheta }\}$ be the solutions of equation (52), which are chosen as the discriminant embedding space ${\mathrm{\Psi }}_L$ , where ${\alpha }_1\ge \cdots \ge {\alpha }_2\ge {\alpha }_{\vartheta }$ .

Once we have obtain the discriminant embedding space ${\mathrm{\Psi }}_L$ , the most distinguishing precodings are preserved in the most suitable projection space. In the low-dimensional discriminative embedding space, neighboring users of the same cluster approach each other, while preventing users of other clusters from entering the neighborhood.

Local discriminant matrix is defined as follows

$$G_{\mathrm{local}}=\frac{\frac{\alpha \prime }{(1-\alpha \prime ){\mathrm{\Omega }}_{\varphi }}}{{\mathrm{\Omega }}_{\phi }}$$

(53)

where $\alpha \prime$ is a scalar parameter. The local discriminant matrix is used as the input of kernel function.

In the general case, the distance in a Grassmann manifold (a particular class of manifolds) is the length of the shortest geodesic connecting two users in lineal subspaces $U_{f_1}$ and $U_{f_2}$ , this is

$$d_G(U_{f_1},U_{f_2})=\Vert \mathrm{\Theta }{\Vert }_2$$

(54)

where $\mathrm{\Theta }=[{\tau }_1,{\tau }_2,\dots ,{\tau }_{\kappa }]$ is the principal angles $0\le {\tau }_1\le {\tau }_2\le \cdots \le {\tau }_{\kappa }\le \pi /2$ .

Having defined projection distance over Grassmann manifold, the distance from user $f_1$ to $f_2$ in the discriminant embedding space ${\mathrm{\Psi }}_L$ also needs to be taken into account. To cover the intrinsic linear subspace, the user-to-user distance is denoted by

$$\frac{{\left({\left({F\prime }_{f_1}\right)}^T{\mathrm{\Psi }}_L{\left({\mathrm{\Psi }}_L\right)}^T{F\prime }_{f_2}\right)}^2}{\left({\left({F\prime }_{f_1}\right)}^T{\mathrm{\Psi }}_L{\left({\mathrm{\Psi }}_L\right)}^T{F\prime }_{f_1}\right)\left({\left({F\prime }_{f_2}\right)}^T{\mathrm{\Psi }}_L{\left({\mathrm{\Psi }}_L\right)}^T{F\prime }_{f_2}\right)}$$

(55)

When the two distances are formalized as above, we arrive at the following form of user-to-user distance metric

$$\begin{array}{c}d(f_1,f_2,{\mathrm{\Psi }}_L)=(1-\sigma )\hfill \\ \frac{{\left({\left({F\prime }_{f_1}\right)}^T{\mathrm{\Psi }}_L{\left({\mathrm{\Psi }}_L\right)}^T{F\prime }_{f_2}\right)}^2}{\left({\left({F\prime }_{f_1}\right)}^T{\mathrm{\Psi }}_L{\left({\mathrm{\Psi }}_L\right)}^T{F\prime }_{f_1}\right)\left({\left({F\prime }_{f_2}\right)}^T{\mathrm{\Psi }}_L{\left({\mathrm{\Psi }}_L\right)}^T{F\prime }_{f_2}\right)}\hfill \\ +\sigma \Vert U_{f_1}-U_{f_2}{U\prime }_{f_2}{U}_{f_1}{\Vert }_F^2\hfill \end{array}$$

(56)

In equation (55), the former describes how far away the origins of the two coordinate systems $U_{f_1}$ and $U_{f_2}$ in the discriminant embedding space ${\mathrm{\Psi }}_L$ , whereas the latter reflects the correlation between two orthonormal basis matrices.

The local discriminant matrix $G_{\mathrm{local}}$ is chosen as the input of Kernel function

$$G_{f_1{f}_2}=\{\begin{array}{c}\exp \left(\frac{-\sum _{f_1{f}_2}{\Vert ({\alpha }_{f_1}+\sum _{f_1}{\beta }_{f_1}{U}_{f_1})-({\alpha }_{f_2}+\sum _{f_2}{\beta }_{f_2}{U}_{f_2})\Vert }^2{G}_{\mathrm{local}}}{t}\right)\hfill \\ \multicolumn{1}{c}{0,\mathrm{otherwise}}\end{array}$$

(57)

where t is the parameter of the kernel function, ${\alpha }_{f_1}$ and ${\alpha }_{f_2}$ are the mean vectors of the manifolds where the users are located, and ${\beta }_{f_1}$ and ${\beta }_{f_2}$ are the vector of free parameters of $F{\prime }_{f_1}$ and $F{\prime }_{f_2}$ .

By minimizing the objective function below, we can find a suitable mapping on which the manifolds belonging to the same subspace can be closer and the manifolds in different subspaces can be further apart

$$\frac{1}{2}\sum _{f_1{f}_2}\Vert {\mathrm{\Gamma }}^T({\alpha }_{f_1}+\sum _{f_1}{\beta }_{f_1}{U}_{f_1})-{\mathrm{\Gamma }}^T({\alpha }_{f_2}+\sum _{f_2}{\beta }_{f_2}{U}_{f_2})\Vert G_{f_1{f}_2}$$

(58)

The above formula can be simplified to

$${\mathrm{\Gamma }}^T{D}_{\mathrm{kernel}}(D-G_{f_1{f}_2})D_{\mathrm{kernel}}^T\mathrm{\Gamma }$$

(59)

where

$$D_{\mathrm{kernel}}=\sum _{f_1{f}_2}\Vert ({\alpha }_{f_1}+\sum _{f_1}{\beta }_{f_1}{U}_{f_1})-({\alpha }_{f_2}+\sum _{f_2}{\beta }_{f_2}{U}_{f_2})\Vert$$

(60)

$D$ is a diagonal matrix, this is, $D(f_1,f_2)=\sum _{f_2}{G}_{f_1{f}_2}$ , $D$ establishes a natural metric between different subspaces. To limit z on a fixed scale, we add the following constraint

$$\mathrm{zD}{z}^T={\mathrm{\Gamma }}^T{D}_{\mathrm{kernel}}{\mathrm{DD}}_{\mathrm{kernel}}^T\mathrm{\Gamma }=1$$

(61)

Therefore, the objective function with constraint is obtained as follows

$$\begin{array}{c}\mathrm{\Gamma }=\arg min{\mathrm{\Gamma }}^T{D}_{\mathrm{kernel}}(D-G_{f_1{f}_2})D_{\mathrm{kernel}}^T\mathrm{\Gamma }\hfill \\ s.t.{\mathrm{\Gamma }}^T{D}_{\mathrm{kernel}}{\mathrm{DD}}_{\mathrm{kernel}}^T\mathrm{\Gamma }=1\hfill \end{array}$$

(62)

Let $J(\mathrm{\Gamma })$ represent the objective function. The hybrid precoding optimization problem based on interference leakage under orthogonal constraints is

$$\begin{array}{c}\arg {min}_{\mathrm{\Gamma }}J\left(\mathrm{\Gamma }\right)=\sum _{i=1}^L{\epsilon }_{G_i}^{\left(1\right)}+\sum _{i=1}^L{\epsilon }_{G_i}^{\left(2\right)}\hfill \\ \mathrm{s}.\mathrm{t}.{\mathrm{\Gamma }}^T{D}_{\mathrm{kernel}}{\mathrm{DD}}_{\mathrm{kernel}}^T\mathrm{\Gamma }=1\hfill \end{array}$$

(63)

Finally, we can obtain the optimal $\mathrm{\Gamma }$ by minimizing eigenvalue solution to the generalized eigenvalue problem.

Therefore, solving the objective function can be transformed into a convex optimization problem. The optimal radio frequency precoding matrix $F{\prime }_{G_i}$ is found to obtain a minimum value of ${\epsilon }_{G_i}^{(1)}$ and ${\epsilon }_{G_i}^{(2)}$ .

The manifold algorithm to find the optimal radio frequency precoding matrix $F{\prime }_{G_i}$ is as follows:

• Step 1: initialize the analog precoding matrix $F{\prime }_{G_i,1}$ , error threshold $\epsilon \in (0,1)$ , the discriminant embedding space ${\mathrm{\Psi }}_L$ .

• Step 2: the learning discriminant embedding space ${\mathrm{\Psi }}_L$ can be derived according to equations (45)–(51).

• Step 3: find the largest $\vartheta$ eigenvalue according to equation (52).

• Step 4: compute local discriminant matrix $G_{\mathrm{local}}$ according to equation (53).

• Step 5: compute subspace-to-subspace distance according to equation (56).

• Step 6: update $\mathrm{\Gamma }$ to subject to ${\mathrm{\Gamma }}^T{D}_{\mathrm{kernel}}{\mathrm{DD}}_{\mathrm{kernel}}^T\mathrm{\Gamma }=1$ , and find the optimal radio frequency precoding matrix $F{\prime }_{G_i}$ according to equation (63). Update the analog precoding matrix until convergence to satisfy the error threshold condition, the algorithm ends.

For the intra-cluster, there is correlation between the channels between users. Users in non-adjacent clusters are much smaller than users in adjacent clusters, but the interference is equal. So, the impact of remote user clusters on users in the cluster can be ignored. The SNR of a user cluster $G_i$ in $b\mathrm{th}$ cell is as follows

$$\begin{array}{c}\mathrm{SIN}{R}_{G_i}=\hfill \\ \frac{{\left|H{\prime }_{G_i}^{H}F{\prime }_{G_i}{W\prime }_{G_i}\right|}^2{P}_{G_i}}{\sum _{k\prime =1,k\prime \ne k}^{\left|G_i\right|}{\left|{\mathrm{IN}}_{G_i,k\prime }\right|}^2{P}_{G_i,k\prime }+\sum _{i\prime =1,i\prime \ne i}^L{\left|{\mathrm{IN}}_{G_{i\prime }}\right|}^2{P}_{G_{i\prime }}+{\sigma }_{G_i}^2}\hfill \end{array}$$

(64)

where ${\mathrm{IN}}_{G_i,k\prime }=H{\prime }_{G_i,k}^{H}F{\prime }_{G_{i,k\prime }}W{\prime }_{G_i,k\prime }$ , ${\mathrm{IN}}_{G_{i\prime }}=H{\prime }_{G_{i\prime }}^{H}F{\prime }_{G_{i\prime }}W{\prime }_{G_{i\prime }}$ , and $P_{G_i}$ are the transmit power of the $G_i\mathrm{th}$ cluster, and $P_{G_i,k\prime }$ and $P_{G_{i\prime }}$ are the transmit power of the $k\prime \mathrm{th}$ user in the $i\mathrm{th}$ cluster and the transmit power of the $G_{i\prime }\mathrm{th}$ cluster, respectively.

The capacity of mmWave massive MIMO system can be expressed as

$$\mathrm{SUM}=\sum _{G_i=G_1}^{G_L}{\log }_2(1+\mathrm{SIN}{R}_{G_i})$$

(65)

Equation (65) can be written as

$$\mathrm{SUM}=\sum _{G_i=G_1}^{G_L}{\log }_2(1+\frac{{\left|H{\prime }_{G_i}^{H}F{\prime }_{G_i}{W\prime }_{G_i}\right|}^2{P}_{G_i}}{\sum _{k\prime =1,k\prime \ne k}^{\left|G_i\right|}{\left|{\mathrm{IN}}_{G_i,k\prime }\right|}^2{P}_{G_i,k\prime }+\sum _{i\prime =1,i\prime \ne i}^L{\left|{\mathrm{IN}}_{G_{i\prime }}\right|}^2{P}_{G_{i\prime }}+{\sigma }_{G_i}^2})$$

(66)

Simulation results

In this section, we will study the SE and bit error rate (BER) performance of the proposed hybrid precoder. We compare the method in this article with several traditional methods, that is, OMP, KDHB, AFHB, MO, and RTRNM. We also consider the performance of this method for non-mobile users and mobile users. The basic simulation parameters are as follows.

The carrier frequency is 60 GHz. The AoAs and AoDs are uniformly distributed in $[0,2\pi ]$ , and a common Angle Spread (AS) $\mathrm{\Delta }=8$ . The complex gain of each path obeys the distribution $\mathrm{CN}(0,1)$ . The ULA is adopted in simulations. ²⁶ Among them, there are many overlapping parts between the channel power azimuth spectrum, which causes inter-cluster interference.

Figure 4 shows the differences of sum-rate performance with traditional schemes in the mmWave massive MIMO system of hybrid precoding. Let $N_t=128$ , $N_{\mathrm{RF}}=32$ , and $K=32$ . In Figure 4, we can see that the proposed scheme can achieve a very high sum-rate than other existing precodings against different SNR. The reason for this situation is that traditional methods have not solved the problem of non-linear resolution of multi-user high-dimensional channels. In our proposed scheme, each user set is represented by manifold, and the cluster-oriented multi-manifold learning scheme is used to solve the above problems. Aiming at the high-density hotspot scene of the cell, the geometric model of clustering users is studied. This method can better solve the interference within and between clusters. Through user clustering and hybrid precoding, the achievable summation rate of the mmWave massive MIMO system is improved.

Figure 4.

Sum-rate comparison of different schemes.

Figure 5 shows the differences of the BER performance of different hybrid precoding schemes, where the channel parameters in the single-cell scenario are the same as above. From Figure 5, in the case of different SNRs, a conclusion similar to Figure 4 can be drawn. We can also see that the proposed-based manifold discriminative learning scheme achieve a better BER performance than other schemes. The proposed scheme improves beamspace resolution and reduces the influence of power leakage on beamspace channel.

Figure 5.

BER performance comparison of different schemes.

Figure 6 compares the average sum-rate of this article and the traditional scheme, and other existing precodings with different numbers of users. We set $n_{\mathrm{RF},i}\ge g_i$ in each cluster. It can be seen from Figure 6 that this method is superior to other methods. Due to the continuous increase in the number of users, the traditional solution does not consider the problem of non-linear resolution of multi-user high-dimensional channels. This scheme can well solve the interference in clusters, between clusters and between cells. The solution proposed in this article can significantly improve the average and rate of mmWave massive MIMO systems.

Figure 6.

Two-tier system average and rate comparison.

Figure 7 shows the change trend of the system average SNR as the SNR changes. Figure 8 shows the average SE when the BS antenna changes. We can learn from the figure that the proposed method achieves an average SE that is significantly higher than other traditional schemes. From Figure 7 that through the proposed manifold learning scheme, each user and its adjacent high-dimensional channels are in a global and local non-linear neighborhood. Aiming at the high-density hotspot scene of the community, the geometric model of clustering users is studied. The proposed scheme manages the multi-user and inter-cell interference and improves the data rate for cell-edge users. From Figure 8, the proposed method can effectively and extensively use antennas in multiple low-dimensional manifolds.

Figure 7.

Average SE versus cell-edge SNR.

Figure 8.

The average SE of the number of different BS antennas.

Conclusion

A hybrid precoding scheme for user clustering is proposed, which can solve the problem of large-scale mmWave MIMO in multiple low-dimensional manifolds, avoiding the high-dimensional complex operations of traditional schemes. For the BS, mmWave massive MIMO obtains a low-dimensional learning channel matrix by manifold. Then user clustering hybrid precoding is studied for the transmitted signal based on the low-dimensional channel matrix. The manifold discriminative learning seek to learn the embedding low-dimensional subspace, where manifolds with different user group labels are easier to distinguish, and the local spatial correlation of high-dimensional channels in each manifold is enhanced. Through proper user clustering, the hybrid precoding is investigated for the sum-rate maximization problem by manifold quasi-conjugate gradient methods. The simulation results show that the method has good robustness on the basis of reducing the computational complexity of the mmWave mass MIMO system.

More realistic precoding for MIMO is expected in the future research. In popular learning after user clustering we just use the traditional method, the choice of user clustering center is crucial, if the user is in high speed of movement will make our method inaccurate. So, more advanced research is needed. In addition, our current method is applied in hybrid precoding of full connections; precoding research is still needed for sub-connections as well as dynamic connections.