A Low-Complexity Block Diagonalization Algorithm for MU-MIMO Two-Way Relay Systems with Complex Lattice Reduction

Abstract

We design a scheme of precoding matrices for two-way multiuser relay systems, where a multiantenna relay station (RS) operating in an amplify-and-forward model simultaneously receives information from all multiple-antenna users. Considering the feasibility in mathematical analysis, users are distributed in two symmetrical groups. To reduce the complexity of proposed precoding scheme, we employ the QR decomposition and complex lattice reduction (CLR) transform to replace the two times singular value decomposition (SVD) of conventional BD-based precoding algorithm by introducing a combined channel inversion to eliminate the multiple users interference (MUI). Simulation and performance analysis demonstrate that the proposed LR-MMSE algorithm has not only a better bit error rate (BER) performance, a higher sum-rate, and simple architecture, but also 89.8% and 35.5% less complexity compared to BD- and MMSE-based scheme.

1. Introduction

Two-way relaying (TWR) cooperation has attracted considerable attention due to the high spectral efficiency. A typical model of the TWR protocol with multiple access channel (MAC) phase and broadcast channel (BC) phase has been investigated in [1]. In the MAC phase, two source nodes transmit their message to the relay node simultaneously. After processing the mixed received signal at the relay, a combined version of the received signal is broadcast to each source node in the BC phase.

Among the numerous two-way relaying protocols, such as amplify-and-forward (AF) and decode-and forward [2], AF protocol gets favored as one or more low-complexity relay nodes are adopted to assist the communication between sources and destinations without decoding the signals.

In wireless sensor networks (WSNs), a large number of static or mobile sensors cooperate to perceive, compute, and transmit message assistant by the relay [3]. Since sensor nodes are usually operated by lightweight batteries that are difficult to replace or recharge, energy becomes one of the most crucial resources in WSNs. Moreover, experimental measurements have shown that the energy consumption of a sensor node is dominated by communications [4]. Thus, a simple and low-complexity transceiver model for sensors is a powerful solution to solve this problem. Recently, a prominent research in [5] has proposed a multilevel physical layer network coding to rule the rate of sensors, which saves the power of networks potentially.

As we known, multiple-input multiple-output (MIMO) relays potentially obtain both spectral efficiency and link reliability by exploiting the multiple antennas, and, jointing the TWR technology, it can improve the system performance dramatically. Recently, the precoding design for TWR MIMO systems is extended to multiuser cases, which can be roughly divided into two categories: symmetric systems and asymmetric systems [6–11]. In symmetric system, all users are supposed to be distributed in two groups in the form of pairings [6–8]. In asymmetric systems, a based station exchanges messages with multiple users [9–11], which is a typical scenario of cellular networks.

Unlike the received signals in single-user MIMO (SU-MIMO) systems, the received signals of different users in multiuser MIMO (MU-MIMO) systems not only suffer from the noise and intra-antenna interference but are also disturbed by the multiuser interference (MUI). However, when we consider two-way relaying multiuser MIMO (TWR MU-MIMO) systems, the MUI becomes more complicated, since there are two groups of users mixing in the received signals.

Channel inversion strategies based on zero forcing (ZF) and minimum mean squared error (MMSE) can be used to cancel the MUI but result in amplifying the noise [12, 13]. Block diagonalization (BD) has been proposed in [14, 15] to improve the system throughput and provide convenience for the power control. Literature [16] studies the precoding scheme of the singular value decomposition (SVD) and employs a technology of channels aligning referring to [17]. However, the complexity of SVD is very high when the number of users and the specification of the antenna are large. In order to reduce the complexity, generalized singular value decomposition (GSVD) has been applied in [18] to design the downlink of MIMO systems. Unfortunately, there is also a great performance loss at low signal to noise ratios (SNRs) when the noise is the dominant factor. In recent research, a systematic scheme is proposed in [7], which imposes a power constraint at the relay node and obtains an approximate result by fixing the transmitting and receiving matrices at the user nodes firstly. In [19], a low complexity precoding algorithm is proposed to reduce the condition number of the effective number by introducing a complex lattice reduction (CLR) transform in one-way relay MU-MIMO systems.

In this paper, we focus on AF TWR MU-MIMO systems and strive to find a lower complexity and better performance precoding scheme. By employing a combined channel inversion and replacing the SVD with QR decomposition, we can cancel the MUI remarkably. Then, the CLR transform is used to design the precoding matrices of users which locate in another group and is in pairs of receiving users.

The rest of the paper is organized as follows. Section 2 describes the conventional system model. Section 3 introduces the BD-based precoding algorithm. The scheme LR-MMSE is proposed in Section 4. Performance analysis and simulation are given in Sections 5 and 6. Finally, we conclude this work in Section 7.

Notation. Throughout this paper, for a matrix A , $\mathrm{T}\mathrm{r}\left\{\mathit{A}\right\}$ , ${\mathit{A}}^T$ , ${\mathit{A}}^H$ , ${\mathit{A}}^{-\mathrm{1}}$ , and ${\mathit{A}}^{†}$ denote the trace, transpose, complex conjugate transpose, inverse, and pseudoinverse, respectively. “E” stands for the expectation of a random variable; the terms ${\mathbb{C}}^{a\times b}$ represent the $\left(a\times b\right)$ -dimensional space with complex-valued elements. The notations ${‖\mathit{A}‖}_F^{}$ and ${‖\mathit{A}‖}_F^{\mathrm{2}}$ denote Frobenius-norm and 2-norm of matrix A . ${\mathit{I}}_{m\times n}$ is a m-by-n identity matrix; ${\mathbf{0}}_m$ is a m-dimensional zero matrix.

2. General System Model

Investigate an uncoded TWR MU-MIMO system, with K pairs of user equipment (UE) and one two-way relay station (RS) which operates in AF protocol. For simplicity, we consider a symmetric model in which users are divided into two groups: $l\in \left\{a,b\right\}$ . The kth user in group l and the relay has equipped $N_{l,k}$ and $N_R$ antennas, respectively. A block diagram of such a system is shown in Figure 1.

Figure 1

System model.

Ruled that there is no direct link between two users and all the channels experience independent and flat MIMO fading. Let ${\mathit{d}}_{l,k}\left(t\right)$ denote the data symbol at time t for the kth user in group l ( $\mathrm{U}{\mathrm{E}}_{l,k}$ ). The $\mathrm{U}{\mathrm{E}}_{l,k}$ performs transmit precoding beamforming with vector ${\mathit{P}}_{l,k}\in {\mathbb{C}}^{N_R\times N_{l,k}}$ and transmits the signal in the following form of matrix:

$$\begin{array}{cc}& {\mathit{x}}_{l,k}\left(t\right)={\mathit{P}}_{l,k}{\mathit{d}}_{l,k}\left(t\right)\in {\mathbb{C}}^{N_{l,k}\times \mathrm{1}},\end{array}$$

(1)

where ${\mathit{d}}_{l,k}\left(t\right)\in {\mathbb{C}}^{N_{l,k}\times \mathrm{1}}$ and satisfies $E\left[{\mathit{d}}_{l,k}\left(t\right){\mathit{d}}_{l,k}^H\left(t\right)\right]={\xi }_{{\mathbf{n}}_{l,k}}^{\mathrm{2}}·{\mathit{I}}_{N_{l,k}}$ . We assume the average power of the transmit signal is 1.

Then, users $\mathrm{U}{\mathrm{E}}_a$ and $\mathrm{U}{\mathrm{E}}_b$ exchange their signals by the assisted relay node RS in two phases. Firstly, in the multiple access channel (MAC) phase, $\mathrm{U}{\mathrm{E}}_a$ and $\mathrm{U}{\mathrm{E}}_b$ transmit their signals simultaneously to relay and the received signals at RS are given by

$$\begin{array}{cc}& \mathit{r}\left(t\right)={\displaystyle \sum }_{k=\mathrm{1}}^K{\mathit{H}}_{a,k}{\mathit{x}}_{a,k}\left(t\right)+{\displaystyle \sum }_{k=\mathrm{1}}^K{\mathit{H}}_{b,k}{\mathit{x}}_{b,k}\left(t\right)+{\mathit{n}}_s\left(t\right),\end{array}$$

(2)

where ${\mathit{H}}_{l,k}\in {\mathbb{C}}^{N_R\times N_{l,k}}$ is the channel coefficient coming from $\mathrm{U}{\mathrm{E}}_{l,k}$ to RS and ${\mathit{n}}_s\left(t\right)\in {\mathbb{C}}^{N_R\times \mathrm{1}}$ is a zero-mean addictive white Gaussian noise (AWGN) at the relay and $E\left[{\mathit{n}}_s\left(t\right){\mathit{n}}_s^H\left(t\right)\right]={\sigma }_{{\mathbf{n}}_s}^{\mathrm{2}}{\mathit{I}}_{N_R}$ .

In the broadcast (the broadcast channel) phase, the RS forwards and broadcasts the mixed signals $\mathit{s}\left(t\right)\in {\mathbb{C}}^{N_R\times \mathrm{1}}$ to all user by multiplying a forward matrix $\mathit{f}\in {\mathbb{C}}^{N_R\times N_R}$ , where

$$\begin{array}{cc}& \mathit{s}\left(t\right)=\mathit{f}\mathit{r}\left(t\right).\end{array}$$

(3)

It is worth noting that the f is an assistant precoding matrix for decoding the signals at receiver sites. Since it is supposed that channel reciprocity (This assumption is reasonable when we consider time division duplex (TDD) systems. Channels in uplink and downlink are static and identical in the frequency domain.) is met within two phases, the received signals ${\mathit{y}}_{l,k}\left(t\right)\in {\mathbb{C}}^{N_{l,k}\times \mathrm{1}}$ at the user $\mathrm{U}{\mathrm{E}}_{l,k}$ side are

$$\begin{array}{cc}& {\mathit{y}}_{l,k}\left(t\right)={\mathit{H}}_{l,k}^T\mathit{s}\left(t\right)+{\mathit{n}}_{l,k}\left(t\right),\end{array}$$

(4)

where ${\mathit{n}}_{l,k}\left(t\right)\in {\mathbb{C}}^{N_{l,k}\times \mathrm{1}}$ is also a zero-mean AWGN at the $\mathrm{U}{\mathrm{E}}_{l,k}$ and $E\left[{\mathit{n}}_{l,k}\left(t\right){\mathit{n}}_{l,k}^H\left(t\right)\right]={\sigma }_{}^{\mathrm{2}}{\mathit{I}}_{N_{l,k}}$ . The $\mathrm{U}{\mathrm{E}}_{l,k}$ combines its own received signal (4) by using a received decoding vector ${\mathit{g}}_{l,k}\in {\mathbb{C}}^{\mathrm{1}\times N_{l,k}}$ to get the estimate

$$\begin{array}{cc}& {\widehat{\mathit{d}}}_{\widehat{l},k}\left(t\right)={\mathit{g}}_{l,k}{\mathit{y}}_{l,k}\left(t\right),\end{array}$$

(5)

where ${\widehat{\mathit{d}}}_{\widehat{l},k}\left(t\right)$ denotes the information coming from $\mathrm{U}{\mathrm{E}}_{\widehat{l},k}$ , the subscripts l and $\widehat{l}\in \left\{a,b\right\}$ , and $l\ne \widehat{l}$ represents the user pairings. For notational convenience, the time index t is henceforth wiped off. Refer to Figure 2(a), which shows the overall procedure for transmit- and receive-beamforming and relaying of multiuser two-way systems. Next, designs of the scheme of precoding matrices are given with the criteria of BD, ZF, and MMSE. Further, the lattice reduction is applied to suppress the bit error rate (BER) of the system. As a result, we can omit the decoding matrix to obtain a simple structure.

Figure 2

(a) The transfer model for conventional precoding algorithms. (b) The transfer model for the proposed LR-MMSE algorithm.

3. Generalized Design of BD-Based Precoding Algorithm

The BD- and SVD-based precoding algorithms are proposed in [10] and [12], respectively. Combining the two ideas, the BD-based scheme can be given as follows.

The received signals at the users $\mathrm{U}{\mathrm{E}}_{l,k}$ and $\mathrm{U}{\mathrm{E}}_{\widehat{l},k}$ sides can be written as

$$\begin{array}{cc}& {\widehat{\mathit{d}}}_{\widehat{l},k}={\mathit{g}}_{l,k}{\mathit{y}}_{l,k}.\end{array}$$

(6)

The detail components of ${\widehat{\mathit{d}}}_{\widehat{l},k}$ are

$$\begin{array}{l}{\widehat{\mathit{d}}}_{\widehat{l},k}=\underset{\mathrm{1}}{\underbrace{{\mathit{g}}_{l,k}{\mathit{H}}_{l,k}^T{\mathit{f}}_{l,k}{\mathit{H}}_{l,k}{\mathit{d}}_{l,k}}}+\underset{\mathrm{2}}{\underbrace{{\mathit{g}}_{l,k}{\mathit{H}}_{l,k}^T{\mathit{f}}_{l,k}{\displaystyle \sum }_{i=\mathrm{1},i\ne k}^K{\mathit{H}}_{l,i}{\mathit{d}}_{l,i}}}\\ \hspace{1em}+\underset{\mathrm{3}}{\underbrace{{\mathit{g}}_{l,k}{\mathit{H}}_{l,k}^T{\mathit{f}}_{l,k}{\displaystyle \sum }_{i=\mathrm{1},i\ne k}^K{\mathit{H}}_{\widehat{l},i}{\mathit{d}}_{\widehat{l},i}}}+\underset{\mathrm{4}}{\underbrace{{\mathit{g}}_{l,k}{\mathit{H}}_{l,k}^T{\mathit{f}}_{l,k}{\mathit{H}}_{\widehat{l},k}{\mathit{d}}_{\widehat{l},k}}}\\ \hspace{1em}+{\mathit{g}}_{l,k}{\mathit{H}}_{l,k}^T{\mathit{f}}_{\widehat{l},k}{\mathit{n}}_s+{\mathit{g}}_{l,k}{\mathit{n}}_{l,k}.\end{array}$$

(7)

The 1st item can be dismissed by the perfect self-interference suppression, and the key in the rest of processing is to eliminate MUI and obtain a good performance of the signal estimation.

Based on the system model, the combined channel matrix is given by $\mathit{H}=[{\mathit{H}}_{l,\mathrm{1}}\cdots {\mathit{H}}_{l,K},{\mathit{H}}_{\widehat{l},\mathrm{1}}\cdots {\mathit{H}}_{\widehat{l},K}]\in {\mathbb{C}}^{N_R\times N}$ , where N denotes the sum of users’ antennas. We define ${\bar{\mathit{H}}}_{l,k}=[{\mathit{H}}_{l,\mathrm{1}}^{}{\mathit{H}}_{l,k-\mathrm{1}}^{}{\mathit{H}}_{l,k+\mathrm{1}}^{}\cdots {\mathit{H}}_{l,K}^{},{\mathit{H}}_{\widehat{l},\mathrm{1}}^{}\cdots {\mathit{H}}_{\widehat{l},K}^{}]$ so that ${\bar{\mathit{H}}}_{l,k}\in {\mathbb{C}}^{N_R\times {\bar{N}}_{l,k}}$ , where ${\bar{N}}_{l,k}=N-N_{l,k}$ . In order to eliminate the MUI in (7), we impose the BD constraint in which

$$\begin{array}{cc}& {\bar{\mathit{H}}}_{\widehat{l},k}{\mathit{f}}_{\widehat{l},k}^{}=\mathrm{0}\hspace{1em}\forall k\in \left\{\mathrm{1},\dots ,K\right\}.\end{array}$$

(8)

Firstly, by employing the SVD to ${\bar{\mathit{H}}}_{\widehat{l},k}$ ,

$$\begin{array}{cc}& {\bar{\mathit{H}}}_{\widehat{l},k}={\bar{\mathit{U}}}_{\widehat{l},k}{\stackrel{-}{\mathit{\Lambda }}}_{\widehat{l},k}{\bar{\mathit{V}}}_{\widehat{l},k}^H={\bar{\mathit{U}}}_{\widehat{l},k}{\stackrel{-}{\mathit{\Lambda }}}_{\widehat{l},k}{\left[{\bar{\mathit{V}}}_{\widehat{l},k}^{\left(\mathrm{1}\right)}\mathrm{\hspace{0.17em}\hspace{0.17em}}{\bar{\mathit{V}}}_{\widehat{l},k}^{\left(\mathrm{0}\right)}\right]}^H,\end{array}$$

(9)

where ${\bar{\mathit{U}}}_{l,k}\in {\mathbb{C}}^{N_R\times N_R}$ and ${\bar{\mathit{V}}}_{\widehat{l},k}^H\in {\mathbb{C}}^{{\bar{N}}_{\widehat{l},k}\times {\bar{N}}_{\widehat{l},k}}$ are unitary matrices. The diagonal matrix ${\bar{\Sigma }}_{l,k}\in {\mathbb{C}}^{N_R\times {\bar{N}}_{l,k}}$ is composed of the singular values of ${\bar{\mathit{H}}}_{\widehat{l},k}$ . Factorizing ${\bar{\mathit{V}}}_{\widehat{l},k}^{}$ into two parts, ${\bar{\mathit{V}}}_{\widehat{l},k}^{(\mathrm{1})}\in {\mathbb{C}}^{N_R\times {\lambda }_{\widehat{l},k}}$ consists of the first $L_{\widehat{l},k}$ none-zero singular column vectors and ${\bar{\mathit{V}}}_{\widehat{l},k}^{(\mathrm{0})}\in {\mathbb{C}}^{N_R\times \left(N_R-L_{\widehat{l},k}\right)}$ holds the last $N_R-L_{\widehat{l},k}$ zero singular column vectors, where $L_{\widehat{l},k}$ is $\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}\left({\bar{\mathit{V}}}_{\widehat{l},k}^{}\right)$ . Thus, ${\bar{\mathit{V}}}_{\widehat{l},k}^{(\mathrm{0})}$ lies in the null space of ${\bar{\mathit{H}}}_{\widehat{l},k}$ . The relay forward matrix ${\mathit{f}}_{\widehat{l},k}^{}$ can be obtained as

$$\begin{array}{cc}& {\mathit{f}}_{\widehat{l},k}^{}={\bar{\mathit{V}}}_{\widehat{l},k}^{\left(\mathrm{0}\right)}.\end{array}$$

(10)

Substituting (10) into (7), we have the effective channel

$$\begin{array}{cc}& {\mathit{H}}_{\widehat{l},k}^{\mathrm{e}\mathrm{f}\mathrm{f}}={\mathit{H}}_{l,k}^T{\mathit{f}}_{\widehat{l},k}^{}{\mathit{H}}_{\widehat{l},k}={\mathit{H}}_{l,k}^T{\bar{\mathit{V}}}_{\widehat{l},k}^{\left(\mathrm{0}\right)}{\mathit{H}}_{\widehat{l},k}.\end{array}$$

(11)

Then, the second SVD operation is used to obtain the precoding and decoding matrix. We have

$$\begin{array}{cc}& {\mathit{H}}_{\widehat{l},k}^{\mathrm{e}\mathrm{f}\mathrm{f}}={\mathit{U}}_{\widehat{l},k}^{}\left(\begin{array}{cc}{\mathit{\Lambda }}_{\widehat{l},k}& \mathrm{0}\\ \mathrm{0}& \mathrm{0}\end{array}\right){\mathit{V}}_{\widehat{l},k}^H.\end{array}$$

(12)

Finally, the user $\mathrm{U}{\mathrm{E}}_{\widehat{l},k}$ 's precoding matrix ${\mathit{P}}_{\widehat{l},k}^{}={\mathit{V}}_{\widehat{l},k}^{}{\Omega }_{\widehat{l},k}$ and the decoding matrix is obtained as ${\mathit{g}}_{l,k}={\mathit{U}}_{\widehat{l},k}^H$ , where ${\Omega }_{\widehat{l},k}$ is a diagonal matrix and its elements stand for the transmit power allocation. To guarantee that the ${\bar{\mathit{V}}}_{\widehat{l},k}^{(\mathrm{0})}$ is not a null matrix, we must have $N>N_R$ . Note that we omit the dimension matching matrices in (11) which will be mentioned in the next section.

4. The Proposed LR-MMSE Precoding Algorithms

In this section, we describe the proposed precoding algorithms based on a strategy that employs a combined channel inversion method, QR decompositions, and lattice reductions in detail. We can solve the questions with ZF and MMSE criteria. For a better BER performance, the complex lattice reduction is proposed to improve the MMSE performance.

4.1. ZF-Based Design

The ZF-based design of precoding matrix termed as ZF can be performed in two steps.

Step 1.

It is to obtain the relay forward matrix ${\mathit{f}}_{l,k,\mathrm{z}\mathrm{f}}^{}$ which eliminates the MUI completely by a QR decomposition of ${\bar{\mathit{H}}}_{\widehat{l},k}^{}$ .

Firstly, by using the ZF inversion to the combined channel matrix H , we have

$$\begin{array}{cc}& {\mathit{H}}_{\mathrm{z}\mathrm{f}}^{†}={\mathit{H}}^H{\left(\mathit{H}{\mathit{H}}^H\right)}^{-\mathrm{1}}=\left[\begin{array}{c}{\mathit{H}}_{l,\mathrm{1},\mathrm{z}\mathrm{f}}\\ {\mathit{H}}_{l,\mathrm{2},\mathrm{z}\mathrm{f}}\\ ⋮\\ {\mathit{H}}_{\widehat{l},K,\mathrm{z}\mathrm{f}}\end{array}\right],\end{array}$$

(13)

where ${\mathit{H}}_{\mathrm{z}\mathrm{f}}^{†}$ is the pseudoinverse of matrix ${\mathbf{H}}_{\mathrm{z}\mathrm{f}}$ and ${\mathit{H}}_{\widehat{l},k,\mathrm{z}\mathrm{f}}\in {\mathbb{C}}^{N_{\widehat{l},k}\times N_R}$ is the submatrix of ${\mathit{H}}_{\mathrm{z}\mathrm{f}}^{†}$ . Note that ${\mathit{H}}_{\mathrm{z}\mathrm{f}}{\mathit{H}}_{\mathrm{z}\mathrm{f}}^{†}={\mathit{I}}_{N_R}$ [12]. Thus, the off-diagonal block matrices of $\mathit{H}{\mathit{H}}_{\mathrm{z}\mathrm{f}}^{†}$ are zero and ${\mathit{H}}_{\widehat{l},k,\mathrm{z}\mathrm{f}}$ is in the null space of ${\bar{\mathit{H}}}_{\widehat{l},k}$ ; that is,

$$\begin{array}{cc}& {\mathit{H}}_{\widehat{l},k,\mathrm{z}\mathrm{f}}{\bar{\mathit{H}}}_{\widehat{l},k}=\mathbf{0}\mathbf{.}\end{array}$$

(14)

Imposing the QR decomposition on ${\mathit{H}}_{\widehat{l},k,\mathrm{z}\mathrm{f}}={\mathit{Q}}_{\widehat{l},k,\mathrm{z}\mathrm{f}}{\mathit{R}}_{\widehat{l},k,\mathrm{z}\mathrm{f}}$ , we have

$$\begin{array}{cc}& {\mathit{Q}}_{\widehat{l},k,\mathrm{z}\mathrm{f}}{\bar{\mathit{H}}}_{\widehat{l},k}=\mathbf{0},\hspace{1em}\forall k\in \left\{\mathrm{1},\dots ,K\right\},\end{array}$$

(15)

where ${\mathit{Q}}_{\widehat{l},k,\mathrm{z}\mathrm{f}}\in {\mathbb{C}}^{N_{\widehat{l},k}\times N_R}$ is an orthogonal matrix. Then, substituting ${\mathit{f}}_{\widehat{l},k}^{\mathrm{1}}$ into (7), the 2nd and 3rd items will be equal to zeros.

Step 2.

It is to obtain the user precoding and receive decoding matrices ${\mathit{p}}_{\widehat{l},k,\mathrm{z}\mathrm{f}}$ and ${\mathit{g}}_{l,k,\mathrm{z}\mathrm{f}}$ by a SVD of the effective channel.

Note that the degree of freedom between ${\mathit{H}}_{l,k}^T$ and ${\mathit{H}}_{\widehat{l},k}$ is $N_R\times N_R$ , we match the dimensions with the help of ${\mathbf{\Pi }}_{\widehat{l},k}\in {\mathbb{C}}^{N_R\times N_{\widehat{l},k}}$ , and ${‖{\mathbf{\Pi }}_{\widehat{l},k}‖}_F^{\mathrm{2}}=\mathrm{1}$ . Then, substituting ${\mathit{f}}_{\widehat{l},k}^{\mathrm{1}}$ and ${\mathbf{\Pi }}_{\widehat{l},k}$ into the 4th item of ${\widehat{\mathit{d}}}_{\widehat{l},k}$ , we have the effective channel

$$\begin{array}{cc}& {\mathit{H}}_{\widehat{l},k,\mathrm{z}\mathrm{f}}^{\mathrm{e}\mathrm{f}\mathrm{f}}={\mathit{H}}_{l,k}^T{\mathbf{\Pi }}_{\widehat{l},k}{\mathit{Q}}_{\widehat{l},k,\mathrm{z}\mathrm{f}}{\mathit{H}}_{\widehat{l},k},\end{array}$$

(16)

where ${\mathit{H}}_{\widehat{l},k,\mathrm{z}\mathrm{f}}^{\mathrm{e}\mathrm{f}\mathrm{f}}\in {\mathbb{C}}^{N_{l,k}\times N_{\widehat{l},k}}$ . The ZF algorithm can be completed by applying the SVD operation to the matrix ${\mathit{H}}_{\widehat{l},k,\mathrm{z}\mathrm{f}}^{\mathrm{e}\mathrm{f}\mathrm{f}}={\mathit{U}}_{\widehat{l},k,\mathrm{z}\mathrm{f}}{\mathit{\Lambda }}_{\widehat{l},k,\mathrm{z}\mathrm{f}}{\mathit{V}}_{\widehat{l},k,\mathrm{z}\mathrm{f}}^H$ . Then, the user precoding and receive decoding matrices ${\mathit{p}}_{\widehat{l},k,\mathrm{z}\mathrm{f}}$ and ${\mathit{g}}_{l,k,\mathrm{z}\mathrm{f}}$ are designed by ${\mathit{p}}_{\widehat{l},k,\mathrm{z}\mathrm{f}}={\mathit{V}}_{\widehat{l},k,\mathrm{z}\mathrm{f}}{\Omega }_{\widehat{l},k}$ and ${\mathit{g}}_{l,k,\mathrm{z}\mathrm{f}}={\mathit{U}}_{\widehat{l},k,\mathrm{z}\mathrm{f}}^H$ , respectively. ( ${\mathit{p}}_{\widehat{l},k,\mathrm{z}\mathrm{f}}$ denotes the precoding processing at the site of ${\mathrm{U}\mathrm{E}}_{\widehat{l},k}$ and ${\mathit{g}}_{l,k,\mathrm{z}\mathrm{f}}$ denotes the receiving processing at the site of ${\mathrm{U}\mathrm{E}}_{l,k}$ . This type of subscript contributes to distinguishing the two matrices.) Finally, we get the systematic ZF precoding scheme for users.

4.2. MMSE-Based Design

Similar to Section 4.1, the MMSE-based design of precoding matrix termed as MMSE is performed by introducing the MMSE inversion to the combined channel matrix; we have

$$\begin{array}{cc}& {\mathit{H}}_{\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{†}={\mathit{H}}^H{\left(\mathit{H}{\mathit{H}}^H+\alpha I\right)}^{-\mathrm{1}}=\left[\begin{array}{c}{\mathit{H}}_{l,\mathrm{1},\mathrm{m}\mathrm{s}\mathrm{e}}\\ {\mathit{H}}_{l,\mathrm{2},\mathrm{m}\mathrm{s}\mathrm{e}}\\ ⋮\\ {\mathit{H}}_{\widehat{l},K,\mathrm{m}\mathrm{s}\mathrm{e}}\end{array}\right],\end{array}$$

(17)

where ${\mathit{H}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{}\in {\mathbb{C}}^{N_{\widehat{l},k}\times N_R}$ is the submatrix of pseudoinverse matrix ${\mathit{H}}_{\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{†}$ and $\alpha =N\sigma$ . Note that the regularization factor α approaches zero when the SNR is high, and thus we have $\mathit{H}{\mathit{H}}_{\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{†}\approx {\mathit{I}}_{N_R}$ [12]. This means that those off-diagonal block matrices of $\mathit{H}{\mathit{H}}_{\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{†}$ converge to zeros with the increase in SNR. Hence, the matrix ${\mathit{H}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}$ is approximately in the null space of ${\bar{\mathit{H}}}_{\widehat{l},k}$ ; that is,

$$\begin{array}{cc}& {\mathit{H}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}{\bar{\mathit{H}}}_{\widehat{l},k}\approx \mathbf{0}\mathbf{.}\end{array}$$

(18)

Through the QR decomposition of ${\mathit{H}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}={\mathit{Q}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}{\mathit{R}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}$ , where ${\mathit{Q}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}\in {\mathbb{C}}^{N_{\widehat{l},k}\times N_R}$ is also an orthogonal matrix and ${\mathit{R}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}$ is an upper triangular matrix, we have

$$\begin{array}{cc}& {\mathit{H}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}{\bar{\mathit{H}}}_{\widehat{l},k}={\mathit{Q}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}{\mathit{R}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}{\bar{\mathit{H}}}_{\widehat{l},k}\approx \mathbf{0}\mathbf{.}\end{array}$$

(19)

Since ${\mathit{R}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}$ is invertible, we have

$$\begin{array}{cc}& {\mathit{Q}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}{\bar{\mathit{H}}}_{\widehat{l},k}\approx \mathbf{0}\mathbf{.}\end{array}$$

(20)

So the relay forward matrix for $\mathrm{U}{\mathrm{E}}_{\widehat{l},k}$ is designed as

$$\begin{array}{cc}& {\mathit{f}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}={\mathit{Q}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}.\end{array}$$

(21)

By the above process and matching matrix ${\mathbf{\Pi }}_{\widehat{l},k}=I_{N_{l,k}\times N_{\widehat{l},k}}$ , the received signals ${\widehat{\mathit{d}}}_{\widehat{l},k}$ can be rewritten as

$$\begin{array}{l}{\widehat{\mathit{d}}}_{\widehat{l},k}={\mathit{g}}_{l,k}{\mathit{H}}_{l,k}^T{\mathbf{\Pi }}_{\widehat{l},k}{\mathit{Q}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}{\mathit{H}}_{\widehat{l},k}{\mathit{d}}_{\widehat{l},k}\\ \hspace{1em}+{\mathit{g}}_{l,k}{\mathit{H}}_{l,k}^T{\mathbf{\Pi }}_{\widehat{l},k}{\mathit{Q}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}{\mathit{n}}_s+{\mathit{g}}_{l,k}{\mathit{n}}_{l,k}.\end{array}$$

(22)

Then, the MMSE-BD algorithm also can be completed by applying the SVD operation to the effective channel matrix

$$\begin{array}{l}{\mathit{H}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{\mathrm{e}\mathrm{f}\mathrm{f}}={\mathit{H}}_{\widehat{l},k}^T{\mathbf{\Pi }}_{\widehat{l},k}{\mathit{Q}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}{\mathit{H}}_{\widehat{l},k}\\ ={\mathit{U}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}{\mathit{\Lambda }}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}{\mathit{V}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^H.\end{array}$$

(23)

The user precoding and receive decoding beamforming matrices of MMSE are obtained as

$$\begin{array}{cc}& {\mathit{P}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}={\mathit{V}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}{\Omega }_{\widehat{l},k},\\ {\mathit{g}}_{l,k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}={\mathit{U}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^H.\end{array}$$

(24)

Finally, the systematic MMSE precoding schemes for all users are followed as

$$\begin{array}{cc}& \mathit{P}=\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}\left\{{\mathit{P}}_{l,\mathrm{1},\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}\cdots {\mathit{P}}_{l,K,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}},{\mathit{P}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}\cdots {\mathit{P}}_{\widehat{l},K,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}\right\},\\ \mathit{F}=\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}\left\{{\mathit{F}}_{l,\mathrm{1},\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}\cdots {\mathit{F}}_{l,K,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}},{\mathit{F}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}\cdots {\mathit{F}}_{\widehat{l},K,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}\right\},\\ \mathit{G}=\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}\left\{{\mathit{g}}_{l,\mathrm{1}}\cdots {\mathit{g}}_{l,K},{\mathit{g}}_{\widehat{l},k}\cdots {\mathit{g}}_{\widehat{l},K}\right\}.\end{array}$$

(25)

The MMSE algorithm is summarized in Algorithm 1.

Algorithm 1: The MMSE precoding algorithm.

Steps and their operations

Applying the MMSE Channel Inversion

$$(\mathit{1})$$    $${\mathit{H}}_{\text{mmse}}^{†}={\mathit{H}}^H{\left(\mathit{H}{\mathit{H}}^H+\alpha I\right)}^{-\mathrm{1}}$$

$$(\mathit{2})$$ for k = 1 : K

$$(\mathit{3})$$   $$\left[{\mathit{Q}}_{\widehat{l},k,\text{mmse}}{\mathit{R}}_{\widehat{l},k,\text{mmse}}\right]=\text{QR}\left({\mathit{H}}_{\widehat{l},k,\text{mmse}},\mathrm{0}\right)$$

$$(\mathit{4})$$   $${\mathit{f}}_{\widehat{l},k,\text{mmse}}^{\mathrm{1}}={\mathit{Q}}_{\widehat{l},k,\text{mmse}}$$

$$(\mathit{5})$$   $${\mathit{H}}_{\widehat{l},k,\text{mmse}}^{\text{eff}}={\mathit{H}}_{\widehat{l},k}^T{\mathbf{\Pi }}_{\widehat{l},k}{\mathit{Q}}_{\widehat{l},k,\text{mmse}}{\mathit{H}}_{\widehat{l},k}={\mathit{U}}_{\widehat{l},k,\text{mmse}}{\mathit{\Lambda }}_{\widehat{l},k,\text{mmse}}{\mathit{V}}_{\widehat{l},k,\text{mmse}}^H$$

$$(\mathit{6})$$   $${\mathit{f}}_{\widehat{l},k,\text{mmse}}={\mathit{Q}}_{\widehat{l},k,\text{mmse}}$$

$$(\mathit{7})$$   $${\mathit{g}}_{l,k}={\mathit{U}}_{\widehat{l},k,\text{mmse}}^H$$ , $${\mathit{P}}_{\widehat{l},k,\text{mmse}}={\mathit{V}}_{\widehat{l},k,\text{mmse}}$$

$$(\mathit{8})$$ end

 Operate similar circulation to obtain  $${\mathit{f}}_{l,k,\text{mmse}},{\mathit{P}}_{l,k,\text{mmse}}$$   and $${\mathit{g}}_{\widehat{l},k}$$ then Compute overall precoding scheme

$$(\mathit{9})$$    $$\mathit{F}=\mathrm{diag}\left\{{\mathit{F}}_{l,\mathrm{1},\text{mmse}}\cdots {\mathit{F}}_{l,K,\text{mmse}},{\mathit{F}}_{\widehat{l},k,\text{mmse}}\cdots {\mathit{F}}_{\widehat{l},K,\text{mmse}}\right\}$$

$$(\mathit{10})$$    $$\mathit{G}=\mathrm{diag}\left\{{\mathit{g}}_{l,\mathrm{1}}\cdots {\mathit{g}}_{l,K},{\mathit{g}}_{\widehat{l},k}\cdots {\mathit{g}}_{\widehat{l},K}\right\}$$ ,   $$\mathit{P}=\mathrm{diag}\left\{{\mathit{P}}_{l,\mathrm{1},\text{mmse}}\cdots {\mathit{P}}_{l,K,\text{mmse}},{\mathit{P}}_{\widehat{l},k,\text{mmse}}\cdots {\mathit{P}}_{\widehat{l},K,\text{mmse}}\right\}$$

$$(\mathit{13}$$ )   $${\widehat{\mathit{d}}}_{\widehat{l},k}={\mathit{g}}_{l,k}{\mathit{d}}_{\widehat{l},k}+{\mathit{g}}_{l,k}{\mathit{n}}_{\widehat{l},k}^{\text{eff}}$$

4.3. LR-MMSE-Based Design

In this subsection, LR is applied to reduce the dimension of the effective channel which replaces SVD in MMSE algorithm. For convenience, we term it as LR-MMSE (see Figure 2(b)). A powerful and famous reduction criterion for arbitrary lattice dimensions was introduced by Lenstra et al. in [20], and the algorithm they proposed is known as the LLL (or ${\mathrm{L}}^{\mathrm{3}}$ ) algorithm [21, 22]. In order to reduce the complexity, a complex LLL (CLLL) algorithm was in [23], which reduces the overall complexity of the LLL algorithm by nearly half without sacrificing any performance. Through reducing the size of channels, CLLL can obtain better BER performance. In this paper, we employ the CLLL algorithm to implement the LR transform.

After the first MMSE precoding, we transform the MU-MIMO channel into parallel or approximately parallel SU-MIMO channels and the effective channel matrix for the $\mathrm{U}{\mathrm{E}}_{l,k}$ is

$$\begin{array}{cc}& {\mathit{H}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{\mathrm{e}\mathrm{f}\mathrm{f}}={\mathit{H}}_{l,k}^T{\mathbf{\Pi }}_{\widehat{l},k}{\mathit{Q}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}{\mathit{H}}_{\widehat{l},k}.\end{array}$$

(26)

We perform the LR transformation on ${\mathit{H}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{\mathrm{e}\mathrm{f}\mathrm{f}}$ in the precoding scenario [24]; that is,

$$\begin{array}{cc}& {\tilde{\mathit{H}}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{\mathrm{e}\mathrm{f}\mathrm{f}}={\mathit{T}}_{\widehat{l},k}{\mathit{H}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{\mathrm{e}\mathrm{f}\mathrm{f}},\hspace{1em}\hspace{1em}{\mathit{H}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{\mathrm{e}\mathrm{f}\mathrm{f}}={\mathit{T}}_{\widehat{l},k}^{-\mathrm{1}}{\tilde{\mathit{H}}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{\mathrm{e}\mathrm{f}\mathrm{f}},\end{array}$$

(27)

where ${\mathit{T}}_{\widehat{l},k}$ is a unimodular matrix with $\mathrm{d}\mathrm{e}\mathrm{t}\left|{\mathit{T}}_{\widehat{l},k}\right|=\mathrm{1}$ and all elements of ${\mathit{T}}_{\widehat{l},k}$ are complex integers. The unimodular feature of ${\mathit{T}}_{\widehat{l},k}$ guarantees that the energy will not change through the LR transform.

The MMSE precoding is actually equivalent to the ZF precoding with respect to an extended system model [25]. The extended channel matrix $\underset{\_}{\mathit{H}}$ for the MMSE precoding scheme is defined as

$$\begin{array}{cc}& \underset{\_}{\mathit{H}}=\left[\begin{array}{c}\mathit{H}\\ \sqrt{\eta }{\mathit{I}}_N\end{array}\right].\end{array}$$

(28)

By introducing the MMSE method, a trade-off between the level of MUI and the noise is introduced (see also [12]). Then, the LR-M-D precoding filter is designed as

$$\begin{array}{cc}& {\mathit{F}}_{\mathrm{m}\mathrm{s}\mathrm{e}}=\eta {\underset{\_}{\mathit{H}}}^H{\left(\underset{\_}{\mathit{H}}{\underset{\_}{\mathit{H}}}^H\right)}^{-\mathrm{1}},\end{array}$$

(29)

where $\eta =[{\mathit{I}}_N,{\mathbf{0}}_{N\times N_R}]$ and the multiplication by η will not result in transmit power amplifying since $\eta {\sigma }^H={\mathit{I}}_N$ . From the mathematical expression in (15), the rows of H determine the effective transmit power amplification of the MMSE precoding. Correspondingly, the LR transformation for the MMSE precoding can be applied to the transpose of the extended channel matrix ${\underset{\_}{\mathit{H}}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{\mathrm{e}\mathrm{f}\mathrm{f}}=[{\mathit{H}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{\mathrm{e}\mathrm{f}\mathrm{f}},\sqrt{\eta }{\mathit{I}}_{N_{l,k}}]$ , and the LR transformed channel matrix ${\underset{\_}{\tilde{\mathit{H}}}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{\mathrm{e}\mathrm{f}\mathrm{f}}$ is obtained as

$$\begin{array}{cc}& {\underset{\_}{\tilde{\mathit{H}}}}_{\widehat{l},k,\mathrm{L}\mathrm{R}}^{\mathrm{e}\mathrm{f}\mathrm{f}}={\underset{\_}{\mathit{T}}}_{\widehat{l},k}{\underset{\_}{\mathit{H}}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{\mathrm{e}\mathrm{f}\mathrm{f}},\end{array}$$

(30)

where ${\underset{\_}{\mathit{T}}}_{\widehat{l},k}$ is the unimodular matrix for ${\underset{\_}{\mathit{H}}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{\mathrm{e}\mathrm{f}\mathrm{f}}$ . Then, the LR-MMSE relay forwarding filter is given by

$$\begin{array}{cc}& {\mathit{P}}_{\widehat{l},k,\mathrm{L}\mathrm{R}}={\eta }_{\widehat{l},k}{\left({\underset{\_}{\tilde{\mathit{H}}}}_{\widehat{l},k,\mathrm{L}\mathrm{R}}^{\mathrm{e}\mathrm{f}\mathrm{f}}\right)}^H{\left[{\underset{\_}{\tilde{\mathit{H}}}}_{\widehat{l},k,\mathrm{L}\mathrm{R}}^{\mathrm{e}\mathrm{f}\mathrm{f}}{\left({\underset{\_}{\tilde{\mathit{H}}}}_{\widehat{l},k,\mathrm{L}\mathrm{R}}^{\mathrm{e}\mathrm{f}\mathrm{f}}\right)}^H\right]}^{-\mathrm{1}}{\Omega }_{\widehat{l},k},\end{array}$$

(31)

where the matrix ${\eta }_{\widehat{l},k}=\left[\begin{array}{c}{\mathit{I}}_{M_{\widehat{l},k}}\\ {\mathbf{0}}_{(N_{\widehat{l},k}-M_{\widehat{l},k})\times M_{\widehat{l},k}}\end{array}\right]$ and $M_{\widehat{l},k}$ is the dimension of ${\underset{\_}{\tilde{\mathit{H}}}}_{\widehat{l},k,\mathrm{L}\mathrm{R}}^{\mathrm{e}\mathrm{f}\mathrm{f}}$ . Since LR transformation can reduce the matrix dimension, there should be $N_{\widehat{l},k}>M_{\widehat{l},k}$ . Finally, the systematic MMSE precoding scheme for all users is followed as

$$\begin{array}{cc}& \mathit{P}=\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}\left\{{\mathit{P}}_{l,\mathrm{1},\mathrm{L}\mathrm{R}}\cdots {\mathit{P}}_{l,K,\mathrm{L}\mathrm{R}},{\mathit{P}}_{\widehat{l},k,\mathrm{L}\mathrm{R}}\cdots {\mathit{P}}_{\widehat{l},K,\mathrm{L}\mathrm{R}}\right\},\\ \mathit{F}=\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}\left\{{\mathit{f}}_{l,\mathrm{1},\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}\cdots {\mathit{f}}_{l,K,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}},{\mathit{f}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}\cdots {\mathit{f}}_{\widehat{l},K,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}\right\}.\end{array}$$

(32)

Since the lattice reduced precoding matrix P has near orthogonal columns, the required transmit power will be reduced compared to the MMSE precoding algorithms. Thus, a better BER performance than that of the BD precoding algorithms can be achieved by the proposed LR-MMSE precoding algorithm. The left work for receiver is to quantize the signals $\widehat{\mathit{d}}$ to the nearest vectors. The LR-MMSE algorithm is summarized in Algorithm 2.

Algorithm 2: The LR-MMSE precoding algorithm.

Steps and their operations

Applying the MMSE Channel Inversion

$$(\mathit{1})$$    $${\mathit{H}}_{\text{mmse}}^{†}={\mathit{H}}^H{\left(\mathit{H}{\mathit{H}}^H+\alpha I\right)}^{-\mathrm{1}}$$

$$(\mathit{2})$$   for k = 1 : K

$$(\mathit{3})$$   $$\left[{\mathit{Q}}_{\widehat{l},k,\text{mmse}}{\mathit{R}}_{\widehat{l},k,\text{mmse}}\right]=\text{QR}\left({\mathit{H}}_{\widehat{l},k,\text{mmse}},\mathrm{0}\right)$$

$$(\mathit{4})$$   $${\mathit{f}}_{\widehat{l},k,\text{mmse}}={\mathit{Q}}_{\widehat{l},k,\text{mmse}}$$

$$(\mathit{5})$$   $${\mathit{H}}_{\widehat{l},k,\text{mmse}}^{\text{eff}}={\mathit{H}}_{\widehat{l},k}^T{\mathbf{\Pi }}_{\widehat{l},k}{\mathit{Q}}_{\widehat{l},k,\text{mmse}}{\mathit{H}}_{\widehat{l},k}={\mathit{U}}_{\widehat{l},k,\text{mmse}}{\mathit{\Lambda }}_{\widehat{l},k,\text{mmse}}{\mathit{V}}_{\widehat{l},k,\text{mmse}}^H$$

$$(\mathit{6})$$   $${\underset{\_}{\mathit{H}}}_{\widehat{l},k,\text{mmse}}^{\text{eff}}=\left[{\mathit{H}}_{\widehat{l},k,\text{mmse}}^{\text{eff}},\sqrt{\sigma }{\mathit{I}}_{N_{l,k}}\right]$$

$$(\mathit{7})$$   $$\left[{\underset{\_}{\mathit{T}}}_{\widehat{l},k},{\underset{\_}{\mathit{H}}}_{\widehat{l},k,\text{LR}}^{\text{eff}}\right]=\text{CLLL}\left({\underset{\_}{\mathit{H}}}_{\widehat{l},k,\text{mmse}}^{\text{eff}}\right)$$

$$(\mathit{8})$$   $${\eta }_{\widehat{l},k}=\left[\begin{array}{c}{\mathit{I}}_{M_{\widehat{l},k}}\\ {\mathbf{0}}_{(N_{\widehat{l},k}-M_{\widehat{l},k})\times M_{\widehat{l},k}}\end{array}\right]$$

$$(\mathit{9})$$    $${\mathit{P}}_{\widehat{l},k,\text{LR}}={\eta }_{\widehat{l},k}{\left({\underset{\_}{\mathit{H}}}_{\widehat{l},k,\text{LR}}^{\text{eff}}\right)}^H{\left[{\underset{\_}{\mathit{H}}}_{\widehat{l},k,\text{LR}}^{\text{eff}}{\left({\underset{\_}{\mathit{H}}}_{\widehat{l},k,\text{LR}}^{\text{eff}}\right)}^H\right]}^{-\mathrm{1}}$$

$$(\mathit{10})$$   end

 Operate similar circulation to obtain $${\mathit{f}}_{l,k,\text{mmse}}$$   and   $${\mathit{P}}_{l,k,\text{mmse}}$$ then Compute overall precoding scheme

$$(\mathit{10})$$    $$\mathit{F}=\mathrm{diag}\left\{{\mathit{F}}_{l,\mathrm{1},\text{mmse}}\cdots {\mathit{F}}_{l,K,\text{mmse}},{\mathit{F}}_{\widehat{l},k,\text{mmse}}\cdots {\mathit{F}}_{\widehat{l},K,\text{mmse}}\right\}$$

$$(\mathit{11})$$    $$\mathit{P}=\mathrm{diag}\left\{{\mathit{P}}_{l,\mathrm{1},\text{LR}}\cdots {\mathit{P}}_{l,K,\text{LR}},{\mathit{P}}_{\widehat{l},k,\text{LR}}\cdots {\mathit{P}}_{\widehat{l},K,\text{LR}}\right\}$$

$$(\mathit{14})$$    $${\widehat{\mathit{d}}}_{\widehat{l},k}={\mathit{T}}_{\widehat{l},k}^{-\mathrm{1}}{\mathit{d}}_{\widehat{l},k}+\sqrt{{\eta }_{\widehat{l},k}}{\mathit{n}}_{\widehat{l},k}^{\text{eff}}$$

5. Simulation and Performance Analysis

In this section, we analyze the proposed LR-MMSE algorithm in terms of computational complexity, architecture, BER, and achievable rates.

5.1. Computational Complexity Analysis

In this section, we measure the computational complexity of the precoding algorithms we have introduced by the total number of floating point operations Per Second (FLOPS). Note that the LR algorithm has complex variable, and the average complexity of CLLL algorithm has been given in [26] by FLOPS. The number of FLOPS for the complex QR decomposition and the real SVD operation are given in [27]. Moreover, the FLOPS number in a $x\times y$ complex SVD operation is equivalent to its extended $\mathrm{2}x\times \mathrm{2}y$ real matrix. The total FLOPS number required by the matrix operations is summarized below:

• (i)

multiplication of $x\times y$ and $y\times z$ complex matrices: $\mathrm{8}xyz-\mathrm{2}xz$ ;

(ii)

QR decomposition of a $x\times y\hspace{0.17em}\hspace{0.17em}\left(x\le y\right)$ complex matrix: $\mathrm{16}\left(y^{\mathrm{2}}x-x^{\mathrm{2}}y+\left(\mathrm{1}/\mathrm{3}\right)x^{\mathrm{3}}\right)$ ;

(iii)

SVD of a $x\times y\hspace{0.17em}\hspace{0.17em}\left(x\le y\right)$ complex matrix by only obtaining Λ and V : $\mathrm{32}\left(y{x}^{\mathrm{2}}+\mathrm{2}{x}^{\mathrm{3}}\right)$ ;

(iv)

SVD of a $x\times y\hspace{0.17em}\hspace{0.17em}\left(x\le y\right)$ complex matrix by obtaining U and Λ and V : $\mathrm{8}\left(\mathrm{4}{y}^{\mathrm{2}}x+\mathrm{8}y{x}^{\mathrm{2}}+\mathrm{9}{x}^{\mathrm{3}}\right)$ ;

(v)

inversion of a $x\times x$ real matrix by Gauss-Jordan elimination: $\mathrm{4}{x}^{\mathrm{3}}/\mathrm{3}$ ;

(vi)

inversion of a $x\times x$ complex matrix by Gauss-Jordan elimination: $\mathrm{16}{x}^{\mathrm{3}}-\mathrm{8}{x}^{\mathrm{2}}+\mathrm{2}x$ .

The FLOPS of three types of precoding schemes are shown in Tables 1, 2, and 3, respectively. We assume the system is composed of 6 users ( $K=3$ ) equipped 2 antennas and one relay station equipped 12 antennas. From Table 1, apparently, we know that the complexity of SVD is very large, resulting in the BD's complexity reaching a level of million. On the contrary, the combined channel inversion needs only 22805.4 FLOPS which operates in merely one time. Meanwhile, the LR-MMSE can save 35% of FLOPS of MMSE, while MMSE has saved 89.8% of FLOPS of BD.

Table 1

Computational complexity of BD in case (6, 2, 12).

Steps	Operations	FLOPS	Number
$$(\mathit{1})$$	$${\bar{\mathit{U}}}_{\widehat{l},k}{\bar{\mathit{\Lambda }}}_{\widehat{l},k}{\bar{\mathit{V}}}_{\widehat{l},k}^H$$	$$\mathrm{64}K\left({\bar{N}}_{\widehat{l},k}{N}_R^{\mathrm{2}}+\mathrm{2}{N}_R^{\mathrm{3}}\right)$$	1880064
$$(\mathit{2})$$	Obtain $${\mathit{H}}_{\widehat{l},k}^{\text{eff}}$$	$$\mathrm{2}K\left(\mathrm{8}{N}_{l,k}{N}_R{\bar{N}}_{\widehat{l},k}-\mathrm{2}{N}_{l,k}{\bar{N}}_{\widehat{l},k}\right)$$	22560
$$(\mathit{3})$$	SVD on $${\mathit{H}}_{\widehat{l},k}^{\text{eff}}$$	$$\mathrm{16}K\left(\mathrm{4}{\bar{N}}_{\widehat{l},k}^{\mathrm{2}}{N}_{l,k}+\mathrm{8}{\bar{N}}_{\widehat{l},k}{N}_{l,k}^{\mathrm{2}}+\mathrm{9}{N}_{l,k}^{\mathrm{3}}\right)$$	114432  Total 2017056

Table 2

Computational complexity of MMSE in case (6, 2, 12).

Steps	Operations	FLOPS	Number
$$(\mathit{1})$$	$${\mathit{H}}_{\text{mmse}}^{†}$$	$$\mathrm{16}{N}_R^{\mathrm{3}}-\mathrm{10}{N}_R^{\mathrm{2}}+\mathrm{8}{N}_R^{\mathrm{2}}N+\mathrm{8}{N}_R{N}^{\mathrm{2}}+\mathrm{2}{N}_R$$	22805.4
$$(\mathit{2})$$	$${\mathit{Q}}_{\text{mmse}}^{†}{\mathit{R}}_{\text{mmse}}^{†}$$	$$\mathrm{32}K\left(N_R^{\mathrm{2}}{N}_{\widehat{l},k}-N_{\widehat{l},k}^{\mathrm{2}}{N}_R+\frac{\mathrm{1}}{\mathrm{3}}{N}_{\widehat{l},k}^{\mathrm{3}}\right)$$	46592.64
$$(\mathit{3})$$	Obtain $${\mathit{H}}_{\widehat{l},k,\text{mmse}}^{\text{eff}}$$	$$\mathrm{2}K\left(\mathrm{8}{N}_{l,k}{N}_R{\bar{N}}_{\widehat{l},k}-\mathrm{2}{N}_{l,k}{\bar{N}}_{\widehat{l},k}\right)$$	22560
$$(\mathit{4})$$	SVD on $${\mathit{H}}_{\text{mmse}}^{\text{eff}}$$	$$\mathrm{16}K\left(\mathrm{4}{\bar{N}}_{\widehat{l},k}^{\mathrm{2}}{N}_{l,k}+\mathrm{8}{\bar{N}}_{\widehat{l},k}{N}_{l,k}^{\mathrm{2}}+\mathrm{9}{N}_{l,k}^{\mathrm{3}}\right)$$	114432 Total 206389

Table 3

Computational complexity of LR-MMSE in case (6, 2, 12).

Steps	Operations	FLOPS	Number
$$(\mathit{1})$$	$${\mathit{H}}_{\text{mmse}}^{†}$$	$$\frac{\mathrm{4}}{\mathrm{3}}{N}_R^{\mathrm{3}}-\mathrm{2}{N}_R^{\mathrm{2}}+\mathrm{12}{N}_R^{\mathrm{2}}N-\mathrm{2}{N}_{R}N$$	22805.4
$$(\mathit{2})$$	$${\mathit{Q}}_{\text{mmse}}^{†}{\mathit{R}}_{\text{mmse}}^{†}$$	$$\mathrm{32}K\left(N_R^{\mathrm{2}}{N}_{\widehat{l},k}-N_{\widehat{l},k}^{\mathrm{2}}{N}_R+\frac{\mathrm{1}}{\mathrm{3}}{N}_{\widehat{l},k}^{\mathrm{3}}\right)$$	46592.64
$$(\mathit{3})$$	Obtain $${\mathit{H}}_{\widehat{l},k}^{\text{eff}}$$	$$\mathrm{2}K\left(\mathrm{8}{N}_{l,k}{N}_R{\bar{N}}_{\widehat{l},k}-\mathrm{2}{N}_{l,k}{\bar{N}}_{\widehat{l},k}\right)$$	22560
$$(\mathit{4})$$	$${\underset{\_}{\mathit{H}}}_{\widehat{l},k,\text{LR}}^{\text{eff}}$$	CLLL transform	38387.97
$$(\mathit{5})$$	$${\mathit{P}}_{\widehat{l},k,\text{LR}}$$	$$\mathrm{2}K\left(\frac{\mathrm{4}}{\mathrm{3}}{N}_{l,k}^{\mathrm{3}}+\mathrm{12}{N}_{l,k}^{\mathrm{3}}-\mathrm{4}{N}_{l,k}^{\mathrm{2}}\right)$$	544 Total 133169.5

Next, we fix the antennas of users and reveal the low complexity of LR-MMSE's property by enlarging the number of users. Ruling $N_R=N$ , we can obtain the performance comparison in Figure 3.

Figure 3

Computational complexity versus K by fixing each user's antennas.

Figure 3 shows that the complexity of BD-type precoding scheme grows rapidly with the increase in k, while it grows gently with respect to MMSE and LR-MMSE. This phenomenon is due to the significant influence of dimensions with regards to SVD. Note that the QR decomposition also needs to be implemented in K times, though there is only one time of the combined channel inversion. But, the QR decomposition is simpler than SVD.

5.2. Performance Analysis in Architecture

Compared to the BD and MMSE scheme, the LR-MMSE has a simple architecture. At the site of each user, the decoding matrix can be omitted, since the precoding processing in formula (31) has directly decoded the effective channel. This outperformance can contribute to saving a significant power of mobile users, even the relay stations. Moreover, it is worth noting that we need not employ the self-interference eliminating technology, due to the combined channel inversion having included the channel information of the receiver.

5.3. BER Performance Analysis

Recall that, with the increase in SNR, we have

$$\begin{array}{cc}& {\mathit{Q}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}{\bar{\mathit{H}}}_{\widehat{l},k}\approx \mathbf{0}\end{array}$$

(33)

in the LR-MMSE algorithm. Thus, the MU-MIMO channel is approximately decoupled into equivalent SU-MIMO channel. In most schemes of considering the channel noise, RBD is a common method. A result is shown in [28], which is like formula (33), which is not converged to zeros. Comparing the above two characters, LR-MMSE has more accurate receiving.

As we known, the condition number is a measurement of receiving error bits. In virtue of the defined condition number in [23],

$$\begin{array}{cc}& \mathrm{c}\mathrm{o}\mathrm{n}\mathrm{d}\left(\mathit{H}\right)={‖\mathit{H}‖}_F·{‖{\mathit{H}}^{-\mathrm{1}}‖}_F,\end{array}$$

(34)

the channel matrix can be detected with respect to orthogonality. Taking an example in a system of linear equations,

$$\begin{array}{cc}& \mathit{A}\mathit{x}=\mathit{b}.\end{array}$$

(35)

There are three cases for different kinds of matrices:

• (i)

  when the channel matrix is an orthogonal matrix, the condition number is 1; x will change a little with tiny changing b ;

• (ii)

  when it is a singular matrix, the condition number is ∞; x will change significantly with tiny changing b ; even b has not changed;

• (iii)

  when it is a nonsingular but not orthogonal matrix, the condition number will be large, and x changes between the above two cases.

As can be seen in Figure 4, the LR-MMSE has a smaller average $\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{d}(\mathbf{H})$ of the effective channel compared to MMSE and BD. Thus, a significant power reduction and better BER performance are obtained.

Figure 4

The $\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{d}(\mathit{H})$ statistics of 1000 6∗6 channel matrices.

5.4. Achievable Sum-Rate Analysis

We have mentioned that LR-MMSE is similar to the LR-ZF. Thus, the received signal can be rewritten as

$$\begin{array}{l}{\widehat{\mathit{d}}}_{\widehat{l},k}={\mathit{H}}_{l,k}^T{\mathit{f}}_{l,k}{\mathit{H}}_{\widehat{l},k}{\mathit{P}}_{\widehat{l},k}{\mathit{d}}_{\widehat{l},k}+{\mathit{H}}_{l,k}^T{\mathit{f}}_{\widehat{l},k}{\mathit{n}}_s+{\mathit{n}}_{l,k}\\ ={\mathit{T}}_{\widehat{l},k}^{-\mathrm{1}}{\mathit{d}}_{\widehat{l},k}+\sqrt{{\psi }_{\widehat{l},k}}{\mathbf{n}}_{\widehat{l},k}^{\mathrm{e}\mathrm{f}\mathrm{f}},\end{array}$$

(36)

where $\sqrt{{\psi }_{\widehat{l},k}}$ is a detecting scalar and ${\psi }_{\widehat{l},k}={‖{\left({\mathit{H}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{\mathrm{e}\mathrm{f}\mathrm{f}}\right)}^{-\mathrm{1}}{\mathit{T}}_{\widehat{l},k}^{-\mathrm{1}}{\mathit{d}}_{\widehat{l},k}‖}_F^{\mathrm{2}}$ . Ordering ${\mathit{z}}_{\widehat{l},k}={\mathit{T}}_{\widehat{l},k}^{-\mathrm{1}}{\mathit{d}}_{\widehat{l},k}$ , we have the normalization factor

$$\begin{array}{l}{\psi }_{\widehat{l},k}={‖{\left({\mathit{H}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{\mathrm{e}\mathrm{f}\mathrm{f}}\right)}^{-\mathrm{1}}{\mathit{z}}_{\widehat{l},k}‖}_F^{\mathrm{2}}\\ ={‖{\left({\mathit{U}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}{\mathit{\Lambda }}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}{\mathit{V}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^H\hspace{0.17em}\right)}^{-\mathrm{1}}{\mathit{z}}_{\widehat{l},k}‖}_F^{\mathrm{2}}\\ ={‖{\mathit{U}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}\hspace{0.17em}{\mathit{\Lambda }}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{-\mathrm{1}}{\mathit{V}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^H{\mathit{z}}_{\widehat{l},k}‖}_F^{\mathrm{2}}.\end{array}$$

(37)

For a unitary matrix, the norm satisfies

$$\begin{array}{cc}& {‖\mathit{Q}\mathit{x}‖}_F^{\mathrm{2}}={\mathit{x}}^H{\mathit{Q}}^H\mathit{Q}\mathit{x}={\mathit{x}}^H\mathit{x}={‖\mathit{x}‖}_F^{\mathrm{2}}.\end{array}$$

(38)

Thus, we have

$$\begin{array}{l}E\left\{{\eta }_{\widehat{l},k}\right\}\\ \hspace{1em}=E\left\{{‖\hspace{0.17em}{\mathit{\Lambda }}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{-\mathrm{1}}{\mathit{V}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^H{\mathit{z}}_{\widehat{l},k}‖}_F^{\mathrm{2}}\right\}\\ \hspace{1em}=E\left\{\mathrm{T}\mathrm{r}\left({\mathit{\Lambda }}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{-\mathrm{1}}{\mathit{V}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^H{\mathit{z}}_{\widehat{l},k}{\mathit{z}}_{\widehat{l},k}^H{\mathit{V}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}{\mathit{\Lambda }}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{-\mathrm{1}}\right)\right\}\\ \hspace{1em}=\mathrm{T}\mathrm{r}\left(\hspace{0.17em}{\mathit{\Lambda }}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{-\mathrm{1}}{\mathit{V}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^H\mathit{E}\left\{{\mathit{z}}_{\widehat{l},k}{\mathit{z}}_{\widehat{l},k}^H\right\}{\mathit{V}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}{\mathit{\Lambda }}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{-\mathrm{1}}\right)\\ \hspace{1em}={\xi }_{\widehat{l},k,i}^{\mathrm{2}}\mathrm{T}\mathrm{r}\left({\mathit{\Lambda }}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{-\mathrm{1}}{\mathit{V}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^H{\mathit{V}}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{}{\mathit{\Lambda }}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{-\mathrm{1}}\right)\\ \hspace{1em}={\xi }_{\widehat{l},k,i}^{\mathrm{2}}\mathrm{T}\mathrm{r}\left({\mathit{\Lambda }}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}^{-\mathrm{2}}\right)\\ \hspace{1em}={\displaystyle \sum }_{i=\mathrm{1}}^{L_{\mathrm{e}\mathrm{f}\mathrm{f}}}\frac{{\xi }_{\widehat{l},k,i}^{\mathrm{2}}}{{\lambda }_{\widehat{l},k,i}^{\mathrm{2}}},\end{array}$$

(39)

where ${\xi }_{\widehat{l},k,i}$ is the transferring power of ith stream of ${\mathit{z}}_{\widehat{l},k}$ and ${\lambda }_{\widehat{l},k,i}$ is the ith singular of ${\mathit{\Lambda }}_{\widehat{l},k,\mathrm{m}\mathrm{m}\mathrm{s}\mathrm{e}}$ . Naturally, the SNR with respect to the ith stream is given by

$$\begin{array}{cc}& \mathrm{S}\mathrm{N}{\mathrm{R}}_{\widehat{l},k}^i=\frac{{\xi }_{\widehat{l},k,i}^{\mathrm{2}}}{{\left({\sigma }_{\widehat{l},k}^{\mathrm{e}\mathrm{f}\mathrm{f}}\right)}^{\mathrm{2}}{\sum }_{i=\mathrm{1}}^{L_{\mathrm{e}\mathrm{f}\mathrm{f}}}\left({\xi }_{\widehat{l},k,i}^{\mathrm{2}}/{\lambda }_{\widehat{l},k,i}^{\mathrm{2}}\right)}.\end{array}$$

(40)

Thus, the $\mathrm{U}{\mathrm{E}}_{l,k}$ 's achievable sum-rate is obtained as

$$\begin{array}{l}{r}_{l,k}={\displaystyle \sum }_{i=\mathrm{1}}^{L_{\mathrm{e}\mathrm{f}\mathrm{f}}}\mathrm{l}\mathrm{o}\mathrm{g}\left(\mathrm{1}+\mathrm{S}\mathrm{N}{\mathrm{R}}_{\widehat{l},k}^i\right)\\ ={\displaystyle \sum }_{i=\mathrm{1}}^{L_{\mathrm{e}\mathrm{f}\mathrm{f}}}\mathrm{l}\mathrm{o}\mathrm{g}\left(\mathrm{1}+\frac{{\xi }_{\widehat{l},k,i}^{\mathrm{2}}}{{\left({\sigma }_{\widehat{l},k}^{\mathrm{e}\mathrm{f}\mathrm{f}}\right)}^{\mathrm{2}}{\psi }_{\widehat{l},k}}\right).\end{array}$$

(41)

Note that $r_{l,k}$ decreases with ${\eta }_{\widehat{l},k}$ and converges to a maximum with ${\xi }_{\widehat{l},k,\mathrm{1}}^{\mathrm{2}}/{\lambda }_{\widehat{l},k,\mathrm{1}}^{\mathrm{2}}={\xi }_{\widehat{l},k,i}^{\mathrm{2}}/{\lambda }_{\widehat{l},k,i}^{\mathrm{2}}=\cdots ={\xi }_{\widehat{l},k,L_{\mathrm{e}\mathrm{f}\mathrm{f}}}^{\mathrm{2}}/{\lambda }_{\widehat{l},k,L_{\mathrm{e}\mathrm{f}\mathrm{f}}}^{\mathrm{2}}$ .

6. Simulation Results

Considering a specification of antenna with $N=8$ , $N_R=\mathrm{10}$ , and $N_{l,k}=\mathrm{2}$ , six users are divided into two symmetrical groups. The perfect channel scenario is applied in which $\sigma =\mathrm{1}$ and all channels’ elements are fetched from the complex Gaussian process. For highlighting the performance of the proposed algorithm, we allocate the transmit power equally to each user.

Figure 5 shows that the LR-MMSE algorithm has the best sum-rate performance when MMSE and BD have a similar achievable sum-rate. In particular, when the SNR is 30 dB, the proposed algorithm has a 3.208 bit/s/Hz gain compared to the BD algorithm. Since the low SNR will affect the diagonality after the precoding processing in MMSE scheme, there is an inevitable performance loss. However, by virtue of the size reduction of CLLL transform, the LR-MMSE has obtained a performance compensation.

Figure 5

The achievable sum-rate versus SNR.

Additionally, in the process of simulating, we discover that choosing the optimal matching matrix Π can achieve nearly 1 bit/s/Hz gain. In this work, we did not study the optimization of power and matching matrices.

7. Conclusion

In this work, we have proposed a low complexity precoding scheme of two-way MU-MIMO relay systems with QR decomposition and complex LLL transform instead of two times SVD. For simplicity, users are distributed in two symmetrical groups. The performance analysis show that the LR-MMSE algorithm has not only a low complexity, but also a better BER, a simple structure at the site of the receiving user, and a theoretical maximum. Finally, a higher achievable sum-rate of the system is confirmed by simulation.