Dynamical Spectrum Sharing and Medium Access Control for Heterogeneous Cognitive Radio Networks

Abstract

This paper tackles the issue of spectrum sharing and medium access control among heterogeneous secondary users. Two solutions are proposed in this paper. The first solution can be used in centralized fashion where a central entity exists which decides transmission power for all secondary users. This solution tries to minimize the time required by secondary users to clear their queues. The second solution assumes the autonomy of secondary users where the decision to update transmission power is distributed among users. Dynamical system approach is used to model system behavior. The trajectory of interference noise level suffered by secondary users is used to update transmission power at the beginning of each time frame based on the proposed dynamic power assignment rule. This rule couples the responses of all secondary users in a way which simplifies future interference noise forecasting. A forecasting engine based on deep neural network is proposed. This engine gives secondary users the ability to acquire useful knowledge from surrounding wireless environment. As a result, better transmission power allocation is achieved. Evaluation experiments have confirmed that adopting deep neural network can improve the performance by 46% on average. All of the proposed solutions have achieved an outstanding performance.

1. Introduction

Cognitive radio is a major step in the evolution of wireless communications [1]. It provides wireless systems with the ability to adapt to the changes in the environment and to increase its utilization. By observing their surroundings, wireless systems should update their communication parameters in a way that increases their performance. The most important issue in system utilization is spectrum usage which can be considered as the main motivation behind cognitive radio paradigm. The available wireless spectrum is a very scarce resource and highly underutilized [2]. Cognitive radio systems (secondary users) should take the opportunity and they should try to use the underutilized spectrum to improve their performance. This is easier said than done. A serious problem will arise if naïve policy of unorganized attempts by multiple cognitive systems to utilize the same spectrum is adopted by these systems. This naïve way of utilizing Idle spectrum will lead to chaos and high competitive situations among cognitive radio systems. Also, other resources will be wasted such as power and computation. Hence, secondary users should figure out a way to share the underutilized spectrum without adding more problems to each other.

Spectrum sharing is a key challenge in cognitive radio technologies. There are two types of spectrum sharing. The first is primary-secondary sharing where the primary users allow secondary users to use their spectrum under some conditions. This first type is divided into two categories which are primary-secondary underlay sharing and primary-secondary overlay sharing [3]. In underlay paradigm, the secondary user operates on the primary user channel with promise of not causing interference higher than some threshold. In overlay paradigm, the primary user allows the secondary user to use its channel in return for some service such as relaying. The second type of spectrum sharing is secondary-secondary sharing where secondary users utilize the CR spectrum based on some sharing policy.

1.1. Related Works

Numerous works in literatures tried to tackle the first type (primary-secondary) of spectrum sharing issue [4–24]. For example, convex optimization was used by [4] to show the possibility of having both of the primary user and secondary user operating in the same spectrum. Their approach is based on using multiple antennas for the secondary user transmission so that spatial multiplexing can be used to guarantee low interference constraints at the primary user. In addition, their approach is able to operate on multiple channels at the same time which means that their approach is capable of having high level of reconfigurability in spatial, time, and frequency domains.

Another example of primary-secondary sharing is provided by [5] that followed a game theory approach based on oligopoly market strategies. In their approach, primary users are dynamically evaluating their prices in which they offer spectrum access to secondary users. The cost for primary users is measured in terms of quality of service degradation of primary transmission. Niyato and Hossain analysis of their approach found that Nash equilibrium of their game is not optimal from primary users perspective. Therefore, they forced primary users to choose strategies which achieve global optimality and they provided a mechanism to punish deviated primary users (i.e., primary users who do not follow globally optimal strategies) to ensure stability. However, their approach is not fully robust because it requires that all primary users be aware of the punishment mechanism.

A similar market-based approach is adopted by [8] where auction theory is used. The proposed mechanism is based on secondary users submitting bids for auctioned spectrum by primary users. Then, the primary user will choose the best bid which achieves stable equilibrium. To generalize the proposed solution to multiple primary users, several conditions have to be satisfied to achieve stability which is the main weakness of this approach. Examples of conditions are the fact that primary users always consider other wireless stations interest; if they do not, there should be an entity that punishes uncooperative primary users. In addition, every primary user is aware of all other primary users. If it is not, the proposed solution will not be stable.

The work in [16] used simulated annealing to form their solution. It is based on primary users allowing secondary users to operate in the spectrum while obeying interference cap. In general their approach had good performance. However, it requires a centralized global optimization where simulated annealing is applied. Reference [15] followed a different path to tackle spectrum sharing issue. They assumed that secondary networks have relay nodes which can be used to ease cognitive radio operations. Each secondary user is supposed to select transmission through a relay node which guarantees interference constraint of primary users. Their solution showed very good performance. Nevertheless, it requires an existing supportive relay infrastructure to be effective.

Literature lacks the needed research to tackle the second type (secondary-secondary) of spectrum sharing. Most of the references worked on the second-type assumed homogeneity of secondary users and Idle primary channels [25–33]. For instance, [26] proposed a mechanism which distributes channels among secondary users in a way that minimizes interference among them. They followed game theoretic approach to develop their solution. To improve the performance, Pigouvian taxation was incorporated into the proposed mechanism. On the other hand, [33] was concerned with spectrum sharing and MAC in wide secondary networks where each secondary user can access a different subset of available channels. They formulated such system as a mixed-integer nonlinear problem which is very hard to solve. As a result, linearization and relaxation techniques were used to ease the problem. The proposed solution is iterative in nature and it was able to achieve near-optimal performance according to their problem formulation.

Some view works tried to handle heterogeneous cognitive radio settings [34–41]. An example of such works is proposed by [34]. Here, the authors suggested a two-phase approach. The first phase distributes channels among secondary users. Then, the second phase assigns transmission power on these channels. Their approach was able to perform well. However, it requires a centralized scheduler which introduces communication and cooperation overhead. Similarly, [35] tried to tackle heterogeneous spectrum sharing and MAC by proposing cross layer approach. It is based on classifying channels probabilistically where Hungarian algorithm is used in the next step to schedule channels among secondary users. The proposed solution was able to outperform greedy techniques.

1.2. Motivation and Contributions

Most of the discussed related works assumed some sort of cooperation among secondary users. These facts motivated the authors of this paper to investigate the second type of spectrum sharing and medium access control in harsher wireless environment. In this environment, secondary users are assumed to be totally heterogeneous without any cooperation among them. In addition, primary user channels are assumed to be heterogeneous as well, where they have different bandwidth and channel characteristics (i.e., fading and shadowing).

This paper tries to contribute to the existing cognitive radio literature by delivering three main contributions:

• (1)

The concept of Spectrum-Time Duality is highlighted which states that it is beneficial for secondary users to reduce their level of competition with other secondary users by giving spectrum in return for time and vice versa. The idea depends on the fact that secondary users can be Idle if they transmitted all their data. For a secondary user which still has data to send, its jobs get easier after the others become Idle. Based on this concept, centralized transmission power mechanism is introduced where the main objective is to clear the secondary users queues as soon as possible so that the competition for other secondary users is reduced.

(2)

Distributed transmission power rule is developed so that secondary users can assign their transmission power according to closed dynamical system evolution. By using this rule, the trajectory evolution of many system parameters can be predicted since it follows strict dynamics. Examples of system parameters are queue backlogs of secondary users, interference noise caused by secondary users, and the actual transmission power of all secondary users.

(3)

Advanced method is proposed to employ deep neural network (DNN) [42] which is a very promising machine learning technique. This method utilizes trajectory evolution of interference noise levels to forecast future noise levels. As a result, performance of transmission power rule is greatly enhanced. By employing DNN in cognitive radio technology, real cognition capabilities can be realized. Secondary users will be able to generate decisions without the intervention of system designer. To the best of our knowledge, this is the first attempt in literature to employ DNN in cognitive radio networks.

Note that the proposed solutions are designed for cognitive radio technology since it is the main technology of next generation wireless communications. Nevertheless, these solutions can be easily implemented in conventional wireless technologies.

This paper is organized as follows: Section 2 defines the adopted system model to develop the proposed solutions; Section 3 introduces the proposed centralized spectrum sharing and access mechanism; Section 4 develops distributed transmission power rule and noise forecasting engine as the second proposed solution; intensive evaluation experiments are discussed in Section 5 and the conclusion is delivered as Section 6.

2. System Model

In this section, the general formulation of the spectrum sharing problem in cognitive radio networks is introduced. Then, this formulation will be modified to incorporate important aspects which can be used to ease the proposed solution development.

2.1. General Spectrum Sharing Problem

Consider a set of several secondary users pairs (transmitter and receiver) $U=\left\{u_{\mathrm{1}},u_{\mathrm{2}},\dots ,u_{\left|U\right|}\right\}$ and a set of primary user channels $C=\left\{c_{\mathrm{1}},c_2,\dots ,c_{\left|C\right|}\right\}$ . Each primary user channel has a state of being either Idle or Busy depending on primary user operations. Every $u_i$ perceives channels states according to its sensing capability where it depends on probabilities of detection and false alarm. Let us take O as the matrix which represents how each user perceives channel occupancy states. Elements in O ( $o_{ij}$ denotes the jth channel state according to the ith user) are either 0 if the channel state is Idle or 1 if it is Busy. For simplicity, $o_{ij}=\mathrm{0}$ only if the actual channel state is Idle and the user thinks channel state is Idle as well. Otherwise, $o_{ij}=\mathrm{1}$ if the actual channel state is Busy or if the user thinks that channel state is Busy even if it is not. Also, $o_{ij}=\mathrm{1}$ if the user thinks that channel state is Idle while the actual state is Busy since any transmission attempt on the channel will not be successful because of the high interference from the primary user. Let ${\overline{o}}_k$ denote the actual state of channel $c_k$ .

The transmitter in every secondary user pair $u_i$ uses $p_{ik}$ as transmission power on channel $c_k$ , where $\mathrm{0}\le p_{ik}\le p_k^{\mathrm{m}\mathrm{a}\mathrm{x}}$ . Assume that primary user transmission power on channel $c_k$ is much larger than the maximum transmission power of secondary users ( ${\overline{p}}_k\gg p_k^{\mathrm{m}\mathrm{a}\mathrm{x}}$ ). The channel gain from transmitter in secondary user pair $u_j$ to the receiver in secondary user pair $u_i$ on channel $c_k$ is denoted by $g_{ij}^k$ , while the channel gain from primary user transmitter to the receiver in secondary user pair $u_i$ is denoted as ${\overline{g}}_i^k$ . Therefore, the Signal-to-Interference-Noise-Ratio (SINR) experienced by the receiver in secondary user pair $u_i$ on channel $c_k$ is calculated as follows:

$$\begin{array}{cc}& {\gamma }_i^k=\frac{\left(\mathrm{1}-o_{ik}\right)g_{ii}^k{p}_{ik}}{{\sum }_{j\ne i}^{\left|U\right|}\left(\mathrm{1}-o_{jk}\right)g_{ij}^k{p}_{jk}+\left(\mathrm{1}-{\overline{o}}_k\right){\overline{g}}_i^k{\overline{p}}_k+n_i},\end{array}$$

(1)

where $n_i$ represents the thermal noise at the receiver. Every ${\gamma }_i^k$ has to be larger than some threshold ( ${\gamma }_i^k\ge {\gamma }_i^{\mathrm{t}\mathrm{h}\mathrm{r}}$ ) for this SINR to be useful. The threshold ${\gamma }_i^{\mathrm{t}\mathrm{h}\mathrm{r}}$ depends on the receiver sensitivity, coding, and error correction mechanisms adopted by $u_i$ . Also, this threshold can be considered as a proxy for the minimum quality of service.

Most works in literature assume homogeneity of channels bandwidth. However, future systems have to deal with heterogeneous channels. Therefore, let us assume that every channel $c_k$ has unique a bandwidth $\mathrm{B}{\mathrm{W}}_k$ . The maximum achievable data rate of user $u_i$ on channel $c_k$ can be measured by Shannon formula $r_i^k=\mathrm{B}{\mathrm{W}}_k\mathrm{l}\mathrm{o}{\mathrm{g}}_{\mathrm{2}}\left(\mathrm{1}+{\gamma }_i^k\right)$ . The general spectrum sharing problem is formulated as follows:

$$\begin{array}{cc}& \underset{p}{\max }\hspace{1em}{\displaystyle \sum }_i^{\left|U\right|}{\displaystyle \sum }_k^{\left|C\right|}{r}_i^k\\ \text{s.t.:}\hspace{1em}{\gamma }_i^k\ge {\gamma }_i^{\text{thr}}\\ \hspace{1em}\mathrm{0}\le p_{ik}\le p_k^{\max }.\end{array}$$

(2)

Keep in mind that this formulation assumes that every secondary user has something to transmit all the time. In other words, secondary users do not have Idle periods. Such assumption may be valid in controlled environment of homogeneous secondary users (i.e., wireless sensor networks). However, it is not valid for heterogeneous secondary users' environment (i.e., mix of multiple femtocells and WLANs). Therefore, we introduce another formulation in the next subsection.

2.2. Spectrum-Time Duality

One of the most important observations in heterogeneous secondary users environment is that every user has its own packet size ( $s_i$ ). As a result, depending on the achieved $r_i^k$ , there is a minimum amount of time that must be guaranteed for successful packet transmission on channel $c_k$ (i.e., $t_i^k\ge s_i/r_i^k$ , where packet size is in bits). Also, each $u_i$ has several packets waiting for transmission in its queue (i.e., $Q_i$ packets). The total time needed to transmit all packets in the queue can be approximated by

$$\begin{array}{cc}& T_i\approx \frac{Q_i·s_i}{{\sum }_k^{\left|C\right|}\left[\mathrm{B}{\mathrm{W}}_k\mathrm{l}\mathrm{o}{\mathrm{g}}_{\mathrm{2}}\left(\mathrm{1}+{\gamma }_i^k\right)\right]}.\end{array}$$

(3)

Equation (3) does not guarantee minimum $t_i^k$ . Hence, it is only an approximation. For the time being, we will use this approximation to reformulate our problem. The main idea in the new formulation is to minimize the time required by $u_i$ to clear its queue. As a result, user $u_i$ will be Idle for some period of time. Consequently, the competition will be reduced on the available channels for other secondary users.

So far, we considered ${\gamma }_i^k$ as a fixed value. Conversely, ${\gamma }_i^k$ is a function of time (i.e., ${\gamma }_i^k(t)$ ) because both of $p_{ik}$ and $g_{ij}^k$ change over time. While $p_{ik}$ is a controllable parameter, $g_{ij}^k$ is a random variable. Let us assume that channel gain is a stationary process, where $\mathbb{E}\left[g_{ij}^k\right]$ represents its fixed expectation. Hence, the change in ${\gamma }_i^k(t)$ is only due to change in $p_{ik}$ (i.e., $p_{ik}(t)$ ). Keep in mind that ${\gamma }_i^k(t)$ not only depends on the transmission power of user $u_i$ , but also depends on the transmission power of other secondary users which are operating on the same channel. Accordingly, the maximum amount of data that can be transferred over channel $c_k$ in interval $[\mathrm{0},\tau ]$ is

$$\begin{array}{cc}& r_i^k={{\displaystyle \int }}_{\mathrm{0}}^{\tau }\mathrm{B}{\mathrm{W}}_k{\log }_{\mathrm{2}}\left(\mathrm{1}+{\gamma }_i^k\left(t\right)\right)dt.\end{array}$$

(4)

If there is a central coordinator which assigns the transmission power to all users during $[\mathrm{0},\tau ]$ , the expected SINR for all users will be fixed during this interval ${\gamma }_i^k(t)={\gamma }_i^k$ since only central coordinator can change the transmission power for users. As a result

$$\begin{array}{cc}& r_i^k={{\displaystyle \int }}_{\mathrm{0}}^{\tau }\mathrm{B}{\mathrm{W}}_k\mathrm{l}\mathrm{o}{\mathrm{g}}_{\mathrm{2}}\left(\mathrm{1}+{\gamma }_i^k\right)dt.\end{array}$$

(5)

Equation (5) is for centralized control, while the previous is for distributed control. As a starting point, we will focus on the centralized version.

The maximum number of packets that can be transferred in interval $[\mathrm{0},\tau ]$ is equal to

$$\begin{array}{cc}& ⌊\frac{{\sum }_k^{\left|C\right|}\left[{\int }_{\mathrm{0}}^{\tau }\mathrm{B}{\mathrm{W}}_k\mathrm{l}\mathrm{o}{\mathrm{g}}_{\mathrm{2}}\left(\mathrm{1}+{\gamma }_i^k\right)dt\right]}{s_i}⌋.\end{array}$$

(6)

The next step is calculating how user's queues will evolve over time. The arrival rate of packets at user $u_i$ is denoted by ${\lambda }_i$ packets/second. The total number of packets received during interval $\left[\mathrm{0},\tau \right]$ at user $u_i$ is ${\int }_{\mathrm{0}}^{\tau }{\lambda }_i\hspace{0.17em}dt$ . Let $Q_i(\mathrm{0})$ and $Q_i(\tau )$ denote the number of packets waiting in user $u_i$ queue at the beginning and at the end of interval $\left[\mathrm{0},\tau \right]$ , respectively. Hence

$$\begin{array}{cc}& Q_i\left(\tau \right)=\max \left[Q_i+\Delta Q_i,\mathrm{0}\right],\end{array}$$

(7)

where

$$\begin{array}{cc}& \Delta Q_i={{\displaystyle \int }}_{\mathrm{0}}^{\tau }{\lambda }_i\hspace{0.17em}dt-⌊\frac{{\sum }_k^{\left|C\right|}\left[{\int }_{\mathrm{0}}^{\tau }\mathrm{B}{\mathrm{W}}_k\mathrm{l}\mathrm{o}{\mathrm{g}}_{\mathrm{2}}\left(\mathrm{1}+{\gamma }_i^k\right)dt\right]}{s_i}⌋.\end{array}$$

(8)

Assuming that the arrival rate of all users is a stationary process where $\mathbb{E}\left[{\lambda }_i\right]$ represents the expectation for user $u_i$ , the stability of all systems queues will depend solely on $\Delta \mathbf{Q}$ . If $\Delta Q_i>\mathrm{0}$ , $u_i$ has more packets in its queue at the end of $[\mathrm{0},\tau ]$ , while if $\Delta Q_i<\mathrm{0}$ , the queue is clearing up and user $u_i$ has reduced the number of packets in its queue. Having $\Delta Q_i=\mathrm{0}$ indicates no change in the queue size. Note that there is a relationship between the achievable data rate $r_i={\sum }_k^{\left|C\right|}{r}_i^k$ and $\Delta Q_i$ . As $r_i$ increases, $\Delta Q_i$ decreases and vice versa. Also, if $-\Delta Q_i>Q_i$ , then $\tau -T_i>\mathrm{0}$ which means that fraction of interval $[\mathrm{0},\tau ]$ of user $u_i$ will be Idle. This fraction is equivalent to $(\tau -T_i)/\tau$ . Other secondary users will benefit from the absence of Idle $u_i$ due to reduced interference which was caused by $u_i$ transmission. However, the transmission power assignment of the remaining secondary users during the absence of $u_i$ is not optimal and can be improved. Therefore, τ has to be chosen to be equal to the minimum $T_i$ among all users. This way, new power allocation can take place where the absence of $u_i$ (with the minimum $T_i$ ) is considered. Thus, the new spectrum sharing problem is formulated based on clearing queues as soon as possible:

$$\begin{array}{cc}& \underset{p,\tau }{\mathrm{m}\mathrm{i}\mathrm{n}}\hspace{1em}{\displaystyle \sum }_i^{\left|U\right|}{Q}_i\left(\tau \right)\\ \text{s.t.:}\hspace{1em}{\gamma }_i^k\ge {\gamma }_i^{\text{thr}}\\ \hspace{1em}\mathrm{0}\le p_{ik}\le p_k^{\max }\\ \hspace{1em}\tau -T_i\le \mathrm{0}.\end{array}$$

(9)

The aim of the last constraint is to guarantee that no user is Idle during the interval $[\mathrm{0},\tau ]$ which results in optimal power allocation for all users during the whole interval.

3. Centralized Medium Sharing and Access

The main purpose of this section is to develop a general approach to allocate transmission power in centralized setting. However, such approach cannot be used in heterogeneous secondary users' environment. Therefore, the next section will develop a distributed mechanism to achieve the same goal. The findings of this section can be easily implemented in classical wireless technologies where centralized control entity is assumed.

First, let us call the interval $[\mathrm{0},\tau ]$ the decision time frame which means that during this time frame the transmission powers of all users are fixed. As a result, the time is slotted into frames where each frame has a period length of ${\tau }_n$ (i.e., n represents the frame index according to the past and future time frames). The periods lengths of time frames are not assumed to be equal. By fixing the transmission power during the time frame, $\Delta Q_i$ at the end of time frame n can be written as follows:

$$\begin{array}{cc}& \Delta Q_i=\frac{{\tau }_n}{s_i}\left[s_i\mathbb{E}\left[{\lambda }_i\right]-{\displaystyle \sum }_k^{\left|C\right|}\left[\mathrm{B}{\mathrm{W}}_k{\log }_{\mathrm{2}}\left(\mathrm{1}+{\gamma }_i^k\right)\right]\right]=\frac{{\tau }_n}{s_i}\left[s_i\mathbb{E}\left[{\lambda }_i\right]-{\displaystyle \sum }_k^{\left|C\right|}{r}_i^k\right].\end{array}$$

(10)

The value inside the brackets can be controlled by only manipulating the transmission power levels. To highlight the relationship between $\Delta Q_i$ and any transmission power manipulation, ${\gamma }_i^k$ as experienced during the new time frame can be rewritten as

$$\begin{array}{cc}& {\gamma }_i^k+\Delta {\gamma }_i^k=\frac{{\tilde{p}}_{ik}+{\widetilde{\Delta p}}_{ik}}{{\tilde{\eta }}_{ik}+{\widetilde{\Delta I}}_{ik}},\end{array}$$

(11)

where

$$\begin{array}{cc}& {\tilde{p}}_{ik}=\left(\mathrm{1}-o_{ik}\right)\mathbb{E}\left[g_{ii}^k\right]p_{ik}\\ {\widetilde{\Delta p}}_{ik}=\left(\mathrm{1}-o_{ik}\right)\mathbb{E}\left[g_{ii}^k\right]\Delta p_{ik}\\ {\tilde{\eta }}_{ik}={\tilde{I}}_{ik}+n_i\\ {\tilde{I}}_{ik}={\displaystyle \sum }_{j\ne i}^{\left|U\right|}\left(\mathrm{1}-o_{jk}\right)\mathbb{E}\left[g_{ij}^k\right]p_{jk}\\ {\widetilde{\Delta I}}_{ik}={\displaystyle \sum }_{j\ne i}^{\left|U\right|}\left(\mathrm{1}-o_{jk}\right)\mathbb{E}\left[g_{ij}^k\right]\Delta p_{jk}.\end{array}$$

(12)

The update of transmission power is represented by ${\widetilde{\Delta p}}_{ik}$ which is decided at the beginning of the time frame. Hence, the new transmission power during the new frame is equal to the transmission power of the previous frame plus ${\widetilde{\Delta p}}_{ik}$ . The same goes for SINR level and interference where ${\widetilde{\Delta I}}_{ik}$ represents the change of experienced interference and $\Delta {\gamma }_i^k$ represents the change of the experienced SINR level in comparison to the previous time frame.

The next step is to calculate the change in data rate ( $\Delta r_i^k$ ). Based on the previous section, $r_i^k=\mathrm{B}{\mathrm{W}}_k\mathrm{l}\mathrm{o}{\mathrm{g}}_{\mathrm{2}}\left(\mathrm{1}+{\gamma }_i^k\right)$ ; hence $r_i^k+\Delta r_i^k=\mathrm{B}{\mathrm{W}}_k\mathrm{l}\mathrm{o}{\mathrm{g}}_{\mathrm{2}}\left(\mathrm{1}+{\gamma }_i^k+\Delta {\gamma }_i^k\right)$ . After several algebraic manipulations, we find out that

$$\begin{array}{cc}& \Delta r_i^k=\mathrm{B}{\mathrm{W}}_k{\log }_{\mathrm{2}}\left(\mathrm{1}+\frac{\Delta {\gamma }_i^k}{\mathrm{1}+{\gamma }_i^k}\right).\end{array}$$

(13)

Let us write both of ${\gamma }_i^k$ and $\Delta {\gamma }_i^k$ in terms of transmission power and interference:

$${\gamma }_i^k=\frac{{\tilde{p}}_{ik}}{{\tilde{\eta }}_{ik}}$$

(14)

$$\begin{array}{l}\Delta {\gamma }_i^k=\frac{{\tilde{p}}_{ik}+{\widetilde{\Delta p}}_{ik}}{{\tilde{\eta }}_{ik}+{\widetilde{\Delta I}}_{ik}}-\frac{{\tilde{p}}_{ik}}{{\tilde{\eta }}_{ik}}\hfill \\ =\frac{{\tilde{\eta }}_{ik}\left({\tilde{p}}_{ik}+{\widetilde{\Delta p}}_{ik}\right)-{\tilde{p}}_{ik}\left({\tilde{\eta }}_{ik}+{\widetilde{\Delta I}}_{ik}\right)}{{\tilde{\eta }}_{ik}\left({\tilde{\eta }}_{ik}+{\widetilde{\Delta I}}_{ik}\right)}.\hfill \end{array}$$

(15)

By substituting both of (14) and (15) in (13) and performing numerous algebraic operations, we have the following equation:

$$\begin{array}{cc}& \Delta r_i^k=\mathrm{B}{\mathrm{W}}_k{\log }_{\mathrm{2}}\left(\frac{{\tilde{\eta }}_{ik}}{{\tilde{p}}_{ik}+{\tilde{\eta }}_{ik}}\left[\frac{{\tilde{p}}_{ik}+{\widetilde{\Delta p}}_{ik}}{{\tilde{\eta }}_{ik}+{\widetilde{\Delta I}}_{ik}}+\mathrm{1}\right]\right).\end{array}$$

(16)

Equation (16) gives us a direct link between the change in data rate and the update of transmission power. It can be shown that the relationship among transmission power updates of all users is governed by

$$\begin{array}{cc}& \left(\mathrm{1}-o_{ik}\right)\mathbb{E}\left[g_{ii}^k\right]\Delta p_{ik}-\left[{\mathrm{2}}^{\left(\Delta r_i^k/\mathrm{B}{\mathrm{W}}_k\right)}\frac{{\tilde{p}}_{ik}}{{\tilde{\eta }}_{ik}}\right]{\displaystyle \sum }_{j\ne i}^{\left|U\right|}\left(\mathrm{1}-o_{jk}\right)\mathbb{E}\left[g_{ij}^k\right]\Delta p_{jk}=\left({\tilde{p}}_{ik}+{\tilde{\eta }}_{ik}\right)\left({\mathrm{2}}^{\left(\Delta r_i^k/\mathrm{B}{\mathrm{W}}_k\right)}-\mathrm{1}\right).\end{array}$$

(17)

Note that (17) can be written in matrix format for every kth channel as ${\mathbf{C}}_{\mathbf{k}}\Delta {\mathbf{p}}_{\mathbf{k}}={\mathbf{B}}_{\mathbf{k}}$ , where $\Delta {\mathbf{p}}_{\mathbf{k}}$ is a vector of all users transmission power updates, ${\mathbf{B}}_{\mathbf{k}}$ is a vector where the ith element is the left hand side of (17), and ${\mathbf{C}}_{\mathbf{k}}$ is the coefficient matrix where its elements are

$$\begin{array}{cc}& {\left[{\mathbf{C}}_{\mathbf{k}}\right]}_{ij}=\left\{\begin{array}{cc}\left(\mathrm{1}-o_{ik}\right)\mathbb{E}\left[g_{ij}^k\right],\hfill & i=j\hfill \\ -{\mathrm{2}}^{\left(\Delta r_i^k/\mathrm{B}{\mathrm{W}}_k\right)}{\gamma }_i^k\left(\mathrm{1}-o_{jk}\right)\mathbb{E}\left[g_{ij}^k\right],\hfill & i\ne j.\hfill \end{array}\right.\end{array}$$

(18)

By using the invertible matrix theorem [43] which states that matrix A is invertible if and only if $\mathbf{A}x=\mathbf{b}$ has one solution, since we derived ${\mathbf{C}}_{\mathbf{k}}$ from unique solution (17), then ${\mathbf{C}}_{\mathbf{k}}$ is invertible. As a result, the required transmission power updates to achieve data rate change $\Delta {\mathbf{r}}^{\mathbf{k}}$ based on the current transmission power ${\mathbf{p}}_{\mathbf{k}}$ are calculated as follows:

$$\begin{array}{cc}& \Delta {\mathbf{p}}_{\mathbf{k}}={{\mathbf{C}}_{\mathbf{k}}}^{-\mathrm{1}}{\mathbf{B}}_{\mathbf{k}}.\end{array}$$

(19)

Based on this finding, we reformulate the optimization problem in (9) by using $\Delta {\mathbf{r}}^{\mathbf{k}}$ with ${\tau }_n$ as the controlling variables instead of ${\mathbf{p}}_{\mathbf{k}}$ . Note that the change in the later variable $({\mathbf{p}}_{\mathbf{k}})$ can be expressed through the change in data rate.

3.1. Reformulation of Power Allocation Problem

At the beginning, we need to revisit optimization problem (9) constraints. The first constraint ( ${\gamma }_i^k\ge {\gamma }_i^{\mathrm{t}\mathrm{h}\mathrm{r}}$ ) ensures that data rate has the minimum acceptable bit error rate. In the new time frame, ${\gamma }_i^k+\Delta {\gamma }_i^k\ge {\gamma }_i^{\mathrm{t}\mathrm{h}\mathrm{r}}\Rightarrow {\tilde{p}}_{ik}+{\widetilde{\Delta p}}_{ik}/{\tilde{\eta }}_{ik}+{\widetilde{\Delta I}}_{ik}\ge {\gamma }_i^{\mathrm{t}\mathrm{h}\mathrm{r}}$ . Since ${\tilde{\eta }}_{ik}+{\widetilde{\Delta I}}_{ik}$ is always positive, we can rewrite the first constraint for the ith user on the kth channel as $-{\widetilde{\Delta p}}_{ik}+{\gamma }_i^{\mathrm{t}\mathrm{h}\mathrm{r}}{\widetilde{\Delta I}}_{ik}+{\gamma }_i^{\mathrm{t}\mathrm{h}\mathrm{r}}{\tilde{\eta }}_{ik}-{\tilde{p}}_{ik}\le \mathrm{0}$ . Hence, the first constraint for the kth channel in matrix notation is ${\mathbf{E}}_{\mathbf{k}}\Delta {\mathbf{p}}_{\mathbf{k}}+{\mathbf{D}}_{\mathbf{k}}\preccurlyeq \mathbf{0}$ , where bit error rate coefficient matrix ( ${\mathbf{E}}_{\mathbf{k}}$ ) has elements as follows:

$$\begin{array}{cc}& {\left[{\mathbf{E}}_{\mathbf{k}}\right]}_{ij}=\left\{\begin{array}{cc}-\left(\mathrm{1}-o_{ik}\right)\mathbb{E}\left[g_{ij}^k\right],\hfill & i=j\hfill \\ {\gamma }_i^{\text{thr}}\left(\mathrm{1}-o_{jk}\right)\mathbb{E}\left[g_{ij}^k\right],\hfill & i\ne j.\hfill \end{array}\right.\end{array}$$

(20)

The vector ${\mathbf{D}}_{\mathbf{k}}$   ith element is ${\gamma }_i^{\text{thr}}{\tilde{\eta }}_{ik}-{\tilde{p}}_{ik}$ . Now, we can state the first constraint in terms of change of data rate instead of transmission power by substituting (19) as follows:

$$\begin{array}{cc}& {\mathbf{E}}_{\mathbf{k}}{{\mathbf{C}}_{\mathbf{k}}}^{-\mathrm{1}}{\mathbf{B}}_{\mathbf{k}}+{\mathbf{D}}_{\mathbf{k}}\preccurlyeq \mathbf{0}.\end{array}$$

(21)

Similarly, the second constraint of optimization problem ( $\mathrm{0}\le p_{ik}\le p_k^{\max }$ ) can be written in terms of data rate change as follows:

$$\begin{array}{cc}& \mathbf{0}\preccurlyeq {\mathbf{p}}_{\mathbf{k}}+{{\mathbf{C}}_{\mathbf{k}}}^{-\mathrm{1}}{\mathbf{B}}_{\mathbf{k}}\preccurlyeq p_k^{\max }·\mathbf{1}.\end{array}$$

(22)

The third constraint needs to be changed to an equivalent constraint since it depends only on ${\tau }_n$ . Note that the max operator in (8) implicitly states that $Q_i+\Delta Q_i\ge \mathrm{0}$ . Both of these inequalities ( $Q_i+\Delta Q_i\ge \mathrm{0}$ and the third constraint ${\tau }_n-T_i\le \mathrm{0}$ ) are equivalent since they guarantee that no channel is left underutilized because some user occupied it and has no more data to transmit. Therefore, after substitution of (10), we can reformulate the third constraint as follows:

$$\begin{array}{cc}& {\tau }_n{\mathbf{s}}^T\left(\Delta \mathbf{r}·\mathbf{1}\right)+{\tau }_n{\mathbf{s}}^T\left(\mathbf{r}·\mathbf{1}\right)-\mathbf{Q}-{\tau }_n\mathbb{E}\left[\lambda \right]\preccurlyeq \mathbf{0},\end{array}$$

(23)

where $\Delta \mathbf{r}$ and r are matrices representing the required change in data rate and the current data rate, respectively. The kth column in these matrices corresponds to the kth channel and the ith row corresponds to the ith user. Expected packets arrival rate of all users is donated by vector $\mathbb{E}[\lambda ]$ and the ith element in vector s represents the reciprocal of the ith user packet size ( $\mathrm{1}/s_i$ ).

The last step is to rewrite the global objective function. Note that each user would like to minimize $Q_i+{\tau }_n\mathbb{E}[{\lambda }_i]-{\tau }_n/s_i{\sum }_k^{\left|C\right|}\left(r_i^k+\Delta r_i^k\right)$ to zero which means that the user has an empty queue and all of its packets were successfully transmitted. To combine the utility functions of all users, the following objective function is used:

$$\begin{array}{cc}& \underset{\Delta \mathbf{r},{\tau }_n}{\min }{\displaystyle \sum }_i^{\left|U\right|}\left(w_i\left[Q_i+{\tau }_n\mathbb{E}\left[{\lambda }_i\right]-\frac{{\tau }_n}{s_i}{\displaystyle \sum }_k^{\left|C\right|}\left(r_i^k+\Delta r_i^k\right)\right]\right).\end{array}$$

(24)

Trying to find the optimal $\Delta \mathbf{r}$ and ${\tau }_n$ of this objective function subject to the mentioned constraints is tricky due to the nonconvexity of this optimization problem. However, we can ease it by fixing ${\tau }_n$ to a specific value since in reality ${\tau }_n$ has to be a discrete variable. It is well known that any communication protocol has to transmit its packets as a whole at once. Therefore, it has to have a minimum period of time to transmit a packet. This minimum period depends on the used technology. For example, if the used technology is LTE-A, then ${\tau }_n$ can be one millisecond which is the minimum frame time according to LTE-A standard [44]. Hereafter, our problem is redefined as follows:

$$\begin{array}{cc}& \underset{\Delta \mathbf{r}}{\mathrm{m}\mathrm{i}\mathrm{n}}\hspace{1em}{\displaystyle \sum }_i^{\left|U\right|}\left(w_i\left[Q_i+{\tau }_n\mathbb{E}\left[{\lambda }_i\right]-\frac{{\tau }_n}{s_i}{\displaystyle \sum }_k^{\left|C\right|}\left(r_i^k+\Delta r_i^k\right)\right]\right)\\ \text{s.t.:}\hspace{1em}{\mathbf{E}}_{\mathbf{k}}{{\mathbf{C}}_{\mathbf{k}}}^{-\mathrm{1}}{\mathbf{B}}_{\mathbf{k}}+{\mathbf{D}}_{\mathbf{k}}\preccurlyeq \mathbf{0}\\ \hspace{1em}\mathbf{0}\preccurlyeq {\mathbf{p}}_{\mathbf{k}}+{{\mathbf{C}}_{\mathbf{k}}}^{-\mathrm{1}}{\mathbf{B}}_{\mathbf{k}}\preccurlyeq p_k^{\max }·\mathbf{1}\\ \hspace{1em}{\tau }_n{\mathbf{s}}^T\left(\Delta \mathbf{r}·\mathbf{1}\right)+{\tau }_n{\mathbf{s}}^T\left(\mathbf{r}·\mathbf{1}\right)-\mathbf{Q}-{\tau }_n\mathbb{E}\left[\lambda \right]\preccurlyeq \mathbf{0}.\end{array}$$

(25)

Having our problem in this form, we can find the best possible $\Delta \mathbf{r}$ by using Interior-Point method [45]. This method is based on the idea of combining the inequality constraints with the objective function so that Newton method can be used (please, refer to [45] for elaborated discussion). Optimizing for $\Delta \mathbf{r}$ needs to be performed only once (at the first time frame), given that the number of secondary users in subsequent time frames is the same and their packet arrival rate process is stationary.

3.2. Balancing Weight

In the problem of (25), due to the heterogeneity of both primary channels and secondary users, the obtained solution may lead to unfair power allocation among secondary users. Therefore, $w_i$ is used to guarantee a level of fairness among them. Several formulas can be used to calculate $w_i$ . This paper proposes three straightforward approaches as follows:

$$w_i=\frac{\mathrm{1}}{\left|U\right|}$$

(26)

$$w_i=\frac{Q_i}{{\sum }_j^{\left|U\right|}{Q}_j}$$

(27)

$$w_i=\frac{\mathbb{E}\left[{\lambda }_i\right]}{{\sum }_j^{\left|U\right|}\mathbb{E}\left[{\lambda }_i\right]}.$$

(28)

Formula (26) describes the default approach where an equal treatment of all users is given. Formula (27) gives users with larger queues higher priority, while formula (28) favors users with faster packets arrival rate.

4. Distributed Medium Sharing and Access

The previous section developed an approach to share and access medium among multiple secondary users. However, such approach requires a centralized coordinator to assign transmission power level to each secondary user. This approach can be implemented only among cooperative secondary users. For example, base stations in mobile telecommunication system (i.e., LTE or WiMAX) which belong to the same provider can adopt the centralized approach since they can coordinate among themselves easily. Conversely, an environment with noncooperative secondary users will require a passive distributed approach due to the lack of coordination among secondary users.

Heterogeneous secondary users shall decide the appropriate transmission power based solely on the local acquired information regarding the surrounding wireless environment. Assuming that every user knows its queue size, expected packets arrival rate, and Channel State Information (CSI), a relationship between the required transmission power and time can be formulated based on the objective of clearing the queue. In other words, each user would like to clear its queue within a specific maximum time frame ( ${\tau }_i^{\max }$ ) to achieve the required quality of service level. Therefore, the transmission power should be assigned based on the current queue size, expected arrival rate, and noise so that the queue will be cleared by the end of the time frame. Noise is calculated based on CSI, while ${\tau }_i^{\max }$ can be computed based on several factors such as delay sensitivity of communication.

Note that using noise level as a parameter to decide the transmission power will lead to coupling all interfering secondary users since most of the noise is interference noise generated by them. As a result, we will have a dynamical system which evolves over time. This dynamical system is discrete due to the fact that all digital communication technologies transmit data in packets. The discrete dynamical system requires global time frame to be defined which can be tricky in heterogeneous environment. To simplify the formulation of transmission power assignment rule, we will assume that such global time frame is defined as $\Delta t$ . Note that τ is used to represent local time frame for each secondary user, while $\Delta t$ represent the global time frame of all secondary users. At the end of this section, a simple approach is proposed to define the start and the end of this global time frame.

Any transmission power formula should try to honor four limits which are maximum queue size for the ith secondary user ( $Q_i^{\max }$ ), the maximum time frame length for the ith secondary user ( ${\tau }_i^{\max }$ ), maximum allowed transmission power on the kth channel ( $p_k^{\max }$ ), and maximum noise for the ith secondary user on the kth channel ( ${\eta }_{ik}^{\max }$ ). The last limit is calculated based on the fact that SINR has to be larger than some threshold ( ${\gamma }_i^k\ge {\gamma }_i^{\text{thr}}$ ). By using $p_k^{\max }$ as the transmission power, ${\eta }_{ik}^{\max }=(\mathbb{E}[g_{ii}^k]p_k^{\max })/{\gamma }_i^{\text{thr}}$ can be considered as the third limit. Both of $Q_i^{\max }$ and ${\tau }_i^{\max }$ depend on factors controlled by upper layers and application requirements such as latency and delay. The third limit $p_k^{\max }$ is channel specific which is assigned by regulators or primary users.

4.1. Time Frame Length Assignment

The first step in designing the transmission power allocation rule is to link the length of time frame ( ${\tau }_{ik}$ ) for the ith secondary user on the kth channel with the current state of the secondary user. Both of the current queue size and current noise level can be used to represent current state. This work proposes a time frame length assignment based on these parameters. By using both ${\eta }_{ik}^{\max }$ and the noise level ( ${\eta }_{ik}(t-\Delta t)$ ) experienced at the previous time frame, a relationship between noise and appropriate time frame length is formed as follows:

$$\begin{array}{cc}& {\tau }_{ik}\left(t\right)=\frac{{\eta }_{ik}\left(t-\Delta t\right)}{{\eta }_{ik}^{\max }}{\tau }_i^{\max }.\end{array}$$

(29)

Formula (29) does not honor maximum time frame limit which means that ${\tau }_{ik}\left(t\right)$ can be larger than ${\tau }_i^{\max }$ . To handle this issue, a modified formula will be used:

$$\begin{array}{cc}& {\tau }_{ik}\left(t\right)=\frac{{\eta }_{ik}\left(t-\Delta t\right)/{\eta }_{ik}^{\max }}{{\eta }_{ik}\left(t-\Delta t\right)/{\eta }_{ik}^{\max }+{\beta }_{ik}{Q}_i\left(t\right)}{\tau }_i^{\max },\hspace{1em}\text{where}\hspace{0.17em}\hspace{0.17em}{\beta }_{ik}>\mathrm{0}.\end{array}$$

(30)

The added parameter ${\beta }_{ik}$ controls the difficulty of assigning ${\tau }_i^{\max }$ as the current time frame length. If ${\beta }_{ik}$ is small, it will be easy to achieve maximum time frame length and vice versa. Note that the current queue size parameter is introduced in calculation to add the effect of this parameter. By using the definition of noise based on (12), the time frame length can be assigned based on

$$\begin{array}{cc}& {\tau }_{ik}\left(t\right)=\frac{{\eta }_{ik}\left(t-\Delta t\right){\gamma }_i^{\text{thr}}{\tau }_i^{\max }}{{\eta }_{ik}\left(t-\Delta t\right){\gamma }_i^{\text{thr}}+{\beta }_{ik}{Q}_i\left(t\right)\mathbb{E}\left[g_{ii}^k\right]p_k^{\max }}=\frac{\left[{\sum }_{j\ne i}^{\left|U\right|}\mathbb{E}\left[g_{ij}^k\right]p_{jk}\left(t-\Delta t\right)+n_i\right]{\gamma }_i^{\text{thr}}{\tau }_i^{\max }}{\left[{\sum }_{j\ne i}^{\left|U\right|}\mathbb{E}\left[g_{ij}^k\right]p_{jk}\left(t-\Delta t\right)+n_i\right]{\gamma }_i^{\text{thr}}+{\beta }_{ik}{Q}_i\left(t\right)\mathbb{E}\left[g_{ii}^k\right]p_k^{\max }}.\end{array}$$

(31)

The term $\left(\mathrm{1}-o_{jk}(t-\Delta t)\right)$ is taken out because it will be used in $p_{jk}$ calculation (Section 4.2). The calculated ${\tau }_{ik}$ tries to reduce the required time to clear the queue when secondary user suffers from large queue or high level of interference. By doing that, the secondary user gets more competitive which results in higher level of transmission power.

4.2. Transmission Power Rules

As mentioned before, the primary goal for any secondary user is to clear its queue as soon as possible. Based on this objective, a general transmission power assignment rule on the kth channel can be formed by algebraically manipulating the following equation:

$$\begin{array}{cc}& \frac{\mathrm{B}{\mathrm{W}}_k}{{\sum }_l^{\left|C\right|}\mathrm{B}{\mathrm{W}}_l}\left(Q_i+{\tau }_{ik}\mathbb{E}\left[{\lambda }_i\right]\right)-\frac{{\tau }_{ik}}{s_i}\mathrm{B}{\mathrm{W}}_k\mathrm{l}\mathrm{o}{\mathrm{g}}_{\mathrm{2}}\left(\mathrm{1}+{\gamma }_i^k\right)=\mathrm{0}.\end{array}$$

(32)

Equation (32) is based on (8). The first term distributes the load (current and new arriving packets) among channels based on their bandwidth. Other approaches to distribute the load can be used. However, this approach is adopted for simplicity. After some algebraic manipulations, general transmission power is calculated as follows:

$$\begin{array}{cc}& p_{ik}\left(t\right)=\left[\mathrm{1}-o_{ik}\left(t\right)\right]\frac{{\sum }_{j\ne i}^{\left|U\right|}\left[\mathbb{E}\left[g_{ij}^k\right]p_{jk}\left(t-\Delta t\right)\right]+n_i}{\mathbb{E}\left[g_{ii}^k\right]}{\delta }_{ik}\left(t\right),\end{array}$$

(33)

where

$$\begin{array}{cc}& {\delta }_{ik}\left(t\right)={\mathrm{2}}^{\left[\left(s_i\left(Q_i\left(t\right)+\mathbb{E}\left[{\lambda }_i\right]\right)\right)/{\tau }_{ik}{\sum }_l^{\left|C\right|}\mathrm{B}{\mathrm{W}}_l\right]}-\mathrm{1}.\end{array}$$

(34)

It is clear that $p_k^{\max }$ limit is not honored by this transmission power rule. A similar approach used to modify time frame length assignment ${\tau }_{ik}$ can be employed here as well. This yields

$$\begin{array}{cc}& p_{ik}\left(t\right)=\frac{\left[\mathrm{1}-o_{ik}\left(t\right)\right]{\eta }_{ik}\left(t-\Delta t\right){\delta }_{ik}\left(t\right)p_k^{\max }}{{\eta }_{ik}\left(t-\Delta t\right){\delta }_{ik}\left(t\right)+{{\alpha }_{ik}}^{{\left[\mathrm{1}-{\eta }_{ik}\left(t-\Delta t\right)/{\eta }_{ik}^{\max }\right]}^{\mathrm{2}}}\mathbb{E}\left[g_{ii}^k\right]}=\frac{\left[\mathrm{1}-o_{ik}\left(t\right)\right]\left({\sum }_{j\ne i}^{\left|U\right|}\left[\mathbb{E}\left[g_{ij}^k\right]p_{jk}\left(t-\Delta t\right)\right]+n_i\right){\delta }_{ik}\left(t\right)p_k^{\max }}{\left({\sum }_{j\ne i}^{\left|U\right|}\left[\mathbb{E}\left[g_{ij}^k\right]p_{jk}\left(t-\Delta t\right)\right]+n_i\right){\delta }_{ik}\left(t\right)+{{\alpha }_{ik}}^{{\left[\mathrm{1}-\left({\sum }_{j\ne i}^{\left|U\right|}\left[\mathbb{E}\left[g_{ij}^k\right]p_{jk}\left(t-\Delta t\right)\right]+n_i\right)/{\eta }_{ik}^{\mathrm{m}\mathrm{a}\mathrm{x}}\right]}^{\mathrm{2}}}\mathbb{E}\left[g_{ii}^k\right]},\hspace{1em}\text{where}\hspace{0.17em}\hspace{0.17em}{\alpha }_{ik}>\mathrm{0}.\end{array}$$

(35)

Term ${{\alpha }_{ik}}^{{\left[\mathrm{1}-{\eta }_{ik}\left(t-\Delta t\right)/{\eta }_{ik}^{\max }\right]}^{\mathrm{2}}}$ is added to introduce an exponential effect of current noise level. Whenever noise becomes very large (i.e., system is in outage), the transmission power reduces exponentially. This behavior favors secondary users which are not in outage by reducing the noise caused by secondary users suffering from outage due to high noise level. Also, whenever noise is very low, the transmission power is primarily computed based on the current queue size through ${\tau }_i$ calculation (assuming ${\alpha }_{ik}$ is chosen properly). Note that when ${\eta }_{ik}\left(t-\Delta t\right)={\eta }_{ik}^{\max }$ , transmission power is computed purely based on queue size.

4.3. Inference Engine

At this point, we constructed a model for heterogeneous cognitive radio networks where transmission power of different secondary users over different channels forms a tightly coupled dynamical system. The next step is to develop an inference engine to forecast future noise level so that better values of transmission power are assigned. This paper proposes the use of deep neural network (DNN) [42] to construct such engine. The capability of DNN to extract features from any series of correlated input data is more strengthened when inputs are coming from a tightly coupled dynamical system. Also, the nonlinearity of any dynamical system can be approximated by neural network. These facts advocate for the use of DNN as base for the proposed engine.

4.3.1. Engine Input

The first step to develop the proposed noise forecasting mechanism is to define the appropriate input design. Any input data should be based on local information only since secondary users are assumed to be heterogeneous with no active cooperation. Inspired by Taken's theorem [46], the input is defined as a vector of delayed noise values over time (i.e., $\mathrm{i}\mathrm{n}\mathrm{p}\mathrm{u}{\mathrm{t}}_{ik}=[{\eta }_{ik}\left(t-\Delta t\right),\mathrm{\hspace{0.17em}\hspace{0.17em}}{\eta }_{ik}\left(t-\mathrm{2}\Delta t\right),\dots ,{\eta }_{ik}\left(t-d\Delta t\right)]$ ). Taken showed that a local image of dynamical system evolution manifold can be constructed based on delayed data from one dimension of the manifold. The mapping between the original manifold and the constructed local image is one to one. By utilizing this finding in our case, every secondary user can construct an image of the manifold describing total system noise level over time. This manifold image can be used as knowledge base to forecast future values of noise level. One question still needs an answer which is how many of the past noise values should we use? In other words, what is the best value for d?

Based on Taken's theorem, the number of delayed value samples should be more than twice the space dimensionality which contains the manifold. In our case, the dynamical system forms $\left|U\right|$ -dimension space. Therefore, $d=\mathrm{2}\left|U\right|+\mathrm{1}$ is the minimum required vector size to guarantee manifold embedding. Assuming that all interfering secondary users are in close range, each one of them can figure out $\left|U\right|$ by overhearing each other's distinct transmission patterns (i.e., modulation or coding settings). Finally, by feeding local image of manifold values to the DNN, useful features of the manifold can be extracted. These features are used to forecast future noise level.

4.3.2. DNN Architecture

Deep neural network can be divided into two main categories: Deep Autoencoder (DAE) [47] and Deep Belief Network (DBN) [48]. Both of these types use neurons where the activation function is a sigmoid function. They differentiate in how to interpret the value of activation function. DAE interprets activation function value as the neuron output, while DBN uses this value as probability of selecting one of binary values as the neuron output. This paper adopts the first type as the main architecture for inference engine due to its deterministic behavior.

Neurons in DNN are arranged in layers. Each layer represents an extracted feature of input data. The first layer is called input layer, while the last layer is the output layer. Any layer between the input layer and the output layer is called a hidden layer. Classical neural networks have one hidden layer because of extreme difficulty in training multiple hidden layers [49] by using the conventional learning techniques. This was the case until 2006 when Hinton et al. [48] proposed a technique to train deep neural networks where several hidden layers are stacked over each other. This finding revolutionized the field of neural networks and its application, especially after similarities of this training technique and how the mammalian brain operates were highlighted.

Stacking several hidden layers allows DNN to extract several features from input data. To illustrate what we mean by features, let us use object recognition example since it is easier to visualize. Imagine the input data is a stream of gray scale images. After feeding these images to DNN, the first layer will be able to distinguish different types of edges, while the second layer will be able to distinguish different combinations of edges recognized by the first layer. By repeating this behavior, the third layer will differentiate between different object subparts based on the second-layer output and the fourth layer will use the third-layer output to recognize the object type. This hierarchical process can be repeated to extract higher abstraction features depending on the application objective.

In our case, a similar approach is used to extract several features of wireless environment surrounding secondary users. The noise level can be forecasted locally by every secondary user based on these extracted features. In general, we do not really care what the exact nature of these features is as long as they enhance the inference engine forecasting capabilities. Similarly, in previous object recognition example, features such as light intensity may not have big impact on how well DNN can recognize objects. Nonetheless, these types of features may help emphasize more important features which leads to improving DNN learning capabilities. The adopted learning technique has the ability to extract any necessary features to achieve learning objectives [47].

4.3.3. Engine Integration

At the beginning, the inference engine will not be able to accurately forecast future noise level because it requires time to learn the surrounding wireless environment dynamics. Therefore, to calculate transmission power using (31) and (35), the forecasted noise level should be integrated in a way which takes forecasting error into consideration. Let ${\widehat{\eta }}_{ik}$ be the forecasted noise level produced by inference engine for the ith secondary user and let ${\check{\eta }}_{ik}$ be the exponential moving average of the actual experienced noise which is calculated as follows:

$$\begin{array}{cc}& {\check{\eta }}_{ik}\left(t\right)={\displaystyle \sum }_{j=\mathrm{1}}^{t-\Delta t}\frac{{\eta }_{ik}\left(t-j\Delta t\right)}{{\omega }^j},\end{array}$$

(36)

where ω can be any positive number larger than one and it represents the dependence of ${\check{\eta }}_{ik}$ on past values of ${\eta }_{ik}$ .

Instead of using ${\eta }_{ik}(t-\Delta t)$ in (31) and (35) to calculate transmission power, ${\stackrel{⃛}{\eta }}_{ik}$ will be used which is a combination of the expected noise level in the current time frame and the exponential moving average of noise level from previous time frames:

$$\begin{array}{cc}& {\stackrel{⃛}{\eta }}_{ik}\left(t\right)=\left(\mathrm{1}-ϵ\right){\widehat{\eta }}_{ik}\left(t\right)+ϵ{\check{\eta }}_{ik}\left(t\right).\end{array}$$

(37)

The weight term ϵ relates to the error in noise forecasting. Its value should be between zero and one where approaching zero means higher confidence in noise forecasting accuracy. The adopted formula for ϵ is

$$\begin{array}{cc}& ϵ=\frac{e^{{\left[{\widehat{\eta }}_{ik}\left(t\right)-{\check{\eta }}_{ik}\left(t\right)\right]}^{\mathrm{2}}/{\check{\eta }}_{ik}\left(t\right)}-\mathrm{1}}{e^{{\left[{\widehat{\eta }}_{ik}\left(t\right)-{\check{\eta }}_{ik}\left(t\right)\right]}^{\mathrm{2}}/{\check{\eta }}_{ik}\left(t\right)}+\mathrm{1}}.\end{array}$$

(38)

It is clear that as ${\left[{\widehat{\eta }}_{ik}\left(t\right)-{\check{\eta }}_{ik}\left(t\right)\right]}^{\mathrm{2}}$ (i.e., forecasting error) decreases, dependence of ${\stackrel{⃛}{\eta }}_{ik}$ on ${\widehat{\eta }}_{ik}$ increases. Another way to look at this approach is by imagining that we have two experts. One is totally past oriented, while the other is future oriented. The forecasting of both of these experts is added in a way which favors the expert with better performance so far.

To summarize, the integrated transmission power allocation rule for the ith secondary user on the kth channel is

$$\begin{array}{cc}& p_{ik}\left(t\right)=\frac{\left[\mathrm{1}-o_{ik}\left(t\right)\right]{\stackrel{⃛}{\eta }}_{ik}\left(t\right){\delta }_{ik}\left(t\right)p_k^{\max }}{{\stackrel{⃛}{\eta }}_{ik}\left(t\right){\delta }_{ik}\left(t\right)+{{\alpha }_{ik}}^{{\left[\mathrm{1}-{\stackrel{⃛}{\eta }}_{ik}\left(t\right)/{\eta }_{ik}^{\max }\right]}^{\mathrm{2}}}\mathbb{E}\left[g_{ii}^k\right]}.\end{array}$$

(39)

Also, computing ${\delta }_{ik}$ requires ${\tau }_i$ which is redefined as follows:

$$\begin{array}{cc}& {\tau }_{ik}\left(t\right)=\frac{{\stackrel{⃛}{\eta }}_{ik}\left(t\right){\gamma }_i^{\text{thr}}{\tau }_i^{\max }}{{\stackrel{⃛}{\eta }}_{ik}\left(t\right){\gamma }_i^{\text{thr}}+{\beta }_{ik}{Q}_i\left(t\right)\mathbb{E}\left[g_{ii}^k\right]p_k^{\max }}.\end{array}$$

(40)

Note that using this rule will result in a tightly coupled dynamical system as well. Consequently, inference engine performance is guaranteed to keep improving with more experience. Both of Figures 1 and 2 show general illustration of how the proposed solutions operate and integrate.

Figure 1

Inference engine integration.

Figure 2

Illustration of the proposed system model.

4.4. Global Time Frame

So far, it is assumed that a global time frame is well defined for all secondary users. However, such assumption is not practical in a heterogeneous environment. Therefore, secondary users need to find out the global time frame boundaries based on local information. The first thing to note is that global time frame boundaries have to align with local time frame boundaries of all secondary users. Thus, from each secondary user perspective, the end of local time frame is the end of the global time frame as well. In addition, each secondary user will update its transmission power by the end of its local time frame. Hence the noise level experienced by other secondary users will change.

Having the last three facts in mind, the global time frame can be approximated locally by secondary users. The simplest approach is to monitor noise level variation. Whenever a sudden change in noise level is noticed, the time of this change will be marked as a boundary of global time frame. Using this approach will lead to rapid transmission power updates for all secondary users. As a result, faster convergence of inference engine learning can be achieved.

5. Evaluation and Discussion

Several simulation experiments were conducted to evaluate the proposed solutions in this paper. Parameters in these simulations were chosen randomly. The reasoning behind random choice is to test the evaluated solutions performance in the most general way. A uniform distribution is used to choose simulation parameters values. Heterogeneous secondary users and heterogeneous wireless channels were assumed during the simulation experiments. Secondary users have different arrival rates and different queue sizes. Also, they have different packet sizes and different quality of service levels which is represented by the minimum allowed SINR. On the other hand, wireless channels have different bandwidth and different fading behaviors.

The simulation time was set to 1000 seconds. Time frame value was chosen randomly between two extreme values. The minimum value is one millisecond and the maximum value is 0.01 seconds. Each simulation experiment was repeated 100 times. The average of simulation results was taken as the final result. Two parameters were chosen as the performance variables (x-axis). These are the number of secondary users and the number of wireless channels. On the other hand, three performance metrics (y-axis) ware selected to study the proposed solutions. These are queue size, achieved data rate, and power consumption.

Secondary user pairs (transmitter and receiver) were distributed randomly where the average distance between any two pairs is 5 m. The average distance between the transmitter and receiver in any secondary user pair is 3 m. Parameters such as minimum SINR threshold, arrival rate, the maximum queue size, and channel bandwidth were assigned in a way that amplifies their effect on the general performance. A random value for each one of these parameters was chosen between two extreme levels at the beginning of each simulation experiment. Then, this chosen value was multiplied by some coefficient which depends on secondary user identity. For example, the third secondary user may use 7 as its coefficient for maximum queue size which means that the maximum queue size for the third secondary user is seven times larger than the chosen value for this parameter at the beginning of simulation experiment.

These coefficients were selected in an ordered fashion which reflects secondary users and wireless channel indexes. For instance, the first secondary user coefficient is 1 and the second secondary user coefficient is 2. By using this coefficient approach to assign parameter values, the effect of the parameter will be apparent in the simulation results. For example, coefficients for the maximum queue size were set in ascending order. Therefore, as the number of coexisting secondary users increases, buffering capability of overall system increases as well. Now, we can see if introducing more heterogeneous secondary users in terms of queue size will improve the performance. Similar to maximum queue size, minimum SINR threshold and channel bandwidth were set in ascending order, where packets arrival rates were set in descending order. The initial maximum queue size is randomly chosen in interval $[10,100]$ packets. The initial minimum SINR threshold is randomly chosen in interval $[0,-10]$  dB. The initial packets arrival rate is randomly chosen in interval $[100,1000]$ packets/s. The initial channel bandwidth is randomly chosen in interval $[10,20]$  kHz.

Packet size for all secondary users is 64 bytes. This parameter is identical for all secondary users to increase the influence of arrival rate and queue size on overall system performance. Maximum transmission power for each wireless channel was set to 0.1 watts. Also, probability detection ( $P_d$ ) for each secondary user, path loss exponent, and fading standard deviation were chosen randomly in these intervals $[0.9,0.99]$ , $[2,5]$ , and $[5,15]$ , respectively. Note that the fading is modeled as Rayleigh distributed random variable. Idle and Busy intervals for each wireless channel were distributed exponentially where the mean (rate parameter) was chosen randomly in interval $[1,10]$ milliseconds. The deep neural network has three layers and moving average factor is 0.5. Alpha parameter in (39) is set to the moving average of SINR and beta parameter in (40) is set to 0.5. Note that the proposed solution in Section 3 is denoted by CMSA in the figures. This solution has three versions depending on weighting approach defined by (26) (CMSA-E), (27) (CMSA-Q), and (28) (CMSA-L). For Section 4 solution, there are two versions. The default version is based on (35) and (31). It is denoted by DMSA-D in the figures, while the second version is based on deep neural network ((39) and (40)) which is denoted by DMSA-N in the figures. Also, naïve transmission power assignment which uses the maximum allowed power is simulated as well. Tables 1 and 2 provide summaries of simulation parameters and neural system parameters, respectively.

Table 1

Simulation parameters.

Parameter name	Value
Time slot	0.001–0.01 seconds
Simulation time	1000 seconds
Queue size	10–100 packets
SINR	$[$ 0, −10] dB
Packets arrival rate	$[$ 100, 1000] packets/s
Carrier frequency	2 GHz
Channel bandwidth	$[$ 10, 20] kHz
Packet size	64 bytes
Transmission power	0.1 watts
Thermal noise density	−174 dBm/Hz
Fading	Rayleigh
Fading standard deviation	$[$ 5, 15]
Path loss exponent	$[$ 2, 5]
$P_d$	$[$ 0.9, 0.99]

Table 2

Neural network parameters.

Parameter name	Value
Number of layers	3
Layer sizes	$[$ 20, 15, 10]
Activation function	Sigmoid
Learning rate	0.1
Momentum	0.9
Regularization type	$L_{\mathrm{2}}$
Regularization factor	0.001
Error function	Root mean squared

Simulation experiments were programmed using MATLAB environment. Several utilities were from LTE system level simulator [50]. This simulator has been developed by the Institute of Telecommunication at Vienna. It is freely available for noncommercial and academic usage. It is very flexible and it has the capability of providing more organized and lesser complex simulation methods.

5.1. Queue Size Evaluation

The first performance metric to investigate is the average queue size of all secondary users. Keep in mind that secondary users have heterogeneous buffering capabilities. Figure 3(a) shows how the average queue size for DMSA-D, DMSA-N, and MAX-POWER increases as the number of coexisting secondary users increases. This behavior is due to the fact that introducing more secondary users increases the competition over the fixed available channels which leads to lower data rate for all users. Furthermore, adding more users with higher maximum queue size as explained in the previous section allows the average queue size to increase. For MAX-POWER, the average queue size increases almost linearly, while both of the DMSA versions have much better performance. It is clear that adopting deep learning approach has improved the performance noticeably.

Figure 3

Queue size performance. (a) Average queue size where the number of channels is 10 and the number of secondary users is increased. (b) Average queue size where the number of secondary users is 10 and the number of channels is increased.

On the other hand, CMSA solutions have achieved the best performance. The average queue size if these solutions were used is very close to zero especially when the number of available channels is larger than the number of secondary users as inferred from Figure 3(b). In the latter (Figure 3(b)), different versions of CMSA have different performance when the number of channels is 2 and the number of secondary users is 10. Here, the Interior-Point method was not able to find feasible transmission power assignment that clears all secondary user's queues. CMSA-Q had the best performance in this harsh environment since its optimization depends on the current queue size of secondary users (27). In Figure 3(b), the average queue size is decreasing as the number of available channels increases. The reduction rate of MAX-POWER is very small compared to other solutions. Note that, in Figures 3(a) and 3(b), DMSA-N performance approaches DMSA-D performance as overall system complexity increases. The achieved improvement for DMSA-D through the use of DMSA-N shrinks as the number of secondary users and channels increases. One way to keep DMSA-N improvement from shrinking is by using deeper architecture. However, deeper architecture requires more computation power and training time. In addition, DMSA-N is able to exploit increased system complexity to find better transmission power assignment. However, as complexity passes some threshold, DMSA-N performance starts degrading similar to other solutions.

5.2. Data Rate Evaluation

Data rate is measured for secondary users only if they are not in outage. Otherwise, it is assumed as zero. From wireless channel perspective, the actual experienced data rate is used in the figures. Fairness evaluations of the proposed solutions were conducted by using Jain's index [51]. However, due to the limited available space for this paper, brief discussion regarding fairness evaluations will be provided without fairness figures. Average data rate achieved by CMSA-L is the highest as seen in Figures 4(a) and 4(b). However, the fairness performance of CMSA-L is the lowest. It can be concluded that optimizing based on the arrival rate of secondary users achieves the highest performance in terms of average data rate. CMSA-Q comes second after CMSA-L. It has better fairness performance than CMSA-L due to its dependency on the current queue size of secondary users.

Figure 4

Data rate performance. (a) Average data rate where the number of channels is 10 and the number of secondary users is increased. (b) Average data rate where the number of secondary users is 10 and the number of channels is increased.

Both of DMSA solutions achieve very high performance in terms of fairness. DMSA-N has very close performance to CMSA-E in terms of average data rate when the system is simple. It is reasonable to hypothesize that increasing inference engine capability by using deeper architecture will reduce the gap between DMSA-N and CMSA-E performance in terms of average data rate. Performance of MAX-POWER is the worst. Secondary users were in outage during most of Figure 4(a) experiments when MAX-POWER was used.

5.3. Power Consumption Evaluation

The average power consumption for MAX-POWER is fixed for all experiments and it solely depends on the number of wireless channels. As this number increases, the average power consumption linearly increases as well with slope of one. Similarly, the fairness for MAX-POWER is fixed at the highest possible value in all experiments since the same transmission power is used by all users on all channels. For DMSA solutions, gradually increasing power consumption is noticed as well. However, the rate of this power consumption growth is shrinking as the number of secondary users and wireless channels increases. In Figure 5(a), this growth rate is decreasing due to the increased competition in the system as a result of the increased number of secondary users. Higher competition forces secondary users to reduce their transmission power so that the interference suffered by other secondary users is mitigated. The behavior observed in Figure 5(a) is a testament for the proposed dynamical system ability to infer the interference and load states of the surrounding secondary users. Such inference ability allows secondary users to be more reasonable when assigning transmission power. Performance in Figure 5(b) states that as the number of wireless channels increases, the need for aggressive competition through higher transmission power decreases. Such behavior can be greatly improved by using deep learning approach.

Figure 5

Power consumption performance. (a) Average power consumption where the number of channels is 10 and the number of secondary users is increased. (b) Average power consumption where the number of secondary users is 10 and the number of channels is increased. (c) Average logarithmic power consumption where the number of channels is 10 and the number of secondary users is increased. (d) Average logarithmic power consumption where the number of secondary users is 10 and the number of channels is increased.

Fairness for DMSA solution is very high compared to CMSA solutions. As the number of secondary users and wireless channels increases, DMSA fairness decreases. Again, such behavior is very desirable since it affirms that DMSA solutions were able to intelligently treat every secondary user differently based on their distinct situations. The average power consumption of CMSA solutions is very tiny compared to other solutions. To show the magnitude and the general behavior of CMSA solutions in terms of average power consumption, logarithmic scale was used as depicted in Figures 5(c) and 5(d). The Interior-Point method was able to find very tiny transmission power levels which achieve high SINR values for secondary users. It seems that the thermal noise was the reason behind keeping transmission power at these levels rather than reducing them more. Keep in mind that SINR is a ratio. It does not depend on the actual values of the numerator and denominator as long as the ratio between them is the same. Therefore, mathematically speaking, lower values of transmission power can be found if thermal noise was not considered which is not realistic.

Both of CMSA-Q and CMSA-L average transmission power levels are decreasing as the number of secondary users and wireless channels increases. CMSA-E has similar behavior when the number of wireless channels is increasing. However, increased competition in Figure 5(c) forces CMSA-E to increase transmission power of secondary users. Both of CMSA-Q and CMSA-L are more sensitive than CMSA-E to load states of secondary users. On the other hand, the equal treatment of CMSA-E for secondary users leads to increasing transmission power when the number of secondary users is increased. Heterogeneity of added wireless channels leads CMSA-E fairness to decrease. As said before, observing this impact of heterogeneity confirms the dynamicity of the proposed solutions.

6. Conclusion

This paper proposed two solutions with five versions to allocate transmission power for cognitive radio systems in heterogeneous environment. Three of these versions are centralized mechanisms where the decision is generated by central entity which has all the necessary information. The remaining two versions are distributed mechanisms where each cognitive system observes its surrounding wireless environment and it uses only its own sensory data to generate transmission power decision. Designs of all proposed solutions were inspired by Spectrum-Time Duality concept. This concept states that cognitive systems may intentionally reduce their spectrum utilization for other cognitive systems in return for longer usage time after these cognitive systems clear their queues. The optimization problem in centralized solutions tries to clear queues of coexisting cognitive radio systems in a way that reduces competition in future time frames. The distributed solutions assign transmission power by taking into consideration the queue size and the interference level of all cognitive radio systems.

Dynamical system theory was used to design the distributed solutions. Transmission power allocation rule was proposed. This rule guarantees the overall system evolution in predicted fashion. In addition, a very powerful machine learning technique was used. This technique is deep neural network. Applying this technique on the proposed dynamical system led to much better performance. To the best of our knowledge, this paper is the first attempt to utilize deep learning in cognitive radio networks. Results suggest that using deeper and more sophisticated neural networks with local information may produce comparable performance to the centralized solutions where all global information is available. One of the most important recommendations of this work for future research is to utilize advanced machine learning techniques (i.e., deep learning) in cognitive radio networks. The increased complexity of these networks requires higher level of sophistication and self-adaptability in any proposed solution which can be provided to some extent by using these techniques.