Selective Sensing and Access Strategy to Maximize Throughput in Cognitive Radio Sensor Network

Abstract

This paper presents a selective spectrum sensing and access strategy in a cognitive radio sensor network (CRSN), in order to maximize the throughput of secondary user (SU) system. An SU senses multiple channels simultaneously via wideband spectrum sensing. To maximize the throughput and reduce the sensing energy consumption, not all of the channels are sensed. The SU selects some channels for spectrum sensing and accesses these channels based on the sensing results. The unselected channels are accessed directly with low transmission power. A selection making algorithm based on partially observable Markov decision process (POMDP) theory is proposed, to make the SU determine which channels are selected for sensing, how long the sensing time, and the transmission powers of channels. An optimal policy and a myopic policy are proposed to solve the proposed POMDP problem. Moreover, an optimization problem is proposed to solve the synchronism problem among the selected channels. Numerical results show that the proposed selective spectrum sensing and access strategy improves the system performance efficiently.

1. Introduction

Wireless sensor networks (WSNs) play a critical role in many research areas, such as machine-to-machine network (M2M), emergency communication, and smart home [1–4]. There are two characteristics about the WSNs: limited energy and scarce spectrum. Generally, the nodes of WSNs are powered by battery; energy efficiency is one of the design factors. In this paper, we focus on the spectrum scarcity problem of WSNs. According to the dramatic increasing number of wireless devices, the problem of available spectrum scarcity becomes more serious. Cognitive radio (CR), which has been introduced as a way to improve the efficiency of spectrum utilization, becomes a research focus in recent years [1–27]. Combined with the advantages of CR and WSNs, cognitive radio sensor network (CRSN) has been studied [1–4, 6–8, 15]. In studies [1, 2], CR has been used in smart grid communication networks. The spectrum access strategy has been proposed, in order to find much available spectrum resource for data collection and transmission. In order to improve the energy efficient of vehicular ad hoc network, CRSN has been studied in [3]. In the CRSN, the wireless nodes are defined as secondary (unlicensed) users (SUs), while the other available spectrum owners are defined as primary (licensed) users (PUs). The important challenge is that the SUs find available channels and adjust their transmission parameters (transmission power, transmission time, carrier frequency, etc.) to access these channels, while avoiding harmful interference to PUs.

Normally, the available spectrum opportunities are found in time [4, 7, 8, 10–12, 15] space [14] and frequency domains [18]. In [10–12], SUs perform spectrum sensing to find access opportunities in time domain. The SUs access the licensed channel when it was not used by PUs. In [4], channel sensing and switching were used for SU to access a set of PU channels. In [7, 8], two spectrum sensing methods were studied in CRSN. In [15], both single-user and collaborative spectrum sensing schemes were proposed in cognitive sensor networks, while quantised sensing observation is used. In [13], the SUs accessed the licensed channel directly at any time; the transmission power was limited to avoid unacceptable interference to PU system. The spectrum access opportunity in space domain was used. In order to find available spectrum in different channels, parallel sensing [19], cooperative sensing [17], and wideband spectrum sensing (WSS) were studied. In [16, 17, 20], cooperative sensing has been studied. The SUs sense other channels if the current channel has been occupied or detect a set of channels simultaneously [21–25], to find available spectrum in a wideband spectrum.

Different from the excessive spectrum sensing and access researches, which focus on utilizing the time or space domain only, a spectrum sensing and access strategy of CRSN is proposed in this paper, in which an SU system uses the available spectrum both in time and space domains in multiple channels. WSS is used for SU to identify the presence of PU signals. Different from the normal spectrum sensing scheme, which an SU senses the channels one by one, in WSS scheme, multiple channels are detected simultaneously by an SU; the sensing time durations of channels are the same. After spectrum sensing, the SU accesses all of the channels with mixed access strategy (MAS) [26, 27]. Under MAS, the SU accesses the channels with different powers based on the sensing results. When the channel is sensed as idle, the SU accesses it with a higher transmission power; the available spectrum in time domain is used. Otherwise, transmission power is lower enough to avoid unacceptable interference to PUs. The available spectrum in space domain is used. Thus, comparing with other spectrum access strategies, the SU in MAS obtains greater throughput.

However, all of the channels are selected for WSS which is not a suitable choice. There are three reasons for the necessity of the sensing channels' selection. First, sensing all of the channels is a huge challenge; great energy is needed [9], which is not a good choice for WSNs. Second, if PU signals in some channels are weak, much more sensing time is needed to guarantee the protection of PUs. For the reason of WSS utilization, the sensing overhead of the system is prolonged. Finally, when the idle probabilities of some channels are low, the SU obtains a larger average throughput when it accesses these channels directly, compared with the average throughout which is obtained when the SU accesses the channels after sensing. Thus, in this paper, the SU selects some channels for spectrum sensing. For the channel which has been selected, the SU accesses it and adjusts its transmission power based on the sensing results. Otherwise, the SU accesses the channels directly via underlay access strategy. Therefore, in our proposed system model, a tradeoff exists between achieving larger throughput and selecting appropriate sensing channels.

According to the dynamic spectrum environment, the SU cannot obtain accurate states of PU channels, due to the imperfect spectrum sensing and not all of the channels are selected for sensing. In this paper, we propose a selection making algorithm by using the partially observable Markov decision process (POMDP) theory. Under the selection making algorithm, the SU determines which channels are selected for spectrum sensing, how long the sensing time, and the transmission powers of the accessed channels. The objective of the selection making algorithm is to maximize the throughput of SU system, while avoiding unacceptable interference to PUs. An optimal policy and a myopic policy are derived to solve the formulated POMDP problem. Moreover, we present an optimization algorithm to solve the synchronism problem among the selected channels. Extensive numerical examples are proposed to demonstrate the merit of the proposed algorithms.

The contributions of this paper can be described as follows.

• (i)

A new selective sensing and access strategy of CRSN based on POMDP theory is proposed, in which at beginning of each slot, an SU selects some channels for wideband spectrum sensing and accesses all of the channels via mixed access strategy.

(ii)

An optimal policy and a myopic policy are proposed to solve the proposed POMDP problem.

(iii)

We consider an optimization problem, in which the decision probability thresholds of the selected channels are jointly optimized, in order to ensure the synchronism among the selected channels.

The rest of this paper is organized as follows. In Section 2, we review the related work. System model is proposed and analyzed in Section 3. In Section 4, we discuss a selection making algorithm via the POMDP framework; an optimal policy and a myopic policy are proposed to solve the proposed POMDP problem. The advantage of the proposed algorithms is illustrated by numerical results in Section 5, and conclusions are drawn in Section 6.

2. Related Work

Wideband spectrum sensing has been discussed in [21–25]. In [21], the detection thresholds of energy detectors in channels were jointly optimized. In order to maximize the throughput of SU, the sensing time and detection thresholds were jointly optimized in [22]. In [23, 24], both the sensing time and transmission powers of channels are jointly optimized. However, in these works, the SU selects all of the channels for spectrum sensing, and accesses these channels only when the sensing result is idle. The selection of sensing channels has not been considered. Different from the previous works, in this paper, not all of the channels are selected for sensing. After spectrum sensing, the SU accesses the channels via mixed access strategy. No matter the sensing result is idle or occupied, the SU accesses the channel with different transmission powers, and greater throughput is obtained. Moreover, different from the previous works in which the WSS was considered in one slot, we consider the sensing channels' selection and spectrum access in multiple slots. The problem becomes complicated.

According to the time-varying character of the dynamic spectrum environment, POMDP is used to formulate the selective spectrum sensing and access problem. In [28, 29], two optimal opportunistic spectrum access MAC protocols were proposed. In [30], a well-known separation principle was proposed to transfer the solution of the POMDP problem from optimal policy to myopic policy. However, both are based on the condition that the SU can sense and access one channel in a slot. In our proposed scheme, the SU selects several channels for spectrum sensing and accesses all of the channels after sensing. The calculation of sensing time becomes complicated. In [31], an adaptive sensing scheduling scheme was proposed, based on POMDP theory. The study [32] is probably the most relevant paper, in which an optimal sensing channels' selection policy is proposed. However, it accesses the channel only when the sensing result is idle. In our work, the SU accesses all of the channels with different transmission powers. Although the previous works take the same mathematical method (POMDP), the problem in this paper becomes quite complicated, and some efficient methods are proposed to solve the problem.

3. System Model

In this section, we present the system model of this paper and the structure of wideband spectrum sensing. Then, a selective sensing and access strategy in multiple channels is proposed.

3.1. System Model

An SU system shares a licensed wideband spectrum assigned to PUs, which can be divided into N nonoverlapping narrowband channels. The channels operate in a time-slotted manner. The traffic of PU system is modeled as a two states ON-OFF process. Figure 1 shows the structure of SU system. We assume the SU obtains the duration of slot and can keep synchronization with PUs [10, 11]. Denote ${\alpha }_n$ as the probability that channel n transits from ON to OFF state and ${\beta }_n$ as the probability that channel n transits from OFF to ON state. We assume ${\alpha }_n$ and ${\beta }_n$ are obtained from the previous long time measurement. $P_n(H_{\mathrm{0}})={\alpha }_n/({\alpha }_n+{\beta }_n)$ and $P_n(H_{\mathrm{1}})={\beta }_n/({\alpha }_n+{\beta }_n)$ , $n\in \{\mathrm{1,2},\dots ,N\}$ , where $P_n(H_{\mathrm{0}})$ and $P_n(H_{\mathrm{1}})$ are the average idle and occupied probabilities. Let a $\mathrm{1}\times N$ vector $\mathbf{S}(m)$ denote the state vector of PU channels in slot m, $\mathbf{S}(m)=(s_{\mathrm{1}}(m),s_{\mathrm{2}}(m),\dots ,s_N(m))$ , where m is the index of slot, $m\in \{\mathrm{1,2},\dots ,M\}$ , $s_n(m)=\mathrm{1}$ indicates the operating channel n in slot m is occupied, and $s_n(m)=\mathrm{0}$ indicates the channel is idle. The state space of $\mathbf{S}(m)$ is

$$\begin{array}{cc}& \mathbf{w}\left(m\right)=\left\{\left(w_{\mathrm{1}},w_{\mathrm{2}},\dots ,w_N\right)\hspace{0.17em}|\hspace{0.17em}{w}_n\in \left\{\mathrm{0,1}\right\}\right\}.\end{array}$$

(1)

Let $P_{w_n{w}_n^{\prime }}$ denote the state transition probability from state $w_n$ to state $w_n^{\prime }$ of channel n. $P_{w_n{w}_n^{\prime }}$  =  $\text{Pr}(s_n(m+\mathrm{1})=w_n\hspace{0.17em}|\hspace{0.17em}{s}_n(m)=w_n^{\prime })$ , where $w_n^{\prime }\in \{\mathrm{0,1}\}$ . For channel n, the state transition probabilities are $P_{\mathrm{01}}={\beta }_n$ , $P_{\mathrm{00}}=\mathrm{1}-P_{\mathrm{01}}$ , $P_{\mathrm{10}}={\alpha }_n$ , and $P_{\mathrm{11}}=\mathrm{1}-P_{\mathrm{10}}$ .

Figure 1

Structure of SU system.

At beginning of each slot, the SU selects some channels for WSS and accesses the channels with MAS. For the other channels, the SU waits the same time, in order to keep synchronism with others. The unselected channels are accessed directly. The SU cannot transmit in one channel while performing spectrum sensing in another one. Denote ${\tau }_n(m)$ as the sensing time duration of channel n in slot m. Under MAS, the SU detects the channel states firstly. When the sensing result is idle, the SU accesses it with power $P_{\mathrm{1}n}$ . When the sensing result is occupied, the transmission power is $P_{\mathrm{2}n}$ . Normally, we assume $P_{\mathrm{1}n}>P_{\mathrm{2}n}$ . The values of $P_{\mathrm{1}n}$ and $P_{\mathrm{2}n}$ are obtained by some power allocation optimization algorithms [27], which are not our focus in this paper. If the SU accesses the channel directly, the transmission power is $P_{\mathrm{2}n}$ . After transmission, the SU receiver announces ACKs to the transmitter of SU. If the transmission is not successful, the SU receiver announces a NAK. The time duration for acknowledges is ignored in our proposed structure. The structure of SU in one slot is shown as Figure 2.

Figure 2

Structure of SU in one slot.

3.2. Wideband Spectrum Sensing

In WSS scheme, we assume an wideband spectrum is occupied by some PUs. The PUs are operating in different spectrum bands; idle probabilities of channels are not the same. WSS is used to identify the presence of PUs in some special channels. In WSS, the wideband is divided into N nonoverlapping channels. The SU receives data through a wideband RF antenna. Then, the received data is passed through a high speed A/D converter [21–25]. The structure of WSS is shown as Figure 3. We can find that the sensing time durations of channels are the same.

Figure 3

Structure of wideband spectrum sensing.

3.3. Selective Sensing and Access Strategy

At the beginning of each slot, the SU determines (1) which channels are selected for spectrum sensing, (2) how long the sensing time, (3) which channels to be accessed with power $P_{\mathrm{1}n}$ , and which channels with power $P_{\mathrm{2}n}$ . The design objective is to maximize the throughput of SU system during a desired period of M slots, while the interferences to PUs are under the predetermined thresholds. Let $\mathrm{\Omega }$ denote the set of the selected channels. For the channel n in slot m, $n\in \mathrm{\Omega }$ , the SU selects it for spectrum sensing and accesses it based on the sensing results. According to the difference between the sensing results and the real states of channel, four cases are considered. When the sensing result is idle and the channel is not occupied by PUs, the transmission is successful with power $P_{\mathrm{1}n}(m)$ ; the transmission rate is $R_{\mathrm{00}n}(m)$ . However, if the real state of the channel is occupied and the SU has not detected it, the transmission of SU cannot be successful, for the reason of the interference from PUs. When the sensing result is occupied, the transmission power is $P_{\mathrm{2}n}(m)$ . The power $P_{\mathrm{2}n}(m)$ is limited to avoid unacceptable interference to PUs. No matter the real state of channel is idle or occupied, the transmission of SU is successful. Therefore, the achievable rate of channel n in slot m is given by

$$\begin{array}{l}{r}_n\left(m\right)=P_n\left(H_{\mathrm{0}}\right)\left(\mathrm{1}-P_{f,n}\left({\tau }_n\left(m\right)\right)\right)R_{\mathrm{00}n}\left(m\right)\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.17em}+P_n\left(H_{\mathrm{0}}\right)P_{f,n}\left({\tau }_n\left(m\right)\right)R_{\mathrm{01}n}\left(m\right)\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.17em}+P_n\left(H_{\mathrm{1}}\right)P_{d,n}\left({\tau }_n\left(m\right)\right)R_{\mathrm{11}n}\left(m\right),\end{array}$$

(2)

where $R_{\mathrm{00}n}(m)=B\text{lo}{\text{g}}_{\mathrm{2}}(\mathrm{1}+(P_{\mathrm{1}n}(m)h_{ssn}/N_{\mathrm{0}}))$ , $R_{\mathrm{01}n}(m)=B\text{lo}{\text{g}}_{\mathrm{2}}(\mathrm{1}+(P_{\mathrm{2}n}(m)h_{ssn}/N_{\mathrm{0}}))$ , and $R_{\mathrm{11}n}(m)$  =  $B\text{lo}{\text{g}}_{\mathrm{2}}(\mathrm{1}+(P_{\mathrm{2}n}(m)h_{ssn}/(P_p{h}_{psn}+N_{\mathrm{0}})))$ . $h_{ssn}$ and $h_{psn}$ are the channel gains, B is the bandwidth of channel, and $P_p$ is the power of PUs. The channel which has not been selected, the SU estimates it as occupied and accesses it directly with transmission power $P_{\mathrm{2}n}(m)$ . The transmission rate is $r_n^{\prime }(m)=P_n(H_{\mathrm{0}})\text{lo}{\text{g}}_{\mathrm{2}}(\mathrm{1}+(P_{\mathrm{2}n}(m)h_{ssn}/N_{\mathrm{0}}))+P_n(H_{\mathrm{1}})\text{lo}{\text{g}}_{\mathrm{2}}(\mathrm{1}+(P_{\mathrm{2}n}(m)h_{ssn}/(P_p{h}_{psn}+N_{\mathrm{0}})))$ . To maximize the throughput of SU system, the optimization problem is formulated as

$$\begin{array}{cc}& \underset{\mathrm{\Omega },\tau \left(m\right)}{\max }{\displaystyle \sum }_{m=\mathrm{1}}^M‍\frac{T-{\tau }_n\left(m\right)}{T}\left({\displaystyle \sum }_{n\in \mathrm{\Omega }}‍r_n\left(m\right)+{\displaystyle \sum }_{n\notin \mathrm{\Omega }}‍r_n^{\prime }\left(m\right)\right)\end{array}$$

(3)

subject to

$$\begin{array}{cc}& P_{d,n}\left({\tau }_n\left(m\right)\right)\ge P_{d,th},\end{array}$$

(4)

$$\begin{array}{cc}& {\tau }_{\mathrm{1}}\left(m\right)={\tau }_{\mathrm{2}}\left(m\right)=\cdots ={\tau }_N\left(m\right)=\tau \left(m\right),\end{array}$$

(5)

$$\begin{array}{cc}& n\in \left\{\mathrm{1,2},\dots ,N\right\}\mathrm{\hspace{0.17em}\hspace{0.17em}}\mathrm{\hspace{0.17em}\hspace{0.17em}}\mathrm{\hspace{0.17em}\hspace{0.17em}}\mathrm{\hspace{0.17em}\hspace{0.17em}}\mathrm{\hspace{0.17em}\hspace{0.17em}}\mathrm{\hspace{0.17em}\hspace{0.17em}}\mathrm{\hspace{0.17em}\hspace{0.17em}}m\in \left\{\mathrm{1,2},\dots ,M\right\}.\end{array}$$

(6)

Constraint (4) is a sensing time constraint, in order to protect PUs. Constraint (5) guarantees the WSS is used for spectrum sensing. It is not easy to solve this problem directly. We present a selection making algorithm based on POMDP theory, to find the optimal and suboptimal solutions.

4. Selection Making Algorithm

In the proposed strategy, because not all of the channels are selected for sensing and the presence of sensing errors, the SU cannot obtain the accurate states of each channel. At the beginning of each slot, decisions are made based on the previous actions and observations. This setting matches well with the POMDP framework [28–35]. Therefore, an optimization selection making problem under the proposed strategy is formulated as a POMDP, which determines an optimal policy for sensing channels' selection, the size of sensing time, and the access decisions. Next, we describe the POMDP framework. An optimal and a myopic policy are proposed to solve the POMDP problem.

4.1. POMDP Framework

4.1.1. Actions

At the beginning of slot m, the actions of SU have three stages: determine which channels to sense, the size of sensing time, and the transmission power. Let $\mathbf{A}(m)$ denote the SU action in slot m

$$\begin{array}{cc}& \mathbf{A}\left(m\right)=\left[{\mathbf{A}}_{\mathrm{1}}\left(m\right),\mathit{\tau }\left(m\right),{\mathbf{A}}_{\mathrm{2}}\left(m\right)\right],\end{array}$$

(7)

where ${\mathbf{A}}_{\mathrm{1}}(m)$ denotes which channels are selected for spectrum sensing, ${\mathbf{A}}_{\mathrm{1}}(m)=[a_{\mathrm{11}}(m),a_{\mathrm{12}}(m),\dots ,a_{\mathrm{1}N}(m)]$ , $a_{\mathrm{1}n}(m)\in \{\mathrm{0,1}\}$ . When channel n is selected for sensing, $a_{\mathrm{1}n}(m)=\mathrm{1}$ . Otherwise, $a_{\mathrm{1}n}(m)=\mathrm{0}$ . $\mathit{\tau }(m)$ denotes the sensing time duration. $\mathit{\tau }(m)=[{\tau }_{\mathrm{1}}(m),{\tau }_{\mathrm{2}}(m),\dots ,{\tau }_N(m)]$ , and ${\tau }_{\mathrm{1}}(m)={\tau }_{\mathrm{2}}(m)=\cdots ={\tau }_N(m)$ . The WSS is used to detect the presence of PU channels. The sensing time durations of the selected channels are the same. If none of the channels are selected, the sensing time durations are zero. ${\mathbf{A}}_{\mathrm{2}}(m)$ denotes which channels are selected to access with power $P_{\mathrm{1}n}(m)$ or $P_{\mathrm{2}n}(m)$ . ${\mathbf{A}}_{\mathrm{2}}(m)=[a_{\mathrm{21}}(m),a_{\mathrm{22}}(m),\dots ,a_{\mathrm{2}N}(m)]$ , $a_{\mathrm{2}n}(m)\in \{\mathrm{0,1}\}$ , $a_{\mathrm{2}n}(m)=\mathrm{1}$ indicates the channel n is accessed with $P_{\mathrm{1}n}(m)$ and $a_{\mathrm{2}n}(m)=\mathrm{0}$ indicates it is accessed with power $P_{\mathrm{2}n}(m)$ . If the channel n has not been selected for sensing, the transmission power is $P_{\mathrm{2}n}(m)$ , $a_{\mathrm{2}n}(m)=\mathrm{0}$ .

4.1.2. Observations

Let $\mathbf{\theta }(m)$ denote the channel observation vector in slot m. $\mathbf{\theta }(m)=[{\theta }_{\mathrm{1}}(m),{\theta }_{\mathrm{2}}(m),\dots ,{\theta }_N(m)]$ . In the proposed scheme, the observation of channel n in slot m has four possible values. ${\theta }_n(m)\in \{\mathrm{0,1},\mathrm{2,3}\}$ .

• (i)

If channel n has been selected for sensing and the sensing result is idle, the SU accesses it with power $P_{\mathrm{1}n}(m)$ . After transmission, the SU transmitter receives ACK1, which indicates the sensing result is correct. We denote this as observation 0, ${\theta }_n(m)=\mathrm{0}$ .

(ii)

However, if the SU transmitter receives NAK, it indicates the sensing result is wrong; the transmission of SU causes unacceptable interference to PU. We denote this as observation 1, ${\theta }_n(m)=\mathrm{1}$ .

(iii)

If the sensing result of channel n is occupied, the SU accesses it with power $P_{\mathrm{2}n}(m)$ . The SU transmitter receives ACK2 after transmission. We denote this as observation 2, ${\theta }_n(m)=\mathrm{2}$ .

(iv)

If the SU accesses channel n directly with power $P_{\mathrm{2}n}(m)$ and receives ACK2 after transmission. We denote this as observation 3, ${\theta }_n(m)=\mathrm{3}$ .

The difference between observations 2 and 3 is that the SU has not sensed the channel in observation 3. Although the announcing signals are the same, the transmitter of SU can distinguish different observations.

4.1.3. Belief Vector

In the POMDP formulation, belief vector is used to infer the channel states at the beginning of each slot. It is a conditional probability for the past history, including the past decisions and observations. At the end of each slot, the belief vector is updated based on different actions and the corresponding observations, in order to obtain accurate information of the dynamic environment. The belief vector of channel n in slot m is denoted as ${\mathbf{b}}_n(m)$ , ${\mathbf{b}}_n(m)=[b_{n,\mathrm{0}}(m),b_{n,\mathrm{1}}(m)]$ , $b_{n,\mathrm{0}}(m)+b_{n,\mathrm{1}}(m)=\mathrm{1}$ . $b_{n,\mathrm{0}}(m)$ is a conditional idle probability. Consider the following:

$$\begin{array}{l}{b}_{n,\mathrm{0}}(m+\mathrm{1})\\ \hspace{1em}=\text{Pr}\left(s_n\left(m+\mathrm{1}\right)=\mathrm{0}\hspace{0.17em}|\hspace{0.17em}{b}_{n,\mathrm{0}}\left(m\right),{\mathbf{A}}_n\left(m\right),{\theta }_n\left(m\right)\right),\end{array}$$

(8)

where ${\mathbf{A}}_n(m)=[{\mathbf{A}}_{\mathrm{1}n}(m),{\mathit{\tau }}_n(m),{\mathbf{A}}_{\mathrm{2}n}(m)]$ is the actions of SU in channel n. Let ${\mathbf{b}}_n(\mathrm{0})$ denote the initial belief vector; it is equal to the stationary probability vector. $b_{n,\mathrm{0}}(\mathrm{0})={\alpha }_n/({\alpha }_n+{\beta }_n)$ , $b_{n,\mathrm{1}}(\mathrm{0})={\beta }_n/({\alpha }_n+{\beta }_n)$ . At the end of slot m, the SU transmitter receives four different observations; the belief vectors $b_{n,\mathrm{0}}(m)$ and $b_{n,\mathrm{1}}(m)$ are updated in different cases. ${\theta }_n(m)=\mathrm{0}$ means the real state of the channel n in slot m is idle. The belief vector in slot $m+\mathrm{1}$ is $b_{n,\mathrm{0}}(m+\mathrm{1})=P_{\mathrm{00}}$ . ${\theta }_n(m)=\mathrm{1}$ means the real state is occupied. $b_{n,\mathrm{0}}(m+\mathrm{1})=P_{\mathrm{10}}$ . When ${\theta }_n(m)=\mathrm{2}$ , the SU senses channel n and the sensing result is occupied, but the SU cannot obtain the real state of that channel. From the Bayes rule, the belief vector updated formula is given as the following equation:

$$\begin{array}{l}{b}_{n,\mathrm{0}}(m+\mathrm{1})\\ =({\displaystyle \sum }_{w_n=\mathrm{0}}^{\mathrm{1}}‍b_{n,w_n}\left(m\right)P_{w_n\mathrm{0}}\\ \phantom{11111111}\times \text{Pr}\left({\theta }_n\left(m\right)\hspace{0.17em}|\hspace{0.17em}{\mathbf{A}}_n\left(m\right),s_n\left(m+\mathrm{1}\right)=\mathrm{0}\right)\phantom{{\displaystyle \sum }_{w_n=\mathrm{0}}^{\mathrm{1}}‍})\\ \hspace{1em}\times ({\displaystyle \sum }_{w_n^{\prime }=\mathrm{0}\hspace{0.17em}\hspace{0.17em}}^{\mathrm{1}}‍{\displaystyle \sum }_{w_n=\mathrm{0}}^{\mathrm{1}}‍b_{n,w_n}\left(m\right)P_{w_n{w}_n^{\prime }}\\ \phantom{11111111111111}\times {\text{Pr}\left({\theta }_n\left(m\right)\hspace{0.17em}|\hspace{0.17em}{A}_n\left(m\right),s_n\left(m+\mathrm{1}\right)=w_n^{\prime }\right)\phantom{{\displaystyle \sum }_{w_n^{\prime }=\mathrm{0}}^{\mathrm{1}}‍})}^{-1}\\ =\left(\left(b_{n,o}\left(m\right)P_{\mathrm{00}}+b_{n,\mathrm{1}}\left(m\right)P_{\mathrm{10}}\right)P_{f,n}\right)\\ \hspace{1em}\times (\left(b_{n,\mathrm{0}}\left(m\right)P_{\mathrm{00}}+b_{n,\mathrm{1}}\left(m\right)P_{\mathrm{10}}\right)P_{f,n}\\ \hspace{1em}\phantom{1111}{+{\left(b_{n,o}\left(m\right)P_{\mathrm{01}}+b_{n,\mathrm{1}}\left(m\right)P_{\mathrm{11}}\right)P}_{d,n}\phantom{\left(b_{n,\mathrm{0}}\left(m\right)P_{\mathrm{00}}+b_{n,\mathrm{1}}\left(m\right)P_{\mathrm{10}}\right)})}^{-1}.\end{array}$$

(9)

Finally, when ${\theta }_n(m)=\mathrm{3}$ , the belief vector is updated as

$$\begin{array}{cc}& b_{n,\mathrm{0}}\left(m+\mathrm{1}\right)=b_{n,\mathrm{0}}\left(m\right)P_{\mathrm{00}}+b_{n,\mathrm{1}}\left(m\right)P_{\mathrm{10}}.\end{array}$$

(10)

4.1.4. Reward Function

Denote $R_n(m)$ as the immediate reward of channel n in slot m; the immediate reward of the SU system is $R(m)={\sum }_{n=\mathrm{1}}^N{R}_n(m)$ . It is associated with actions and observations. Denote ${\tilde{R}}_n(m)$ as the expected reward of channel n. When the channel n has not been selected for spectrum sensing, we have

$$\begin{array}{cc}& {\tilde{R}}_n\left(m\right)=R_n\left(m\right)=r_n^{\prime }\left(m\right).\end{array}$$

(11)

Otherwise, the expected reward of the SU system $\tilde{R}(m)$ is calculated as

$$\begin{array}{l}\tilde{R}\left(m\right)\\ \hspace{0.5em}={\displaystyle \sum }_{n=\mathrm{1}}^N‍{\tilde{R}}_n\left(m\right)\\ \hspace{0.5em}={\displaystyle \sum }_{n=\mathrm{1}\hspace{0.17em}\hspace{0.17em}}^N{\displaystyle \sum }_{z=\mathrm{0}}^{\mathrm{2}}‍‍\text{Pr}\left({\theta }_n\left(m\right)=z\hspace{0.17em}|\hspace{0.17em}{\mathbf{b}}_n\left(m-\mathrm{1}\right),{\mathbf{A}}_n\left(m\right),s_n\left(m\right)\right)R_n\left(m\right),\end{array}$$

(12)

where $z\in \{\mathrm{0,1},\mathrm{2}\}$ denotes the observation value space. $\text{Pr}({\theta }_n(m)=z\hspace{0.17em}|\hspace{0.17em}{\mathbf{b}}_n(m-\mathrm{1}),{\mathbf{A}}_n(m),s_n(m))$ is the probability of observation; it is associated with the belief vector in last slot, the actions and the real state of the current slot. The expression of $\tilde{R}(m)$ is shown as follows:

$$\begin{array}{cc}& \tilde{R}\left(m\right)=\{\begin{array}{cc}\frac{\left(T-{\tau }_n\left(m\right)\right)}{T}{R}_{\mathrm{00}n}\hfill & \hfill \\ \times \hspace{0.17em}(b_{n,\mathrm{0}}(m-\mathrm{1})P_{\mathrm{00}}\hfill & \hfill \\ \phantom{\mathrm{111}}+b_{n,\mathrm{1}}(m-\mathrm{1})P_{\mathrm{10}})(\mathrm{1}-P_{f,n}),\hfill & \text{if}\hspace{0.17em}{\theta }_n\left(m\right)=\mathrm{0}\hfill \\ \mathrm{0},\hfill & \text{if}\hspace{0.17em}{\theta }_n\left(m\right)=\mathrm{1}\hfill \\ \frac{(T-{\tau }_n\left(m\right))}{T}{R}_{\mathrm{01}n}\hfill & \hfill \\ \times (b_{n,\mathrm{0}}(m-\mathrm{1})P_{\mathrm{00}}\hfill & \hfill \\ \phantom{\mathrm{111}}+b_{n,\mathrm{1}}(m-\mathrm{1})P_{\mathrm{10}})P_{f,n}\hfill & \hfill \\ \hspace{1em}+\frac{\left(T-{\tau }_n\left(m\right)\right)}{T}{R}_{\mathrm{11}n}\hfill & \hfill \\ \times (b_{n,\mathrm{0}}(m-\mathrm{1})P_{\mathrm{01}}\hfill & \hfill \\ \phantom{\mathrm{111}}+b_{n,\mathrm{1}}(m-\mathrm{1})P_{\mathrm{11}})P_{d,n},\hfill & \text{if}\hspace{0.17em}{\theta }_n\left(m\right)=\mathrm{2}.\hfill \end{array}\end{array}$$

(13)

Based on the above discussion and analysis, the procedure in the POMDP framework is shown as Figure 4.

Figure 4

The procedure in the POMDP framework.

4.2. Solution to POMDP

In the proposed scheme, the design objective is to develop an optimal selective sensing and access policy in each slot, in order to maximize the expected total reward obtained in the finite M slots. The complete problem formulation based on POMDP is given by

$$\begin{array}{cc}& \underset{\mathbf{A}\left(m\right)}{\max \mathrm{\hspace{0.17em}\hspace{0.17em}}}{\displaystyle \sum }_{n=\mathrm{1}\hspace{0.17em}}^N‍{\displaystyle \sum }_{m=\mathrm{1}}^M‍R_n\left(m\right)\end{array}$$

(14)

subject to

$$\begin{array}{cc}& P_{d,n}\left({\tau }_n\left(m\right)\right)\ge P_{d,th}\\ n\in \left\{\mathrm{1,2},\dots ,N\right\}\mathrm{\hspace{0.17em}\hspace{0.17em}}\mathrm{\hspace{0.17em}\hspace{0.17em}}\mathrm{\hspace{0.17em}\hspace{0.17em}}m\in \left\{\mathrm{1,2},\dots ,M\right\}.\end{array}$$

(15)

It is a constraint POMDP problem, which requires an intractable randomized policy to achieve optimality. However, the objective function can be separated from the constraint, if the SU trusts the current sensing result and accesses the channel based on the sensing result in the current slot [28, 30]. The sensing time is obtained from $P_{d,n}({\tau }_n(m))=P_{d,th}$ , $n\in \{\mathrm{1,2},\dots ,N\}$ . The problem becomes an unconstrained POMDP problem.

After obtaining the sensing time, the problem reduces to a simple one. Two questions are considered: which channels are selected for spectrum sensing and which transmission powers are selected for transmission. The narrowband channels are independent with each other. The actions of SU system can be divided into the combination of each channel's actions. Then, we can calculate the optimal actions of each channel independently.

4.2.1. Optimal Policy

In order to calculate the optimal policy of channel n effectively, a value function $J_m({\mathbf{b}}_n(m-\mathrm{1}))$ is proposed, which denotes the maximum expected remaining reward accumulated from slot m to the frame horizon slot M, when the current belief vector is ${\mathbf{b}}_n(m-\mathrm{1})$ . Using Bellman equation, we have

$$\begin{array}{l}{J}_m\left({\mathbf{b}}_n(m-\mathrm{1})\right)\\ =\underset{{\mathit{A}}_n\left(m\right)}{\max }(\phantom{\left(\Gamma \left({\mathbf{b}}_n\left(m-\mathrm{1}\right),{\mathbf{A}}_n\left(m\right),{\theta }_n\left(m\right)=z\right)\right)}{\tilde{R}}_n\left(m\right)\\ \phantom{\text{hhhhhhh}}+{\displaystyle \sum }_{z=\mathrm{0}}^{\mathrm{3}}‍\text{Pr}\left({\theta }_n\left(m\right)=z\hspace{0.17em}|\hspace{0.17em}{\mathbf{b}}_n\left(m-\mathrm{1}\right),{\mathbf{A}}_n\left(m\right),s_n\left(m\right)\right)\\ \phantom{\text{hhhhhhhhhh}}\times J_{m+\mathrm{1}}\left(\Gamma \left({\mathbf{b}}_n\left(m-\mathrm{1}\right),{\mathbf{A}}_n\left(m\right),{\theta }_n\left(m\right)=z\right)\right)),\end{array}$$

(16)

where

$$\begin{array}{cc}& {\mathbf{b}}_n\left(m\right)=\Gamma \left({\mathbf{b}}_n\left(m-\mathrm{1}\right),{\mathbf{A}}_n\left(m\right),{\theta }_n\left(m\right)=z\right).\end{array}$$

(17)

It represents the updated knowledge of channel state based on the actions and observations of SU in slot m. ${\tilde{R}}_n(m)$ is given by (13). The value function contains two parts: the immediate expected reward obtained in slot m and the maximum expected future reward. The optimal policy is obtained via a fast point-based solution method [35].

4.2.2. Myopic Policy

The solution of optimal policy leads to great computational complexity, especially when the number of channels is large. In order to address this problem, a myopic policy is proposed, in which the SU maximizes the immediate expected reward in the current slot m. The myopic policy solution is given by

$$\begin{array}{cc}& J_m\left({\mathit{b}}_n\left(m-\mathrm{1}\right)\right)=\underset{{\mathit{A}}_n\left(m\right)}{\max }{\tilde{R}}_n\left(m\right).\end{array}$$

(18)

Generally, the myopic policy balances the computational complexity and the optimality of solution. Dynamic programming can be used to find the solution.

4.2.3. Synchronism among Channels

In the proposed solutions, we calculate an optimal policy and a myopic policy of each channel independently, instead of calculating all of the channels at the same time. The access point is $P_{d,n}({\tau }_n(m))=P_{d,th}$ , and the sensing time ${\tau }_n(m)$ is obtained. They may be different from each other. However, in this paper, the SU senses the selected channels with WSS; the sensing time durations of the selected channels should be the same. Therefore, a question exists: how can one ensure the synchronism among the selected channels?

Denote $P_{d,th}^n$ as the detection probability threshold of the selected channel n. In order to maximize the current reward of the slot and ensure the synchronism of the selected channels, we adjust the $P_{d,th}^n$ according to the difference between the selected channels. The sensing time of the selected channel n is obtained from

$$\begin{array}{cc}& P_{d,n}\left({\tau }_n\left(m\right)\right)=P_{d,th}^n.\end{array}$$

(19)

Then, the $P_{d,th}^n$ is formulated as a function of sensing time, $P_{d,th}^n(\tau (m))$ . The optimization problem is formulated as

$$\begin{array}{cc}& \underset{\tau \left(m\right)}{\max }{\displaystyle \sum }_{n\in \mathrm{\Omega }}‍{\tilde{R}}_n\left(m\right)\end{array}$$

(20)

subject to

$$\begin{array}{cc}& P_{d,th}^n\left(\tau \left(m\right)\right)\ge P_{d,th},\end{array}$$

(21)

$$\begin{array}{cc}& {\tau }_n\left(m\right)=\tau \left(m\right),\hspace{1em}n\in \mathrm{\Omega },\\ m\in \left\{\mathrm{1,2},\dots ,M\right\}.\end{array}$$

(22)

The detection probability thresholds of each channel are adjusted, based on the sensing time duration. The $P_{d,th}^n(\tau (m))$ should be set as larger than the predefined threshold $P_{d,th}$ , in order to ensure the protection of PUs. Constraint (21) is converted to the interval of sensing time $\tau (m)$ , based on the convex character of the function $P_{d,th}^n(\tau (m))$ . One-dimensional exhaustion search method is used to solve this problem. The sensing time in slot m is optimized to balance maximizing the immediate current expected reward and keeping synchronism among the selected channels.

5. Numerical Results

In this section, the proposed optimal selective sensing and access policy will be compared with myopic policy and random policy under different simulation conditions. In random policy, the SU selects channels for spectrum sensing, and the access actions are also selected randomly. The sensing time durations of the three policies are the same, which are obtained from the proposed optimization algorithm (20). The slot size of PUs is fixed, the same as the sensing period of SU system, $T=\mathrm{10}$  ms. The channel power gains of channels are ergodic stationary. For the sake of simplicity, we assume the adaptive modulations are not used [33, 34]. When the transmission power is $P_{\mathrm{1}n}$ , the transmission rate is $R_{\mathrm{00}n}(m)=\mathrm{0.06}$  Mbps. When the power is $P_{\mathrm{2}n}$ , the transmission rates are $R_{\mathrm{01}n}(m)=R_{\mathrm{11}n}(m)=\mathrm{0.02}$  Mbps. The bandwidth of each narrowband channel is $B=\mathrm{1}$  MHz and the sampling frequency is $f_s=\mathrm{2}$  MHz. The decision threshold is ${\varepsilon }_n=\mathrm{1.5}$ , the predefined detection probability is $P_{d,th}=\mathrm{0.9}$ , and the number of slot is $M=\mathrm{30}$ .

Figure 5 shows the performance of SU's aggregate throughput under different total channel number N. We consider two cases: $N=\mathrm{6}$ and $N=\mathrm{2}$ ; N denotes the number of subchannels. The idle probability $P_n(H_{\mathrm{0}})$ and the signal-to-noise ratio (SNR) ${\gamma }_n$ are shown in Table 1. It is found that the performance of SU in the optimal policy is better than others. The reason is that the SNR ${\gamma }_n$ affects the sensing time of that channel. When the channel number is large, the channel with smaller SNR affects the whole sensing time duration efficiently. In optimal policy, the SU obtains greater throughput gains when it selects some channels for spectrum sensing, not all of the channels. Comparing with the myopic policy, the selection is more accurate. The SU in random policy may select the channels with lower SNR, which affects the sensing time and the throughput of the SU system. When the channel number is large, the SU in the optimal policy obtains greater throughput than others.

Table 1

Denotation	$N=\mathrm{6}$	$N=\mathrm{2}$	Definition
$P_n(H_{\mathrm{0}})$	0.3 + 0.1( $n-1$ )	0.3 + 0.4( $n-1$ )	Idle probability
${\gamma }_n$	$\mathrm{2.1}$ + 0.1( $n-1$ )	2.1 + 0.2( $n-1$ )	SNR (dB)

Figure 5

SU's throughput performance comparison under different total channel number N.

In Figure 6, we study the performance of SU's aggregate throughput under different idle probability of two adjust PU channels. The idle probability of channel n is $P_n(H_{\mathrm{0}})=\mathrm{0.3}+(n-\mathrm{1})(\mathrm{0.1}+\mathrm{0.05}*(\lambda -\mathrm{1}))$ . When $\lambda =\mathrm{1,2},\mathrm{3}$ , the idle probabilities of the channels are $\mathrm{0.3,0.4,0.5,0.6}$ , $\mathrm{0.3,0.45,0.6,0.75}$ , and $\mathrm{0.3,0.5,0.7,0.9}$ , respectively. It is found that the SU in the optimal policy obtains greater throughput than others under different cases. This is because the idle probabilities of channels affect the updating of belief vector. Under the optimal policy, the SU selects the suitable sensing channels and access actions.

Figure 6

SU's throughput performance comparison under different idle probability λ between two adjust channels.

Figure 7 illustrates the performance of aggregate throughput under optimal sensing time and fixed sensing time. Both are under the myopic policy. In the fixed sensing time case, the SU senses the channels in each slot with ${\tau }_n(m)=\mathrm{2}$  ms, $n\in \{\mathrm{1,2},\mathrm{3,4}\}$ , $m\in \{\mathrm{1,2},\dots ,\mathrm{30}\}$ . It is found that the SU with the optimal sensing time obtains greater throughput. The reason is that the sensing time in each slot is optimized; it balances the tradeoff between maximizing the throughput and sensing efficiency.

Figure 7

SU's throughput performance comparison under different sensing time duration.

Figure 8 illustrates the performance of aggregate throughput under different access strategies. All of them are under the optimal policy. The number of channels is four, and the probability of channel n is $P_n(H_{\mathrm{0}})=\mathrm{0.3}\hspace{0.17em}+\hspace{0.17em}\mathrm{0.2}\hspace{0.17em}*\hspace{0.17em}(n-\mathrm{1})$ ; the SNR is $r_n=\mathrm{2.1}+\mathrm{0.2}(n-\mathrm{1})$ , $n\in \{\mathrm{1,2},\mathrm{3,4}\}$ . It is found that the SU with the mixed access strategy can obtain larger throughput than others. In the underlay access strategy, the SU accesses the channels directly without sensing; the sensing time is saved. However, the transmission power is fixed as $P_{\mathrm{2}n}$ . In the overlay access strategy, the SU accesses the channels just when the sensing result is idle. The SU in the mixed access strategy can access the channels, no matter the sensing result is idle or occupied. Thus, the SU can obtain greater throughput.

Figure 8

SU's throughput performance comparison under different access strategy.

6. Conclusion

In this paper, we propose a selective spectrum sensing and access strategy in cognitive radio sensor networks. In order to maximize the aggregate throughput of SU system and reduce the spectrum sensing energy consumption, the SU selects some channels for spectrum sensing, accesses these channels based on the sensing results, and accesses the other channels directly. According to the dynamic spectrum environment, a selection making algorithm based on PODMP theory is proposed. An optimal policy and a myopic policy are proposed to solve the POMDP problem. Theoretical analysis and numerical results show that the proposed selection making algorithm can better balance maximizing the throughput of SU system and avoiding unacceptable interference to PUs.