Optimal linear quadratic control for wireless sensor and actuator networks with random delays and packet dropouts

Abstract

In this article, the optimal linear quadratic control problem is considered for the wireless sensor and actuator network with stochastic network-induced delays and packet dropouts. Considering the event-driven relay nodes, the optimal solution is obtained, which is a function of the current plant state and all past control signals. It is shown that the optimal control law is the same for all locations of the controller placement. Since the perfect plant state information is available at the sensor, the optimal controller should be collocated with the sensor. In addition, some issues such as the plant state noise and suboptimal solution are also discussed. The performance of the proposed scheme is investigated by an application of the load frequency control system in power grid.

Introduction

Networked control systems (NCSs) in which the shared communication medium is used for the connection between the plant and the controller have recently attracted much attention due to their potential applications in various areas such as power grids, dc motors, and robotic networks.^1,2 In such systems, the network-induced delay and the packet dropout introduced by the communication network cause performance degradation and potential system instability.

In the literature, network-induced delays have been modeled in various forms such as constant delays, ³ short stochastic delays, ⁴ and long stochastic delays. ⁵ The typical approaches to modeling, analysis, and synthesis of NCSs with delays are summarized in Hespanha et al. ⁶ and Zhang et al. ⁷ On the other hand, the packet dropout can be modeled either as a stochastic process ⁸ or a deterministic one. ³ If the packet containing the current state information is dropped, the typical solution is to either predict the lost state, ⁹ or use the previous state values, ³ or simply set the lost state to zero. ¹⁰ With full state information, the optimal control problem can be solved. In particular, the optimal linear quadratic (LQ) controller is developed for the NCS in Nilsson et al. ⁴ and Hu and Zhu ⁵ with stochastic network-induced delays.

Compared with the traditional NCS, wireless NCSs, especially wireless sensor and actuator networks (WSANs), offer architectural flexibility and additional degrees of freedom. ¹¹ Several new standards have recently been introduced for multi-hop WSANs, for example, WirelessHART, ¹² ISA-100. ¹³ Due to the flexibility of placing the controller in multi-hop WSANs, two significant challenges arise: the optimal placement of the controller node and the optimal control law design, which have been studied by a significant amount of works on WSANs. In particular, assuming the packets to be arbitrarily long, the problem of optimal controller placement in multi-hop WSANs is addressed in Robinson and Kumar, ¹⁴ and it is shown that placing the controller at the actuator node achieves better performance than at the sensor node under certain conditions. Over the lossy networks with packet dropouts, the optimal control problem is decomposed into an LQ optimal regulator design and state estimation problem in Gupta et al., ¹⁵ and then the optimal control algorithm is obtained by a backward recursion for every time step. Assuming the sensor and the actuator can exchange data with the controller independently, the LQ Gaussian optimal control problem is addressed in Garone et al. ¹⁶ for both finite and infinite horizon cases. The LQ Gaussian problem has also been studied in Lin et al. ¹⁷ for systems with packet losses but without acknowledgment signals of status of control packet losses. In Xin et al., ¹⁸ the necessary and sufficient conditions under which the closed-loop WSAN is guaranteed to be stable for arbitrary controller location are studied in both one-hop and multi-hop network cases. Given the predesigned control strategy, the location distribution for the optimal controller placement is found in Quevedo et al., ¹⁹ where the actuator node is assumed to have the capability to broadcast the plant state information to the intermediate nodes over parallel unreliable links. However, only the effect of the packet dropout is considered in all the above works, and in this case the optimal control law is linear with the plant state (or estimated state). On the other hand, in practice, the network-induced delay and the packet dropout are the two most important characteristics that should be jointly studied in wireless NCSs.^6,7 Taking the delay into consideration, not only the plant state but also the previous control signals should be used in the design of the optimal controller.^4,5,20,21

In this article, we focus on the general case in which both the stochastic network-induced delays and the packet dropouts are considered, and the optimal control problem is investigated for a linear WSAN with event-driven relay nodes. Using the quadratic cost function, the optimal solution is derived as a function that is linear with the current plant state and all past control signals, and the corresponding suboptimal solution is also proposed. Then, the optimal controller location is deserved to be at the sensor since the perfect plant state is available. The performance of the proposed algorithm is assessed using the application of load frequency control (LFC) in power grid systems.

The remainder of this article is organized as follows. The system model and problem formulation are given in section “System model and problem formulation.” We then derive the optimal controller with stochastic delays and packet dropouts in section “Optimal control law design.” Section “Extension to the long delay case” extends the results to the case of long stochastic network-induced delays. Numerical results and conclusions are given in sections “Simulation results” and “Conclusion,” respectively.

System model and problem formulation

In general, one sensor (or relay) node is selected as the controller to maintain the system stability in WSANs. As shown in Figure 1, node 5 is selected as the controller to coordinately generate control signals which are fed back to the plant. Actually, the WSAN considered in Figure 1 can be simplify converted to be the equivalent wireless multi-hop WSAN as shown in Figure 2,^14,19 where the plant, sensor, actuator, and a number of relay nodes together form a closed-loop NCS.

Figure 1.

The structure of the WSAN.

Figure 2.

The wireless multi-hop WSAN.

We assume that the plant is a continuous-time linear time-invariant system, the sensor is time-driven with a constant sampling period $T$ , while the relay nodes and the actuator are event-driven. At each sampling instant, the measurement of the plant state is sent to the actuator through the multi-hop wireless network, and the controller can be located at either one of the sensor, relay nodes, or actuator to maintain the system stability.^14,15,18,19 The transmission in the WSAN always suffers network-induced delays and packet dropouts. For a given transmission in the $l\mathrm{-}\mathrm{th}$ hop, that is, from node $l$ to node $l+1$ , we denote the network-induced delay ${\tau }_l$ and the packet dropout ${\theta }_l$ , which is a random switch such that ${\theta }_l=1$ indicates that the packet is successfully transmitted, whereas ${\theta }_l=0$ indicates that the packet is lost. Here, we consider a general case that both network-induced delays and packet dropouts are taken into consideration, and all of them are assumed to be independent and stochastic, such that only their distributions are known.^4,5,18,19

For a given controller located at the $l\mathrm{th}$ node, the sensor-to-controller (S/C) delay ${\tau }_l^{\mathrm{sc}}$ , the controller-to-actuator (C/A) delay ${\tau }_l^{\mathrm{ca}}$ , and the sensor-to-actuator (S/A) delay ${\tau }^{\mathrm{tot}}$ are given respectively as

$$\begin{array}{c}{\tau }_l^{\mathrm{sc}}=\sum _{i=1}^{l-1}{\tau }_i,{\tau }_l^{\mathrm{ca}}=\sum _{i=l}^{M\prime -1}{\tau }_i,\hfill \\ \multicolumn{1}{c}{{\tau }^{\mathrm{tot}}={\tau }_l^{\mathrm{sc}}+{\tau }_l^{\mathrm{ca}}=\sum _{i=1}^{M\prime -1}{\tau }_i}\end{array}$$

(1)

where $M\prime$ is the number of nodes through the sensor-to-actuator transmission and the S/C, C/A, and S/A packet dropouts ${\theta }_l^{\mathrm{sc}}$ , ${\theta }_l^{\mathrm{ca}}$ , and ${\theta }^{\mathrm{tot}}$ are given respectively as

$$\begin{array}{c}{\theta }_l^{\mathrm{sc}}=\underset{i=1}{\overset{l-1}{\Pi }}{\theta }_i,{\theta }_l^{\mathrm{ca}}=\underset{i=l}{\overset{M\prime -1}{\Pi }}{\theta }_i,\hfill \\ \multicolumn{1}{c}{{\theta }^{\mathrm{tot}}={\theta }_l^{\mathrm{sc}}{\theta }_l^{\mathrm{ca}}=\underset{i=1}{\overset{M\prime -1}{\Pi }}{\theta }_i}\end{array}$$

(2)

Note that, from equations (1) and (2), we can see that the S/A delay and packet dropout are independent of the controller node location, that is, unrelated to the parameter $l$ .

In this section, we first assume that the S/A delay is smaller than one sampling period, that is, ${\tau }^{\mathrm{tot}}\le T$ , and then the results are extended to the large delay case in section “Extension to the long delay case.” Considering the network-induced delay, the dynamic of the controlled plant is given by the following linear continuous-time model

$$\stackrel{·}{x}(t)=\mathrm{Ax}(t)+{\mathrm{Bu}}_l^a(t-{\tau }^{\mathrm{tot}})$$

(3)

where $x(t)$ is an $M$ -dimensional plant state vector, $u_l^a\left(t\right)$ is the $K$ -dimensional actuator input, and $A$ and $B$ are known matrices of appropriate sizes.

Then, the corresponding discrete-time version of equation (1) is given as [4]

$$x_{k+1}=\Phi x_k+{\Gamma }_k^0{u}_{l,k}^a+{\Gamma }_k^1{u}_{l,k-1}^a$$

(4)

where $x_k=x(\mathrm{kT})$ , $\Phi =e^{\mathrm{AT}}$ , ${\Gamma }_k^0={\int }_0^{T-{\tau }_k^{\mathrm{tot}}}{e}^{\mathrm{As}}\mathrm{dsB}$ , ${\Gamma }_k^1={\int }_{T-{\tau }_k^{\mathrm{tot}}}^T{e}^{\mathrm{As}}\mathrm{dsB}$ , ${\tau }_k^{\mathrm{tot}}$ , and $u_{l,k}^a$ denote the S/A delay and the actuator input, respectively, in response to the sensor signal $x_k$ .

If the packet dropout is also considered, $u_{l,k}^a$ can be expressed as

$$u_{l,k}^a={\theta }_k^{\mathrm{tot}}{u}_{l,k}+(1-{\theta }_k^{\mathrm{tot}})u_{l,k-1}$$

(5)

where $u_{l,k}$ and ${\theta }_k^{\mathrm{tot}}$ are the control signal of node $l$ and the value of switch ${\theta }^{\mathrm{tot}}$ , respectively, at the $k\mathrm{th}$ step. Equation (5) indicates that the actuator remains to use the latest control signal $u_{l,k-1}$ when $u_{l,k}$ is not generated due to the S/C dropout or lost due to the C/A dropout; otherwise, the newly arrived control signal $u_{l,k}$ is used.

If the consecutive packet dropout happens, iteratively using equation (5), the general formulation of $u_{l,k}^a$ is given as

$$u_{l,k}^a={\theta }_k^{\mathrm{tot}}{u}_{l,k}+\sum _{j=0}^{k-1}\left(\underset{i=j+1}{\overset{k}{\Pi }}(1-{\theta }_i^{\mathrm{tot}})\right){\theta }_j^{\mathrm{tot}}{u}_{l,j}$$

(6)

which means that $u_{l,k}^a=u_{l,k-m}$ when ${\theta }_l^{\mathrm{tot}}=0,l=k,k-1,\cdots ,k-m+1$ and ${\theta }_{k-m}^{\mathrm{tot}}=1$ , that is, the latest available control signal $u_{l,k-m}$ is used when $u_{l,k}$ and the previous $(m-1)$ control signals are all lost or not generated.

Substituting equations (6) into (4), the discrete-time control process with delays and packet dropouts can be expressed as

$$x_{k+1}=\Phi x_k+\sum _{j=0}^k{\beta }_{j,k}{u}_{l,k-j}$$

(7)

where

$${\beta }_{j,k}=\{\begin{array}{cc}{\Gamma }_k^0{\theta }_k^{\mathrm{tot}},\hfill & j=0,\hfill \\ \multicolumn{1}{c}{[{\Gamma }_k^0(1-{\theta }_k^{\mathrm{tot}})+{\Gamma }_k^1]{\theta }_{k-1}^{\mathrm{tot}},}& j=1,\hfill \\ \multicolumn{1}{c}{\sum _{q=0}^1\left({\Gamma }_k^q\underset{i=k-j+1}{\overset{k-q}{\Pi }}(1-{\theta }_i^{\mathrm{tot}})\right){\theta }_{k-j}^{\mathrm{tot}},}& \mathrm{others}\hfill \end{array}$$

(8)

which is determined by the S/A delay ${\tau }_k^{\mathrm{tot}}$ and packet dropouts ${\theta }_i^{\mathrm{tot}},i=k,k-1,\cdots ,k-j$ .

Note that there are many works to model the discrete-time control process with packet dropouts. However, most of them ignore the network-induced delay or directly set the actuator input to be zero when the packet dropout happens, which is the typical way to simplify the analysis. Here, considering the event-driven controller and actuator, the plant state loss and the control signal loss can result in no signal (i.e., no event) arriving at the controller and the actuator, respectively. In this case, the actuator input remains the same as the last control signal, but cannot be directly set to zero. In addition, the effect of network-induced delays is also considered. Thus, we formulate the discrete-time control process as in equation (7).

In this article, we focus on the optimal control problem of the WSAN with stochastic network-induced delays and packet dropouts. Using a quadratic cost function as in Robinson and Kumar, ¹⁴ Xin et al., ¹⁸ and Quevedo et al., ¹⁹ the design of the optimal control is to minimize the cost function, that is

$$\underset{\left\{u_{l,k}\right\}}{min}{J}_N=\mathbb{E}[x_N^T{Q}_N{x}_N+\sum _{k=0}^{N-1}(x_k^{T}Q{x}_k+u_{l,k}^{T}R{u}_{l,k})]$$

(9)

where $\mathbb{E}$ is the expectation operator over the distributions of all delays and packet dropouts, $N$ is the total number of sampling instants, $Q_N\cancel{\succcurlyeq }0$ is a symmetric positive semi-definite weight matrix, and $Q\succ 0$ and $R\succ 0$ are symmetric positive definite weight matrices.

Optimal control law design

In this section, the optimal control law is derived for the WSAN in equation (9) subject to the constraint equation (7).

Optimal control law derivation

Define

$$z_{l,k}={\left[\begin{array}{ccccc}{x}_k^T\hfill & u_{l,k-1}^T\hfill & u_{l,k-2}^T\hfill & \cdots \hfill & u_{l,0}^T\hfill \end{array}\right]}^T\in R^{M+\mathrm{kK}}$$

(10)

Then, equation (7) can be rewritten as

$$z_{l,k+1}=C_k{z}_{l,k}+D_k{u}_{l,k}$$

(11)

where the time-varying coefficient matrices $C_k$ and $D_k$ are given as

$$C_k=\left[\begin{array}{ccccc}\Phi \hfill & {\beta }_{1,k}\hfill & {\beta }_{2,k}\hfill & \cdots \hfill & {\beta }_{k,k}\hfill \\ \multicolumn{1}{c}{0}& 0\hfill & 0\hfill & \cdots \hfill & 0\hfill \\ \multicolumn{1}{c}{0}& I_K\hfill & 0\hfill & \cdots \hfill & 0\hfill \\ \multicolumn{1}{c}{0}& 0\hfill & I_K\hfill & \cdots \hfill & 0\hfill \\ \multicolumn{1}{c}{⋮}& ⋮\hfill & ⋮\hfill & \ddots \hfill & ⋮\hfill \\ \multicolumn{1}{c}{0}& 0\hfill & 0\hfill & \cdots \hfill & I_K\hfill \end{array}\right],D_k=\left[\begin{array}{c}{\beta }_{0,k}\hfill \\ \multicolumn{1}{c}{I_K}\\ \multicolumn{1}{c}{0}\\ \multicolumn{1}{c}{⋮}\\ \multicolumn{1}{c}{0}\end{array}\right]$$

(12)

where $0$ is the zero matrix with appropriate size, and $I_K$ is the $K\times K$ identity matrix. Note that $C_k$ and $D_k$ are independent of the location of the controller.

Then, the optimization problem in equation (9) can be rewritten as

$$\begin{array}{cc}\multicolumn{1}{c}{\underset{\left\{u_{l,k}\right\}}{min}}& J_N=\mathbb{E}[z_{l,N}^T{\overline{Q}}_N{z}_{l,N}+\sum _{k=0}^{N-1}(z_{l,k}^T\overline{Q}{z}_{l,k}+u_{l,k}^{T}R{u}_{l,k})],\hfill \\ \multicolumn{1}{c}{s.t.}& z_{l,k+1}=C_k{z}_{l,k}+D_k{u}_{l,k}\hfill \end{array}$$

(13)

where

$${\overline{Q}}_N=\left(\begin{array}{cccc}{Q}_N\hfill & 0\hfill & \cdots \hfill & 0\hfill \\ \multicolumn{1}{c}{0}& 0\hfill & \cdots \hfill & 0\hfill \\ \multicolumn{1}{c}{⋮}& ⋮\hfill & \ddots \hfill & ⋮\hfill \\ \multicolumn{1}{c}{0}& 0\hfill & \cdots \hfill & 0\hfill \end{array}\right),\overline{Q}=\left(\begin{array}{cccc}Q\hfill & 0\hfill & \cdots \hfill & 0\hfill \\ \multicolumn{1}{c}{0}& 0\hfill & \cdots \hfill & 0\hfill \\ \multicolumn{1}{c}{⋮}& ⋮\hfill & \ddots \hfill & ⋮\hfill \\ \multicolumn{1}{c}{0}& 0\hfill & \cdots \hfill & 0\hfill \end{array}\right)$$

(14)

Lemma 1

Assume that function $l(x,y,u)$ has a unique minimum with respect to $u\in U$ for all $x\in X$ and $y\in Y$ . ²² Then, we have

$$\underset{u(x,y)}{min}\mathbb{E}[l(x,y,u)]=\mathbb{E}{min}_u\{l(x,y,u)\}$$

(15)

Define

$$V_L=\underset{u_{l,k},k=L,\cdots ,N-1}{min}\mathbb{E}[z_{l,N}^T{\overline{Q}}_N{z}_{l,N}+\sum _{k=L}^{N-1}(z_{l,k}^T\overline{Q}{z}_{l,k}+u_{l,k}^{T}R{u}_{l,k})]$$

(16)

We next derive the expressions for the optimal control law.

L = N

When $L=N$ , we have

$$V_N=\mathbb{E}\left[z_{l,N}^T{S}_N{z}_{l,N}\right]$$

(17)

where $S_N={\overline{Q}}_N$

L=N–1

When $L=N-1$ , based on Lemma 1, we get

$$\begin{array}{c}\multicolumn{1}{c}{V_{N-1}=\underset{u_{l,N-1}}{min}}\\ \multicolumn{1}{c}{\mathbb{E}\left[(z_{l,N-1}^T\overline{Q}{z}_{l,N-1}+u_{l,N-1}^{T}R{u}_{l,N-1})+V_N|z_{l,N-1}\right]}\\ \multicolumn{1}{c}{=\underset{u_{l,N-1}}{min}\mathbb{E}\{(z_{l,N-1}^T\overline{Q}{z}_{l,N-1}+u_{l,N-1}^{T}R{u}_{l,N-1})}\\ \multicolumn{1}{c}{+\mathbb{E}\left[W^T{S}_N\left(W\right)\right]|z_{l,N-1}\}}\\ \multicolumn{1}{c}{=\mathbb{E}{min}_{u_{l,N-1}}\left[{\left(\begin{array}{c}{z}_{l,N-1}\hfill \\ \multicolumn{1}{c}{u_{l,N-1}}\end{array}\right)}^{T}H\left(\begin{array}{c}{z}_{l,N-1}\hfill \\ \multicolumn{1}{c}{u_{l,N-1}}\end{array}\right)|z_{l,N-1}\right]}\end{array}$$

(18)

where

$$\begin{array}{c}\multicolumn{1}{c}{W=(C_{N-1}{z}_{l,N-1}+D_{N-1}{u}_{l,N-1}),}\\ \multicolumn{1}{c}{H=\left(\begin{array}{cc}\overline{Q}+\mathbb{E}\left[C_{N-1}^T{S}_N{C}_{N-1}\right]\hfill & \mathbb{E}\left[C_{N-1}^T{S}_N^T{D}_{N-1}\right]\hfill \\ \multicolumn{1}{c}{\mathbb{E}\left[D_{N-1}^T{S}_N{C}_{N-1}\right]}& R+\mathbb{E}\left[D_{N-1}^T{S}_N{D}_{N-1}\right]\hfill \end{array}\right)}\end{array}$$

(19)

Then, the optimal control law minimizing equation (18) with respect to $u_{l,N-1}$ is given as [23]

$$u_{l,N-1}=-L_{N-1}{z}_{l,N-1}$$

(20)

where

$$L_{N-1}={[R+\mathbb{E}\left[D_{N-1}^T{S}_N{D}_{N-1}\right]]}^{-1}\mathbb{E}\left[D_{N-1}^T{S}_N{C}_{N-1}\right]$$

(21)

At the same time, substituting $u_{l,N-1}$ in equation (20) into equation (18), we have

$$V_{N-1}=\mathbb{E}\left[z_{l,N-1}^T{S}_{N-1}{z}_{l,N-1}\right]$$

(22)

where

$$\begin{array}{c}\multicolumn{1}{c}{S_{N-1}=\overline{Q}+\mathbb{E}\left[C_{N-1}^T{S}_N{C}_{N-1}\right]}\\ \multicolumn{1}{c}{-L_{N-1}^T[R+\mathbb{E}\left[D_{N-1}^T{S}_N{D}_{N-1}\right]]L_{N-1}}\end{array}$$

(23)

which is symmetric, that is, $S_{N-1}^T=S_{N-1}$

$L=N-2,...,1,0$

When $L=N-2$ , from equations (16) and (22), we have

$$V_{N-2}=\underset{u_{l,N-2}}{min}\mathbb{E}\left[(z_{l,N-2}^T\overline{Q}{z}_{l,N-2}+u_{l,N-2}^{T}R{u}_{l,N-2})+V_{N-1}|z_{l,N-2}\right],$$

(24)

We can see that $V_{N-2}$ has the same form as equation (18). Repeating the derivation from equations (18) to (23), the optimal control law of $u_{l,N-2}$ can be obtained. And similarly for other steps of $u_{l,k},\forall k$ . We then have

$$u_{l,k}=-L_k{z}_{l,k},k=0,1,\cdots ,N-1$$

(25)

where

$$\begin{array}{c}\multicolumn{1}{c}{L_k={[R+\mathbb{E}\left[D_k^T{S}_{k+1}{D}_k\right]]}^{-1}\mathbb{E}\left[D_k^T{S}_{k+1}{C}_k\right],}\\ \multicolumn{1}{c}{S_k=\overline{Q}+\mathbb{E}\left[C_k^T{S}_{k+1}{C}_k\right]-L_k^T[R+\mathbb{E}\left[D_k^T{S}_{k+1}{D}_k\right]]L_k}\end{array}$$

(26)

Here $S_k$ is symmetric, and the corresponding value of $V_k=\mathbb{E}\left[z_{l,k}^T{S}_k{z}_{l,k}\right]$ .

From equations (25) and (26), we observe that the optimal control law $L_k$ is obtained by backward recursion, and the optimal control strategy is linear with the plant state and all past control signals.

Some discussions

In this subsection, some issues are discussed for design of the optimal controller.

Optimal location for the controller

The design of the optimal controller is obtained in equation (25) and we observe that $L_k$ is independent of the controller location $l$ , since all parameters such as $C_k$ and $D_k$ are independent of $l$ . In other words, the optimal control law is the same for all possible placements of the controller. Then, as the full plant state information is needed for the optimal control law, the optimal location should be collocated with the sensor, because the sensor is able to obtain the perfect plant state information, while only partial state information is available for other nodes due to the S/C delays and packet dropouts. Note that some articles assume that the actuator also has the perfect state information directly obtained from the plant. In this case, either the sensor or the actuator can be chosen as the optimal location for the placement of the controller.

State noise

The disturbance usually exists at the plant. If the state noise is considered, equation (4) can be rewritten as

$$x_{k+1}=\Phi x_k+{\Gamma }_k^0{u}_{l,k}^a+{\Gamma }_k^1{u}_{l,k-1}^a+\omega$$

(27)

where $\omega$ is the white noise with zero mean and covariance matrix ${\sigma }_{\omega }$ .

Using equations (27) instead of (4) as the dynamic process for design of the optimal controller, similar to that in subsection “Optimal control law derivation,” we can derive the same optimal control law as equation (25), and the value of $V_k$ is changed to be

$$V_k=\mathbb{E}\left[z_{l,k}^T{S}_k{z}_{l,k}\right]+{\alpha }_k$$

(28)

where ${\alpha }_k=\mathrm{tr}\left(S_{k+1}^{11}{\sigma }_{\omega }\right)+{\alpha }_{k+1}$ , ${\alpha }_N=0$ , and $S_k^{\mathrm{ij}}$ denote the $(i,j)\mathrm{th}$ block of $S_k$ .

The cost function in equation (9) can be derived as $J_N=V_0=1E\left[x_0^T{S}_0^{11}{x}_0\right]+{\alpha }_0$ , where $x_0$ is the initial value of the plant state. Thus, the state noise introduces an extra cost ${\alpha }_0$ , but has no influence on the design of the optimal controller.

Suboptimal solution

From equations (25) and (26), a drawback with the optimal control law is the complicated matrix $L_k$ , since the dimension of $z_{l,k}$ increases when $k$ becomes larger. In order to reduce the computational complexity, one approach is to truncate the effect of packet dropouts that only keep the ones with most contributions. Then, equation (6) is modified to be

$$u_{l,k}^a={\theta }_k^{\mathrm{tot}}{u}_{l,k}+\sum _{j=k-W}^{k-1}\left(\underset{i=j+1}{\overset{k}{\Pi }}(1-{\theta }_i^{\mathrm{tot}})\right){\theta }_j^{\mathrm{tot}}{u}_{l,j}$$

(29)

which means that at most the effect of $W$ consecutive packet dropouts is considered.

Using equation (29), it is equivalent to set $u_{l,j}\equiv 0$ when $j<k-W$ . In this case, the optimal control law in equation (25) can be simplified as

$$u_{l,k}=-L_k{\overline{z}}_{l,k}$$

(30)

where

$$\begin{array}{c}{\overline{z}}_{l,k}={\left[\begin{array}{ccccc}{x}_k^T\hfill & u_{l,k-1}^T\hfill & u_{l,k-2}^T\hfill & \cdots \hfill & u_{l,k-W}^T\hfill \end{array}\right]}^T\in R^{M+\mathrm{WK}},\hfill \\ \multicolumn{1}{c}{L_k={[R+\mathbb{E}\left[D_k^T{S}_{k+1}{D}_k\right]]}^{-1}\mathbb{E}\left[D_k^T{S}_{k+1}{C}_k\right]}\end{array}$$

(31)

and the related parameters $D_k$ and $S_k$ have the same expressions defined in equations (12) and (26), which are reduced to be $(W+1)\times 1$ -block and $(W+1)\times (W+1)$ -block matrices, respectively, and the other parameter $C_k$ is simplified to be $(W+1)\times (W+1)$ -block matrix as follows

$$C_k=\left[\begin{array}{cccccc}\Phi \hfill & {\beta }_{1,k}\hfill & {\beta }_{2,k}\hfill & \cdots \hfill & {\beta }_{W-1,k}\hfill & {\beta }_{W,k}\hfill \\ \multicolumn{1}{c}{0}& 0\hfill & 0\hfill & \cdots \hfill & 0\hfill & 0\hfill \\ \multicolumn{1}{c}{0}& I_K\hfill & 0\hfill & \cdots \hfill & 0\hfill & 0\hfill \\ \multicolumn{1}{c}{0}& 0\hfill & I_K\hfill & \cdots \hfill & 0\hfill & 0\hfill \\ \multicolumn{1}{c}{⋮}& ⋮\hfill & ⋮\hfill & \ddots \hfill & ⋮\hfill & ⋮\hfill \\ \multicolumn{1}{c}{0}& 0\hfill & 0\hfill & \cdots \hfill & I_K\hfill & 0\hfill \end{array}\right].$$

(32)

From the definition of ${\overline{z}}_{l,k}$ in equation (31), its dimension $R^{M+\mathrm{WK}}$ is independent of $k$ . Thus, the drawback that the computational complexity is determined by the value of $k$ is solved.

Extension to the long delay case

In this section, we extend the result with short network-induced delay in section “Optimal control law design” to the case when the delay is larger than one sampling period.

Here, we assume that the S/A network-induced delay is bounded that ${\tau }^{\mathrm{tot}}<\mathrm{hT}$ , where $h$ is a positive integer. Then, during one sampling interval, the number of received control signals on the actuator is between 0 and $h$ . The signal timing is illustrated in Figure 3. to show that the actuation time set is $\{t_{k-h-2},t_{k-h-1},...,t_k,t_{k-1}\}$ . Note that the actuation time $t_j,j=k-h-2,k-h-1,...,k$ only depends on the beginning of the sampling $\mathrm{jT}$ and the corresponding S/A delay ${\tau }_j^{\mathrm{tot}}$ . Thus, $t_j$ is independent of the location of the controller. In addition, the long delay results in a disorder in the control signal arrivals at the actuator, for example, the control signal $u_{l,k}$ arrives before $u_{l,k-1}$ . In this case, we will ignore the control signal $u_{l,k-1}$ and still use the newer control signal $u_{l,k}$ even after $u_{l,k-1}$ arrives at the actuator. Then, for a given sampling interval $[\mathrm{kT},(k+1)T)$ , suppose that there are $m_k$ effective control signals from controller $i$ , where $0\le m_k\le h$ , which arrive at time instants $t_{{\eta }_1},t_{{\eta }_2},...,t_{{\eta }_{m_k}}$ , where $k-h<{\eta }_0<{\eta }_1<{\eta }_2<\cdots <{\eta }_{m_k}<k+1$ , and ${\eta }_0$ denotes the index of the active control signal at time $\mathrm{kT}$ (see Figure 4 for an example). Then, the control signals will be actuated in the order of $u_{l,{\eta }_0},u_{l,{\eta }_1},...,u_{l,{\eta }_{m_{l,k}}}$ , and the corresponding actuation periods are $[\mathrm{kT},t_{{\eta }_1}),[t_{{\eta }_1},t_{{\eta }_2}),\cdots ,[t_{{\eta }_{m_k}},(k+1)T)$ . Thus, integrating equation (3) over a sampling interval $[\mathrm{kT},(k+1)T)$ , we obtain the following discrete-time linear system parameterized by the network-induced delays

$$x_{k+1}=\Phi x_k+\sum _{j=0}^h{\Gamma }_k^j{u}_{l,k-j}^a$$

(33)

where

$${\Gamma }_k^j=\{\begin{array}{cc}{\int }_{\mathrm{kT}}^{t_{{\eta }_1}}{e}^{A(t_{{\eta }_1}-s)}\mathrm{dsB},\hfill & j=k-{\eta }_0,\hfill \\ \multicolumn{1}{c}{{\int }_{t_{{\eta }_1}}^{t_{{\eta }_2}}{e}^{A(t_{{\eta }_2}-s)}\mathrm{dsB},}& j=k-{\eta }_1,\hfill \\ \multicolumn{1}{c}{⋮}& ⋮\hfill \\ \multicolumn{1}{c}{{\int }_{t_{{\eta }_{m_k}}}^{(k+1)T}{e}^{A((k+1)T-s)}\mathrm{dsB},}& j=k-{\eta }_{m_k},\hfill \\ \multicolumn{1}{c}{0,}& \mathrm{otherwise}\hfill \end{array}$$

(34)

Note that ${\Gamma }_k^j$ is random due to the stochastic S/A delays.

Figure 3.

Signal timings.

Figure 4.

An example of arrivals in $[\mathrm{kT},(k+1)T)$ . In this case, ${\eta }_0=k-3,{\eta }_1=k-1,{\eta }_2=k$ . As $u_{k-3}$ is active in the beginning, $u_{k-2}$ arrives later than $u_k$ (and $u_{k-1}$ ), it is ignored. The arrival instants of effect control signals are $t_{k-1}$ and $t_k$ .

Based on equations (6) and (33), the discrete-time control process with long delays is given as

$$x_{k+1}=\Phi x_k+\sum _{j=0}^k{\overline{\overline{\beta }}}_{j,k}{u}_{l,k-j}$$

(35)

where

$${\overline{\beta }}_{j,k}=\{\begin{array}{cc}{\Gamma }_k^0{\theta }_k^{\mathrm{tot}},\hfill & j=0,\hfill \\ \multicolumn{1}{c}{{\rho }_j{\Gamma }_k^j{\theta }_{k-j}^{\mathrm{tot}}+\sum _{q=0}^h\left({\Gamma }_k^q\underset{i=k-j+1}{\overset{k-q}{\Pi }}(1-{\theta }_i^{\mathrm{tot}})\right){\theta }_{k-j}^{\mathrm{tot}},}& \mathrm{others}\hfill \end{array}$$

(36)

and ${\rho }_j=1$ when $j=1,2,...,h$ , and otherwise ${\rho }_j=0$ .

We observe that equations (7) and (35) have the same form. Then, using ${\overline{\beta }}_{j,k}$ instead of ${\beta }_{j,k}$ , the same expression of the optimal control law can be derived as in equation (25). In addition, it can be seen that, ${\overline{\beta }}_{j,k}$ , same as ${\overline{\beta }}_{j,k}$ , is independent of the location of the controller. Thus, for the case of long stochastic delay, similar properties for the short delay case also hold: The optimal control law is the same for all controller placements, and the optimal location should be at the sensor.

Simulation results

In this section, we provide simulation studies on the application of the proposed optimal controller scheme with stochastic network-induced delays and packet dropouts to the LFC system in power grid.^24,25 The typical LFC system is composed of speed governor, turbine, generator, and LFC controller (included in the wireless sensor and actuator network), which is illustrated in Figure 5. The objective is to optimally adjust LFC controller $u$ to keep the frequency deviation $\Delta f$ within prescribed limits. The deviations of the generator-turbine-governor system can be represented by two time constants, $T_t$ of the turbine and $T_g$ of the governor. The generator response is considered to be instantaneous in comparison with the time constants of turbine and governor, which can be written as

$$\begin{array}{c}\multicolumn{1}{c}{\frac{d}{\mathrm{dt}}\Delta P_g=-\frac{1}{T_t}\Delta P_g+\frac{1}{T_t}\Delta X_g,}\\ \multicolumn{1}{c}{\frac{d}{\mathrm{dt}}\Delta X_g=-\frac{1}{T_g}\Delta X_g+\frac{1}{T_g}\Delta P_c}\end{array}$$

(37)

where $\Delta P_g$ , $\Delta X_g$ , and $\Delta P_c$ are the deviations of generator mechanical output, valve position, and generator output, respectively.

Figure 5.

WSAN-based LFC system for power grid.

The deviation of frequency $\Delta f$ is given as

$$\frac{d}{\mathrm{dt}}\Delta f=-\frac{1}{T_p}\Delta f+\frac{K_p}{T_p}\Delta P_g$$

(38)

where $K_p$ is the electric system gain and $T_p$ is the electric system time constant.

The system state vector can be defined as

$$x(t)={\left[\begin{array}{cccc}\Delta P_c\hfill & \Delta f\hfill & \Delta P_g\hfill & \Delta X_g\hfill \end{array}\right]}^T$$

(39)

The sensor is required to measure the four-dimensional corresponding system state, and then send to the LFC controller. Once the state signal has arrived at the controller, a new control signal is calculated and sent to the actuator node.

Then, the linear dynamic control model can be described as

$$\stackrel{·}{x}(t)=\mathrm{Ax}(t)+B{u}^a(t-\tau )$$

(40)

where $u^a$ is the actuator input corresponding to the LFC control signal $u$ , which is affected by the packet losses introduced by the WSAN as shown in equation (6).

Based on equations (37) and (38) and Figure 5, we obtain the system parameters $B=\left[\begin{array}{cccc}1\hfill & 1\hfill & 0\hfill & 0\hfill \end{array}\right]$ and

$$A=\left[\begin{array}{cccc}0\hfill & 0\hfill & 0\hfill & 0\hfill \\ \multicolumn{1}{c}{0}& -1/T_p\hfill & K_p/T_p\hfill & 0\hfill \\ \multicolumn{1}{c}{0}& 0\hfill & -1/T_t\hfill & 1/T_t\hfill \\ \multicolumn{1}{c}{1/T_g}& 0\hfill & 0\hfill & -1/T_g\hfill \end{array}\right]$$

(41)

In the simulation, a 4-hop wireless network composed of the sensor, actuator, and three relay nodes is chosen for simulation. For each hop, it is assumed that the packet dropout follows the Bernoulli distribution with $p=0.9$ , and the network-induced delay is uniform in $[0,\alpha T],0\le \alpha \le 0.5$ where the maximum S/A delay is $2T$ .

In addition, we set the initial value of the plant state $x_0={\left[\begin{array}{cccc}0.25\hfill & 0.15\hfill & 0.2\hfill & 0.1\hfill \end{array}\right]}^T$ , $K_p=1,T_p=0.2,T_t=0.3,T_g=0.08$ , and

$$Q_N=Q=\left[\begin{array}{cccc}1\hfill & 0\hfill & 0\hfill & 0\hfill \\ \multicolumn{1}{c}{0}& 1\hfill & 0\hfill & 0\hfill \\ \multicolumn{1}{c}{0}& 0\hfill & 1\hfill & 0\hfill \\ \multicolumn{1}{c}{0}& 0\hfill & 0\hfill & 1\hfill \end{array}\right],R=1$$

(42)

and the sampling period and duration are chosen as T = 0.05 and N = 50, respectively.

Figure 6 shows the performance comparison when the controller is located at different nodes such as the sensor (the first node) and relay nodes (the second, third, and fourth nodes). We observe that the cost can be reduced when the controller collocates with the sensor, which indicates that the imperfect plant state information caused by the network-induced delays and packet dropouts introduces performance degradation, and the sensor is the optimal controller placement. In addition, the system performance is degraded significantly with the network-induced delay increasing. This reveals that the effect of delays should be considered in the design of the optimal control law.

Figure 6.

Performance comparison for various controller locations.

Figure 7 shows the performance comparison for three cases: our proposed scheme with the optimal controller location, the approach in Robinson and Kumar ¹⁴ where the controller collocates with the actuator, and the LQ optimal control strategy proposed in Garone et al. ¹⁶ ignoring the network-introduced delay. We can observe that our proposed scheme with the optimal controller location outperforms the other two schemes in the sense that it has a lower total cost. It can also be seen that the cost of the approach proposed in Garone et al. ¹⁶ without considering the effect of network-induced delay increases more rapidly than the others. This is because this scheme cannot effectively deal with the network-induced delay so that both network-induced delays and packet losses should be investigated.

Figure 7.

Performance comparison of the proposed scheme with existing approaches.

The performance of the suboptimal approach is shown in Figure 8, where the controller is assumed to be collocated with the sensor. It can be seen that when we truncate the effect of packet dropouts, the performance degradation is significantly decreased with larger $W$ , and the cost is close to the optimal one even when the value of $W$ is small $(W=4)$ . Thus, keeping a proper length of the consecutive packet dropout is also an effective approach to reducing the computational complexity and meanwhile maintaining the near-optimal performance.

Figure 8.

Performance comparison for suboptimal control design.

In Figure 9, the effect of the plant state noise is investigated. Here the covariance matrix of the noise is set as ${\sigma }_{\omega }=\sigma I_M$ , where $\sigma$ is chosen as 0.0005, 0.001, 0.0015, and 0.002. It can be seen that the noise introduces an extra cost to degrade the system performance, and the extra cost increases with the value of $\sigma$ , which well matches the discussion on the cost function in equation (28).

Figure 9.

The effect of plant state noise.

Conclusion

We have considered the design of the optimal control for the WSAN in the presence of stochastic network-induced delays and packet dropouts. In particular, the optimal control solution is derived as a linear function of the current plant state and all past control signals, and the optimal controller should be collocated with the sensor. Other issues such as the state noise and suboptimal solution are also discussed, and the corresponding performances are investigated by the LFC system in power grid.