Optimal Distributed Tracking Control for Nonlinear Cooperative Wireless Sensor Networks

Abstract

The optimal distributed tracking control algorithms over nonlinear cooperative wireless sensor networks (WSNs) are presented in this paper. In order to solve transfer delay and packet loss problem, the architecture of wireless active sensor (WAS) is employed, where a state estimator is embedded, which can provide the needed state information. Furthermore, the optimal distributed tracking control algorithm is proposed. By solving the matrix equations and the adjoint difference equations, the optimal control law can be obtained easily, in which an increment integral regulator is designed to implement tracking target without steady-state error and a nonlinearity compensation term is designed to compensate for the effect produced by system nonlinearity. Moreover, the observer-based dynamical algorithm is given considering the physically unrealizable disturbance states and the unavailable sensor states. Finally, computer simulations are carried out with application to two nonlinear pendulums, which prove that the algorithm is effective and easy to implement, and the system achieves the desired performance based on tradeoff between tracking error and control energy consumption.

1. Introduction

Target tracking has valuable application in scenarios of military, industry, architecture, natural resources detection, search and rescue operation, civil surveillance system, mobile robot, autonomous underwater vehicle, unmanned aerial vehicle, and so forth. For instance, during ocean resource exploration, towed vehicle is required to track the given route in order to load the detection device getting underwater data [1]; the hydrofoil craft tracks the predesigned route to implement the marine task [2]; in space, attitude tracking control for airplane or spacecraft has always been a significant research [3]; the studies of mobile robot on trajectory tracking [4], tour tracking [5], and formation control [6] have sprung up worldwide. With the increasing development of science technology, the requirement for tracking stability, accuracy, real-time quality, and so forth, becomes higher and higher. And thus, WSNs grow to be adopted widely in tracking tasks due to their convenience, flexibility, mobility, and low cost, which play an irreplaceable role in practical engineering by now.

However, some limitations of WSNs, such as noise, interference, limited bandwidth and power, and time-varying gain, bring the factors of uncertainty, nonlinearity, time-delay, or limited energy to WSN-based control systems. Thereby, it is still an emerging and challengeable research to study tracking control problems on WSNs. Currently, there are some results reported on this field. As for the analysis for system characteristics, [7] showed that the tradeoff between data rate, time-delay, and packet loss will greatly affect the performance of distributed wireless networked system; [8] developed the result in [7] for a control system where data exchange between sensor-controller and controller-actuator are all completed over WSNs; [9] analyzed the input-output stability of nonlinear control system on wireless network with disturbance; [10] presented a testbed for ambient intelligence under the constraints of low power, limited spectrum and resources; [11] discussed the effect on an ambient intelligent system by network-induced delay, random uncertainty, and limited packet information and gave three compensation strategies. On the other hand, as for the tracking control issues in WSNs environment, [12] designed the fuzzy observer-based tracking control for time-delay nonlinear distributed parameter system; [13] proposed variable structure control for consensus tracking problem of autonomous vehicle formation; [14] gave an intelligent Fuzzy control for real-time tracking; [15] used range and range-rate measurement for multitarget tracking relating with the uncollected information; in [16], an adaptive predictive control reduced the energy consumption and enhanced tracking accuracy for wireless network system; [17] built a multimodality framework and gave n-step predictive tracking algorithm to conserve the energy consumption. Seeing that, previous works on tracking control problems have given some algorithms or solutions to solve the relevant issues. However, during these reports, some concerned the models with simple representations but not from a large-scale system viewpoint; others tended to ignore the specifics in such WSNs environment, for example, nonlinearity, uncertainty, time-delay, limited power, or steady-state error of tracking.

Alternatively, this paper explores to design an optimal tracking control without steady-state errors with distributed systems in a large-scale system viewpoint, taking into account the disturbance, nonlinearity, and time-delay faced by the system. The key contributions of this paper are twofold: first, to model an appropriate system, an interconnected large-scale system combined by N subsystems is built, in which the interconnected terms represent cooperative communications between the sensors; then, to design the zero steady-state error controller, the relevant optimal tracking control algorithm is presented, which is derived from the matrix equations and adjoint difference equations, in which an increment integral regulator is designed to eliminate the steady-state error. On the basis of the control performance, the tracking error and control energy are designed in the performance index prior so that the goal of the wireless networked control system is ensured to be achieved.

The paper incorporates three novel features: (1) it gives a distributed architecture. The traditional centralized architecture (Figure 1) is practically proved unsuitable to a large-scale environment for its limited communication bandwidth and power supplies [18–21]. Instead of it, we focus on the distributed architecture, for example, Figure 2, where each sensor can be viewed as an intelligent agent with some degree of autonomy in decision-making, and information is exchanged between nodes to compute an almost optimal sensor-target assignment, which makes it more robust and efficient than the centralized one. (2) It presents a cooperative algorithm for the WSNs-based control design. In the cooperative distributed approach, the sensors are rational; that is, each sensor will not only consider its own benefit over a permissible action space, but also the other sensors' behavior [21, 22]. According to the presented cooperative algorithm, the sensors can know the others' states from the wireless network and consider the tradeoff between tracking performance and the expense of it. (3) It proposes an optimal algorithm for tracking control. As it is known, limited power supply is a challenge which is inherent from the nature of sensors. Although some methods were introduced to deal with this problem, for example, [23, 24], fundamentally, the tracking algorithm should be improved to adapt to this reality. The proposed optimal tracking control design is capable of minimizing the control energy of each sensor.

Figure 1

Two centralized control systems supported by WSNs.

Figure 2

Two distributed cooperative control systems supported by WSNs.

The organization of this paper is as follows. After this introduction, in Section 2, systems description and problem formulation have been done. Section 3 proposes the design procedure of optimal distributed tracking algorithms. Computer simulations are demonstrated in Section 4. Concluding remarks are given in Section 5.

Notation.

Throughout this paper the following notations are used. N denotes the set of integers and $N_0=N\cup \{0\}$ . $R^n$ denotes the n-dimensional Euclidean space with vector norm $\parallel ·\parallel$ . $R^{n\times m}$ is the set of all $n\times m$ matrices. $R^{+}$ denotes the positive real numbers. $C(R^n)$ stands for the set of continuously differentiable functions. $A^T$ indicates the transpose of matrix A. $\parallel A\parallel$ denotes the Euclidean norm of a $n\times n$ of matrix A. I represents the identity matrix and $\mathbf{0}$ represents the zero matrix of appropriate dimensions. $\mu (·)$ denote the eigenvalues of a matrix and ${\mu }_{\min }$ is the minimum eigenvalue. $\mathrm{diag}\{\cdots \}$ stands for a diagonal matrix and $\text{Diag}\{\cdots \}$ stands for a block one. $x_t^{[i]}$ represents the vector of the $i\text{th}$ subsystem of a large-scale distributed system in discrete-time form. $x_t^i$ represents the $i\text{th}$ iterative vector and $\{x_t^i\}$ are the vector sequences for $i=\mathrm{1,2},\dots$ in discrete-time forms.

2. Problem Statement

A cooperative distributed control system over WSNs is illustrated in Figure 2, which consists of two subsystems. As it is shown, the 1st subsystem is combined by three sensors and the 2nd subsystem by two sensors, which are communicated with each other to exchange data by WSNs. In each of these subsystems, a base station acts as the controller which transfers the command signal to the actuator ensuring it be executed by the plant.

2.1. Distributed WSNs Model

In the first place, the distributed control system is modeled and the problem is formulated. The distributed WSNs system is modeled by a discrete-time interconnected large-scale system consisting of N subsystems, which is described by

$$\begin{array}{cc}& x_{t+1}^{\left[i\right]}=A^{\left[i\right]}{x}_t^{\left[i\right]}+{\mathit{A}}^{\left[i\right]}{\mathit{x}}_t+B^{\left[i\right]}{u}_t^{\left[i\right]}+D^{\left[i\right]}{v}_t^{\left[i\right]}+f^{\left[i\right]}\left(x_t^{\left[i\right]}\right),\\ y_t^{\left[i\right]}=C^{\left[i\right]}{x}_t^{\left[i\right]},\hspace{1em}t\in N_0,\hspace{0.17em}\hspace{0.17em}i=\mathrm{1,2},\dots ,N,\end{array}$$

(1)

where ${\mathit{x}}_t=[x_t^{[1]T},x_t^{[2]T},\dots ,x_t^{[N]T}{]}^T$ is the state vector of the large-scale system, and

$$\begin{array}{cc}& {\mathit{A}}^{\left[i\right]}=\left[A^{\left[i,1\right]},\dots ,A^{\left[i,i-1\right]},\mathit{0},A^{\left[i,i+1\right]},\dots ,A^{\left[i,N\right]}\right]\end{array}$$

(2)

is a constant matrix representing information communications within the sensors, by which the $i\text{th}$ subsystem enables to know the other subsystems' states such that they can cooperate with each other to accomplish tasks, such as track target, balance tradeoff, and avoid obstacle.

For the $i\text{th}$ subsystem, $x_t^{[i]}=[x_t^{[i,1]},x_t^{[i,2]},\dots ,x_t^{[i,n_i]}{]}^T$ is the state vector of the $i\text{th}$ sensor, $y_t^{[i]}$ is the output vector, $u_t^{[i]}$ is the distributed controller, and $v_t^{[i]}$ is some exogenous signal suffered by the system, for example, disturbance, noise, or uncertainties affecting the plant, which can be represented by some periodical or attenuated signal. $A^{[i]},B^{[i]},C^{[i]}$ ,and $D^{[i]}$ are constant matrices of appropriate dimensions. $f^{[i]}(·):C(R^{n_i})\to R^{n_i}$ represents the nonlinearity displayed by the $i\text{th}$ subsystem, which satisfies Lipschitz condition and $f^{[i]}(0)=\mathit{0}$ . The initial state vector of the $i\text{th}$ system (1) is $x_0^{[i]}$ . The following assumptions are needed for the derivation process.

Assumption 1.

The pair $(A^{[i]},B^{[i]})$ is completely controllable.

Assumption 2.

The pair $(A^{[i]},C^{[i]})$ is completely observable.

For physical implement, Assumptions 1 and 2 guarantee the communicability and connectivity of information over a network. Under these conditions, the data can flow and be observed over network among controllers, actuators, and sensors, which enable information from sensors to reflect the information of system states completely. On the other hand, for mathematical derivation, the two assumptions guarantee the stabilizability and observability of the $i\text{th}$ system.

Since every subsystem is described by system (1), in what follows, the superscript $[i]$ is dropped for simplification reason except when needed for clarification.

Supposing that the dynamical characteristic of disturbance is unknown, thus it can be described in the form of an exosystem:

$$\begin{array}{cc}& w_{t+1}=G{w}_t,\hspace{1em}\hspace{1em}{v}_t=F{w}_t,\end{array}$$

(3)

where the pair $(G,F)$ is assumed completely observable. System (3) can describe several kinds of disturbance signals, such as step, sinusoidal [25], or random [26].

Furthermore, the tracked target can be denoted by a reference vector $r_t$ , which is assumed asymptotically stable (a.s.) or stable. Then, the tracking error is defined by $e_t=r_t-y_t$ .

2.2. Problem Formulation

In order to design a control law to track the target without steady-state error, an integrator is to be designed. Therefore, the control increment ${\stackrel{-}{u}}_t\triangleq \mathrm{\Delta }{u}_t=u_t-u_{t-1}$ is defined. And then, regarding the $i\text{th}$ system, the target should be tracked with ${\lim }_{t\to \infty }{e}_t=0$ , and the control energy consumption should be minimized as well. Thus, choose tracking error $e_t$ and control ${\stackrel{-}{u}}_t$ in the performance index. Consider two cases of disturbance: (i) when disturbance and reference signals are attenuated, that is, a.s., the infinite-time performance index can be adopted:

$$\begin{array}{cc}& J=\sum _{t=0}^{\infty }\left(e_t^{T}Q{e}_t+{\stackrel{-}{u}}_t^{T}R{\stackrel{-}{u}}_t\right),\end{array}$$

(4)

where Q is a positive semidefinite matrix and R a positive definite one; (ii) alternatively, when they are periodical, that is, stable, the infinite-time performance index (4) will not be convergent, since the control in it includes periodical disturbance signal; in this case, the following average performance index can be taken:

$$\begin{array}{cc}& J=\underset{T\to \infty }{\lim }\frac{\mathrm{1}}{T}\sum _{t=\mathrm{0}}^T\left(e_t^{T}Q{e}_t+{\stackrel{-}{u}}_t^{T}R{\stackrel{-}{u}}_t\right).\end{array}$$

(5)

Our objective is to design the distributed OTDC to minimize performance index (4) or (5).

Consequently, denoting the augmented state vector

$$\begin{array}{cc}& z_t\triangleq \left[\begin{array}{c}\mathrm{\Delta }{x}_t\\ e_t\end{array}\right],\hspace{1em}\mathrm{\Delta }{x}_t=x_t-x_{t-1},\end{array}$$

(6)

the augmented system is produced:

$$\begin{array}{cc}& z_{t+1}=\stackrel{-}{A}{z}_t+\stackrel{-}{B}{\stackrel{-}{u}}_t+\stackrel{-}{D}F\mathrm{\Delta }{w}_t+\stackrel{-}{\mathit{A}}\mathrm{\Delta }{\mathit{x}}_t+\mathrm{\Delta }f\left(z_t\right),\end{array}$$

(7)

where

$$\begin{array}{cc}& \stackrel{-}{A}=\left[\begin{array}{cc}A& \mathit{0}\\ \mathit{0}& I\end{array}\right],\hspace{1em}\hspace{1em}\stackrel{-}{B}=\left[\begin{array}{c}B\\ \mathit{0}\end{array}\right],\hspace{1em}\hspace{1em}\stackrel{-}{D}=\left[\begin{array}{c}DF\\ \mathit{0}\end{array}\right],\hspace{1em}\hspace{1em}\stackrel{-}{\mathit{A}}=\left[\begin{array}{c}\mathit{A}\\ \mathit{0}\end{array}\right],\\ \mathrm{\Delta }f\left(z_t\right)\triangleq \left[\begin{array}{c}f\left(x_t\right)-f\left(x_{t-1}\right)\\ \mathit{0}\end{array}\right],\hspace{1em}\hspace{1em}\mathrm{\Delta }{\mathit{x}}_t={\mathit{x}}_t-{\mathit{x}}_{t-1}.\end{array}$$

(8)

From Assumption 1, it can be proved that $\text{Rank}[\stackrel{-}{B}\hspace{0.5em}\stackrel{-}{A}\hspace{0.17em}\stackrel{-}{B}\hspace{0.5em}{\stackrel{-}{A}}^2\stackrel{-}{B}\hspace{0.5em}\cdots \hspace{0.5em}{\stackrel{-}{A}}^{n-1}\stackrel{-}{B}]=n$ , which implies that the pair $(\stackrel{-}{A},\stackrel{-}{B})$ is completely controllable. Then the purpose of designing OTDC to minimize the performance index (4) or (5) is equivalent to that of designing an optimal regulation control law to minimize the performance index

$$\begin{array}{cc}& J=\sum _{t=0}^{\infty }\left(z_t^T\tilde{Q}{z}_t+{\stackrel{-}{u}}_t^{T}R{\stackrel{-}{u}}_t\right)\end{array}$$

(9)

or

$$\begin{array}{cc}& J=\underset{T\to \infty }{\lim }\frac{1}{T}\sum _{t=0}^T\left(z_t^T\tilde{Q}{z}_t+{\stackrel{-}{u}}_t^{T}R{\stackrel{-}{u}}_t\right),\end{array}$$

(10)

where $\tilde{Q}=\text{Diag}\{\mathit{0},Q\}$ . And the pair $(\stackrel{-}{A},\tilde{D})$ is assumed completely observable for an arbitrary matrix $\tilde{D}$ which satisfies ${\tilde{D}}^T\tilde{D}=\tilde{Q}$ . Hence, the original tracking problem of performance index (4) or (5) subject to the dynamical constraint (1) is converted into an optimal regulation problem of performance index (9) or (10) subject to the dynamical constraint (7).

3. Optimal Distributed Tracking Control Design

3.1. Design of Estimators at Sensors

In WSNs, information collected from sensors is always not complete, which is attributable to transfer delays and packet losses. A kind of wireless estimation sensor is introduced to avoid challenges associated with transfer delays and packet losses [27–29]. As shown in Figure 3, a WAS consists of four functional components: sensing interface, computational core, wireless communication channel, and actuation interface. The sensing interface connects the perfectly estimated analog signals, converts them to digital ones, and then transfers them to computational core ready to process. The designed algorithm is embedded in computational core which outputs control command signals to actuation interface. In actuation interface, the control signals are converted into analog ones while driving actuators or active sensors to execute control forces. Wireless communication unit allows the connectivity among this and other sensors and remote data servers within WSNs. Through a series of laboratory tests, such a kind of wireless active sensor is attested that not only its architecture is simple and easy to design but also it enhances the communication performance for its direct connection within sensor-controller-actuator. In this study, this kind of useful WAS architecture will be employed in distributed WSNs modeling. In addition to these reports, we will provide an estimator design method and the corresponding algorithm for the wireless estimator, which can be embedded in the WAS. Consequentially, the unavailable states will be entirely estimated and the transfer delays and packet losses can be effectively avoided as well. After this procedure, the perfect state information can be collected by the active sensors over the WSNs and all information is ready in the sensing interfaces for controller computations.

Figure 3

Architecture of WAS.

Consider that the $j\text{th}\hspace{0.17em}\hspace{0.17em}(j=\mathrm{1,2},\dots ,n_i)$ state vector of the $i\text{th}$ subsystem, which is denoted by $x_t^{[i,j]}$ (simplified by $x_t^{[j]}$ ), is measured by the $ij\text{th}$ sensor, assuming it is partial or unavailable. In order to estimate the full information of it and thus to send the overall state values to the controller, the full-order observer construction theory is applied and the state estimator is constructed in form of

$$\begin{array}{cc}& {\widehat{x}}_{t+1}^{\left[j\right]}=\left(a^{\left[j\right]}-l^{\left[j\right]}\right){\widehat{x}}_t^{\left[j\right]}+l^{\left[j\right]}{x}_t^{\left[j\right]},\end{array}$$

(11)

where $a^{[j]}$ is the original constant of the $ij\text{th}$ sensor, ${\widehat{x}}_t^{[j]}$ is the estimated state of $x_t^{[j]}$ , and $l^{[j]}$ is the estimator gain. By regulating the gain and applying the pole assignment principle to make $|\mu (a^{[j]}-l^{[j]})|<1$ , the estimation error enables approaching to zero; that is, ${\lim }_{t\to \infty }{\widehat{x}}_t^{[j]}=x_t^{[j]}$ . Consequently, all the estimated states of the WSN-based system will be finally achieved with ${\lim }_{t\to \infty }{\widehat{\mathit{x}}}_t={\mathit{x}}_t$ through each state estimator, in which ${\widehat{\mathit{x}}}_t=[{\widehat{x}}_t^{[1]T},{\widehat{x}}_t^{[2]T},\dots ,{\widehat{x}}_t^{[N]T}{]}^T$ is the estimated state of the large-scale system. Therefore, entire state information can be collected over the WSNs and will be sent to the relevant computational core for control usage.

3.2. Optimal Tracking and Disturbance Rejection Control (OTDC) Design

3.2.1. Global Optimal Control of Nonlinear System

This section is devoted to outlining the sufficient and necessary condition for the optimality of a tracking and disturbance rejection control of the quadratic optimization problem (4) or (5) subject to the nonlinear dynamical constraint (1). As above mentioned, in order to simplify the derivation, one might consider the equivalent optimal regulation problem (9) or (10) subject to (7). Then Theorem 3 can be gotten.

Theorem 3.

Given the nonlinear large-scale system (1) and the cost functional (4) or (5), where the distributed control $u_t$ is unconstrained, then a distributed optimal tracking and disturbance rejection control law is given by

$$\begin{array}{cc}& u_t^{*}=-K{x}_t-\tilde{K}\sum _{j=1}^t{e}_j-R^{-1}{\stackrel{-}{B}}^T{\stackrel{-}{A}}^{\hspace{0.17em}-T}\left(\tilde{P}{w}_t+g_t\right),\end{array}$$

(12)

where $K,\tilde{K}$ are denoted by

$$\begin{array}{cc}& \left[\begin{array}{cc}K& \tilde{K}\end{array}\right]\triangleq R^{-1}{\stackrel{-}{B}}^T{\stackrel{-}{A}}^{\hspace{0.17em}-T}\left(P-\tilde{Q}\right),\end{array}$$

(13)

where  P is the unique positive definite solution of the Riccati matrix equation

$$\begin{array}{cc}& {\stackrel{-}{A}}^{T}P{\left(I+\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^{T}P\right)}^{-1}\stackrel{-}{A}+\tilde{Q}=P,\end{array}$$

(14)

where   $\tilde{P}$ is the unique solution of the Stein matrix equation

$$\begin{array}{l}{\stackrel{-}{A}}^T\left[I-P{\left(I+\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^{T}P\right)}^{-1}\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^T\right]\tilde{P}G\\ \hspace{1em}=\tilde{P}-{\stackrel{-}{A}}^{T}P{\left(I+\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^{T}P\right)}^{-1}\stackrel{-}{D}F\end{array}$$

(15)

and $g_t$ is the unique solution of the adjoint difference equation

$$\begin{array}{l}{g}_t=-{\stackrel{-}{A}}^T\left[I-P{\left(I+\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^{T}P\right)}^{-1}\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^T\right]g_{t+1}+\stackrel{-}{\mathit{A}}{\mathit{x}}_t\\ \hspace{1em}\hspace{1em}+{\stackrel{-}{A}}^{T}P{\left(I+\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^{T}P\right)}^{-1}f\left(z_t\right)\\ g_{\infty }=\mathit{0}.\end{array}$$

(16)

The optimal state $x_t^{*}$ is the solution of the closed-loop system

$$\begin{array}{l}{x}_{t+1}^{*}={\stackrel{-}{A}}^T\left[I-P{\left(I+\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^{T}P\right)}^{-1}\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^T\right]x_t\\ \hspace{1em}\hspace{1em}\hspace{1em}+\mathit{A}{\mathit{x}}_t+f\left(x_t\right)+\left(DF-B{R}^{-1}{\stackrel{-}{B}}^T{\stackrel{-}{A}}^{\hspace{0.17em}-T}\tilde{P}\right)w_t\\ \hspace{1em}\hspace{1em}\hspace{1em}-B\tilde{K}\sum _{j=1}^t{e}_j-B{R}^{-1}{\stackrel{-}{B}}^T{\stackrel{-}{A}}^{\hspace{0.17em}-T}{g}_t.\end{array}$$

(17)

Proof.

In analogy to classical linear quadratic regulator (LQR) optimal control theory from minimum principle, the Hamiltonian for the linear quadratic regulation problem (7) with respect to (9) or (10) becomes

$$\begin{array}{l}H\left(z_t,u_t,{\lambda }_t\right)=\frac{1}{2}\left[z_t^T\tilde{Q}{z}_t+{\stackrel{-}{u}}_t^{T}R\stackrel{-}{u}\right]\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.17em}+{\lambda }_{t+1}^T\left[\stackrel{-}{A}{z}_t+\stackrel{-}{B}{\stackrel{-}{u}}_t+\stackrel{-}{D}F\mathrm{\Delta }{w}_t+\stackrel{-}{\mathit{A}}\mathrm{\Delta }{\mathit{x}}_t+\mathrm{\Delta }f\left(z_t\right)\right],\end{array}$$

(18)

which satisfies the canonical equations

$$\begin{array}{cc}& z_{t+1}=\frac{\partial H_t}{\partial {\lambda }_{t+1}}=\stackrel{-}{A}{z}_t+\stackrel{-}{D}F\mathrm{\Delta }{w}_t+\stackrel{-}{\mathit{A}}\mathrm{\Delta }{\mathit{x}}_t+\mathrm{\Delta }f\left(z_t\right)-\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^T{\lambda }_{t+1}\end{array}$$

(19)

$$\begin{array}{cc}& {\lambda }_t=\frac{\partial H_t}{\partial z_t}=\tilde{Q}{z}_t+{\stackrel{-}{A}}^T{\lambda }_{t+1}\end{array}$$

(20)

with the transversality condition

$$\begin{array}{cc}& {\lambda }_{\infty }=\mathbf{0}\end{array}$$

(21)

and the control equation

$$\begin{array}{cc}& \frac{\partial H_t}{\partial {\stackrel{-}{u}}_t}=R{\stackrel{-}{u}}_t+B^T{\lambda }_{t+1}=\mathit{0}\end{array}$$

(22)

giving

$$\begin{array}{cc}& {\stackrel{-}{u}}_t^{*}=-R^{-1}{\stackrel{-}{B}}^T{\lambda }_{t+1}\end{array}$$

(23)

along an optimal trajectory, where ${\lambda }_t$ is an introduced costate vector. Then the canonical equations (19) and (20) result in the coupled nonlinear two-point boundary value (TPBV) problem

$$\begin{array}{l}\left[\begin{array}{l}{z}_{t+1}\\ {\lambda }_t\end{array}\right]=\left[\begin{array}{ll}\stackrel{-}{A}& -\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^T\\ \tilde{Q}& {\stackrel{-}{A}}^T\end{array}\right]\left[\begin{array}{l}{z}_t\\ {\lambda }_{t+1}\end{array}\right]\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+\left[\begin{array}{l}\stackrel{-}{D}F\\ \mathit{0}\end{array}\right]\mathrm{\Delta }{w}_t+\left[\begin{array}{l}\stackrel{-}{\mathit{A}}\mathrm{\Delta }{\mathit{x}}_t+\mathrm{\Delta }f\left(z_t\right)\\ \mathit{0}\end{array}\right],\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\left[\begin{array}{l}{z}_0\\ {\lambda }_{\infty }\end{array}\right]=\left[\begin{array}{l}{z}_0\\ \mathit{0}\end{array}\right].\end{array}$$

(24)

Adopt the Riccati transformation

$$\begin{array}{cc}& {\lambda }_t=P{z}_t+\tilde{P}\mathrm{\Delta }{w}_t+\mathrm{\Delta }{g}_t\end{array}$$

(25)

which combines the terms designed to compensate for the disturbance and the nonlinearity effects, respectively, where P, $\tilde{P}$ , and $g_t$ are unknown matrices and vector to solve. Then, on one hand, from (25) and the state equation (19), it follows

$$\begin{array}{l}{z}_{t+1}={\left(I+\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^{T}P\right)}^{-1}\\ \hspace{1em}\hspace{1em}\hspace{1em}\times \left[\stackrel{-}{A}{z}_t+\left(\stackrel{-}{D}F-\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^T\tilde{P}G\right)\mathrm{\Delta }{w}_t-\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^T\mathrm{\Delta }{g}_{t+1}\right]\\ \hspace{1em}\hspace{1em}\hspace{1em}+\stackrel{-}{\mathit{A}}\mathrm{\Delta }{\mathit{x}}_t+\mathrm{\Delta }f\left(z_t\right).\end{array}$$

(26)

On the other hand, from (25) and the costate equation (20), the following equation holds:

$$\begin{array}{cc}& \left(P-\tilde{Q}\right)z_t={\stackrel{-}{A}}^{T}P{z}_{t+1}+\left({\stackrel{-}{A}}^T\tilde{P}G-\tilde{P}\right)\mathrm{\Delta }{w}_t+{\stackrel{-}{A}}^T{g}_{t+1}-g_t.\end{array}$$

(27)

Substituting (26) into (27) yields

$$\begin{array}{l}\left[{\stackrel{-}{A}}^{T}P{\left(I+\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^{T}P\right)}^{-1}\stackrel{-}{A}+\tilde{Q}-P\right]z_t\\ \hspace{1em}+{\stackrel{-}{A}}^{T}P{\left(I+\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^{T}P\right)}^{-1}\mathrm{\Delta }f\left(z_t\right)\\ \hspace{1em}+[\phantom{\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^T\tilde{P}G-\stackrel{-}{D}F}{\stackrel{-}{A}}^T\tilde{P}G-\tilde{P}-{\stackrel{-}{A}}^{T}P{\left(I+\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^{T}P\right)}^{-1}\\ \phantom{\hspace{1em}vb+}\times \left(\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^T\tilde{P}G-\stackrel{-}{D}F\right)\phantom{{\stackrel{-}{A}}^T\tilde{P}G-\tilde{P}-{\stackrel{-}{A}}^{T}P{\left(I+\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^{T}P\right)}^{-1}}]\mathrm{\Delta }{w}_t\\ \hspace{1em}+\stackrel{-}{\mathit{A}}\mathrm{\Delta }{\mathit{x}}_t-{\stackrel{-}{A}}^T\left[I-P{\left(I+\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^{T}P\right)}^{-1}\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^T\right]\mathrm{\Delta }{g}_{t+1}\\ \hspace{1em}-\mathrm{\Delta }{g}_t\equiv \mathit{0}.\end{array}$$

(28)

Since (28) is arbitrarily true, thus the Riccati equation (14), the Sylvester equation (15), and the adjoint vector difference equation (16) can be directly produced from it.

Furthermore, substituting the costate vector (25) into the costate equation (20) yields

$$\begin{array}{cc}& {\lambda }_{t+1}={\stackrel{-}{A}}^{\hspace{0.17em}-T}\left[\left(P-\tilde{Q}\right)z_t+\tilde{P}\mathrm{\Delta }{w}_t+\mathrm{\Delta }{g}_t\right],\end{array}$$

(29)

and then substituting (29) into (23) leads to the optimal control increment

$$\begin{array}{cc}& {\stackrel{-}{u}}_t^{*}=-R^{-1}{\stackrel{-}{B}}^T{\stackrel{-}{A}}^{\hspace{0.17em}-T}\left\{\left(P-\tilde{Q}\right)\left[\begin{array}{c}\mathrm{\Delta }{x}_t\\ e_t\end{array}\right]+\tilde{P}\mathrm{\Delta }{w}_t+\mathrm{\Delta }{g}_t\right\}.\end{array}$$

(30)

Using denotation (13) and integrating the optimal control increment (30) yields the optimal control law

$$\begin{array}{cc}& u_t^{*}=-K{x}_t-\tilde{K}\sum _{j=1}^t{e}_j-R^{-1}{\stackrel{-}{B}}^T{\stackrel{-}{A}}^{\hspace{0.17em}-T}\left(\tilde{P}{w}_t+g_t\right).\end{array}$$

(31)

Consequently, substituting (31) into subsystem (1) results in its closed-loop system

$$\begin{array}{l}{x}_{t+1}=\left(A-BK\right)x_t+\mathit{A}{\mathit{x}}_t\\ \hspace{1em}\hspace{1em}\hspace{1em}+\left(DF-B{R}^{-1}{\stackrel{-}{B}}^T{\stackrel{-}{A}}^{\hspace{0.17em}-T}\tilde{P}\right)w_t+f\left(x_t\right)\\ \hspace{1em}\hspace{1em}\hspace{1em}-B{R}^{-1}{\stackrel{-}{B}}^T{\stackrel{-}{A}}^{\hspace{0.17em}-T}{g}_t-B\tilde{K}\sum _{j=1}^t{e}_j,\end{array}$$

(32)

which gives the optimal state trajectory (17). The proof of Theorem 3 is completed.

3.2.2. Approximations of Sequences of Adjoint and State Equations

However, noting that the adjoint equation (16) and the optimal state equation (17) are coupled nonlinear difference equations, they are complex and seldom have analysis solutions. So, in this paper, the sequence approximation method in continuous-time domain [30] is developed into discrete-time domain to solve the coupled nonlinear equations (16) and (17).

Firstly, replace the difference equations (16) and (17) by the following sequences of linear time-invariant (LTIV) approximations:

$$\begin{array}{l}{g}_t^0=\mathit{0},\\ g_t^i=-{\stackrel{-}{A}}^T\left[I-P{\left(I+\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^{T}P\right)}^{-1}\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^T\right]g_{t+1}^i+\stackrel{-}{\mathit{A}}{\mathit{x}}_t^i\\ \hspace{1em}\hspace{1em}+{\stackrel{-}{A}}^{T}P{\left(I+\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^{T}P\right)}^{-1}f\left(z_t^{i-1}\right),\\ g_{\infty }^i=\mathit{0},\hspace{1em}i\in N_0,\end{array}$$

(33)

$$\begin{array}{l}{x}_t^0=\mathit{0},\\ x_{t+1}^i={\stackrel{-}{A}}^T\left[I-P{\left(I+\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^{T}P\right)}^{-1}\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^T\right]x_t^i\\ \hspace{1em}\hspace{1em}\hspace{0.17em}+\mathit{A}{\mathit{x}}_t^i+f\left(x_t^{i-1}\right)+\left(DF-B{R}^{-1}{\stackrel{-}{B}}^T{\stackrel{-}{A}}^{\hspace{0.17em}-T}\tilde{P}\right)w_t\\ \hspace{1em}\hspace{1em}\hspace{0.17em}-B\tilde{K}\sum _{j=1}^t{e}_j^i-B{R}^{-1}{\stackrel{-}{B}}^T{\stackrel{-}{A}}^{\hspace{0.17em}-T}{g}_t^i,\\ x_0^i=x_0,\end{array}$$

(34)

in which $i\in N$ represents the $i\text{th}$ iteration sequence. In what follows, with the purpose of proving that sequences $\{g^i\}$ and $\{x^i\}$ of (33) and (34) converge to the solutions of (16) and (17), respectively, some preliminary work will be carried out. Notice that the approximation sequences (33) and (34) are represented as inhomogeneous linear difference equations, which have the following solutions given by the variation of constants' formulae:

$$\begin{array}{l}{g}_t^0=\mathit{0},\\ g_t^i=\sum _{j=0}^{t-1}{\mathrm{\Phi }}_{t,j+1}\left[\stackrel{-}{\mathit{A}}{\mathit{x}}_j^{i-1}+{\stackrel{-}{A}}^{T}P{\left(I+\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^{T}P\right)}^{-1}f\left(z_j^{i-1}\right)\right],\\ g_{\infty }^i=\mathit{0},\end{array}$$

(35)

$$\begin{array}{l}\hspace{1em}{x}_t^0=\mathit{0},\\ x_{t+1}^i={\mathrm{\Phi }}_{t,0}{x}_0\\ \hspace{1em}\hspace{1em}\hspace{1em}+\sum _{j=0}^{t-1}{\mathrm{\Phi }}_{t,j+1}[\phantom{\sum _{l=1}^j{e}_l^i-B{R}^{-1}{\stackrel{-}{B}}^T{\stackrel{-}{A}}^{\hspace{0.17em}-T}{g}_j^i}\mathit{A}{\mathit{x}}_j^{i-1}+f\left(x_j^{i-1}\right)\\ \hspace{1em}\hspace{1em}\hspace{1em}\phantom{+\sum _{j=0}^{t-1}{\mathrm{\Phi }}_{t,jb+1}}+\left(DF-B{R}^{-1}{\stackrel{-}{B}}^T{\stackrel{-}{A}}^{\hspace{0.17em}-T}\tilde{P}\right)w_j\\ \hspace{1em}\hspace{1em}\hspace{1em}\phantom{+\sum _{j=0}^{t-1}{\mathrm{\Phi }}_{t,jb+1}}-B\tilde{K}\sum _{l=1}^j{e}_l^i-B{R}^{-1}{\stackrel{-}{B}}^T{\stackrel{-}{A}}^{\hspace{0.17em}-T}{g}_j^i],\\ x_0^i=x_0,\end{array}$$

(36)

in which $\mathrm{\Phi }$ denotes the transition matrix of ${\stackrel{-}{A}}^T[I-P(I+\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^{T}P{)}^{-1}\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^T]$ . Without loss of generality, combine (16) and (17) in a compact form of

$$\begin{array}{cc}& x_{t+1}=A_c{x}_t+h\left(t,x_t\right),\end{array}$$

(37)

where $x_t$ is a state vector and $A_c$ is a constant matrix of appropriate dimension. Meanwhile, denote (35) and (36) in another compact form of

$$\begin{array}{l}{x}_t^0={\mathrm{\Psi }}_{t,0}{x}_0,\\ x_t^i={\mathrm{\Psi }}_{t,0}{x}_0+\sum _{j=0}^{t-1}{\mathrm{\Psi }}_{t,j+1}h\left(j,x_j^{i-1}\right),\\ x_0^i=x_0,\hspace{1em}i\in N,\end{array}$$

(38)

where $\mathrm{\Psi }$ denotes the state transition matrix corresponding to matrix $A_c$ and the initial state vector $x_0$ .   $h(·)\in C(R^{+}\times R^{n_i})\to R^{n_i}$ with $h(t,\mathrm{0})=\mathbf{0}$ satisfies Lipschitz condition; that is, for every $x_t\in R^{n_i}$ and a positive constant β, there exits the following inequality:

$$\begin{array}{cc}& \parallel h\left(t,x_{t1}\right)-h\left(t,x_{t2}\right)\parallel \le \beta \parallel x_{t1}-x_{t2}\parallel .\end{array}$$

(39)

Hence, we only need to prove that the sequences $\{x^i\}$ of (38) converge to the solution of (37). As a result, Lemma 4 is produced.

Lemma 4.

Let $h(t,x_t)$ satisfy the Lipschitz condition and be bounded in its arguments. Then, the limits of the solutions of the approximating sequences (38) on $C(R^{+};R^{n_i})$ converge to the unique solution of (37) on $R^{+}$ .

Proof.

Denote

$$\begin{array}{cc}& \rho =\sup \parallel {\mathrm{\Psi }}_{t,j}\parallel ,\hspace{1em}\hspace{1em}\gamma =\parallel x_0\parallel ,\end{array}$$

(40)

where $\rho ,\gamma$ are positive constants. Noting that $\parallel {\mathrm{\Psi }}_{j,j}\parallel =\parallel I\parallel =1$ , thus $\rho \ge \mathrm{1}$ . From (38), it follows

$$\begin{array}{cc}& x_t^1-x_t^0=\sum _{j=0}^{t-1}{\mathrm{\Psi }}_{t,j+1}h\left(j,x_j^0\right).\end{array}$$

(41)

Due to (39) and (40), (41) gives

$$\begin{array}{cc}& \parallel x_t^1-x_t^0\parallel \le \beta \rho \sum _{j=0}^{t-1}\parallel x_j^0\parallel \le \beta \gamma t{\rho }^2.\end{array}$$

(42)

Further, from (38), (39), and (42), it yields

$$\begin{array}{cc}& \parallel x_t^2-x_t^1\parallel \le \beta \rho \sum _{j=0}^{t-1}\parallel x_j^1-x_j^0\parallel \le \frac{1}{2!}{\beta }^2\gamma t^2{\rho }^3.\end{array}$$

(43)

Note that

$$\begin{array}{cc}& \frac{1}{t+1}\sum _{j=1}^{t-1}{\left(\frac{j}{t+1}\right)}^l\le \frac{1}{l+1},\hspace{1em}l\in N_0.\end{array}$$

(44)

Then, the following holds:

$$\begin{array}{cc}& \parallel x_t^i-x_t^{i-1}\parallel \le {\beta }^l\gamma {\rho }^{l+1}\sum _{j=1}^{t-1}\frac{j^{l-1}}{\left(l-1\right)!}\le \frac{1}{l!}{\beta }^l\gamma t^l{\rho }^{l+1},\hspace{1em}l\in N.\end{array}$$

(45)

Through the trigonometry inequality, there exists

$$\begin{array}{cc}& \parallel x_t^i-x_t^{i-m}\parallel \le \gamma \rho \sum _{j=i-m+1}^i\frac{{\left(\beta \rho t\right)}^j}{j!}\end{array}$$

(46)

for any $m<i,\hspace{0.17em}\hspace{0.17em}m\in N$ , which leads to

$$\begin{array}{cc}& \underset{i\to \infty }{\lim }\parallel x_t^i-x_t^{i-m}\parallel =0.\end{array}$$

(47)

Consequentially, $\{x^i\}$ is the sequence of Cauchy in Banach space $C^1(R^{+};R^{n_i})$ . Therefore, $x_t^i\to x_t$ on $C(R^{+};R^{n_i})$ . The proof of Lemma 4 is completed.

From Lemma 4, Lemma 5 is directly produced.

Lemma 5.

Let $f(·)$ be bounded and satisfy the condition. Then, the limits of the solutions of the approximating sequences (33) and (34) (or (35) and (36)) on $C(R^{+};R^{n_i})$ globally converge to the unique solutions of (16) and (17), respectively, on $R^{+}$ .

Hence, the implication of the sufficient and necessary condition for a global distributed optimal solution of the nonlinear quadratic tracking and disturbance rejection control problem (1) with respect to (4) or (5) is given by the following theorem.

Theorem 6.

Given the nonlinear system (1) and the cost functional (4) or (5), where $u_t$ is unconstrained, then the distributed optimal tracking and disturbance rejection control is given by the limit of the sequence

$$\begin{array}{cc}& u_t^{i*}=-K{x}_t^i-\tilde{K}\sum _{j=1}^t{e}_j^i-R^{-1}{\stackrel{-}{B}}^T{\stackrel{-}{A}}^{\hspace{0.17em}-T}\left(\tilde{P}{w}_t+g_t^i\right),\end{array}$$

(48)

where $K,\tilde{K}$ are denoted by (13), P is the unique positive definite solution of the Riccati matrix equation (14), $\tilde{P}$ is the unique solution of the Sylvester matrix equation (15), and $g_t^i$ is given by the converged unique solution of the LTIV difference equation sequence (33) or (35). The optimal state $x_t^{i*}$ is the solution of the closed-loop system (34) or (36).

It is obvious that (33) and (34) (or (35) and (36)) are the TPBV problem of the LQR TPBV problem of the sequence (49) or (50) subject to the constraint (51). Following from Theorem 6, an alternative equivalent nonlinear OTDC problem of (1) is summarized as the following corollary.

Corollary 7.

The nonlinear quadratic optimization problem to find $u_t^{*}$ minimizing cost functional (4) or (5) subject to the constraint (1) is equivalent to the LTIV problem to find $u_t^{*i}$ minimizing cost functional $J^i$

$$\begin{array}{cc}& J^i=\sum _{t=0}^{\infty }\left(e_t^{Ti}Q{e}_t^i+{\stackrel{-}{u}}_t^{Ti}R{\stackrel{-}{u}}_t^i\right)\end{array}$$

(49)

or

$$\begin{array}{cc}& J^i=\underset{T\to \infty }{\lim }\frac{1}{T}\sum _{t=0}^T\left(e_t^{Ti}Q{e}_t^i+{\stackrel{-}{u}}_t^{Ti}R{\stackrel{-}{u}}_t^i\right)\end{array}$$

(50)

subject to the constraint

$$\begin{array}{cc}& x_{t+1}^i=A{x}_t^i+\mathit{A}{\mathit{x}}_t^i+B{u}_t^i+D{v}_t^i+f\left(x_t^i\right),\\ x_0^i=\mathit{0}.\end{array}$$

(51)

Remark 8.

The optimal control $u_t^{*}$ is unique.

To prove the uniqueness of optimal control $u_t^{*}$ , the following useful lemma is needed.

Lemma 9 (see [31]).

Assume that $\mathrm{\Lambda }\in R^{n\times n}$ , $\mathrm{\Theta }\in R^{n\times n}$ , $\mathrm{\Xi }\in R^{n\times n}$ , the Stein matrix equation

$$\begin{array}{cc}& \mathrm{\Lambda }X\mathrm{\Theta }-X=\mathrm{\Xi }\end{array}$$

(52)

has a unique solution X if and only if

$$\begin{array}{cc}& \left|\mu \left(\mathrm{\Lambda }\right)·\mu \left(\mathrm{\Theta }\right)\right|\ne 1.\end{array}$$

(53)

Then, we will use Lemma 9 to prove the uniqueness of $\tilde{P}$ firstly.

Proof.

Since the triple $(\stackrel{-}{A},\stackrel{-}{B},\tilde{D})$ is controllable-observable, according to LQR theory, there exists the unique positive definite solution P such that the matrix

$$\begin{array}{cc}& {\stackrel{-}{A}}^T\left[I-P{\left(I+\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^{T}P\right)}^{-1}\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^T\right]\end{array}$$

(54)

is Hurwitz; that is,

$$\begin{array}{cc}& \left|\mu \left({\stackrel{-}{A}}^T\left[I-P{\left(I+\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^{T}P\right)}^{-1}\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^T\right]\right)\right|<1.\end{array}$$

(55)

Moreover, the disturbance is supposed a.s. or stable; that is, $|\mu (\stackrel{-}{G})|\le 1$ . Therefore, the following inequality holds:

$$\begin{array}{cc}& \left|\mu \left({\stackrel{-}{A}}^T\left[I-P{\left(I+\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^{T}P\right)}^{-1}\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^T\right]\right)·\mu \left(\stackrel{-}{G}\right)\right|<1.\end{array}$$

(56)

According to Lemma 9, the Stein equation (15) has the unique solution $\tilde{P}$ .

Additionally, $g_t,x_t$ are unique solutions of difference equations (16), (17), respectively, for satisfying their uniqueness condition. Consequentially, the uniqueness of OTDC (12) is proved.

Remark 10.

It should be noted that, in practice, the exact adjoint vector $g_t^i$ in (48) is usually impossible to obtain when designing the controller in case of $i\to \infty$ . Generally, ∞ can be replaced by a positive integer M, and thus the following distributed suboptimal tracking and disturbance rejection control (SOTDC) is proposed:

$$\begin{array}{cc}& u_t^{M*}=-K{x}_t^M-\tilde{K}\sum _{j=1}^t{e}_j^M-R^{-1}{\stackrel{-}{B}}^T{\stackrel{-}{A}}^{\hspace{0.17em}-T}\left(\tilde{P}{w}_t+g_t^M\right),\end{array}$$

(57)

where $M\in N$ is determined by a small enough concrete error criterion $\epsilon >\mathrm{0}$ . Then, the relevant performance index is calculated by

$$\begin{array}{cc}& J^M=\underset{T_M\to \infty }{\lim }\frac{1}{T_M}\sum _{t=0}^{T_M}\left(e_t^{TM}Q{e}_t^{iM}+{\stackrel{-}{u}}_t^{TM}R{\stackrel{-}{u}}_t^M\right)\end{array}$$

(58)

until $|(J^M-J^{M-1})/J^M|<\epsilon$ .

Hence, the synthesis algorithm of SOTDC for systems (1) is schemed out as follows.

Algorithm 1.

SOTDC of system (1).

Step 1. Regulate $l^{i,j}$ at the respective estimator; provide appropriate state $x^{[i,j]}$ to the controller.

Step 2. Judge the controllability-observability of $(A,B,C)$ .

Step 3. Determine matrices in (8).

Step 4. Get the augmented system (7).

Step 5. Solve P and $\tilde{P}$ from (14) and (15); let $x_t^0=g_t^0=\mathit{0}$ , $J^0=0$ , and $i=1$ .

Step 6. Obtain the $i\text{th}$ adjoint vector $g_t^i$ from (33) or (35).

Step 7. Letting $M=i$ , calculate $u_t^M$ from (57).

Step 8. Determine $J^M$ from (58).

Step 9. When $|(J^M-J^{M-1})/J^M|<\epsilon$ , then output $u_t^M$ .

Step 10. Calculate $x_t^i$ from (34) or (36).

Step 11. Letting $i=i+1$ , go to Step 5.

3.2.3. Closed-Loop Stability Analysis

Taking $i\to \infty$ in (30) yields the increment OTDC and then substituting it into the augmented system (7) yields the closed-loop augmented system

$$\begin{array}{l}{z}_{t+1}=A_c{z}_t+\left(\stackrel{-}{D}F-\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^T{\stackrel{-}{A}}^{\hspace{0.17em}-T}\tilde{P}\right)\mathrm{\Delta }{w}_t\\ \hspace{1em}\hspace{1em}\hspace{1em}+\tilde{\mathit{A}}{\mathit{z}}_t+\mathrm{\Delta }f\left(z_t\right)+\underset{i\to \infty }{\lim }\mathrm{\Delta }{g}_t^i\end{array}$$

(59)

with

$$\begin{array}{cc}& A_c\triangleq {\stackrel{-}{A}}^T\left[I-P{\left(I+\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^{T}P\right)}^{-1}\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^T\right],\\ \tilde{\mathit{A}}=\left[\begin{array}{cc}\stackrel{-}{\mathit{A}}& \mathit{0}\\ \mathit{0}& \mathit{0}\end{array}\right],\hspace{1em}\hspace{1em}{\mathit{z}}_t=\left[\begin{array}{c}\mathrm{\Delta }{\mathit{x}}_t\\ e_t\end{array}\right].\end{array}$$

(60)

In (59), the term $-\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^T{\stackrel{-}{A}}^{\hspace{0.17em}-T}\tilde{P}\mathrm{\Delta }{w}_t$ is the feedforward term to suppress the disturbance. Moreover, since there exists ${\lim }_{t\to \infty }\mathrm{\Delta }{g}_t=\mathit{0}$ , thus suppose $w_t\equiv 0$ and ${\lim }_{t\to \infty }\mathrm{\Delta }{g}_t^i=\mathit{0}$ in (59). Select $V_t=z_t^{T}P{z}_t$ as a Lyapunov function candidate. The increment of it along the trajectories of system (59) gives

$$\begin{array}{l}\mathrm{\Delta }{V}_t=z_t^T\left(A_c^{T}P{A}_c-P\right)z_t+2z_t^T{A}_c^{T}P\left[\tilde{\mathit{A}}{\mathit{z}}_t+\mathrm{\Delta }f\left(z_t\right)\right]\\ \hspace{1em}\hspace{1em}\hspace{1em}+{\left[\tilde{\mathit{A}}{\mathit{z}}_t+\mathrm{\Delta }f\left(z_t\right)\right]}^{T}P\left[\tilde{\mathit{A}}{\mathit{z}}_t+\mathrm{\Delta }f\left(z_t\right)\right].\end{array}$$

(61)

Since

$$\begin{array}{cc}& A_c^{T}P{A}_c-P=-\tilde{Q}-\mathit{K}{B}^T{R}^{-1}B\mathit{K},\end{array}$$

(62)

where $\mathit{K}\triangleq R^{-1}{\stackrel{-}{B}}^T{\stackrel{-}{A}}^{\hspace{0.17em}-T}(P-\tilde{Q})$ and function vector $\mathrm{\Delta }f(z)$ has the property that for any $\eta >\mathrm{0}$ , there exists $r>0$ such that

$$\begin{array}{cc}& \parallel \mathrm{\Delta }f\left(z\right)\parallel <\eta \parallel z\parallel ,\hspace{1em}\forall \parallel z\parallel \le r;\end{array}$$

(63)

thus the following holds

$$\begin{array}{l}\parallel \mathrm{\Delta }V\parallel \le -\{{\mu }_{\min }\left(\tilde{Q}+\mathit{K}{B}^T{R}^{-1}B\mathit{K}\right)\\ \hspace{1em}\hspace{0.5em}-[{\eta }^2+2\eta \left(\parallel A_c^T\parallel +\parallel \tilde{\mathit{A}}\parallel \right)\\ \hspace{1em}\hspace{0.5em}\hspace{1em}\hspace{0.5em}+\left(\parallel \tilde{\mathit{A}}\parallel +2\parallel A_c^T\parallel \right)\parallel \tilde{\mathit{A}}\parallel ]\parallel P\parallel \}{\parallel z\parallel }^2.\end{array}$$

(64)

Choosing η which satisfies

$$\begin{array}{l}{\mu }_{\min }\left(\tilde{Q}+\mathit{K}{B}^T{R}^{-1}B\mathit{K}\right)\\ \hspace{1em}-\left[{\eta }^2+2\eta \left(\parallel A_c^T\parallel +\parallel \tilde{\mathit{A}}\parallel \right)+\left(\parallel \tilde{\mathit{A}}\parallel +2\parallel A_c^T\parallel \right)\parallel \tilde{\mathit{A}}\parallel \right]\parallel P\parallel >0\end{array}$$

(65)

ensures $\mathrm{\Delta }V$ being negative definite. Hence, the closed-loop system (59) is a.s., which leads to

$$\begin{array}{cc}& \underset{t\to \infty }{\lim }\mathrm{\Delta }{x}_t=\mathit{0},\hspace{1em}\hspace{1em}\underset{t\to \infty }{\lim }{e}_t=\underset{t\to \infty }{\lim }\left(r_t-y_t\right)=\mathit{0}.\end{array}$$

(66)

Consequentially, (66) indicates that the target is perfectly tracked without steady-state error.

3.2.4. Physical Realization of SOTDC

Notice that SOTDC (57) includes the disturbance state w which is physically unrealizable. Moreover, in WSNs, states of x might not reach the controller completely. Therefore, we can reconstruct these states through a reduced-order observer by using the output vectors v and y. Defining ${\zeta }_t={[x_t^T\hspace{0.5em}{w}_t^T]}^T$ , the observer can be constructed in the form of

$$\begin{array}{l}{\psi }_{t+1}=\left(A_{22}-H{A}_{12}\right){\psi }_t\\ \hspace{1em}\hspace{1em}\hspace{1em}+\left[\left(A_{22}-H{A}_{12}\right)H+A_{12}-H{A}_{11}\right]{\varphi }_t\\ \hspace{1em}\hspace{1em}\hspace{1em}+\left(B_2-H{B}_1\right)u_t+\left(E_2-H{E}_1\right)\tilde{f}\left(x_t\right),\\ \hspace{1em}{\widehat{\zeta }}_t={\Pi }_2{\psi }_t+\left({\Pi }_1+{\Pi }_{2}H\right){\varphi }_t,\end{array}$$

(67)

where

$$\begin{array}{cc}& \tilde{f}\left(x_t\right)={\left[\begin{array}{cc}{f}^T\left(x_t\right)& \mathit{0}\end{array}\right]}^T,\hspace{1em}\hspace{1em}{\varphi }_t={\left[\begin{array}{cc}{y}_t^T& v_t^T\end{array}\right]}^T.\end{array}$$

(68)

$A_{ij}$ , ${\Pi }_i$ , $B_i$ , $E_i$    $(i,j=\mathrm{1,2})$ are constant matrices of appropriate dimensions, $\widehat{\zeta }$ is the estimated state of ζ, ψ is the observer state, and H is the observer gain. By regulating the gain to make $|\mu (A_{22}-H{A}_{12})|<1$ , the estimation error can approximate to zero. Therefore, the system state can be reconstructed and the feedforward compensation can be implemented physically. For simplification reason, the observer construction process is omitted which can be referred to in [25, 26]. Therefore, in the basis of observer (67), a dynamical distributed suboptimal tracking and disturbance rejection control (DSOTDC) is obtained:

$$\begin{array}{l}{\psi }_{t+1}^M=\left(A_{22}-H{A}_{12}\right){\psi }_t^M\\ \hspace{1em}\hspace{1em}\hspace{1em}+\left[\left(A_{22}-H{A}_{12}\right)H+A_{12}-H{A}_{11}\right]{\varphi }_t^M\\ \hspace{1em}\hspace{1em}\hspace{1em}+\left(B_2-H{B}_1\right)u_t^M+\left(E_2-H{E}_1\right)\tilde{f}\left(x_t^{M-1}\right),\\ u_t^{dM}=\left[\begin{array}{ll}-K& -R^{-1}{\stackrel{-}{B}}^T{\stackrel{-}{A}}^{\hspace{0.17em}-T}\tilde{P}\end{array}\right]\left[{\Pi }_2{\psi }_t^M+\left({\Pi }_1+{\Pi }_{2}H\right){\varphi }_t^M\right]\\ \hspace{1em}\hspace{1em}\hspace{1em}-\tilde{K}\sum _{j=1}^t{e}_j^M-R^{-1}{\stackrel{-}{B}}^T{\stackrel{-}{A}}^{\hspace{0.17em}-T}{g}_t^M.\end{array}$$

(69)

And then, the relevant algorithm of DSOTDC is outlined.

Algorithm 2.

DSOTDC for system (1).

Step 1. Regulate $l^{i,j}$ at the respective estimator; provide appropriate state $x^{[i,j]}$ to the controller.

Step 2. Determine the matrices in observer (67).

Step 3. Judge the controllability-observability of $(A,B,C)$ .

Step 4. Determine matrices (8).

Step 5. Get augmented system (7).

Step 6. Solve P and $\tilde{P}$ from (14) and (15); let $x_t^0=g_t^0=\mathit{0}$ , $J^0=0$ , and $i=1$ .

Step 7. Obtain the $i\text{th}$ adjoint vector $g_t^i$ from (33) or (35).

Step 8. Letting $M=i$ , calculate $u_t^{dM}$ from (69).

Step 9. Determine $J^M$ from (58).

Step 10. When $|(J^M-J^{M-1})/J^M|<\epsilon$ , then output $u_t^M$ .

Step 11. Calculate $x_t^i$ from (34) or (36).

Step 12. Letting $i=i+1$ , go to Step 7.

3.3. OTC Design

In some cases, regarding a tracking problem, disturbance effect is minor and the key of analysis and synthesis is speediness and accuracy of tracking. Consequentially, ignoring disturbance effect on a system, the following distributed optimal tracking control (OTC) is presented. In the same way, using the maximum principle and letting ${\lambda }_t=P{z}_t+\mathrm{\Delta }{g}_t$ in the Riccati transformation (25), through the similar derivation procedure, the distributed OTC is obtained as the follows.

Theorem 11.

Given the nonlinear large-scale system (70)

$$\begin{array}{cc}& x_{t+1}^{\left[i\right]}=A^{\left[i\right]}{x}_t^{\left[i\right]}+{\mathit{A}}^{\left[i\right]}{\mathit{x}}_t+B^{\left[i\right]}{u}_t^{\left[i\right]}+f^{\left[i\right]}\left(x_t^{\left[i\right]}\right),\\ y_t^{\left[i\right]}=C^{\left[i\right]}{x}_t^{\left[i\right]},\hspace{1em}t\in N_0,\hspace{0.17em}\hspace{0.17em}i=\mathrm{1,2},\dots ,N\end{array}$$

(70)

and the cost functional (4) or (5), where the distributed control $u_t$ is unconstrained, then a distributed optimal tracking control law is given by

$$\begin{array}{cc}& u_t^{*}=-K{x}_t-\tilde{K}\sum _{j=1}^t{e}_j-R^{-1}{\stackrel{-}{B}}^T{\stackrel{-}{A}}^{\hspace{0.17em}-T}{g}_t,\end{array}$$

(71)

where $K,\tilde{K}$ are denoted by (13), P is the unique positive definite solution of the Riccati matrix equation (14), and $g_t$ is the unique solution of the adjoint difference equation (16). The optimal state $x_t^{*}$ is the solution of the closed-loop system

$$\begin{array}{l}{x}_{t+1}^{*}={\stackrel{-}{A}}^T\left[I-P{\left(I+\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^{T}P\right)}^{-1}\stackrel{-}{B}{R}^{-1}{\stackrel{-}{B}}^T\right]x_t\\ \hspace{1em}\hspace{1em}\hspace{1em}+\mathit{A}{\mathit{x}}_t+f\left(x_t\right)-B\tilde{K}\sum _{j=1}^t{e}_j-B{R}^{-1}{\stackrel{-}{B}}^T{\stackrel{-}{A}}^{\hspace{0.17em}-T}{g}_t.\end{array}$$

(72)

For briefness, the distributed suboptimal tracking control and the observer-based dynamical suboptimal tracking control are omitted.

4. Simulation Examples

Consider two identical pendulums, which are controlled by the forces $T_c$ , and track target signals, respectively. The dynamic equation is given by [32]:

$$\begin{array}{cc}& m{l}^2\ddot{\theta }\left(t\right)=-mgl\sin \theta \left(t\right)-k{l}^2\dot{\theta }\left(t\right)+T_c\left(t\right),\end{array}$$

(73)

where m is the mass of the bob, l is the length of the rod, θ is the angle subtended by the rod and the vertical axis through the pivot point, g is the acceleration due to gravity, k is the coefficient of friction, and $T_c$ is the applied torque. Letting $x_1=\theta$ , $x_2=\dot{\theta }$ , and $u=T_c$ , model (73) is represented in the state-space form of

$$\begin{array}{cc}& A^{\left[1\right]}=A^{\left[2\right]}=\left[\begin{array}{cc}0& 1\\ 0& -\text{k}/\text{m}\end{array}\right],\hspace{1em}\hspace{1em}f\left(x_1\right)=-g/l\sin x_1,\\ B^{\left[1\right]}=B^{\left[2\right]}=\left[\begin{array}{c}0\\ 1/\left(\text{m}{\text{L}}^{\mathrm{2}}\right)\end{array}\right],\hspace{1em}\hspace{1em}{C}^{\left[1\right]}=C^{\left[2\right]}=\left[\begin{array}{cc}1& 0\end{array}\right],\\ {\mathit{A}}^{\left[\mathrm{1,2}\right]}={\mathit{A}}^{\left[\mathrm{2,1}\right]}=\mathrm{diag}\left\{0,1\right\},\hspace{1em}\hspace{1em}{x}_0^{\left[1\right]}=x_0^{\left[2\right]}={\left[\begin{array}{cc}0& 0\end{array}\right]}^T,\end{array}$$

(74)

where ${\mathit{A}}^{[\mathrm{1,2}]}$ and ${\mathit{A}}^{[\mathrm{2,1}]}$ represent the pendulums receiving information from each other over WSNs. The parameter values are adopted as follows:

$$\begin{array}{cc}& m=0.1\hspace{0.17em}\text{Kg},\hspace{1em}\hspace{1em}k=0.1\hspace{0.17em}\text{N}·\text{m}/\text{rad}/\text{sec},\\ l=1\hspace{0.17em}\text{m},\hspace{1em}\hspace{1em}g=9.8\hspace{0.17em}\text{m}/\text{se}{\text{c}}^{\mathrm{2}}.\end{array}$$

(75)

Taking the sampling time $T_s=0.1\hspace{0.17em}\text{sec}$ , system (74) is discretized into the discrete-time system in form as (1). The performance index matrices of (4) are chosen taking

$$\begin{array}{cc}& Q^{\left[1\right]}=Q^{\left[2\right]}=\mathrm{diag}\left\{1\right\},\hspace{1em}\hspace{1em}{R}^{\left[1\right]}=R^{\left[2\right]}=1.\end{array}$$

(76)

One wireless sensor is installed upon each of the pendulums; in total, two wireless sensors are installed. The role of the wireless sensor is to measure the angle velocity of the pendulum, calculate an OTDC force, apply a command signal to actuator, and wirelessly exchange data with other wireless sensors sharing a wireless network. To determine the angle velocity, each wireless sensor measures the absolute velocity from a velocity meter equipped with the pendulum, see Figure 4.

Figure 4

Pendulums equipped with velocity meters and wireless active sensors.

Then, in the first place, we will validate the effectiveness of the designed OTDC. Command no. 1 pendulum to keep its angle θ tracking a step signal target as $r_t^{[1]}=1$ and no. 2 pendulum tracking a sinusoidal signal target as $r_t^{[2]}=\sin t$ . In light of Theorem 6, the distributed OTDCs are designed. The closed-loop structural diagram is shown in Figure 5. The computer simulations are demonstrated. The pendulums' responses of angular displacements, velocities, control forces, and tracking errors are described as in Figure 6.

Figure 5

The closed-loop structural diagram of controlled pendulum system on WSNs.

Figure 6

Pendulums' responses tracking step and sinusoidal signals.

They receive the information of themselves and that of others, respectively, by communicating over the WSNs and obtain it through the matrices $A^{[1]},A^{[2]}$ and the interconnection matrices ${\mathit{A}}^{[\mathrm{1,2}]},{\mathit{A}}^{[\mathrm{2,1}]}$ and then calculate the data using (14)–(17) so that the distributed control laws (12) are determined. Hence, they complete their separate tasks tracking the respective targets such that, in the whole system point, the entire tasks consisting of the two subtasks are accomplished also. From the figure of tracking errors, one can see that the tracking errors of the two pendulums are completely with zero steady-state errors.

Consequently, in order to demonstrate the effect of zero steady-state error with OTDC approach, we compare with a feedforward and feedback optimal control (FFOC). The relevant tracking errors are exhibited in Figure 7.

Figure 7

No. 2 pendulum’ responses tracking sinusoidal signals by OTDC and FFOC.

Figure 7 reveals that the tracking error approaches zero by employing OTDC. However, the one through FFOC approximates some stable sinusoidal signal but not zero, since the increment integral regulator enables eliminating the steady-state error especially for high-order signals. Contrarily, the feedforward compensator can only stabilize system states while tracking the target; in most cases, it does not possess the property to eliminate steady-state errors.

5. Conclusions

In this paper, we have presented the optimal tracking control algorithms for distributed nonlinear systems on WSNs. The optimization algorithm has been derived from increment equations and obtained by solving discrete-time matrix equations and difference equation sequences. Illustrated by the numerical simulations, it has shown that the targets can be tracked without steady-state errors and the design goal of balancing the tradeoff between track error and energy consumption can be archived over WSNs.