A novel approach to integrate potential field and interval type-2 fuzzy learning for the formation control of multiple autonomous underwater vehicles

Abstract

Underwater vehicles coordination and formation have attracted increasingly attentions since their great potential on the real-world applications. However, usually such vehicles are underactuated and with very different environmental difficulties, which are different from those vehicles (robots) on the land. This study proposes a novel approach to integrate potential field and interval type-2 fuzzy learning algorithm for autonomous underwater vehicles formation control based on formation system framework. For the system nonlinearity and complicated environment, support vector machine has been applied to generate optimal rules for the type-2 fuzzy systems. This approach can generate optimal and reasonable formation rules on the face of different situations through classification. Furthermore, reinforcement learning has been combined with fuzzy systems to deal with limited communication state during formation. Therefore, autonomous underwater vehicles can not only execute actions through the evaluation, but also can avoid coupling character between communication state and potential field. Finally, simulations and experiments results have been extensively performed to validate the proposed methods.

Introduction

In compare with the robots on the land, underwater vehicles will confront with more complicated and extensive environment. It is more important for the vehicles to autonomously adapt to the environment and cooperate in the operations. The collaboration of multiple comparatively cheap and small autonomous underwater vehicles (AUVs) can accomplish complex task more effectively than an expensive and multi-functional AUV. ¹

Recently, researchers have made significant efforts on the multiple AUVs coordinative formation. Different approaches such as generalized coordinate approach,^2,3 the leader–follower approach,^4,5 virtual structure approach, ⁶ the potential field approach, ⁷ and behavior-based approach. Comparing with robot formation on the land, AUVs submarine formation confronts with acoustic communications with propagating delays and channel noisy. Moreover, for the purpose of distant cruising and operation, AUV is usually underactuated. ⁸ It is propelled with one thruster with vertical rudders for the heading and horizontal wings for the diving. In order to realize the formation of multiple AUVs, heterogeneous underwater vehicles are usually employed, particularly with leader–follower formation form, to enhance spatial coverage and improve operation accuracy. The leaders usually equip with higher performance navigation instruments systems and perform the leading role, while the followers obtain position information with comparatively lower precise navigation instruments and relative positioning from the leaders. Strategies of relative positioning showed up in the work of Matsuda et al. ⁹ and Allotta et al. ¹⁰

Hou and Cheah from Singapore have proposed an adaptive proportional and derivative controller with multi-layered potential field on the basis of formation structure ¹¹ and realized 6 degree of freedom AUVs formations in the simulations. ¹² Parlangeli and Indiveri ¹³ from Italy have established a time varying nonlinear observability models with state augmentation on the basis of global observability model, positioned underactuated AUVs during coordinate motion control, and realized formation cruising. Since those AUVs groups are composed by heterogeneous AUVs with navigation systems of different accuracies, leader–follower approach is often used for the formation control. ¹⁴ Researchers from Portugal and Germany have proposed coordinate tracking strategy of heterogeneous AUV on the basis of line of sight and verified the strategy with oceanic experiments. ¹⁵ Millán et al. ¹⁶ from Spain have established the linearization model for the formation maintenance through the AUVs kinematics and Taylor series expansion and proposed an H2/H∞ controller in order to improve the formation robustness with time delayed communication. Cui et al. ¹⁷ from China have constructed a virtual leader AUV in the formation of underactuated AUVs, in order to realize the trajectory convergence. Moreover, a position tracking controller is proposed for the followers to track the virtual leader using Lyapunov and backstepping synthesis.

However, since the formation of multi-AUV in the oceanic environment confronts with limited communication,^9,18 system nonlinearity and complicated unknown environment, high efficiency interaction between AUVs may be difficult for nonlinear formation control algorithm. Therefore, the establishment of formation rules according to current state and environment may improve robustness through fuzzy inference and machine learning. ¹⁹ In compare with type-1 fuzzy logic system, type-2 fuzzy logic system can handle rule uncertainties with antecedent or consequent membership functions. Although it is samely characterized with IF-THEN rules, its antecedent or consequent sets are type-2. ²⁰ The type-2 fuzzy sets allow researchers to model and minimize the effects of uncertainties of changing environment in rule-based systems. Tsai and Tai ²¹ present a distributed consensus formation control law through recurrent interval type-2 fuzzy neural networks. Using online learning and Lyapunov stability theory, intelligent formation has been carried out. Ali et al. ²² propose a type-2 fuzzy ontology simulator to calculate and provide accurate information about collision risk and the marine environment during real-time marine operations. Such decision-making system provides high-level autonomy for AUVs formation. On the complicated oceanic environment and formation state, this article will propose a support vector machine (SVM)–based type-2 fuzzy learning approach for the formation control of multiple AUVs.

The rest of this study is organized as follows. In section “Multiple AUVs formation system framework,” a multiple AUVs formation system framework will be established. A SVM-based type-2 fuzzy learning approach for the formation control of multiple AUVs will be proposed in section “Interval type-2 fuzzy learning approach for AUVs formation.” Simulations and experiments will be discussed and analyzed in section “Simulations and experiments.” We will make conclusion in section “Conclusion.”

Multiple AUVs formation system framework

In order to realize cooperative control and interaction, the framework of formation control has been established (see Figure 1) with propulsion subsystem, oceanic survey subsystem, motion control subsystem, navigation subsystem, acoustic communication subsystem, and formation strategy subsystem. Each subsystem is independent with correspondent hardware. Communication between each subsystem is realized through PC104 bus–based TCP/IP protocol.

Figure 1.

Multiple AUVs formation system framework.

Communication between AUVs is realized through AquaComm network acoustic communication module. This multiple channel communication module is developed in the Innovation in Digital Signal Processing and Processing Company of Australia. It can release broad frequency signal through extended frequency spectrum under low energy consumption state.

In order to establish the topology for multiple AUVs coordinate, an $n$ order weighted directed graph $\mathit{G}=(\mathit{V},\mathbf{\xi },\mathit{A})$ has been defined, where $\mathit{V}=\{{\mathrm{s}}_1,{\mathrm{s}}_2,...,{\mathrm{s}}_n\}$ is the set of AUV nodes, $\mathbf{\xi }\subseteq \mathit{V}\times \mathit{V}$ is the set of edges, and $\mathit{A}=\left[a_{\mathrm{ij}}\right]$ is a weighted adjacency matrix with nonnegative adjacency elements $a_{\mathrm{ij}}$ . Each edge ( $s_i$ , $s_j$ ) corresponds to an available information channel between the AUV $s_i$ and the AUV $s_j$ at time t. The node indexes belong to a finite index set $\Omega =\{1,2,...,n\}$ . The neighbors set of node $s_i$ is $N_i=\{s_j\in \mathit{S}:({\mathrm{s}}_i,{\mathrm{s}}_j)\in \mathbf{\xi }\}$ . ${\mathbf{\xi }}_{\mathrm{ij}}=(s_i,s_j)$ is defined as the edge of $\mathit{G}$ . The $a_{\mathrm{ij}}$ corresponds with the fact that edges of the graph are positive, that is, ${\mathbf{\xi }}_{\mathrm{ij}}\in \mathbf{\xi }\iff a_{\mathrm{ij}}>0$ .

If $(x_i,y_i,z_i)$ represents the local coordinate of the ith AUV, ${\mathit{v}}_i$ represents the cruising speed vector of the ith AUV. One obtains

$$\{\begin{array}{c}{\stackrel{·}{x}}_i={\mathit{v}}_i^H\cos {\theta }_i\hfill \\ \multicolumn{1}{c}{{\stackrel{·}{y}}_i={\mathit{v}}_i^H\sin {\theta }_i}\\ \multicolumn{1}{c}{{\stackrel{·}{z}}_i={\mathit{v}}_i\sin {\psi }_i}\\ \multicolumn{1}{c}{{\stackrel{·}{\theta }}_i={\rho }_i}\\ \multicolumn{1}{c}{{\stackrel{·}{\psi }}_i={\omega }_i}\end{array}$$

(1)

where ${\psi }_i$ denotes the ith AUV pitch angle, ${\theta }_i$ denotes the ith AUV heading angle, ${\mathit{p}}_i=[x_i,y_i,z_i,{\theta }_i,{\psi }_i,{\varphi }_i{]}^T$ denotes the ith AUV speed and position vectors; thus, ${\stackrel{·}{\mathit{p}}}_i={\mathit{v}}_i$ , ${\mathit{v}}_i=[{\stackrel{·}{x}}_i,{\stackrel{·}{y}}_i,{\stackrel{·}{z}}_i,{\stackrel{·}{\varphi }}_i,{\stackrel{·}{\psi }}_i,{\stackrel{·}{\theta }}_i{]}^T$ . ${\mathit{v}}_i^H={\mathit{v}}_i\cos {\psi }_i$ represents the ith AUV horizontal velocity. Since ${\mathit{v}}_i^H$ , ${\theta }_i$ , and ${\psi }_i$ correspond with the control input of the thruster, vertical wings, and horizontal rudders, respectively, a cylinderal coordinate $(r_i,{\theta }_i,z_i)$ for the underactuated AUV have been established

$$\{\begin{array}{c}{r}_i=\sqrt{x_i^2+y_i^2}\hfill \\ \multicolumn{1}{c}{z_i=r_i\tan {\psi }_i}\end{array}\mathrm{where}\{\begin{array}{c}{x}_i=r_i\cos {\theta }_i\hfill \\ \multicolumn{1}{c}{y_i=r_i\sin {\theta }_i}\\ \multicolumn{1}{c}{{\stackrel{·}{r}}_i={\mathit{v}}_i^H}\end{array}$$

(2)

In order to investigate the dynamic state during multiple AUVs formation, multiple AUVs formation is taken as a constraint multibody system; adjacent AUVs are connected through a virtual spring (see Figure 2). Therefore, the virtual spring between the ith and the jth AUV is set to be at natural length $R_{\mathrm{ij}}$ with the stiffness $K_{\mathrm{ij}}$ , when these AUVs are cruising at expected positions. When the actual distance $L_{\mathrm{ij}}$ between the ith AUV and the jth AUV is greater than expected distance, thus the attracted force $F_{\mathrm{ij}}$ is generated

$$F_{\mathrm{ij}}=K_{\mathrm{ij}}(L_{\mathrm{ij}}-R_{\mathrm{ij}})$$

(3)

Figure 2.

Model of multi-AUV formation.

On the contrary, the rejected force is generated

$$F_{\mathrm{ij}}=K_{\mathrm{ij}}(R_{\mathrm{ij}}-L_{\mathrm{ij}})$$

(4)

Since the formation of multiple AUVs needs to maintain vertical and horizontal distance at the same time, the natural length of virtual spring $R_{\mathrm{ij}}$ is decomposed into ${x_R}_{\mathrm{ij}}$ , ${y_R}_{\mathrm{ij}}$ , and ${z_R}_{\mathrm{ij}}$ at three directions of coordinates. Thus, the attractive and repulsive forces can be obtained as

$$\{\begin{array}{c}{F}_{\mathrm{att}}{(x)}_{\mathrm{ij}}=K_{\mathrm{xij}}({x_L}_{\mathrm{ij}}-{x_R}_{\mathrm{ij}})\hfill \\ \multicolumn{1}{c}{F_{\mathrm{att}}{(y)}_{\mathrm{ij}}=K_{\mathrm{yij}}({y_L}_{\mathrm{ij}}-{y_R}_{\mathrm{ij}})}\\ \multicolumn{1}{c}{F_{\mathrm{att}}{(z)}_{\mathrm{ij}}=K_{\mathrm{zij}}({z_L}_{\mathrm{ij}}-{z_R}_{\mathrm{ij}})}\end{array}\mathrm{and}\{\begin{array}{c}{F}_{\mathrm{rep}}{(x)}_{\mathrm{ij}}=K_{\mathrm{xij}}({x_R}_{\mathrm{ij}}-{x_L}_{\mathrm{ij}})\hfill \\ \multicolumn{1}{c}{F_{\mathrm{rep}}{(y)}_{\mathrm{ij}}=K_{\mathrm{yij}}({y_R}_{\mathrm{ij}}-{y_L}_{\mathrm{ij}})}\\ \multicolumn{1}{c}{F_{\mathrm{rep}}{(z)}_{\mathrm{ij}}=K_{\mathrm{zij}}({z_R}_{\mathrm{ij}}-{z_L}_{\mathrm{ij}})}\end{array}$$

The potential energy derived from the attractive and repulsive forces is

$$\{\begin{array}{c}{P}_{\mathrm{att}}{(x)}_{\mathrm{ij}}=\frac{K_{\mathrm{xij}}}{2}{({x_L}_{\mathrm{ij}}-{x_R}_{\mathrm{ij}})}^2\hfill \\ \multicolumn{1}{c}{P_{\mathrm{att}}{(y)}_{\mathrm{ij}}=\frac{K_{\mathrm{yij}}}{2}{({y_L}_{\mathrm{ij}}-{y_R}_{\mathrm{ij}})}^2}\\ \multicolumn{1}{c}{P_{\mathrm{att}}{(z)}_{\mathrm{ij}}=\frac{K_{\mathrm{zij}}}{2}{({z_L}_{\mathrm{ij}}-{z_R}_{\mathrm{ij}})}^2}\end{array}\mathrm{and}\{\begin{array}{c}{P}_{\mathrm{rep}}{(x)}_{\mathrm{ij}}=\frac{K_{\mathrm{xij}}}{2}{({x_R}_{\mathrm{ij}}-{x_L}_{\mathrm{ij}})}^2\hfill \\ \multicolumn{1}{c}{P_{\mathrm{rep}}{(y)}_{\mathrm{ij}}=\frac{K_{\mathrm{yij}}}{2}{({r_y}_{\mathrm{ij}}-{r_y}_{\mathrm{ij}})}^2}\\ \multicolumn{1}{c}{{\mathrm{P}}_{\mathrm{rep}}{(z)}_{\mathrm{ij}}=\frac{K_{\mathrm{zij}}}{2}{({z_R}_{\mathrm{ij}}-{z_L}_{\mathrm{ij}})}^2}\end{array}$$

where ${x_L}_{\mathrm{ij}}$ , ${y_L}_{\mathrm{ij}}$ , and ${z_L}_{\mathrm{ij}}$ are the horizontal and vertical distance between the ith and the jth AUV

$$\{\begin{array}{c}{z_L}_{\mathrm{ij}}=|z_i-z_j|\hfill \\ \multicolumn{1}{c}{{r_L}_{\mathrm{ij}}=\sqrt{{(x_i-x_j)}^2+(y_i-y_j}{)}^2}\end{array}$$

Since multiple AUVs formation is facing with complicated environmental disturbance and limited communication, multiple layered stiffness virtual spring has been established. In Figure 3, $r_{1i}$ is the innermost layer and $r_{\mathrm{Ri}}$ is the outermost layer. For the ith AUV, attractive forces will be derived in the layer of $r_{\mathrm{Ri}}$ through virtual spring, while repulsive forces will be derived out of the layer of $r_{\mathrm{Ri}}$ through virtual spring.

Figure 3.

Multi-layer region of the ith AUV.

The attractive or repulsive forces exert on the jth AUV should be the sum of attractive or repulsive forces derived from virtual spring from the Rth layer to its located layer. Moreover, when the jth AUV is located at the innermost layer, the repulsive energy exerted from the ith AUV to the jth AUV will be the sum of virtual spring repulsive energies from the Rth layer to the first layer. Considering the AUV is underactuated, its vertical and horizontal movements are realized through thruster with wings or rudders, potential functions are order reduced using log function so as to improve the convergence of the controller (see Figure 6 in the simulation)

$$\{\begin{array}{c}{P}_{\mathrm{rep}}{(x)}_{\mathrm{ij}}=\frac{K_{\mathrm{xij}}}{2}\mathrm{In}\left[{(x_{1i}-x_{\mathrm{Lij}})}^2\right]+\frac{K_{\mathrm{xij}}}{2}\mathrm{In}\left[{(x_{2i}-x_{\mathrm{Lij}})}^2\right]\hfill \\ \multicolumn{1}{c}{+\cdots \frac{K_{\mathrm{xij}}}{2}\mathrm{In}\left[{(x_{\mathrm{Ri}}-x_{\mathrm{Lij}})}^2\right]}\\ \multicolumn{1}{c}{P_{\mathrm{rep}}{(y)}_{\mathrm{ij}}=\frac{K_{\mathrm{yij}}}{2}\mathrm{In}\left[{(y_{1i}-y_{\mathrm{Lij}})}^2\right]+\frac{K_{\mathrm{yij}}}{2}\mathrm{In}\left[{(y_{2i}-y_{\mathrm{Lij}})}^2\right]}\\ \multicolumn{1}{c}{+\cdots \frac{K_{\mathrm{yij}}}{2}\mathrm{In}\left[{(y_{\mathrm{Ri}}-y_{\mathrm{Lij}})}^2\right]}\\ \multicolumn{1}{c}{P_{\mathrm{rep}}{(z)}_{\mathrm{ij}}=\frac{K_{\mathrm{zij}}}{2}\mathrm{In}\left[{(z_{1i}-z_{\mathrm{Lij}})}^2\right]+\frac{K_{\mathrm{zij}}}{2}\mathrm{In}\left[{(z_{2i}-z_{\mathrm{Lij}})}^2\right]}\\ \multicolumn{1}{c}{+\cdots \frac{K_{\mathrm{zij}}}{2}\mathrm{In}\left[{(z_{\mathrm{Ri}}-z_{\mathrm{Lij}})}^2\right]}\end{array}$$

(5)

Thus, a multi-layered region for multiple AUV formation can be defined in Figure 4, in order to improve the formation stability. ${\mathit{p}}_c^d$ represents geometric center of the formation shape. The attractive or repulsive forces range of the ith AUV ${\mathit{p}}_i$ is set according to the formation requirement.

Figure 4.

The multi-layer region for AUV formation.

Interval type-2 fuzzy learning approach for AUVs formation

As a type of effective and reliable machine learning techniques, SVM has been widely used in classification problems. It constructs a hyper plane to separate two classes of data so that the margin is largest. ²³ In compare with neural fuzzy systems, the basic SVM requires less prior knowledge of the problem, but can solve the learning problem with smaller number of samples. Instead of minimizing an objective function on the basis of training, SVM attempts to minimize one bound on the generalization error. ²⁴ Since the problem of multiple AUVs formation confronts with tracking and formation maintain, collision avoidance, oceanic current and uncertain disturbance, and so on, the enumeration of fuzzy linguistic rules cannot cover all the situations. Although SVM is computationally expensive, it can generate optimal and reasonable rules on the current situations through classification. Therefore, the interval type-2 fuzzy learning approach has integrated SVM to generate optimal rules for the complicated formation conditions. Furthermore, in the combination with reinforcement learning, limited communication state will be dealt with for the fuzzy systems during multiple AUVs formation. Its structure is illustrated in Figure 5.

Figure 5.

Structure of interval type-2 fuzzy learning approach.

SVM rules generation algorithm

For the formation state between the ith and the jth AUV, the rules are set in the following: IF ( $P(x{)}_{\mathrm{ij}}$ , $P(y{)}_{\mathrm{ij}}$ , $P(z{)}_{\mathrm{ij}}$ ) is ( $\tilde{D}(x{)}_{\mathrm{ij}}^I$ , $\tilde{D}(y{)}_{\mathrm{ij}}^I$ , $\tilde{D}(z{)}_{\mathrm{ij}}^I$ ), AND the expected cruising trajectory is ( $x_{\mathrm{di}}$ , $y_{\mathrm{di}}$ , $z_{\mathrm{di}}$ ), AND oceanic current state is ${\mathit{v}}_c$ THEN The output includes expected state of the ith AUV ( $v_i^H$ , ${\theta }_i$ , ${\psi }_i$ ).

Apparently, the states include permutation and combination of various conditions of potential field, cruising tracking trajectory state, uncertain oceanic current, and so on. It is difficult to enumerate the linguistic rules and input output data pairs to cover all the fuzzy situations. This article will apply the SVM as a tool in the fuzzy rules determination. For the SVM rule generation approach, the determine function is defined in the following

$$\mathit{f}({\mathit{X}}^I)={\mathit{w}}^T\varphi \left({\mathit{X}}^I\right)+\mathit{b}$$

(6)

where ${\mathit{X}}^I$ is the set of input signals ${\mathit{X}}^I=(x_1^I,x_2^I,...,x_{\mathrm{ni}}^I)$ . $\varphi \left({\mathit{X}}^I\right)$ is a nonlinear function which maps the input vector ${\mathit{X}}^I$ into higher dimension feature space, $\mathit{w}$ is ni dimensional weights vector, and $\mathit{b}$ is the scalar.

The obtaining of a hyper plane separating two classes will lead to the following optimal problem ²⁴

$$\{\begin{array}{c}min\frac{1}{2}{\mathit{w}}^T\mathit{w}\hfill \\ \multicolumn{1}{c}{\begin{array}{cc}\mathrm{subject}\hfill & \mathrm{to}{y}_l^I({\mathit{w}}^T\varphi \left({\mathit{X}}^{\mathrm{I}}\right)+b_l)\ge 1\begin{array}{c}\forall l\hfill \end{array}\hfill \end{array}}\end{array}$$

(7)

Thus, optimal hyper plane will be formulated from the following optimization problem

$$\{\begin{array}{c}min(\frac{1}{2}{\mathit{w}}^T\mathit{w}+C\sum _{i=1}^{\mathrm{nr}}{\xi }_i)\hfill \\ \multicolumn{1}{c}{\begin{array}{cc}\mathrm{subject}\hfill & \mathrm{to}{y}_i^I({\mathit{w}}^T\varphi \left({\mathit{X}}^{\mathrm{I}}\right)+b_i)\ge 1-{\xi }_i,\begin{array}{c}{\xi }_i\ge 0,i=1,2,\dots ,\mathrm{nr}\hfill \end{array}\hfill \end{array}}\end{array}$$

(8)

where C > 0 is the regularization parameter which control the trade-off between margin and error for the classification. The primal problem of equation (8) can be solved by the following Lagrangian function

$$\begin{array}{l}\mathit{L}=\frac{1}{2}{\mathit{w}}^T\mathit{w}+C{\displaystyle \sum _{i=1}^{nr}{\xi }_i}-{\displaystyle \sum _{i=1}^{nr}{\alpha }_i{\xi }_i}\\ -{\displaystyle \sum _{i=1}^{nr}{\beta }_i\left[y_i^I\left({\mathit{w}}^T\varphi \left({\mathit{X}}^{\mathbf{I}}\right)+b_i\right)+{\xi }_i-1\right]}\end{array}$$

(9)

where ${\alpha }_i$ and ${\beta }_i$ ( $0\le {\beta }_i,{\alpha }_i\le C$ ) are Lagrange multipliers. Therefore, the following dual quadratic problem can be obtained from equations (8) and (9)

$$\{\begin{array}{c}max\left[\sum _{j=1}^{\mathrm{ni}}\sum _{i=1}^{\mathrm{ni}}[{\beta }_i-\frac{1}{2}{\beta }_i{\beta }_j{y}_i^I{y}_j^{I}K(x_i^I,x_j^I)]\right]\hfill \\ \multicolumn{1}{c}{\begin{array}{cc}\mathrm{subject}\hfill & \mathrm{to}\sum _{i=1}^{\mathrm{ni}}{y}_i^I{\beta }_i=0,\begin{array}{c}0\le {\beta }_i\le C,i=1,2,\dots ,\mathrm{ni}\hfill \end{array}\hfill \end{array}}\end{array}$$

(10)

where $K(x_i^I,x_j^I)$ is a kernel function which is defined as

$$K(x_i^I,x_j^I)=\varphi \left(x_i^I\right)·\varphi \left(x_j^I\right)=\exp (-\gamma {\Vert x_i^r-x_j^r\Vert }^2)$$

where $\gamma$ is the scaling factor. Therefore, the decision function is obtained as

$$f\left({\mathit{X}}^I\right)=\mathrm{sgn}[\sum _{i=1}^{\mathrm{ni}}{\beta }_i{y}_i^{I}K(x_i^I,{\mathit{X}}^I)+b]$$

(11)

Interval type-2 fuzzy learning approach

The system contains two components: the type-reducer, which maps a type-2 fuzzy set into a type-1 fuzzy set and a normal defuzzifier, which transforms a fuzzy output into a crisp output. A type-2 fuzzy set $\tilde{\mathit{D}}$ is defined as

$$\tilde{\mathit{D}}=\{((x^I,u),{\mu }_{\tilde{D}}(x^I,u))|\forall x^I\in {\mathit{X}}^I,\forall u\in {\mathit{J}}_x\subseteq [0,1]\}$$

(12)

where ${\mu }_{\tilde{\mathit{D}}}(x^I,u)\in [0,1]$ , $x^I$ represents the input element of the type-2 fuzzy logic system, the set $\mathit{X}$ includes current potential field, communication state, expected cruising trajectory, and oceanic current, and ${\mathit{J}}_x$ represents the main membership of $x^I$ in the $\tilde{\mathit{D}}$

$$\tilde{\mathit{D}}=\underset{x^I\in \mathit{X}}{\int }\underset{u\in {\mathit{J}}_x}{\int }{\mu }_{\tilde{\mathit{D}}}(x^I,u)/{\mu }_{\tilde{\mathit{D}}}(x^I,u)(x^I,u)(x^I,u),\begin{array}{c}{\mathit{J}}_x\subseteq [0,1]\hfill \end{array}$$

(13)

where ∫∫ indicates union over all admissible $x^I$ and u. If we define the lth input element as $x_l^I$ and the fuzzy set as ${\tilde{\mathit{D}}}_l^I$ , $l=1,2,...,\mathrm{ni}$ is the input signal number. A Gaussian primary membership function with uncertain mean and fixed standard deviation ${\sigma }_l^k$ is expressed as

$${\mu }_{{\tilde{\mathbf{D}}}_{\mathrm{kl}}}=\exp [-\frac{1}{2}{\left(\frac{x_{\mathrm{kl}}^I-m_{\mathrm{kl}}}{{\sigma }_{\mathrm{kl}}}\right)}^2]$$

(14)

where $m_{\mathrm{kl}}\in [m_{\mathrm{kl}1},m_{\mathrm{kl}2}]$ . The membership degree ${\mu }_{{\tilde{\mathit{D}}}_{\mathrm{kl}}}$ is represented as a bounded interval set of ${\mu }_{{\tilde{\mathbf{D}}}_{\mathrm{kl}}}\in [{\underset{¯}{\mu }}_{{\tilde{\mathit{D}}}_{\mathrm{kl}}},{\overline{\mu }}_{{\tilde{\mathbf{D}}}_{\mathrm{kl}}}]$ , where ${\underset{¯}{\mu }}_{{\tilde{\mathbf{D}}}_{\mathrm{kl}}}$ and ${\overline{\mu }}_{{\tilde{\mathit{D}}}_{\mathrm{kl}}^I}$ represent the lower and upper bound, respectively, k = 1, 2,…, nr is the number of the rules

$$\begin{array}{l}{\overline{\mu }}_{{\tilde{\mathit{D}}}_{kl}}=\{\begin{array}{cc}{\mu }_{{\tilde{\mathit{D}}}_{kl}}\left(m_{kl1},{\sigma }_{kl},x_{kl}\right),& x_{kl}<m_{kl1}\\ 1,& m_{kl1}\le x_{kl}\le m_{kl2}\\ {\mu }_{{\tilde{\mathit{D}}}_{kl}}\left(m_{kl2},{\sigma }_{kl},x_{kl}\right),& x_{kl}>m_{kl2}\end{array}\mathrm{and}\\ {\underset{\_}{\mu }}_{{\tilde{\mathit{D}}}_{kl}}=\{\begin{array}{cc}{\mu }_{{\tilde{\mathit{D}}}_{kl}}\left(m_{kl2},{\sigma }_{kl},x_{kl}\right)& x_{kl}^I\le \frac{m_{kl1}+m_{kl2}}{2}\\ {\mu }_{{\tilde{\mathit{D}}}_{kl}}\left(m_{kl1},{\sigma }_{kl},x_{kl}\right)& x_{kl}^I>\frac{m_{kl1}+m_{kl2}}{2}\end{array}\end{array}$$

(15)

Using an algebraic product operation, the fuzzy operation is implemented. The result is an interval type-1 fuzzy set. For the ith rule, the firing strength is computed as

$${\mathit{F}}_k=[{\underset{l=1}{\overset{\mathrm{ni}}{\Pi }}}_{{\tilde{\mathbf{D}}}_{\mathrm{kl}}},\underset{l=1}{\overset{\mathrm{ni}}{\Pi }}{\overline{\mu }}_{{\tilde{\mathbf{D}}}_{\mathrm{kl}}}]=[{\underset{¯}{f}}_k,{\overline{f}}_k]$$

(16)

where ${\underset{¯}{f}}_k$ is the lower firing strength and ${\overline{f}}_k$ is the upper firing strength. If we set ${\mathit{v}}_L$ as the left most point and ${\mathit{v}}_R$ as the right most point of the interval type-1 fuzzy set, they can be expressed as

$$\begin{array}{l}{\mathit{v}}_L=\frac{{\displaystyle {\sum }_{k=1}^{Lc}{\overline{f}}_k{a}_k}+{\displaystyle {\sum }_{k=Lc+1}^{nr}{\underset{\_}{f}}_k{a}_k}}{{\displaystyle {\sum }_{k=1}^{Lc}{\overline{f}}_k}+{\displaystyle {\sum }_{k=Lc+1}^{nr}{\underset{\_}{f}}_k}},\\ {\mathit{v}}_R=\frac{{\displaystyle {\sum }_{k=1}^{Rc}{\underset{\_}{f}}_k{a}_k}+{\displaystyle {\sum }_{k=Rc+1}^{nr}{\overline{f}}_k{a}_k}}{{\displaystyle {\sum }_{k=1}^{Rc}{\underset{\_}{f}}_k}+{\displaystyle {\sum }_{k=Rc+1}^{nr}{\overline{f}}_k}}\end{array}$$

(17)

where Lc and Rc represent the left and right crossover points, respectively, and nr is the number of the rules. On the basis of the center sets type reduction, the defuzzified output is the average of ${\mathit{v}}_L$ and ${\mathit{v}}_R$

$${\mathit{v}}_o=\frac{({\mathit{v}}_R+{\mathit{v}}_L)}{2}$$

(18)

Since reinforcement learning is a suitable algorithm for solving optimization problems through online interactive learning rather than depending on existence database, ²⁵ it will help AUVs issue and improve formation control commands with uncertain conditions. ²⁶ For the multiple AUVs formation, uncertain acoustic communication conditions may affect great on the understanding of the adjacent AUV position. The interval type-2 fuzzy learning approach will apply the Q-learning algorithm to optimize the ${\mathit{v}}_o$ according to current acoustic communication state $s(t)$ through maximized reward function. Thus, the ith AUV can not only execute an action $a_k$ through the evaluation, but also can avoid coupling character between communication state and potential field. According to Q-learning algorithm, the action $a_k$ will be updated as follows ¹⁹

$$\begin{array}{c}Q(s(t),a_k(t))=Q(s(t),a_k(t))+\alpha [r(t+1)\hfill \\ +\gamma Q^{*}(s(t+1))-Q(s(t),a_k(t))\hfill \end{array}]$$

(19)

where $r(t+1)$ is the reinforcement reward and $Q^{*}(s(t+1))$ is the optimal estimation in the set of possible actions. The expected Q value for the action output is

$$Q(s+1))=\frac{1}{2}\sum _{i=1}^{\mathrm{nr}}(\frac{{\underset{¯}{f}}_i(s(t))}{\sum _{k=1}^{\mathrm{nr}}{\underset{¯}{f}}_k(s(t))}+\frac{{\overline{f}}_i(s(t))}{\sum _{k=1}^{\mathrm{nr}}{\overline{f}}_k(s(t))})q_i$$

(20)

According to equation (19), the Q value will be updated as

$$\Delta Q=r(t+1)+\gamma Q^{*}(s(t+1))-Q(s(t),a(t))$$

(21)

For each rule and action, the maximum Q value can be estimated as

$$Q^{*}(s(t+1))=\frac{1}{2}\sum _{i=1}^{\mathrm{nr}}(\frac{{\underset{¯}{f}}_i(s(t))}{\sum _{k=1}^{\mathrm{nr}}{\underset{¯}{f}}_k(s(t))}+\frac{{\overline{f}}_i(s(t))}{\sum _{k=1}^{\mathrm{nr}}{\overline{f}}_k(s(t))})q_i^{*}$$

(22)

where $q_i(t+1)=q_i(t)+\epsilon \Delta q_i(t)\begin{array}{c}i=1,...,\mathrm{nr}\hfill \end{array}$ , and $\epsilon$ is the learning rate

$$\Delta q_i(t)=\frac{1}{2}\Delta Q(\frac{{\underset{¯}{f}}_i(s(t))}{\sum _{k=1}^{\mathrm{nr}}{\underset{¯}{f}}_k(s(t))}+\frac{{\overline{f}}_i(s(t))}{\sum _{k=1}^{\mathrm{nr}}{\overline{f}}_k(s(t))})$$

(23)

Simulations and experiments

Experimental setup

In order to validate the proposed interval type-2 fuzzy learning approach, simulation platform with three heterogeneous underactuated AUVs has been established. These AUVs are connected in series with correlated parameters presented in Table 1. These parameters are corresponding to three real AUVs available of our institutions. Simulation platform includes three different AUV control platforms and maneuverability simulation platforms.

Table 1.

Size and maneuverability parameters for the three AUVs.

AUV	Length (m)	Maximal rotary body diameter (m)	Weight in the air (kg)	Maximal truster of main thruster (kg)
Leader (AUV1)	5.6	0.8	1300	180
Follower 1 (AUV2)	2.3	0.32	160	26
Follower 2 (AUV3)	1.2	0.3	80	20

AUV: autonomous underwater vehicle.

In the AUV control platforms, S surface control scheme ²⁷ has been applied for the AUV control, and the control commands of propeller, heading rudder, and horizontal wing are brought forth through thrust allocation in order to realize AUV cruising velocity, heading control, and diving control.

The maneuverability simulation platform has been established on the basis of dynamic equations with hydrodynamic parameters of correspondent AUV. In the maneuverability simulation platform, the inputs are control voltage of propeller, angle control commands of heading rudder and horizontal wing, control voltage is converted into propeller thrust through thrust–volt curves from thruster experiments, the outputs are the AUV real-time positions deduced from hydrodynamic equations. Formation control and planning sections are set up on the basis of the leader AUV.

Figure 6 illustrates the formation simulation trajectory results from the three AUVs of Table 1. Three AUVs are set out at initial positions (0,0,0), (40,40,0), and (60,60,0), respectively. Then make a 6 m dive and 90° heading control, afterward move along the north direction. Since different AUVs are different in size, gyration radius, and maneuverability, the formation process involves the coupling effects of diving and heading control. By taking underactuation into consideration, potential functions are order reduced using log function so as to improve the convergence of the controller. Therefore, the AUVs formation is stabilized after fluctuation and adjustment.

Figure 6.

Potential field comparisons between second order and “ln.”

In the formation simulations of Figure 7, comparisons have been made between the adaptive potential formation scheme of Huang et al. ⁷ and formation approach of this article. The disturbance is set with current speed v_cur = 0.2 m/s, west. In Figure 7(a) and (b), the three AUVs plans to follow folding line and round curve with line shape, for example, the followers are planned to maintain the same distance one after another. The membership function of type-2 fuzzy logic system includes the potential field, expected cruising trajectory, oceanic current state as inputs (see Figure 8). From comparisons, for the underactuated AUVs, great change of potential energy plus current field disturbance may cause formation shape fluctuation, and even failure, since limited controlled freedom is available. However, this article has proposed an interval type-2 fuzzy learning approach integrated with potential field. This approach automatically generates fuzzy rules according to current conditions, reduces the magnitude of potential energy through SVM rule generation algorithm. As a result, it not only maintains the formation shape more stable, but also improves the controller robust against current disturbance.

Figure 7.

Comparison between adaptive potential field (APF) and type-2 fuzzy learning approach integrated with potential field (T2FLP): (a) formation along folding line and (b) formation along round curve.

Figure 8.

Membership functions.

Figure 9 illustrates the simulation of interval type-2 fuzzy learning approach with communication state evaluation. In the simulations, three AUVs are making formation on the basis of topology, the communication state such as packages losses, time delay, and throughput is prescribed in Figure 9(a)–(c). Before reinforcement learning, the formation of multiple AUVs cannot be realized due to communication deficiency. After reinforcement learning in Figure 9(d), the formation approach has been optimized through the communication evaluation, taking actions and rewards; thus, the formation of multiple AUVs has been greatly improved through reinforcement learning.

Figure 9.

Simulation results from communication state evaluation: (a) package delay, (b) package loss, (c) throughput, and (d) formation trajectory.

Results from offshore experiments of AUVs formation on the water surface are illustrated in Figure 10. The AUVs formation traces were given pre-programmed missions. In the first scenario, AUVs were planned 90° yaw trace to test formation performance of heterogeneous vehicles. In the second scenario, AUVs were planned linear traces. In the third scenario, AUVs were planned with polygon path to validate the performance of regional search. From the experiments, vehicles of different sizes, gyration radius, and maneuverability can realize formation cruising under the proposed approach with.

Figure 10.

Surface experiment on strait and folding trajectory.

Conclusion

In the consideration with the AUVs operation and formation framework, this study has developed a novel approach to integrate potential field and interval type-2 fuzzy learning for the formation control of multiple AUVs based on formation system framework. Contributions and novelties of this article are twofold:

1. For the system nonlinear character and complicated formation state, SVM has been integrated into the type-2 fuzzy learning systems to generate optimal rules. In the simulations, this approach automatically generates fuzzy rules, reduces the magnitude of potential energy, and as a result, not only maintains the formation shape more stable, but also improves the controller robust against current disturbance.

2. Reinforcement learning has been combined with type-2 fuzzy systems to deal with acoustic communication conditions during the formation process. Therefore, AUVs can not only execute actions through the evaluation, but also can avoid coupling character between communication state and potential field.

Simulations and comparisons for the cruising in disturbances have been made and illustrated the effectiveness of the proposed formation strategies. Offshore experiments with heterogeneous AUVs surface formation testify its performance of regional coverage.