Cognition-based hybrid path planning for autonomous underwater vehicle target following

Abstract

Intelligent path planning is one of the key techniques for autonomous underwater vehicles for the purpose of target detection, environmental survey and so on. In order to realize automatic motion plan, an intelligent cognitive architecture for autonomous underwater vehicle motion planning has been proposed to realize complicated target detection and mobile target following in the disturbance environment. A novel adaptive ant colony optimization and particle swarm optimization fusion-based fuzzy rules optimization algorithm has been proposed to generate optimized fuzzy rules. Through this optimization algorithm, the preliminary fuzzy rules can be optimized to realize intelligent motion planning for complicated operation tasks. Experiments of channel following for wall detection and mobile target following in the oceanic environment have verified the validity of path planning method in the implementation of detection and operation tasks.

Introduction

Currently, autonomous underwater vehicles (AUVs) are increasingly attractive for various underwater tasks such as environment exploration, ¹ seabed survey, ² harbour protection ³ and submarine search and rescue. ⁴ With the development of artificial intelligence and computer science, autonomous path planning can not only help the vehicle initialize the operation route but also realize reasonable reaction on different states of obstacles and target during operation. Path planning methods include global path planning and local path planning. Global path is generated on the basis of preliminary environmental understanding and modelling. ⁵ On the other hand, local path planning is responsible for the generation of local or regional path to handle unknown and temporary obstacles under environmental disturbances. ⁶

In the last decade, a variety of solutions have been developed for the underwater vehicle path planning problems. ⁷ The fast marching global plan algorithm includes graph-based algorithm, ⁸ heuristic search algorithm, ⁹ evolutionary optimization algorithm ¹⁰ and so on. Zhuang et al. ¹¹ proposed a hybrid optimization algorithm to integrate particle swarm optimization (PSO) algorithm with Legendre pseudo spectral method (LPM) for AUV operating in cluttered and uncertain environments, and the searching process is accelerated through LPM. Cheng et al. ² proposed a dynamic programming-based genetic path planner algorithm in which the random-based crossover operator is replaced with a deterministic crossover operator.

Local path planner algorithm includes artificial potential field (APF), ¹² fuzzy path planner ¹³ and so on. Compared with surface vehicles, ^14,15 the local path planning of AUV is more eligible to be disturbed by the environmental disturbance. Potential field algorithms are mostly applied because they can generate the forward path by constructing an APF to weigh the influences of obstacles and goal points with better consistency and convergence for AUV local path planning. ¹⁶ But local minima may stop the objective from being achieved. In order to overcome this problem, Melingui et al. ¹⁷ proposed a novel plan and navigation approach to integrate APF and fuzzy logic into a common framework, which utilizes both the heuristic knowledge and the sampled input–output data pairs. Park et al. ¹² presented an advanced fuzzy potential field method for mobile robot obstacle avoidance. The method primarily generates the repulsive forces of surrounding obstacles, and secondly handles linguistic variables with fuzzy rules. However, unknown and complicated coastal environment often causes difficulties in the formulation of fuzzy rules. ¹⁸

However, AUV path planning in applications often confronts with complicated difficulties. Global path planning is proposed to fulfil missions like regional coverage, target search and tracking ^19,20 in limited time. Local path planning is proposed to evade obstacles, make close observation and even operation on specified target, which may sometimes meet unpredicted difficulties and require infinite efforts. ²¹ The AUV should adjust its route and coordinate with global and local task within the consideration of uncertain disturbances, operation time, local sampling cost and so on.

In comparison with the reflection and reasoning of human beings, the intelligence of AUVs is still in their early stages. Invariable rules-based fuzzy reasoning cannot adapt for various and complicated environment. ²² The cognition of human brain, on the contrary, operates on the basis of working memory, generates different fuzzy rules in the face of different conditions, ²³ through the integration of perceiving, understanding, reasoning and learning. ²⁴ This article will propose a novel intelligent AUV path planning algorithm on the basis of cognitive architecture for unknown obstacle avoidance, target following and detection. The contributions of this study are described as follows:

This study has proposed an intelligent fuzzy path planning method on the basis of cognitive architecture for AUV target following to avoid unknown obstacles in the disturbance environment.

A novel adaptive ant colony optimization (ACO) and PSO algorithm-based fuzzy rules optimization algorithm has been proposed to generate optimized fuzzy rules to obtain optimal path for proposed operation tasks.

Canal wall following and oceanic mobile target following experiments have verified the validity of path planning method in the implementation of detection and operation tasks.

The rest of this study is organized as follows. The cognitive architecture for AUV path planning is issued in the second section. A novel intelligent path planning method is proposed in the third section with the adaptive ACO-PSO algorithm for fuzzy rule optimization. Tank and typical environmental experiments will be discussed and analysed in the fourth section. We will make conclusion in the last section.

Cognitive architecture for AUV path planning

The proposed cognitive architecture has been designed on the basis of the torpedo shape and intelligent AUV in Figure 1. It is an AUV with the length at 5.5 m, the greatest gyrator diameter at 0.63 m, greatest submergence depth at 2000 m, greatest cruising speed for 5 kn and greatest continuation journey at 350 km. The AUV is equipped with a ‘Tritech Super SeaKing DST’ digital scanning sonar, a 740 TV line camera, two vertical channel thrusters, one propeller, vertical rudders and horizontal wings and so on. The exploration range of the digital scanning sonar is 300 m with its vertical beam width at 20° and horizontal beam width at 3°. Its navigation and position reckoning can be realized through Doppler Velocity Log (DVL) and inertial navigation system. The operation system is on the basis of PC104 embedded with C++ as the software language.

Figure 1.

Torpedo shape and intelligent AUV construction. (1) Digital scanning sonar. (2) Vertical channel thruster. (3) GPS and wireless antenna. (4) Propeller. (5) Vertical rudders and horizontal wings. (6) Multi-beam Bathymeter. (7) Sidescan sonar. (8) Underwater CCD. AUV: autonomous underwater vehicle.

The proposed cognitive architecture of the AUV includes a high level of knowledge-based path planner, a low level of vehicle control and executive module and environmental perception module (see Figure 2). The high-level autonomous planner and low-level control module interact each other simultaneously through environmental perception and state awareness. The executive module fulfils the AUV control command with current situation feedbacks.

Figure 2.

Cognitive architecture for AUV path planning. AUV: autonomous underwater vehicle.

The high level of knowledge-based path planner includes the global path planner and the local path planner. The global waypoints are initialized in advance according to the marine map and mission tasks before the mission start. The global path planner is realized through a revised heuristic search and coverage algorithm. ²⁵ Through the global path planner, the AUV tasks and path waypoints are arranged and organized.

In order to realize local path planning for target detection and obstacle avoidance in the disturbance environment, the oceanic environment is modelled not only with the digital scanning sonar of mid and far distant obstacles but also with a camera carefully observing the ambient environment and operation target. Fuzzy rules are initially proposed and optimized through adaptive ACO and PSO rule optimization algorithm.

Intelligent path planning method

Obstacles and target modelling

The digital scanning sonar is the device that can not only detect the target and the obstacle but also measure their distances from the vehicle. The range of scanning sonar is 20° with the radius as R. The representation of sonar detection space is important for environmental modelling and target tracking. In the sonar detection space, an occupancy grid map of environment has been constructed. Each cell of the map in the discrete region is characterized through two states: empty and occupied. The state is obtained through the projection of sonar profile. From the data flow of projection profile, one can obtain whether the cell is occupied or empty. The sonar can be applied to model and find uncertain static and dynamic object with Gaussian distribution. The operation profile of scanning sonar is shown in Figure 3.

Figure 3.

The operation range of the digital scanning sonar (the space of sonar includes seven radial directions, that is, left (l), front small left (Sfl), front left (fl), front (f), front right (fr), front large right (fr), right).

Through Demster–Shafer theory, the goal of the occupancy grids is to determine the cell possibilities of empty and occupied. If we define O as the states of occupation by obstacles, and E as the empty states, one can obtain the set of discernment Θ as

$\Theta \text{={}O,E\}$

1

Each grid in the workspace is defined by the cell state $U(i,j)$ , which is used to describe the assignment of basic probability $m_{i,j}$ to each label in Ω

$\Omega =\left\{\varphi ,E,O,\left\{E,O\right\}\right\}=2^{\Theta }$

2

Each cell $U(i,j)$ in the grid is defined as

$m_{i,j}(\varphi )+m_{i,j}(E)+m_{i,j}(O)+m_{i,j}(\left\{E,O\right\})=1$

3

with every cell in this map is initialized as

$m_{i,j}(O)=m_{i,j}(E)=0\hspace{0.17em}\text{and}\hspace{0.17em}{m}_{i,j}(\left\{E,O\right\})=1$

4

In the path planning process of potential field, the attractive forces come from the goal and the observing targets, while the repulsive forces come from the obstacles. On the basis of sonar profile, the repulsive force between the vehicle and the obstacles can be obtained to prevent collisions through the sum of gradients of the potential field

$f_{\text{rep}}=\{\begin{array}{c}{k}_{\text{rep}}{\displaystyle \sum _{i=1}^m(\frac{1}{\Vert P_a-P_{oi}\Vert }-\frac{1}{d_{\text{max}}})\frac{P_a-P_{oi}}{\Vert P_a-P_{oi}\Vert }\begin{array}{cc}& \text{if}\Vert P_a-P_{oi}\Vert <d_{\text{max}}\end{array}}\\ 0\begin{array}{cccc}& & & \text{otherwise}\end{array}\end{array}$

5

where m is the number of ambient obstacles, $k_{\text{rep}}$ is the positive constant of the repulsive forces and $d_{\text{max}}$ is the obstacle influence distance. From equation (5), the repulsive forces are the sum of several subrepulsive ones. From the obstacles modelling through scanning sonar, three components of obstacle information are expressed with obstacle position ( $P_{oi}=(x_{oi},y_{oi},z_{oi})$ ). If the current position of AUV is set as $P_a=(x_a,y_a,z_a)$ , the distance and angular relationship between the AUV and the obstacle can be described as

$\{\begin{array}{c}|d_{A-Ob}|=\sqrt{{\left(x-x_{op}\right)}^2+{\left(y-y_{op}\right)}^2+{\left(z-z_{op}\right)}^2}\\ {\psi }_{A-Ob}=a\text{tan}\frac{y-y_{op}}{x-x_{op}}\\ {\theta }_{A-Ob}=a\text{tan}\frac{z-z_{op}}{\sqrt{{\left(x-x_{op}\right)}^2+{\left(y-y_{op}\right)}^2}}\end{array}$

6

In the path planning process, the obstacles must be avoided once they are found, while the targets should be observed and detected with the existence possibilities. According to the theory of potential field, the attractive forces come from the goal and the observing targets. On the other hand, the position of observed target can be described with $P_t=(x_t,y_t,z_t)$ , target radius rate ( ${\Theta }_{tr}$ ) and target uncertain rate ( ${\Theta }_{Ur}$ ). The target uncertainty is modelled with uncertain radius which is varied with normal distribution ${\Theta }_{tr}\sim \left(P_t,\sigma \right)$ . If M ₁, M ₂ and M ₃ are used to define the uncertainty difference on the object motion, the relationship between target change rate from t − 1 to t can be described through normal distribution and probability density functions of radius rate as follows

${\Theta }_{tr}(t)=M_1{\Theta }_{tr}(t-1)+M_2{X}_{(t-1)}+M_3$

7

where $X_{(t-1)}=N(0,\sigma )$ is the normal distribution for the target, $M_2={\left[\begin{array}{ccc}0& 1& 1\end{array}\right]}^T$ , $M_3={\left[\begin{array}{ccc}0& 0& {\Theta }_{Ur}\end{array}\right]}^T$ and

$M_1=\left[\begin{array}{ccc}1& {\Theta }_{Ur}(t)& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right]$

8

In the local planning, a novel multilayered potential field approach has been proposed for the target detection. The attractive force includes the goal and the observed target

$f_{\text{att}}=\{\begin{array}{c}\begin{array}{l}{k}_{\text{att}}\left(\lambda \left(\text{sgn}(P_g-P_a){\Vert P_g-P_a\Vert }^2\right)+(1-\lambda ){\Theta }_{Ur}(t)\left(\text{sgn}(P_t-P_a){\Vert P_t-P_a\Vert }^2\right)\right)\\ \text{if}\Vert P_g-P_a\Vert >{\gamma }_g\hspace{0.17em}\text{and}\hspace{0.17em}\Vert P_t-P_a\Vert >{\gamma }_t\end{array}\\ \begin{array}{l}{k}_{\text{att}}\left(\lambda \left(\text{sgn}(P_g-P_a)\Vert P_g-P_a\Vert \right)+(1-\lambda ){\Theta }_{Ur}(t)\left(\text{sgn}(P_t-P_a){\Vert P_t-P_a\Vert }^2\right)\right)\\ \text{if}\Vert P_g-P_a\Vert \le {\gamma }_g\hspace{0.17em}\text{and}\hspace{0.17em}\Vert P_t-P_a\Vert >{\gamma }_t\end{array}\\ \begin{array}{l}{k}_{\text{att}}\left(\lambda \left(\text{sgn}(P_g-P_a)\Vert P_g-P_a\Vert \right)+(1-\lambda ){\Theta }_{Ur}(t)\left(\text{sgn}(P_t-P_a)\Vert P_t-P_a\Vert \right)\right)\\ \text{if}\Vert P_g-P_a\Vert \le {\gamma }_g\hspace{0.17em}\text{and}\hspace{0.17em}\Vert P_t-P_a\Vert \le {\gamma }_t\end{array}\\ \begin{array}{l}{k}_{\text{att}}\left(\lambda \left(\text{sgn}(P_g-P_a){\Vert P_g-P_a\Vert }^2\right)+(1-\lambda ){\Theta }_{Ur}(t)\left(\text{sgn}(P_t-P_a)\Vert P_t-P_a\Vert \right)\right)\\ \text{if}\Vert P_g-P_a\Vert >{\gamma }_g\hspace{0.17em}\text{and}\hspace{0.17em}\Vert P_t-P_a\Vert \le {\gamma }_{t2}\end{array}\end{array}$

9

where $k_{\text{att}}$ is the positive constant of the attractive forces, $P_g=(x_g,y_g,z_g)$ represents the goal position, $P_t=(x_t,y_t,z_t)$ represents the target position, λ represents the coefficient of the attractive forces between the target and the goal and ${\gamma }_g$ and ${\gamma }_t$ represent the distance limit for the layer of attractive force.

Therefore, the artificial force can be obtained as

$f=f_{\text{rep}}+f_{\text{att}}$

10

From equations (5) to (10), the local obstacle and the target can be remodelled and the path can be planned through Figure 3.

Fuzzy logic descriptions

For the AUV cruising in the complicated environment, it should not only detect the target and avoid obstacles but also overcome oceanic current and uncertain disturbance. The fuzzy logic path planning method includes the following steps:

1. Fuzzification: all the fuzzy logic inputs, including AUV current positions $P_a=(x_a,y_a,z_a)$ , detected the ith obstacle positions $P_{oi}=(x_{oi},y_{oi},z_{oi})$ , target positions $P_t=(x_t,y_t,z_t)$ , AUV current velocity, heading angle, relative position between the AUV and the target and relative position between AUV and obstacles, are all interpreted into fuzzy linguistic values. And the fuzzy membership functions vary with the fuzzy inputs. The fuzzy outputs are the velocity and expect angle of the AUV.

2. Rule generation and optimization through adaptive ACO and PSO algorithm.

The initial fuzzy rules are the combinations of these corresponding forces and reactions. The number of fuzzy rules n_r should be correspondent with the sum and combinations of environmental conditions such as obstacles, targets, current and disturbances. The number of the fuzzy rules is related to the number of the input variables and the logical relationships between them. Although the former is easy to be get, the latter is hard to be established. In this article, the interplay between those AUV state variables and the outside information such as velocity, angle and position will aggravate the difficulty of the establishment of the fuzzy rules. The sum and combinations of the environmental conditions refer to the internal relationship of the fuzzy inputs.

The n_r fuzzy rules are designed by using Takagi–Sugeno fuzzy model of ‘IF-THEN’ rules as follows.

The ith rule R_p :

IF $\Vert P_a-P_{oi}\Vert =d_{pi}$ , $\Vert P_g-P_a\Vert =d_{ti}$ , the AUV speed is $v_a$ and the current speed is $v_c$ , THEN

$y_p=f_{\text{rep}}^p+f_{\text{att}}^p$

11

where $f_{\text{rep}}^p=\{\begin{array}{c}{k}_{\text{rep}}{\displaystyle \sum _{i=1}^m(\frac{\xi \Vert v_a-v_c\Vert }{\Vert P_a-P_{oi}\Vert }-\frac{1}{d_{\text{max}}})\frac{P_a-P_{oi}}{\Vert P_a-P_{oi}\Vert }\begin{array}{cc}& if\Vert P_a-P_{oi}\Vert <d_{\text{max}}\end{array}}\\ 0\begin{array}{cccc}& & & \text{otherwise}\end{array}\end{array}$

$\begin{array}{l}{f}_{\text{att}}=\{\begin{array}{c}\begin{array}{l}{k}_{\text{att}}\Vert v_a-v_c\Vert \left(\lambda \left(\text{sgn}(P_g-P_a){\Vert P_g-P_a\Vert }^2\right)+(1-\lambda ){\Theta }_{Ur}(t)\left(\text{sgn}(P_t-P_a){\Vert P_t-P_a\Vert }^2\right)\right)\\ \text{if}\Vert P_g-P_a\Vert >{\gamma }_g\hspace{0.17em}\text{and}\hspace{0.17em}\Vert P_t-P_a\Vert >{\gamma }_t\end{array}\\ \begin{array}{l}{k}_{\text{att}}\Vert v_a-v_c\Vert \left(\lambda \left(\text{sgn}(P_g-P_a)\Vert P_g-P_a\Vert \right)+(1-\lambda ){\Theta }_{Ur}(t)\left(\text{sgn}(P_t-P_a){\Vert P_t-P_a\Vert }^2\right)\right)\\ \text{if}\Vert P_g-P_a\Vert \le {\gamma }_g\hspace{0.17em}\text{and}\hspace{0.17em}\Vert P_t-P_a\Vert >{\gamma }_t\end{array}\\ \begin{array}{l}{k}_{\text{att}}\Vert v_a-v_c\Vert \left(\lambda \left(\text{sgn}(P_g-P_a)\Vert P_g-P_a\Vert \right)+(1-\lambda ){\Theta }_{Ur}(t)\left(\text{sgn}(P_t-P_a)\Vert P_t-P_a\Vert \right)\right)\\ \text{if}\Vert P_g-P_a\Vert \le {\gamma }_g\hspace{0.17em}\text{and}\hspace{0.17em}\Vert P_t-P_a\Vert \le {\gamma }_t\end{array}\\ \begin{array}{l}{k}_{\text{att}}\Vert v_a-v_c\Vert \left(\lambda \left(\text{sgn}(P_g-P_a){\Vert P_g-P_a\Vert }^2\right)+(1-\lambda ){\Theta }_{Ur}(t)\left(\text{sgn}(P_t-P_a)\Vert P_t-P_a\Vert \right)\right)\\ \text{if}\Vert P_g-P_a\Vert >{\gamma }_g\text{and}\Vert P_t-P_a\Vert \le {\gamma }_{t2}\end{array}\end{array}\end{array}$

12

$d_{\text{max}}$ is the influence distance of the obstacles and ξ is the speed coefficient, and p = 1, 2,…n_p , represent the number of rules.

3. Inference mechanism: in this process, the reasoning process includes optimized fuzzy inference rules.

4. Defuzzification: in this process, the defuzzification equation is the weighted average of the fuzzy outputs

$y=\frac{{\displaystyle \sum _{p=1}^{n_p}{w}_p{y}_p}}{{\displaystyle \sum _{p=1}^{n_p}{w}_p}}$

13

where $w_p$ is the weight value.

Adaptive rules generation and optimization algorithm

If the fuzzy rules can complete all the possible conditions and fuzzy inputs, one should take the considerations with $n_r\times m^{n_r}$ rules and conditions, where m is the input numbers and $n_r$ represents the number of fuzzy rules. Since these rules involve the states of various conditions of target positions, obstacle positions, AUV state and current disturbance state, it is difficult to enumerate all the appropriate linguistic fuzzy rules beforehand to cover all the possible situations. The following algorithm will adaptively fuse the ACO and PSO algorithms in order to optimize designed fuzzy rules. The following will introduce the steps of the rule generation and optimization method in Figure 4.

Figure 4.

The block diagram of adaptive PSO and ant rule optimization algorithm. PSO: particle swarm optimization.

The first step is to code each rule into a particle or an ant, before the application of ACO and PSO. For the ith input variable and the jth rules solution, the coded result can be obtained as the rules solution individually represented in the first row of Figure 4. For PSO, the rule particles can move and expand quickly in the search space but without global communication. Each rule particle has a position and velocity vector. Each rule particle updates its position and velocity through every learning cycle.

While in continuous ACO, ant rules are randomly distributed among particles. Not only the information can be exchanged among the ant individuals but the ACO Gaussian sampling can push individuals to the end of search from local optimum.

In the beginning of this algorithm, if we set $K_N$ as the total rules, N_aN as the solutions of ant rules individuals, one obtains $K_N-N_{aN}$ as the particle rules individuals. At the end of this algorithm, $K_{\text{end}}=K_N-N_{aN}$ represents the increase number of ant rules individuals and N_aN represents the decrease number of particle rules individuals. If $I_{\text{max}}$ is set as the maximum iteration number, $I_N$ is the iteration number, one can obtain the number of ant individuals from

$K_{Ic}\hspace{0.17em}\text{=}\hspace{0.17em}\text{int}\left(\text{(}{K}_{\text{end}}-N_{aN})\frac{I_N}{I_{\text{max}}}\right)+N_{aN}$

14

In the second step, the particle rules individuals are generated and expanded to the search space; the generated particle individual number is $K_N-K_{Ic}$ with the iteration as $I_N+1$ . Each particle swarm includes many particles which is used to define and assign all particles initialized positions. For the ith particle, the position and velocity vectors of the ith particle are set as $p_p$ and $v_p$ , $p_i$ is the best solution for a certain particle and $p_g$ is the global best solution for the particle individuals. Thus, the ant solutions can guide the particles to better particle solution. The particle velocity can be obtained as

$\begin{array}{l}{v}_i^p(I_N+1)=\varsigma (v_i^p(I_N)+k_1^p{\psi }_1\otimes (p_i(I_N)-p_p(I_N))\\ +k_2^p{\psi }_2\otimes (p_g(I_N)-p_p(I_N))\end{array}$

15

where ς is the constrict coefficient, $k_1^p$ and $k_2^p$ are positive parameters for particle acceleration and ${\psi }_i$ is a random vector with the entries as the random distribution in [0,1]. $\varsigma =0.7$ , $v_1^p=v_2^p=1$ , with each particle changes according to

$p_p(I_N+1)=p_p(I_N)+v_i^p(I_N+1),i=1,\mathrm{...},K_N-K_{Ic}$

16

where $K_N-K_{Ic+1}$ is retained as best performance for the next iteration.

From this approach diagram in Figure 4, the generated fuzzy rules solutions $s_1,\mathrm{...},s_N$ are sorted from best to the worst according to their performance. In the graph, every row represents the fuzzy rules solution obtained from particle or ant. The route from node to level can be searched by ant. Each route is correspondent to a classification rule, while each node represents the solution vector. The nodes connected through path segment corresponding to the pheromone level ${\tau }_i$ and they are selected with high probabilities

${\tau }_i(t+1)=\rho {\tau }_i(t)+\Delta {\tau }_i(t)$

17

where ρ is a parameter between 0 and 1, and $\Delta {\tau }_i(t)$ is selected from the quality of solution performance. The stronger the pheromone level ${\tau }_i$ , the better the solution is. The order of pheromone levels is sorted from the best to the worst.

The third step is ACO algorithm, and the ant generation process includes three stages:

1. Ant path selection: the new fuzzy rules solution is generated through ant path which is generated through the elite tournament technique. At the $I_N+1$ iteration, the total $K_{Ic}$ solutions are generated in the ant path selection process. The elite selection approach selects $L_{Ic}$ best solutions for temporary solutions. Among the ant path segments, the node value with highest pheromone level is selected. The value of $L_{Ic}$ in the elite selection varies with iteration

$L_{Ic}=\text{int}\left(K_{Ic}\frac{I_N}{I_{\text{max}}}\right)$

18

The fuzzy rules solutions are generated through elite selection with the increase of global search. In the selection process, the node can be selected from fuzzy rules particle or fuzzy rules ant.

2. In the Gaussian sampling process, a new value near the temporary solution via Gaussian sampling which means the standard deviation is computed as follows

$d_{ij}=\zeta {\displaystyle \sum _{l=1}^N\frac{|{\stackrel{\sim }{s}}_{ij}-s_{lj}|}{N-1}}$

19

where ${\stackrel{\sim }{s}}_{ij}$ is the mean of probability density function and ζ is the constant value. The Gaussian sampling operation process introduces new parameter values to every active rule in the temporary solutions through the mean of probability density function ${\stackrel{\sim }{s}}_{ij}$ . The standard deviation will dynamically change according to iteration number N.

3. In the solution refinement process, through the consideration with the attraction of ants to the optimal individual in the overall population, the solution can be obtained as

${\stackrel{\sim }{s}}_i={\widehat{s}}_i+{\psi }_3\otimes ({\widehat{s}}_1-{\widehat{s}}_i)$

20

where ${\psi }_3$ is random vector with uniform distribution random number in the interval [0,1]. The random vector is used for the promotion towards the best solution.

In order to realize rules solution iteration and optimization, reinforcement learning is applied to execute and evaluate the AUV path planning actions. The Q-function can realize mapping from rules-action pairs to predict return. ²⁶ The Q-function-based learning will speed up the convergence speed and improve system performance with disturbance. The output of the Q-function can be updated as follows

$Q(R(t),a_k(t))=Q(R(t),a_k(t))+\alpha \left[R(t+1)+\gamma Q^{\ast }(R(t+1))-Q(R(t),a_k(t))\right]$

21

where $R(t)$ represent current state of rules, $r(t+1)$ is the reinforcement reward for the rules, $Q^{\ast }(R(t+1))$ is the optimal estimation for the possible actions and γ is discount factor. At each step, the expected Q values are updated through

$Q(R+1))=\frac{1}{2}{\displaystyle \sum _{j=1}^{nr}(\frac{{\underset{\_}{f}}_j(R(t))}{{\displaystyle {\sum }_{k=1}^{nr}{\underset{\_}{f}}_k(R(t))}}+\frac{{\overline{f}}_j(R(t))}{{\displaystyle {\sum }_{k=1}^{nr}{\overline{f}}_k(R(t))}})}{q}_j$

22

According to equation (22), the Q value can be updated as follows:

$\Delta Q=\text{r}(t+1)+\gamma Q^{\ast }(R(t+1))-Q(R(t),a(t))$

23

The best Q values correspond with the quality of the rules actions above. For each rule, there will be a greedy action to achieve maximum Q values. For each rules solution and path planning action, the maximum Q value can be obtained from the estimation as

$Q^{*}(R(t+1))=\frac{1}{2}{\displaystyle \sum _{j=1}^{n_R}(\frac{{\underset{\_}{f}}_i(R(t))}{{\displaystyle {\sum }_{k=1}^{n_R}{\underset{\_}{f}}_k(R(t))}}+\frac{{\overline{f}}_i(R(t))}{{\displaystyle {\sum }_{k=1}^{n_R}{\overline{f}}_k(R(t))}})}{q}_i^{*}$

24

where $q_j(t+1)=q_j(t)+\epsilon \Delta q_j(t)\begin{array}{cc}& j=1,\mathrm{...},n_R\end{array}$ and ε is the learning rate.

$\Delta q_j(t)=\frac{1}{2}\Delta Q(\frac{{\underset{\_}{f}}_j(R(t))}{{\displaystyle {\sum }_{k=1}^{n_R}{\underset{\_}{f}}_k(R(t))}}+\frac{{\overline{f}}_j(R(t))}{{\displaystyle {\sum }_{k=1}^{n_R}{\overline{f}}_k(R(t))}})$

25

Experimental results

The following section describes experimental results of two different cases of motion plan with the AUV of Figure 1. In the experiments, the number of fuzzy rules is set to 8. The population of ACO and PSO was set to 60. The total iteration number was set as 150. Thus, the total evaluation trials per run were $60\times 150\text{=}9000$ . The real-time motion trajectories results were obtained from control. ²⁷

Case 1: Canal following for the wall detection

For the inland canal, the wall following is very important for the wall detection. Although AUV can carry out delicate detection on underwater target, it should cruise close to the wall of complicated contour under disturbances. Therefore, the motion plan should be made according to the detected wall contour and current distance between canal walls. In order to follow the canal wall, distant measurement sonars have been equipped on the AUV. In order to make comparisons, three kinds of motion planning algorithms including the common fuzzy rules-based motion plan of 3.1 and 3.2, PSO optimized fuzzy rules-based motion plan and adaptive ACO-PSO optimized fuzzy rules-based motion plan are used.

In Figure 5(a), the start point is (0, 0) and the end point is (−120, 1000). The expected distance between the AUV and the canal wall is 4 m. The adaptive ACO-PSO fuzzy planner can obtain the most reasonable trajectory for the canal detection. Since all the planned trajectories are obtained according to the AUV real-time state such as headings, speed, obstacle positions and disturbance, the planned trajectories are complicated curves. The motion control of AUV was realized according to the results of the adaptive ACO-PSO fuzzy planner. From Figure 5, the fuzzy planned motion cannot adapt to the environmental disturbance and the dynamic environment, the fixed rules usually lead to late and inflexible responses, thus the more complicated conditions for the disturbance and the turning of the wall the greater will be the errors. The PSO algorithm with fuzzy rules optimization embodies its advantage in the rule changing according to the real state, but the algorithm cannot converge very quickly at some local regions which cause the planning errors great at some time and even joggle at the place of complicated disturbance and turnings. While the adaptive ACO-PSO fuzzy planner is more adaptable for bend or angular intensively changing canals, because the ACO and reinforcement learning can help the algorithms with regional rules generation and quick convergence respectively.

Figure 5.

Comparisons on the motion plan of canal following. (a) Comparisons on the motion plan of wall following of the canal. (b) Heading plan comparisons of motion plan. (c) Distance of AUV from the canal wall. AUV: autonomous underwater vehicle.

Case 2: Moving target following plan

Moving target following is very important but difficult for AUV advanced detection. The AUV should keep correspondent distance with the same speed and heading angle under real-time disturbance. The target following experiments for motion plan have been made in the Penglai offshore of Chinese Shandong province. The oceanic environmental disturbances were obtained from DVL in Figure 6. In the experiments, the planned trajectories from adaptive ACO-PSO fuzzy planner are obtained according to the AUV real-time state such as headings, speed, target speed and positions and disturbance. The motion control of AUV was realized according to the results of the real-time adaptive ACO-PSO fuzzy planning. The ideal distance between the AUV and the mobile target for the target following is 8.5 m.

Figure 6.

Environmental disturbance obtained from DVL. DVL: Doppler Velocity Log.

From Figure 7, the AUV can realize mobile target following through motion plan of ACO-PSO fuzzy planning algorithm. The distance between the AUV and the mobile target is from 7.5 m to 10 m.

Figure 7.

Motion plan of target following. (a) Following trajectories. (b) Heading plan for target following. (c) Speed plan for target following. (d) Distance from the target during following.

Conclusions

The AUV can realize high-accuracy submarine detection and target following through intelligent path planning. This article has proposed a novel adaptive PSO and ACO fusion-based fuzzy rules optimization algorithm to realize intelligent motion planning. Through this algorithm, the preliminary fuzzy rules can be optimized through continuous selection, sampling, refinement and learning. Two typical cases of experiments including canal wall following and mobile target following have been made in order to verify the validity of intelligent path planning method to implement the complicated operation tasks.