Abstract
In this study, an efficient navigation control method of mobile robot is proposed. The proposed navigation control method consists of behavior manager, toward goal behavior, and wall-following behavior. According to the relative position between the mobile robot and the environment, the behavior manager switches to determine toward goal behavior or wall-following behavior of mobile robot. A novel recurrent fuzzy cerebellar model articulation controller based on an improved dynamic artificial bee colony is proposed for performing wall-following control of mobile robot. The proposed improved dynamic artificial bee colony algorithm uses the sharing mechanism and the dynamic identity update to improve the performance of optimization. A reinforcement learning method is adopted to train the wall-following control of mobile robot. Experimental results show that the proposed method obtains a better navigation control than other methods in unknown environment.
Keywords
Introduction
In recent years, mobile robot has become one of the popular researches. It is widely used in many applications to assist people solving many problems, such as environmental exploration, suspicious objects removal, object handling, and navigation.1–3 In order to achieve these objectives, the popular research topics of the mobile robot control include the obstacle avoidance, wall-following, navigation, and target tracking.
Recently, many researchers have successfully used various methods4,5 on mobile robot control. While keeping a constant distance to the wall, Turennout et al. 4 solved the WF control problem by moving the mobile robot along a wall in a desired direction. Based on the sensorial information from two infra-red (IR) sensors, Gavrilut and Tiponut 5 proposed a local navigation method for mobile robot. Together with the advantages of simple and low-cost sensing modalities, the IR sensors successfully resolve navigation tasks in environments with obstacles or WF problems. The external noise interference and sensitivity of sensing module hinder the traditional control methods4,5 for stable control of mobile robot.
To solve the above-mentioned problems, the fuzzy logic control6–9 has been successfully applied on mobile robot control. Fuzzy logic is proposed by Zadeh 6 in 1965, which can be used on processing the situations in imprecision, ambiguity, multiple definitions, or uncertainty. A fuzzy controller consists of four parts: fuzzification, rule base, fuzzy inference engine, and defuzzification. Thongchai et al. 7 described a sonar-based HelpMate robot using fuzzy control. The proposed fuzzy controller combines different information from all sonar sensors as a sensor mechanism. A novel fuzzy logic-based navigation algorithm was presented to effectively model correct environment and efficiently collect the noisy and uncertain sensory data with low-cost hardware equipment. 8 Al-Sahib and Ahmed 9 proposed a fuzzy logic controller by means of fuzzy decision making method. The controller detects obstacles and leads on the mobile robot from the origin to the destination. Although fuzzy control has good performance on WF, the disadvantage is that the rule must be designed by experts. In addition, Alitappeh et al. 10 proposed a multi-objective optimization method to deploy/redeploy robots in the environment by considering two objectives for the deployment. To deal with the deployment problem, a discrete setup of locational optimization framework and Voronoi partitioning technique were employed. Morales et al. 11 applied support vector machines (SVM) to the path planning problem. The entire obstacle course is divided into two classes of data sets and a separating class boundary is obtained using SVM. This non-linear class boundary line determines the heading of the robot for a collision-free path. Pandey and Parhi 12 used the minimum rule-based adaptive neuro-fuzzy inference system (ANFIS) controller for the safe navigation of single and multiple mobile robots in the cluttered environment using the sensor-based steering angle control technique.
An evolutionary robotic (ER) becomes popular increasingly.13–15 A reinforcement ant optimized fuzzy controller (RAOFC) method, proposed by Juang and Hsu, 13 was designed for controlling mobile robot WF under reinforcement learning environments. A multi-objective, rule-coded, advanced, continuous-ant-colony optimization (MO-RACACO) algorithm 14 was proposed for fuzzy controller (FC) design and was applied on multi-objective WF control for a mobile robot. Based on interval type-2 fuzzy controller (IT2FC), Hsu and Juang proposed an evolutionary WF control for mobile robot and defined a cost function to assess the WF performance of an evolutionary IT2FC. 15 Contreras-Cruz et al. 16 combined the artificial bee colony (ABC) algorithm as a local search procedure and the evolutionary programming algorithm to refine the feasible path found by a set of local procedures for solving the mobile robot path planning problem. According to the above-mentioned methods, these learning approaches avoid the exhaustive collection of supervised input–output training pairs and saves execution time.
The main purpose of the navigation control is that the mobile robot can effectively explore in unknown environment and reach the designated target successfully. The above-mentioned methods can effectively implement WF control, but some drawbacks exist during the navigation control, such as easily falling into a dead zone. In order to escape the dead zone, the mobile robot needs to rely on other mechanisms. The mobile robot requires at least two control behaviors to switch in the unknown environment. Therefore, this study set the behavior manager (BM) to switch corresponding behaviors according to the relative position between the mobile robot and the environment. Two behavior patterns are proposed as follows: (1) toward goal (TG) behavior and (2) WF behavior. If mobile robot encounters obstacles or wall on the way, it uses the WF behavior to avoid collision. The proposed two behaviors can avoid the dead zone during the mobile robot mining the unknown environment.
This study proposes a recurrent fuzzy cerebellar model articulation controller (RFCMAC) based on the improved dynamic artificial bee colony (IDABC) algorithm to achieve mobile robot WF control. The sensor module of the mobile robot is used to capture environment information as the inputs of the proposed RFCMAC controller. In addition, the proposed IDABC is used to develop the mobile robot autonomous controller. Experimental results show the effectiveness of the proposed navigation control method.
The proposed RFCMAC model
Figure 1 shows the proposed six-layer RFCMAC structure.17,18 The Takagi–Sugeno–Kang-type (TSK-type) RFCMAC model realizes a similar fuzzy IF–THEN rule (hypercube cell) as follows.
Rule j: IF
where
Layer 1. The inputs are crisp values and any operation is not performed. Each node directly transmits input values to the next layer
Layer 2. Each fuzzy set
where
Layer 3. Each node receives one-dimensional membership degree of the related rule from the nodes of a set in layer 2 and executes an algebraic product operation as follows
where ∏ denotes the product operator, ND represents the number of the receptive filed functions for each input state, and
Layer 4. A recurrent rule node consists of an internal feedback and an external interaction feedback loop in layer 4. A temporal firing strength depends on both the current spatial and the previous temporal firing strengths to represent the output of a recurrent rule node. Equation (4) presents the temporal firing strength as a linear combination function as follows
where
Layer 5. A function of linear combination of input variables represents an optional node in layer 5. The value of its corresponding association output of interaction feedback loop was used as matching degree to generate a partial fuzzy output for each linear combination. The formula expressed as
where
Layer 6. In order to obtain the actual output y, the centroid of area (COA) is used to convert the fuzzy output into a scalar output
where NL represents the number of the hypercube cells.

Structure of the RFCMAC model.
Proposed IDABC
In this section, the proposed IDABC is used to adjust the parameters of RFCMAC model effectively. Each bee represents all control parameters of RFCMAC, which is shown in Figure 2. A fuzzy hypercube cell contains the center

Encoding of a bee.
The ABC algorithm was proposed for parameter optimization which simulates the foraging behavior of bee colony. 19 It consists of employed bees, onlooker bees, and scout bees. Three kinds of bees cooperate with each other to find the food source. This article proposes the modifications of ABC to enhance the performance of optimization, including best information sharing mechanism and dynamic identity update. Figure 3 shows the flowchart of the proposed IDABC algorithm. The detailed descriptions are introduced as follows:
Initialization phase. Each bee represents a problem solution and is initialized randomly from the solution space.
Fitness evaluation phase. The fitness value of each bee is calculated and represents the corresponding performance.
Employed bees phase. In traditional ABC, employed bees look for new food sources and adopt random search in their neighborhood. To enhance the convergence speed, the information of the best solution is added to the searching direction. Search formula is described as follows
Onlooker bees phase. Onlooker bees choose a new food source according to the probability value associated with that fitness value of the employed bee. The roulette wheel is used to ballot the employed bee k.
Scout bees phase. When employed or onlooker bees are not updated during a specific period of time, the scout bees adopt a random search to find new food source
Bees identity update phase. In traditional ABC, the proportions of employed bee and onlooker bee are equal. This property leads the employed bees searching direction to the colony convergence over speed. Therefore, this study adopts the bee identity update method to become employed or onlooker according to the last searching result

The flowchart of the proposed IDABC algorithm.
Navigation control of a mobile robot
Mobile robot description
The mobile robot of Pioneer 3-DX is shown in Figure 4. The characteristics of the mobile robot are high payload, high endurance, strong extensibility, and support cross-platform Software Development Kit (SDK). The SDK consists of the motion control, the client-server architecture, and the various accessories. Therefore, the mobile robot allows users to integrate all peripheral equipment and fulfill research goals.

Pioneer 3-DX mobile robot and arrangement of sonar sensors.
The Pioneer 3-DX mobile robot has two-wheel drive with the vehicle of a personal computer (PC) and is fully autonomous intelligence. The Advanced Robot Control and Operations Software (ARCOS)-based robot supports four sonar arrays, each with up to eight sonar sensors that provide obstacle detection and surrounding information for collision avoidance.
WF behavior
First, the training environment is set to 11 m × 8 m and is shown in Figure 5. It consists of the straight line, the bevel-angle, and the right-angle problems. Figure 6 shows the control diagram of the mobile robot using a RFCMAC controller with IDABC learning algorithm.

Training environment of wall-following control.

Block diagram of mobile robot wall-following control.
In the training process, the mobile robot WF control includes three terminal conditions. In the evolutionary process, when the terminal condition occurs, the number of evolution counts and the robot returns to the initial point of the training environment and re-runs again. The evolutionary process is not terminated until the mobile robot successfully takes a turn round in an unknown environment. Three terminal conditions are described as follows:
Mobile robot collides with the wall. For example, when the distance from the sonar sensor is less than 0.1 m.
The robot is far away from the wall. For example, when the distance
The mobile robot can successfully take a turn round in an unknown environment.
The objective function of a mobile robot controller depends on an unknown environment. In this study, the assessment factors consist of four objective functions
(a) Objective function
If the actual moving distance of mobile robot is larger than the pre-defined walking maximum distance
(b) Objective function
where
(c) Objective function
where θ is an angle between
(d) Objective function

Illustration of the
If the average moving speed is greater than the pre-defined speed, set
The fitness function
In the training process, the goal is the mobile robot can keep a fixed distance to the wall and take a turn round in an unknown environment. Thus, CF1 and CF2 are more important than CF3 and CF4 for performing the mobile robot WF control. Therefore, the weighting coefficients of control factors are set as
TG behavior
In TG behavior, the mobile robot needs to compute the angle with the mobile robot and the goal
where

Illustration of the goal angle
BM
The BM switches TG behavior and WF behavior according to the relative position between the mobile robot and environment. First, we divide the mobile robot into four regions (

Division of the mobile robot into four regions.
If the mobile robot detects obstacles
Experimental results
In this section, a RFCMAC based on an IDABC is proposed for controlling the mobile robot WF in an unknown environment. Four testing environments are generated to verify the proposed method for navigation control.
WF learning
The proposed IDABC algorithm is used to train the mobile robot WF control. Table 1 shows the initial parameters of the proposed algorithm. Figure 10 presents the moving path of mobile robot by the proposed RFCMAC model with IDABC learning algorithm.
Initial parameters of the proposed algorithm.

The moving path of mobile robot using the proposed algorithm in training environment.
We also compare the proposed IDABC algorithm with particle swarm optimization (PSO), 20 quantum-behaved particle swarm optimization (QPSO), 21 differential evolution (DE), 22 adaptive DE with optional external archive (JADE), 23 and traditional ABC. 19 In all, 10 replications are run in this experiment. The performance measurements include the best fitness value, the worst fitness value, the average fitness value, the STD of fitness value, and the number of success runs. The STD of fitness value represents the square root of the average of the squared deviations of each fitness values from the mean fitness value. The number of success runs represents how many times mobile robot can detour around the training environment successfully in 10 replications. Comparison results are shown in Table 2. The proposed IDABC has a better fitness value than other methods19–23 under the same conditions. Moreover, the STD of the proposed IDABC is the smallest, which represents that the proposed algorithm is more stable during learning process.
Comparison results of various evolutionary methods for the mobile robot wall-following control.
PSO: particle swarm optimization; QPSO: quantum-behaved particle swarm optimization; DE: differential evolution; ABC: artificial bee colony; STD: standard deviation.
Navigation control of mobile robot
The navigation control of mobile robot uses the BM to determine TG behavior or WF behavior of mobile robot according to the relative position between the mobile robot and the environment. Four different environments are used for testing the proposed navigation control method. Figure 11 shows the trajectory of a mobile robot in four different unknown environments. In Figure 11(a), the path of blue arrow represents the mobile robot executing the TG behavior, whereas the others represent the mobile robot executing the WF behavior. Experimental results demonstrate that the proposed method successfully implements the mobile robot navigation control in unknown environments.

Navigation control results using the proposed approach in four testing environments.
In order to verify the performance of the proposed navigation control method, two evaluation indices are used. One index is the total moving distance of mobile robot. If the mobile robot has a better WF behavior, the total moving distance is reduced effectively. The other index is the time spending of mobile robot for completing the navigation control. That is to evaluate the moving speed of the mobile robot in an unknown environment. The detailed best performance analysis is shown in Table 3. In this table, the proposed navigation control method is better than other methods19–23 in the total moving distance and the time spending.
The best performance analysis of various evolutionary methods in navigation control.
PSO: particle swarm optimization; QPSO: quantum-behaved particle swarm optimization; DE: differential evolution; ABC: artificial bee colony; STD: standard deviation; JADE: adaptive DE with optional external archive.
Conclusion
This study proposes a novel RFCMAC based on IDABC learning algorithm for the mobile robot navigation control. The proposed navigation control method consists of BM, TG behavior, and WF behavior. According to the relative position between the mobile robot and the environment, the BM switches to determine TG behavior or WF behavior of mobile robot. The proposed IDABC algorithm uses the sharing mechanism and the dynamic identity update to improve the performance of optimization. The advantages of the proposed method include the global search ability, the convergent speed, and the solution accuracy. Thus, the proposed method prevents falling into a local solution. Experiment results show the effectiveness and stability of the proposed navigation control method.
Footnotes
Academic Editor: Stephen D Prior
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This paper was sponsored by the Program of Tianjin Science and Technology Commissioner of China (no.16JCTPJC50700).
