Abstract
In this paper, online gait control system is designed for walking-up-stairs movement according to the features of humanoid robot, the hybrid evolutionary approach based on neural network optimized by particle swarm is employed for the offline training of the movement process, and the optimal gait of the stability is generated. Additionally, through embedded monocular vision, on-site environmental information is collected as neural network input, so necessary joint trajectory is output for the movement. Simulations and experiment testify the efficiency of the method.
Introduction
Mobility of multi-legged robot on uneven ground or other complex environment is higher than traditional wheeled robot. As its design is based on human features, the design of all complex motions of humanoid robot captures more and more attention. During the movement of humanoid robot, two issues need to be considered firstly, which are stability and real-time characteristic. As an important index to comment on stability, ZMP (Zero-Moment-Point) proposed by Jugoslavian Scholar Vukobratovic accelerates the development of the motion control of humanoid robot [1–3]. Ferreira[4] proposed the method based on human ZMP to the ZMP reference trajectory of the robot. The method that adds an inertial parameter from center of gravity to ZMP point can achieve different reference trajectories. Tomomichi[5] defined the stable area of standing to explain the stability assessment of posture balance controller for different grounds. Stability control for the grounds in different environments is available in this method.
Real-time walking control for robot becomes available through visual feedback. Robert studied the closed-loop iteration between bipedal walking and visual observation, and developed the monocular machine vision system [6], which achieved good control effect.
In the motion control of humanoid robot, inverse kinematics problem becomes the numerical solution problem of nonlinear transcendental equation. To fit in real-time control, robot control system usually needs real-time calculation of inverse kinematics, but iterative nature of conventional numerical solution makes the accuracy of the solution low or makes the solutions process spend too much time. Artificial neural networks have powerful ability of approximation to solve the nonlinear mapping problem, which can be used to solve a series of problems: manipulator kinematics, path optimization, robot control, etc. For the lack of real-time controller to control robot balance during dynamic walking, Jimmy Or et al.[7] proposed the mixed CPG-ZMP control system. It realized the real-time control for the walking of biped robot with different shapes of foot. Reza Ghorbani et al[8] studied the standing of biped robot. They simplified biped robot to the inverted pendulum with foot joints, and used general regression neural network feedback controller to control the vertical attitude of the robot. The genetic algorithm is used for the optimization of standing attitude. It achieved good performance according to the experiment. Ching-Long Shih analyzed the movement of walking up and down stairs for humanoid robot. Cubic spine function approximation of the track for the ankle is used for the trajectory planning of foot of the movement of walking up and down stairs. And there is the balance weight to control the transition between static balance and dynamic balance during the movement of walking up and down stairs [9].
BP neural network is employed in this paper, and nonlinear mapping of the robot from position variable space to joint variable space is realized by training for the sample generated from the forward kinematics. It overcomes the complicated process of formula deriving and programming calculation while calculating the inverse kinematics solution, and meets the real-time requirements of the control process.
While using neural network, it is not sure that the network structure whose neurons is fully connected can generate the optimal performance in the given training cycle. Because of the limitations in the ability of information processing, smaller network structure is also difficult toachieve good performance, while between the nodes of larger network, there are more connection redundancy[10]. The solution accuracy of conventional BP network can't meet robot's requirements in precise control. Evolutionary algorithm should be used to optimize the neural network in order to increase the accuracy of solution and speed of learning. In the document [11], Leung et al used the improved GA algorithm, and optimized the structure and weights of the neural network. The number of nodes in the implicit strata increased, until the neural network model with very good performances (such as fitness values) is achieved.
PSO Algorithm is brand-new evolutionary algorithm for global optimization, which is developed by Eberhart and Kennedy. As the simple and effective random search algorithm, its principle makes it win the bigger advantage than genetic algorithm and so on in the treatment of real-valued optimization problem. The parameter learning of neural network is just the multi-peak Real-Valued optimization problem, so PSO algorithm applies in neural network learning successfully[13].
Firstly, feasibility analysis is used for walking-up-stairs movement of humanoid robot, and then kinematics model is built. Two BP neural networks are used for the offline training respectively for the cycle of supporting with two legs and the cycle of supporting with one leg during walking-up-stairs movement of humanoid robot. Particle swarm optimization of natural selection is used for the weight optimization of neural network according to the objective function during the training, which makes neural network output the expected value quickly and accurately. At last, the feedback information from vision is used for as the network input information to execute the real-time gait control of the movement of walking up and down stairs.
The model of walking-up-stairs of humanoid robot
The movement of walking up and down stairs of humanoid robot is a kind of complex motion. We design the model of walking up and down stairs according to human movement of walking up and down stairs and the features of humanoid robot itself. Because the control method for walking up stairs and walking down stairs are similar, this paper focuses on walking up stairs. See Figure 1 for the geometric constraint model of walking up stairs.

Geometric constraint model of walking up stairs of humanoid robot
Humanoid robot has many degrees of joint freedom and different joint lengths. Not all humanoid robots fit in the movement of walking up stairs, so we make out the feasibility analysis of walking up stairs of humanoid robot.
Geometric constraints
In sagittal plane, the geometric constraints of walking up stairs of humanoid robot can be described by bidimensional geometric constraint[14]. From the fundamental theorem of computational geometry, we can see:
Given three points of
Stability constraints
Stability constraints of humanoid robot focuses on the issue that the friction force between robot feet and steps should be big enough and makes sure that robot will not slide during the movement. Main evidence of stability constraints is whether robot's ZMP is within the support range. According to the above fundamental theorem of computational geometry and with geometric model of walking up stairs of humanoid robot, the constraints of the cycle of supporting with two legs are achieved, just as Formula (1):
In which,
Cycle T of walking up stairs of humanoid robot is similar with walking on the ground, which can be divided to the cycle

Diagram of walking up stairs of humanoid robot
To makes sure that the robot can walk up stairs smoothly, we assume that the movement of the ankle joints of swinging leg makes the leg walk up stair vertically to the step. It is based on the above feasibility analysis and simplifies the model. Additionally, it not only makes sure the smooth movement of walking up stairs, but also reduces instability caused by the too high swinging height and reduces energy consumption. Just as Figure 2, we assume the motion cycle
In which,

Diagram of joint parameters of walking up stairs of humanoid robot
In which,
In the cycle of supporting with one leg, the robot can be regarded to the motion of multi-stage inverted pendulum[15]. Assume the weight of robot's connecting rod is even, so the coordinate (
In which,
In which,
Neural network design
According to the features of the control model for walking up stairs of humanoid robot, two three-forward neural networks are designed. Input layer of the first network is two parameters X1 and X2 for the cycle of supporting with two legs while robot walking up stairs; hidden layer is M neural units; output layer is the horizontal distances of L1 and L2 and vertical distances of H1 and H2 between robot's hip and the centers of both left leg and right leg. By different input X1 and X2, take

Three-forward neural network
When stable L1, L2, H1 and H2 are confirmed, the value of θ

Three-forward neural network
BP network has strict theoretic prove and approximation ability of any nonlinear function, but conventional BP algorithm has many problems such as long training time, slow training speed, easy to trap into local extremum, etc. The core principle of BP algorithm is gradient descent, so BP algorithm and the improved algorithm based on the gradient are both local search algorithms without the global search ability. Compared with BP algorithm, the strength of PSO training neural network is that it doesn't use gradient information. Additionally, it has better global convergence, but in the anaphase, convergence speed becomes slow. Therefore, the PSO algorithm based on selection principle is used to train the weight of neural network.
For given neural network structure, just code the connection weight, and map it as the individual expressed by the string code. At the same time, assess each series of weight by fitness function and calculate the fitness value, and then the training issue of neural network becomes optimization problem to find the optimal connecting weight with maximum fitness function value.
SPSOBP combines natural selection principle and PSO algorithm. Sort the whole particle swarm by the fitness value in every iteration. Substitute position and velocity of the worse half of the particle with the other half, and retain the historical optimal value remembered by each individual. Specific process is just as follows:
Select proper unit in hidden layer and initialize neural network; Carry out vector coding for particles in the swarm, which is composed of Initialize the position Map each particle as the network weight, and form the network. Randomly select samples from the sample space and form the sample set for the training. Assess each particle, and calculate the fitness value Update the velocity For each particle, compare the fitness value and the experienced best position. If it is better, then take it as the existing best position. Compare all existing pbest values and gbest values, and update gbest values. Sort the whole particle swarm by the fitness value. Substitute position and velocity of the worse half the particle with the other half, and retain pbest and gbest. If the stopping requirements are met, then stop searching. As the optimized results, output weight is used by neural network; otherwise, the velocity and position of each particle would be updated. Return to Step (4) to continue the search.
Real-time control based on embedded vision
While walking up stairs, the foot positioning for the humanoid robot should be very precise, but the positioning is mainly generated by the vision. According to the kinematic model in this paper, information such as X1, X2, step height and width should be collected, so online real-time control by the on-site real information can be realized. See Figure 6 for the principle of monocular vision positioning: on-site environmental image information and robot's vision screen information can be linked just by the simple geometric Formula (11).

The geometric diagram of monocular vision positioning
Embedded monocular vision is used according to the HSV color space. Through offline data collection, clustering and training, online identification requirements are met. After the treatment of decision-making system, the collected environmental information uses control system to realize the robot real-time gait control. Specific vision treatment and control process is realized in the document [16]. Space lacks for a detailed description of it. The control process is just as follows:
Collect, cluster and train image information offline. Use neural network control system for the offline training and study of the gait. Collect image information online, and provide it to the decision-making system. According to the treatment results of image information, the decision-making system uses motion control system for the robot real-time gait control. Collect images online, and feed back the information to the decision-making system. If the goal is reached, then stop, or turn to Step (3).
In case of offline, use Matlab to simulate the control method in this paper, and make the experiment on real robot MOS2007. Parameters of the humanoid robot in the simulation are
See Figure (7–13) for the simulation results, and see Figure 14 for experimental screenshot. Figure 7 is BP network training diagram, and Figure 8 is the training diagram of weight optimized by SPSO. The training uses the data from 1200 sets of experiments, and training error is set 10−9. From the figures we can see that the decrement speed of BP network is quicker than SPSOBP at the beginning, but it will not decrease with the training times when the precision reaches to a certain degree. After about 1700 times of training, SPSOBP converges to the regulated precision. From the simulation we can see that the method of SPSOBP neural network has better effect to control the gait. Figure 9 and Figure 10 show the trajectories of H1, H2, L1 and L2 while robot walking up stairs and the cycle is 3 seconds. Figure 11 shows the trajectories of hip joint and swinging ankle joint. Figure 12 shows the trajectory of joint θ

Training process of BP neural network

Training process of SPSOBP-based neural network

Trajectories of H1 and H2

Trajectories of L1 and L2

Trajectories of hz, hx, fz and fx

The trajectories of joint angle θ


Screenshot of the process of walking up and down stairs of robot MOS2007
But it costs too much time, so it can't be applied to the real-time control. The control method proposed in this paper can train the gait of walking up stairs offline. The usage of SPSOBP network training weight for the control can dramatically decrease the training time and meet the requirements of real-time system control.
The feasibility analysis of walking up stairs is carried out in this paper by the features of humanoid robot, and the kinematics model of the gait cycle is established. Input and output of original position and joint trajectory which are necessary for the control of walking up and down stairs are realized by two neural networks. Selection-based PSO algorithm is used to optimize the weight of BP neural network. Gait trajectory with the best ZMP stability is achieved by Offline training. The process of walking up stairs is controlled online by the method that embedded monocular vision collects the on-site environmental information. Issues that should be solved in the future include the real-time gait control on the irregular ground and obstacle-space-based navigation for humanoid robot.
