Bipedal walking pattern generation and control for humanoid robot with bivariate stability margin optimization

Abstract

This article concentrates on the problem of walking pattern generation and online control for humanoid robot. However, it is challenging and thus still remains open so far in the field of bipedal locomotion control. In this article, we solve this problem by proposing a bivariate-stability-margin-based control scheme, in which a random vector function-link neural networks mechanism is additionally contained. By utilizing opposition-based learning algorithm to generate walking patterns and designing random vector function-link neural networks for compensating the combination of zero-moment point error and modeling error, the new walking controller exhibits good performance. Moreover, a bivariate-stability-margin-based fuzzy logic system is proposed to assign a weight to each training sample according to locomotion stability. With these results, a walking control system is successfully established and experiments validate the proposed control scheme.

Introduction

Research of walking control for humanoid robot has collected a great deal of attention over the past decades.^1–8 Roughly, these approaches can be classified into two groups, namely, model-based method and stability-criterion-based method. The former considers the reconstruction of robot configuration while the latter is directly concentrated on the locomotion stability criteria. Kajita et al. ⁴ defined an inverted pendulum model and proposed a walking control method for biped robot with an energy conserving orbit. Ha and Choi ⁵ presented a walking pattern generation method such that the deviations from the desired trajectories can be eliminated with the use of the virtual height inverted pendulum model. In Kajita et al., ⁶ a biped walking pattern generation approach was developed based on cart-table model using preview control. Moreover, Sun et al. ⁷ further proposed an extended cart-table model, and an excellent locomotion stability was guaranteed. Note that the methods based on inverted pendulum model ignore the inertia moment of robot and suffer from an inaccurate dynamics model. Bessonnet et al. ⁸ raised a parametric walking approach based on connective-link model with the use of spline functions. In addition, Fu et al. directly established a gait planning system so that stable gaits were achieved.

To improve locomotion stability, stability-criteria-based methods were proposed. Zero-moment point (ZMP) is one of the most popular locomotion stability criteria, which was first proposed by Vukobratović and Borovac. ⁹ Some interesting results have been reported in the literature.^10–12 In Lee et al., ¹² a modified walking pattern generation algorithm was developed by allowing a variation of ZMP over the convex hull of foot sole. Subsequently, by modifying the foot landing position, Nishiwaki and Kagami ¹³ developed a walking control system to reinforce the control performance. In Erbatur and Kurt, ¹⁴ a ZMP-criterion-based approach was proposed to generate natural gaits with Fourier series approximation technique. In Shin and Kim, ¹⁵ an online gait synthesis algorithm was proposed using the allowable ZMP region, which extended traditional fixed ZMP concept and exhibited a good performance. Moreover, Hu et al. ¹⁶ have fused iterative learning control with k-NN regression algorithm to enhance the robustness performance. By combining polynomial regression technique with principal component analysis, an online walking pattern generation algorithm with a low number of parameters was further proposed in Gasparri et al. ¹⁷ Despite these contributions, the methods established in the literature^13–15 have a common assumption that yaw moment can be ignored during robot walks. However, this assumption is not easy to be met in practical application. As pointed out in the literature,^18–20 yaw moment would inevitably exist due to the fact that the components of robot don’t move in the same plane. Usually, the moment equilibrium in vertical direction holds on for the reason that yaw moment can be counteracted by the ground reaction force when the walking speed is slow enough. Yet, slippage may happen when the ground reaction moment failed to compensate the yaw moment.

To cope with the yaw moment compensation problem of humanoid robot, some pioneering works have been reported in the literature.^21–24 In Yu et al., ²¹ a waist-rotation-based method was proposed to counteract the undesired yaw moment. As an extended research, Ugurlu et al. ²² further developed a yaw moment control scheme using Eulerian ZMP resolution such that rotational inertia of robot is considered. Subsequently, Xing and Su ²³ presented a motion generation method to compensate for the yaw moment, by utilizing trunk spinning and arms swinging. However, the controllers constructed in the literature^21–23 commonly ignored the high energy consumption problem, which is a key difficulty in practical application and may result in the relative short battery life. Until now, it is still a challenging problem to maintain the dynamic balance in both horizontal and vertical directions with energy efficiency.

Inspired by this observation, we are dedicated to deal with the control problem of humanoid robot. The control objective is to control the multi-degrees-of-freedom (DoF) motion of biped robot such that the yaw moment can be effectively counteracted while ZMP trajectory is kept within the allowable ZMP region. The novelties of this work are characterized as follows:

1. Differs from those schemes in the literature,^21–23 the proposed one in this article provides a systematic control scheme, in which multi-joint coordinated control approach is included and a random vector function-link neural networks (RVFLNNs) mechanism is newly designed. By approximating the nonlinear mapping function between ZMP and joints positions/velocities, ZMP error can be compensated online.

2. A weighted bivariate-stability-margin optimization algorithm based on the fact that walking samples with good locomotion stability and energy efficiency need to be emphasized is proposed to deal with the uncertainties of training data.

Finally, we present the constructed the walking control system for humanoid robot. The walking experiments have been performed to validate our proposed scheme.

System description and some preliminaries

Overview of the humanoid robot BRZ-4

The task of bipedal locomotion is carried out by the humanoid robot “BRZ-4,” which contains leg links connected by 10 motorized joints as displayed in Figure 1. BRZ-4 is a half-sized humanoid robot designed and constructed at Zhongshan Institute, University of Electronic Science and Technology of China, Zhongshan, China.

Figure 1.

Three-dimensional model for BRZ-4.

To facilitate the analysis, BRZ-4 is simplified as connective-link-model. Specially, thigh, shank, and trunk are simplified as connective links. As illustrated in Figure 1, BRZ-4 is 61.5 cm in height and weights about 1.436 kg. Specifically, there are five degrees in each leg, where $q_i$ represents the ith joint angle; $l_{\mathrm{sh}}$ and $l_{\mathrm{th}}$ denote the length of shank and thigh, respectively; $l_w$ denotes the distance between two hip joints; $l_a$ is the distance from foot sole to ankle joint; and $l_b$ represents the distance of CoM position and the center of two hip joints. The specific parameters are listed in Table 1. Moreover, the robot is equipped with an embedded control board and a universal asynchronous receiver–transmitter (UART) interface. Thus, specific tasks can be executed remotely.

Table 1.

The configuration of BRZ-4.

Link	Trunk	Thigh	Shank	Foot
Length (m)	0.13	0.225	0.175	0.05
Mass (kg)	0.67	0.144	0.156	0.083

Analysis of locomotion stability and bivariate stability margin

Foot sole is the only contact surface between humanoid robot and the ground during walking. Thus, the ground reaction force is a key factor to maintain the dynamic balance. As illustrated in Figure 2, $F_A$ and $T_A$ denote the moment and force acting on ankle joint by the rest parts of robot above ankle joint. In this figure, m is the mass of the supporting foot and point G is the center of the mass of the supporting foot.

Figure 2.

Forces acting on the supporting foot.

To achieve stable bipedal locomotion of humanoid robot, two aspects need to be considered, which are, respectively, horizontal moment balance and vertical moment balance. To maintain the horizontal moment equilibrium, ZMP criterion has been widely used. According to the literature,^9,25,26 ZMP is a point $P_{\mathrm{zmp}}=(x_{\mathrm{zmp}},y_{\mathrm{zmp}},0)$ on the sole, in which the horizontal components of the net moment $T_{\mathrm{zmp}}=(T_x,T_y,T_z)$ of gravity and inertial forces equal to zero, as displayed in Figure 2. Thus, the following equation holds

$$\begin{array}{c}{T}_x=T_y=0\hfill \end{array}$$

(1)

where $T_{\mathrm{zmp}}$ is the net balance moment of gravity and inertial forces acting on ZMP; $T_x$ and $T_y$ denote the horizontal components of $T_{\mathrm{zmp}}$ . According to Yang et al., ²⁴ ZMP position can be computed by the following equations

$$\{\begin{array}{c}\begin{array}{c}{x}_{\mathrm{zmp}}=x_c-\frac{z_c}{{\stackrel{··}{z}}_c+g}{\stackrel{··}{x}}_c\hfill \\ \multicolumn{1}{c}{y_{\mathrm{zmp}}=y_c-\frac{z_c}{{\stackrel{··}{z}}_c+g}{\stackrel{··}{y}}_c}\end{array}\hfill \end{array}$$

(2)

where $(x_{\mathrm{zmp}},y_{\mathrm{zmp}})$ denotes the coordinate of ZMP; g is gravity acceleration; $z_c$ and ${\stackrel{··}{z}}_c$ are, respectively, the position and acceleration of the center of the mass of robot. The yaw moment $T_z$ is defined as below ²⁰

$$\begin{array}{c}{T}_z=\sum _{i=1}^n{m}_i{\stackrel{··}{y}}_i(x_i-x_{\mathrm{zmp}})-\sum _{i=1}^n{m}_i{\stackrel{··}{x}}_i(y_i-y_{\mathrm{zmp}})\hfill \end{array}$$

(3)

where $x_i$ and $y_i$ denotes the x-axis and y-axis positions of the center of the ith link, $m_i$ represents the mass of the ith link. To evaluate the locomotion stability during robot walks, ZMP stability margin and yaw moment stability margin are designed as follows, which were first proposed in Yang et al. ²⁷

$$F_{\mathrm{zmp}}=\{\begin{array}{c}\begin{array}{cc}\frac{1}{2}\frac{r_{\mathrm{zx}}}{r_{\mathrm{cx}}}+\frac{1}{2}\frac{r_{\mathrm{zy}}}{r_{\mathrm{cy}}}\hfill & \mathrm{if}{p}_{\mathrm{zmp}}\in {\Omega }_{\mathrm{zmp}}\hfill \\ \multicolumn{1}{c}{0}& \mathrm{if}{p}_{\mathrm{zmp}}\notin {\Omega }_{\mathrm{zmp}}\hfill \end{array}\hfill \end{array}$$

(4)

$$F_{\mathrm{yaw}}=\{\begin{array}{c}\begin{array}{cc}1-\frac{|T_z|}{|T_{\mathrm{fr}}^{\max }|}\hfill & \mathrm{if}|T_z|<|T_{\mathrm{fr}}^{\max }|\hfill \\ \multicolumn{1}{c}{0}& \mathrm{if}|T_z|\ge |T_{\mathrm{fr}}^{\max }|\hfill \end{array}\hfill \end{array}$$

(5)

where ${\Omega }_{\mathrm{zmp}}$ is the contact zone between sole and the ground; $r_{\mathrm{zx}}$ and $r_{\mathrm{zy}}$ are, respectively, the distances between ZMP and boundaries along x-axis and y-axis. Specially, $l_f$ and $w_f$ are the length and width of the sole, $r_{\mathrm{zx}}$ and $r_{\mathrm{zy}}$ are given as $r_{\mathrm{cx}}=0.5l_f$ and $r_{\mathrm{cy}}=0.5w_f$ , and $T_{\mathrm{fr}}^{\max }$ denotes the maximum static friction moment. Utilizing the Lagrangian approach, the dynamics equation of humanoid robot is expressed as follow

$$\begin{array}{c}{\tau }_i={\mathbb{H}}_i(q_i){\stackrel{··}{q}}_i+(\frac{1}{2}{\stackrel{·}{\mathbb{H}}}_i(q_i)+{\mathbb{C}}_i(q_i,{\stackrel{·}{q}}_i)){\stackrel{·}{q}}_i+{\mathbb{G}}_i(q_i)\hfill \end{array}$$

(6)

where ${\mathbb{H}}_i(q_i)\in {\mathbb{R}}^{n\times n}$ denotes positive inertial matrix, $(1/2){\stackrel{·}{\mathbb{H}}}_i(q_i)+\mathbb{C}(q_i,{\stackrel{·}{q}}_i)\in {\mathbb{R}}^{n\times n}$ is Coriolis and centrifugal matrix, $G_i(q_i)$ represents the gravitational force vector, and ${\tau }_i$ is the designed control torque acting on the ith joint.

Walking control system design

One of the control objectives is to achieve stable bipedal locomotion. The main difficulties in the implementation of the proposed control scheme arise from two aspects: (1) modeling uncertainties and (2) maintaining the moment equilibrium in both horizontal and vertical directions. Compared with single-layer feedforward neural networks, RVFLNN provides an excellent generalization capability with fast learning property in classification and regression. In this article, an RVFLNNs mechanism is designed to compensate the combination of ZMP error and modeling error. Moreover, a bivariate-stability-margin-based optimized approach is integrated in the proposed RVFLNNs mechanism to deal with the data uncertainties, in which the samples with better stability performance are emphasized. Visual illustration of above control design is given in Figure 3.

Figure 3.

Block diagram of the proposed control scheme.

Gait plan and optimization

To generate smooth and stable gaits, a spline-based parametric technique is adopted. The main idea is to approximate the coordinates of locomotion utilizing spline function fitted at key walking states uniformly distributed along the whole motion. Then, parametric optimization approach is designed to improve the locomotion stability. The procedure of parametric modeling of hip trajectory is listed as follows. At first, we define several parameters to describe hip joint position at specific motion time, as illustrated in Figure 4.

Figure 4.

Kinematic parameters.

By analyzing the relationship between hip joint and ankle joint, we have

$$x_{\mathrm{hip}}=\{\begin{array}{c}\begin{array}{cc}{L}_{\mathrm{ae}}+d_{\mathrm{sb}}\hfill & t\in T_s\hfill \\ \multicolumn{1}{c}{L_{\mathrm{ae}}+\frac{D_s}{2}-d_{\mathrm{sf}}}& t\in T_p\hfill \end{array}\hfill \end{array}$$

(7)

where $T_s$ and $T_p$ denote supporting and swinging phases, respectively; $L_{\mathrm{ae}}$ represents the distance between ankle joint and heel along x-axis; $D_s$ is step length; $d_{\mathrm{sb}}$ and $d_{\mathrm{sf}}$ are the distances between hip joint and ankle joint at supporting phase and swinging phase, respectively. Then, the hip joint position is expressed as below

$$x_{\mathrm{hip}}(t)=G_{\mathrm{xhip}}(L_{\mathrm{ae}},d_{\mathrm{sb}},d_{\mathrm{sf}},D_s)$$

(8)

Similarly, the trajectories of knee and ankle joints can be obtained by the above steps. Till now, the procedure of gait planning is finished. However, it is usually not enough to generate stable gaits. Optimization method needs to be employed to find optimal parameters. To improve the locomotion stability, the following multi-objective optimization functions are established as follows

$$\begin{array}{c}\{\begin{array}{c}\begin{array}{c}\min F_{\mathrm{zmp}}+\beta F_{\mathrm{yaw}}\hfill \\ \multicolumn{1}{c}{s.t.g_i(x_i)\le 0,i=1,2,\dots ,m}\end{array}\hfill \end{array}\hfill \end{array}$$

(9)

where $g_i(x_i)$ denotes the ith physical restriction, m is the total number of constrains, and $\beta$ is a scale constant.

This is a typical quadratic programming problem. To find the optimal solution, an opposition-based particle swarm optimization method is adopted, which was proposed in Yang et al. ²⁷ For the limitation of space, the specific implementation is omitted. For the details, please refer to Yang et al. ²⁷

Bivariate-stability-margin-based RVFLNNs design

Obviously, ZMP error would exist due to modeling uncertainties and disturbances from environment. Equation (2) implies that ZMP is related to center of mass (CoM) position. In other words, ZMP can be modified by adjusting CoM position. However, the mapping between ZMP and CoM is not easily linearized. In addition, both ZMP and yaw moment are related to the position, velocity, and acceleration of each joint. How to compensate ZMP error by controlling robot is a difficulty we need to tackle. From the aforementioned observation, we have

$$\Delta \mathbf{q}=\mathfrak{F}(\mathbf{q},\stackrel{·}{\mathbf{q}},\stackrel{··}{\mathbf{q}},\Delta x_{\mathrm{zmp}},\Delta y_{\mathrm{zmp}},T_z)$$

(10)

where $\mathfrak{F}(•)$ is a nonlinear mapping function, $\Delta \mathbf{q}=(\Delta q_{\mathrm{hip}},\Delta q_{\mathrm{knee}},\Delta q_{\mathrm{ankle}})$ , $(\Delta x_{\mathrm{zmp}},\Delta y_{\mathrm{zmp}})$ denotes ZMP error. To approximate the nonlinear mapping function equation (10) and eliminate ZMP error, RVFLNNs are employed due to their outstanding generalization ability. According to the literature,^28–30 RVFLNNs can be formulated as

$$\mathbb{Y}=[\mathrm{X}|\mathrm{H}]\mathrm{G}$$

(11)

where $\mathrm{X}$ is input vector; $\mathrm{H}$ represents enhanced components; $\mathbb{Y}$ is output matrix, and $\mathrm{G}$ is weight vector connecting input layer and output layer.

Compared with traditional single-layer feedforward neural networks, in our algorithm, the weight matrix is randomly initiated with the proper dimensions. The input data set is given as $\mathrm{X}=(\mathbf{q},\stackrel{·}{\mathbf{q}},\stackrel{··}{\mathbf{q}},\Delta x_{\mathrm{zmp}},\Delta y_{\mathrm{zmp}},T_z)$ . And, the enhanced components can be computed by the following equation

$$\mathrm{H}=\phi (\mathrm{X}{\mathrm{W}}_e+\delta )$$

(12)

Moreover, in supervised learning tasks, the control performance usually suffers from data uncertainties and imbalanced data distribution. To remove this obstacle, we assign a weight to each sample according to bivariate stability margin, which are ZMP stability margin and yaw moment margin. Based on expert knowledge, the weight of each sample is assigned using fuzzy logic system (FLS). Based on the given expert knowledge, the FLSs rules are given as

$$\mathrm{Rule}i:\mathrm{If}x\mathrm{is}{\tilde{A}}_{i,1},y\mathrm{is}{\tilde{A}}_{i,2},\mathrm{then}w\mathrm{is}{\tilde{G}}_i$$

where x and y denote ZMP stability margin and yaw moment margin, respectively; ${\tilde{A}}_{i,j}$ and ${\tilde{G}}_i$ are the linguistic variables of the fuzzy sets; $i=1,2,\dots ,M$ . M represents the total number of the fuzzy rules. By utilizing fuzzy logic inference, the weight of each training sample is deduced. For the limitation of space, this part is omitted. For the details, please refer to Fu et al. ³¹ Hence, the approximation task can be reformulated as

$$\begin{array}{c}\{\begin{array}{c}\begin{array}{c}\min \frac{1}{2}\parallel \mathrm{G}{\parallel }_2^2+\lambda \frac{1}{2}\sum _{i=1}^N{w}_i{\zeta }_i^2\hfill \\ \multicolumn{1}{c}{s.t.\mathrm{A}G=\mathbb{Y}-\zeta }\end{array}\hfill \end{array}\hfill \end{array}$$

(13)

where $\mathrm{A}=[\mathrm{X}|\mathrm{H}]$ denotes input matrix with enhanced components, $\lambda$ is a penalty parameter for regulation, $w_i$ is the weight of the ith sample, and $\zeta$ is training error. By applying the Karush–Kuhn–Tucker (KKT) optimality conditions, the solution of equation (13) can be obtained as below

$$\mathrm{G}={\mathrm{A}}^T{(\frac{1}{\lambda }+\mathrm{WA}{\mathrm{A}}^T)}^{-1}\mathrm{W}\mathbb{Y}$$

(14)

where $\mathrm{W}=(w_1,w_2,\dots ,w_i)$ is weight matrix. After that, an adaptive controller in Yang et al. ²⁰ is developed to track the desired trajectories.

Experiment and discussion

Mechanical setup

We have established a test bed in our laboratory for the validation of the proposed control approach, see in Figure 5. Roughly, the whole walking control scheme is composed of three parts: biped robot BRZ-4, the ground workstation, and onboard system, respectively.

Figure 5.

Basic components of the test bed.

To control the motion of BRZ-4, the onboard system is equipped with a control unit Arduino MEGA 2560, a gyroscope. Control command can be transmitted to the onboard system via UART interface. Specifically, the online control algorithm runs on the onboard system. To alleviate the computing burden of the onboard system, offline training are implemented on the ground workstation, which comprised an Intel Core i5 processor, 8G DDR3, and 1 TB SSD disk. The mechanical structure of BRZ-4 is designed and implemented by three-dimensional (3D) print technique, which is equipped with 10 DYNAMIXEL MX-64-T motors and rechargeable battery.

Simulation and experimental results

To generate smooth and stable gaits, gait planning is the first task to be implemented. By utilizing the opposition-based particle swarm optimization algorithm proposed in Yang et al., ²⁷ the simulation results of walking gait reference are presented in Figure 6, where red solid lines denotes the right leg and trunk, and blue dashed lines represent the left leg. As indicated in Figure 6, it is noted that the generated gaits are smooth and stable.

Figure 6.

Simulation results of 3D walking.

The walking experiment is performed in our laboratory. To verify the control performance under different gaits, the whole walking procedure consists of three kinds of gaits, which are start gait, normal gait, and stop gait. The test bed provides a relatively ideal environment without many external disturbances, as given in Figure 5. It is worth mentioning that the parameter $\beta$ in equation (9) is a scale constant to tackle the trade-off problem between ZMP stability margin and yaw moment margin. Actually, it is still an open problem about how to choose this parameter. In the beginning, $\beta$ is initiated as $\beta =1$ . With applying the trial and error procedure, this scale constant is finally chosen as 1.5. To construct the proposed RVFLNNs, Gaussian functions are chosen as membership functions due to the fact that it is continuously differentiable and has clear physical significance. In this experiment, the central points and the standard deviation are randomly initiated. To reduce the computation complexity, the whole control procedure is divided into two parts, which are offline training and online control. In the offline training phase, we use our algorithm to train RVFLNNs in the ground workstation such that the ideal weight matrix can be obtained. Subsequently, the trained weight matrix is transferred to onboard system via UART interface and the online control is implemented in the onboard system. As indicated in Figure 7, the snapshot of walking gaits is presented.

Figure 7.

Snapshots of walking experiment.

To evaluate bipedal locomotion stability, the evolutions of both ZMP trajectory and yaw moment are recorded in Figures 8 and 9. Specifically, in these figures, both x-axis and y-axis ZMP response trajectories are plotted in solid blue line and ZMP boundaries are illustrated in dashed lines. As demonstrated in Figures 8 and 9, all the x-axis and y-axis ZMP response trajectories are observed to be within ZMP boundaries. Thus, with our proposed control algorithm, horizontal moment equilibrium is achieved. Moreover, the comparison results on the yaw moment response trajectories with and without control are given in Figure 10. From this figure, it is apparent that the yaw moment is effectively counteracted. Compared with the case of without yaw moment compensation, the peak of yaw moment is reduced by 34.1%, which corroborates the effect of the proposed algorithm. Overall, in practical application, the bipedal locomotion stability is well guaranteed by the presented control scheme, as validated by the experimental results.

Figure 8.

x-axis trajectory.

Figure 9.

y-axis trajectory.

Figure 10.

Yaw moment trajectory.

Conclusion

In this article, the problems of walking pattern generation and online control for humanoid robot are investigated. To overcome the difficulty arising from modeling uncertainties and external disturbance, an RVFLNNs-based control scheme is proposed such that ZMP error can be effectively compensated. Moreover, the learning weight is assigned to each training sample by developing an FLS. Subsequently, a bivariate-stability-margin optimization approach, which is additionally integrated in RVFLNNs mechanism, makes that the presented control scheme is more efficient. Finally, the experiment results validate our proposed scheme.