Sage Journals: Discover world-class research

Abstract

In order to better adapt to human living environment for improving the ability of serving people on various occasions, humanoid robots need to prevent themselves from being severely damaged during falling backward. In this article, we have study the law of human falling motion with a motion capture system and propose a minimized falling damage method for humanoid robots. Falling backward is divided into two phases: the falling phase and the touchdown phase. The parametric optimal strategy based on inverted pendulum with flywheel is used to plan the motion of robot in the first phase to reduce the impact. In the second phase, to prevent the robot from bouncing and rolling over, the heuristic strategy including the best ratio of leg length inspired by biomechanical is adopted. The experiments have been tested on the BIT Humanoid Robot 6 prototype platform and the presented method has been validated.

Keywords

Humanoid robots falling backward optimization algorithm inverted pendulum

Introduction

It is always being dreamed of to have a machine or a robot that can serve people in daily life. A domestic robot would be found useful when attending to the household duties. It is necessary to be able to move around the house, where many obstacles such as the entrance, a staircase, doors, and furniture exist. So the most practical and functional distinguish of a robot is bipedal or humanoid which has been produced within the past few decades.¹

Biped humanoid robots have several advantages over wheeled mobile robots, such as the ability of stepping over obstacles and going up or down stairs. However, the humanoid robot may fall down if getting disturbed seriously during walking and the execution of a task. Imbalance and instability will eventually lead to the falling which would bring severely damage for humanoid robots. Thus, it attaches great importance to analyze falling and protection methods for humanoid robots.

Generally speaking, the falling strategies for humanoid robots consist of heuristic strategies, optimal control strategies based on dynamic models, and falling protection for typical parts.

Heuristic strategies are inspired by the motion of human beings. Humanoid robots will be set to imitate human falling action to protect themselves from falling to the ground without reaction. Fujiwara et al.^2
–4 developed a falling motion control method named UKEMI. It introduced a method to minimize damage to a humanoid robot when it fell over to the ground. There were six processes in falling: Standby, Squatting, Extend-1, Touchdown, Extend-2, and Finish. The strategy which was inspired by the similar protective Judo skill UKEMI involved controlling the attitude of the robot while it was falling over so that it landed on the ground at one of shock-absorbing parts of the robot. Ma et al. introduced a falling back control strategy based on human protective falling motion,⁵ which included “knee flexion,” “torso flexion forward,” “torso translation backward,” and “knee stretched” to do negative work. Thus, the collision energy was greatly reduced. Meng et al.⁶ discovered a fall method based on the motion capture system. Besides, Ishida et al.⁷ described the analysis of various dynamic motions, such as falling over motion, standing up motion, and turning wide-angle motion, which were capable of assisting the robot in standing up after falling down if aiming at the small biped entertainment robot.

In dynamic model-based optimal control strategies, dynamic equations have been built to control the motion of joints on humanoids. Fujiwara et al. researched on preventing physical damage that would disable the locomotive ability of the robot, then proposed a safe falling motion control strategy to minimize damage to humanoid robots. They developed a human-sized robot with seven degree of freedom (DOF).⁸ Ogata et al. proposed a technique for selecting an optimal strategy to handle disturbances while walking and described a method of generating Ukemi motion using the three-dimensional (3-D) linear inverted pendulum mode.⁹ Yun and Goswami reported further theoretical developments culminating in a successful hardware implementation of this fall strategy conducted on the Aldebaran NAO robot. This includes new algorithms for humanoid kinematics and Jacobian involving coupled joints and a complete estimation of the body frame attitude using an additional inertial measurement unit.¹⁰ They present a fall strategy which rapidly modifies the robots fall direction in order to avoid hitting a person or an object in the vicinity.¹¹

Changing the falling direction and protecting the robots using specific materials are used in falling protection for typical parts. Lee and Goswami dealt with the problem of reducing the impact damage to a robot associated with a fall.¹² Yun et al. proposed a strategy for planning and control of fall. A direction-changing fall controller was presented.¹¹ Then, they introduced a generalized approach to humanoid falling direction control among multiple objects.¹³ Yun and Goswami addressed a new control strategy to reduce the damage to a humanoid robot during a fall named as tripod fall.¹⁴ Tamagawa et al.¹⁵ focused on the head protection and did the experiment in JARI, which simulated the human accidental injury.

Heuristic action plan is relatively simple but will not achieve effective protection. Meanwhile, it cannot protect the specific parts of robots.

In this article, we propose a minimized falling damage method for humanoid robots, consisting of the parametric optimal strategy based on inverted pendulum with flywheel, and the heuristic strategy with the best ratio of leg length (RLL) inspired in biomechanical by the falling motion of human beings analyzed and obtained via the motion capture system, which are respectively relative to the two motion phase: the falling phase and the touchdown phase. Meanwhile, protective cushions have been applied to the specific parts to reduce the damage.

This article is organized as follows. In “Law of human falling motion” section, we introduce the pattern of human falling motion. In “The minimized falling damage method” section, the minimized damage method of falling backward is presented. “Experiments and results” section shows the experiment on the BIT Humanoid Robot 6 prototype (BHR-6p). Finally, we conclude the article and future work in “Conclusion” section.

Law of human falling motion

Bipedal humanoids robots behaving like human beings are always the direction for researchers. Thus, the study of humanoid robots can be turned to the study of human beings. It is quite common for people to get injured when they fall down in daily life. Although the body movement of human beings is highly complex, it is still possible to get specific motion patterns through a series of experiments.

As is mentioned in the study by Moeslund et al.,¹⁶ the human motion capture system continues to be an increasingly active research area in computer vision. Motion capture system is widely used in capturing human motion with the process of initialization, tracking, pose estimation, and recognition as is shown in the study by Moeslund and Granum.¹⁷

In this article, motion capture system is used to obtain the data of human motion when falling down. And experiments are made on the human falling experiment platform.

As is shown in Figures 1 and 2, the human falling experiment platform consists of the motion capture system and the falling platform. There are 12 cameras around the experimental site to catch the real-time location and posture of human body. And there is one high jump mat in the middle of the experimental site to ensure the safety of the volunteer.

Figure 1.

Human falling experiment platform.

Figure 2.

System structure of human falling experiment platform and process of the experiment.

In our experiments, human model will be simplified and 15 markers will be set to describe human motion. As is shown in Figures 3 and 4, there are three markers on the back, three markers on each arm, and three markers on each leg. Cameras around the top floor catch the markers and send the data of markers trajectory to the computer. The motion capture system records the positions of markers at the speed of 100 fps. The data from motion capture are the 3-D coordinates from the absolute coordinate.

Figure 3.

The volunteer dressed in suit with 15 markers.

Figure 4.

The volunteer dressed in suit with 15 markers and the skeleton is shown in computer.

In the experiment, a force will be given on the chest of the volunteer standing back to the high jump mat, without preparedness, and dressed in the suit with markers, then the volunteer will fall back and make fall protection action with instinct, which is shown in Figure 5. And next, as shown in Figure 2, the motion of volunteer falling back will be detected by the motion capture system, and a series of data that could be shown graphically (Figure 6) will be collected, after which the motion trajectory could be obtained and analyzed to get the law of human falling motion.

Figure 5.

Human falling backward by a force on the chest. Time was set by 0, 0.25, 0.4, 0.6, 0.9, 1.0, 1.2, 1.7, and 3.0 s.

Figure 6.

The process of falling backward recorded by the motion capture system.

From the falling experiment on the volunteer shown in Figure 5, falling motion can be divided into two parts: the falling process and the touchdown process. During the falling process, waist has to rotate reversely to keep the back from touching the ground immediately (Figure 5). Meanwhile, legs begin to shrink so that the center of mass (CoM) is gradually move backward and down. Then, in the touch down process, hip will touch the ground first then the back. At the same time, legs have to elongate to prevent legs turning over the trunk.

The law of falling backward is applied to our humanoid robots. Multipoint contacts can reduce the collision impact on robots. Under ideal conditions, the hip will keep close to the ground when the robot falls down, before the back touching ground. The core for robot falling protection is to minimize the kinetic energy to mitigate the impact and the angular momentum in case of rolling over. Thus, dynamic model of falling backward is built to generate a falling optimization and leg extensive movement is employed to protect the robot from getting greatly damaged and rolling over.

The minimized falling damage method

Parametric optimization control in falling phase

Linear inverted pendulum with flywheel model is used to create the dynamic equation for BHR-6p. According to the linear inverted pendulum, the robot can be equivalent to a massless link with a particle which occupies the whole mass. The particle is only affected by the gravity and the supporting force along the link. The link is connected on the ground with a passive joint. In addition, the flywheel is added to reflex the rotation of upper body as shown in Figure 7.

Figure 7.

One frame of the process of falling backward. The volunteer rotated his waist and tilt forward like a flywheel rotating.

Figure 8 shows the inverted pendulum with flywheel model. The CoM is fixed on the waist of the robot as simplification determined by the mass distribution of the whole robot body.

Figure 8.

The inverted pendulum with flywheel model.

The dynamics of the model is given by

r^{2} \ddot{q} + 2 r \dot{r} \dot{q} - g r sin q = \frac{τ}{m}

\ddot{r} + g r cos q - r q^{2} = \frac{f}{m}

J_{b} \overset{..}{θ} = τ

where r is the distance between the fulcrum and the center of gravity, q is the angle between the pendulum and vertical line, θ is the angle of upper body measured from vertical line, τ is the inertia moment of the flywheel, m is the total mass, f is the force which changed the length of the link, g is the acceleration of gravity, and J_b is the inertia of the whole body of BHR-6p.

The state equations of system can be obtained according to equations (1) to (3) as follows

x = [\begin{matrix} q & \dot{q} & r & \dot{r} & θ & \dot{θ} \end{matrix}]

u = [\begin{matrix} τ \\ f \end{matrix}]

\dot{x} = f (x, u, t) = [\begin{matrix} \begin{matrix} \dot{q} \\ τ / m r^{2} - 2 \dot{r} \dot{q} / r + g sin q / r \\ \dot{r} \end{matrix} \\ \begin{matrix} r {\dot{q}}^{2} - g cos q + f / m \\ \dot{θ} \\ τ / J_{b} \end{matrix} \end{matrix}]

Optimization index function is as follows

J (u) = Φ (x (t_{f})) + \int_{0}^{t_{f}} L (x, u) d t

where t_f is the terminal time and

{\begin{array}{l} Φ (x (t_{f})) = K_{1} J_{1}^{2} + K_{1} J_{2}^{2} + K_{3} J_{3}^{2} + K_{4} J_{4}^{2} \\ L (x, u) = 0 \\ J_{1} = r (t_{f}) \dot{q} (t_{f}) \\ J_{2} = q (t_{f}) - π / 2 \\ J_{3} = \dot{θ} (t_{f}) \\ J_{4} = θ (t_{f}) \end{array}

then the optimization index function can be written as

\begin{matrix} J (u) = K_{1} {(r (t_{f}) \dot{q} (t_{f}))}^{2} + K_{1} {(q (t_{f}) - π / 2)}^{2} \\ + K_{3} \dot{θ} {(t_{f})}^{2} + K_{4} θ {(t_{f})}^{2} \end{matrix}

with respect to constraints

{\begin{array}{l} \dot{q} \geq 0 \\ r_{MIN} \leq r \leq r_{MAX} \end{array}

where r_MIN and r_MAX are the distance limitation decided by the mechanical parameters of robot.

In equations (8) and (9), K₁ is set to decrease the critical speed of CoM, K₂ is the index guaranteeing the robot can touch down, and the angle of upper body would reach the reference value (0) at a speed of 0 with proper K₃ and K₄, when the robot is falling onto the ground.

Then, the control parameterization enhancing technique (CPET) introduced in the study by Lee et al.¹⁸ is adopted to solve this optimal control problem.

According to the CPET, define the input set as

\begin{array}{l} U & = {\begin{matrix} u_{1}, & u_{2}, & \dots, & u_{m} \end{matrix}} \\ = {\begin{matrix} [\begin{matrix} τ_{1} \\ f_{1} \end{matrix}], & [\begin{array}{l} τ_{2} \\ f_{2} \end{array}], & \dots, & [\begin{matrix} τ_{m} \\ f_{m} \end{matrix}] \end{matrix} \end{array}

Construct a set

{b_{i}}_{i = 1}^{M} = {{[\begin{matrix} τ_{i} \\ f_{i} \end{matrix}]}_{i = 1}^{M}

where $M = (N + 1) m, b_{i} = u_{((i - 1) mod m) + 1}, i = 1, 2, ..., M$ , and N is the switching time.

Introduce new time variable $s \in [0, M]$ defined by

\begin{matrix} \frac{d t}{d s} = v (s), & t (0) = 0 \end{matrix}

where $v (s) = t_{i} - t_{i - 1} for s \in [i - 1, i]$ . v(s) is a nonpiecewise constant scalar function and it satisfies

\int_{0}^{M} v (s) d s = t_{f}

The optimal problem is transformed as follows

\frac{d x}{d s} = v (s) f (x (s), u (s))

\begin{matrix} u (s) = b_{i}, & s \in [i - 1, i] \end{matrix}

\begin{matrix} J (u (s)) = K_{1} {(r (M) \dot{q} (M))}^{2} + K_{1} {(q (M) - π / 2)}^{2} \\ + K_{3} \dot{θ} {(M)}^{2} + K_{4} θ {(M)}^{2} \end{matrix}

with respect to constraints

{\begin{matrix} \int_{0}^{M} v (s) d s = t_{f} \\ \dot{q} (s) \geq 0 \\ r_{MIN} \leq r (s) \leq r_{MAX} \end{matrix}

Then, the optimal control problem could be figured out through finding v(s) defined on [0, M] which could minimize equation (17) with universal methods.

Using inverted pendulum with flywheel model, the movement of the upper body, the legs, and CoM could be obtained. As is shown in Figure 9, the red part represents the changeable length of the inverted pendulum, the green part indicates the trunk of robot, and the circle is the CoM.

Figure 9.

Schematic diagram of inverted pendulum with flywheel when falling backward. On ideal condition, the trunk will keep upright to prevent the back from getting damaged but in simulations and experiments, the trunk has to tilt and touch the ground eventually.

Heuristic strategy in touchdown phase

When BHR-6p is touching the ground, the robot may roll over because of the inertia. What’s worse, the great impact caused by collision may destroy the structure of the robot. Thus, the motion trajectory in touchdown phase is very important. However, it cannot be solved by the inverted pendulum with flywheel model which is used in the parametric optimal control strategy in falling phase. Thus, in touchdown phase, heuristic strategy is considered, in which we use the motion capture data of human falling motion to assist the trajectory planning of legs.

As shown in Figure 5 and described in “Law of human falling motion” section, leg extensive movement is adopted in the trajectory planning of leg in touchdown phase, which is inspired by the law of human falling motion obtained through the motion capture system. To describe this trajectory planning, we introduce an important parameter: RLL, which is defined by

RLL = \frac{r}{L_{Crus} + L_{Thigh}}

where r is the same variable in equation (1), L_Crus is the length of crus, and L_Thigh is the length of thigh of robot.

Then, we need to research the effect of RLL on impact and find the optimal value via simulations with different values of RLL. Figure 10 shows the process of falling backward in simulation using inverted pendulum with flywheel.

Figure 10.

Falling simulation of inverted pendulum with flywheel when the force is 200 N and RLL is 70%. RLL: ratio of leg length.

Simulations have been made to analyze the effect of RLL on impact of hip and trunk. The impact force is set to 100, 200, and 300 N and lasts 0.2 s, as shown in Figures 11 to 13. According to these simulations, we could obtain some useful information about the proper value of RLL as follows. When the force is small (100 N), the impact of back and hip significantly reduced at the interval of 65–75%. When the force is large (300 N), the impact of back gradually reduces and the impact of hip drops after the rise, and the best RLL is between 65% and 75%. When the force is medium (200 N), the best RLL is between 70% and 80%.

Figure 11.

Falling simulation data of inverted pendulum with flywheel when the force is 100 N. The impact of hip and trunk changes with the RLL from 45% to 80%. RLL: ratio of leg length.

Figure 12.

Falling simulation data of inverted pendulum with flywheel when the force is 200 N. The impact of hip and trunk changes with the RLL from 45% to 80%. RLL: ratio of leg length.

Figure 13.

Falling simulation data of inverted pendulum with flywheel when the force is 300 N. The impact of hip and trunk changes with the RLL from 45% to 80%. RLL: ratio of leg length.

Experiments and results

The experiments are made on BHR-6p developed for this falling protection research with a height of 1.5 m and a weight of 60 kg, while BHR 1–5 have been developed for humanoid walking pattern generation and control.^19,20 For comparison, human has six DOF in each leg and six DOF in each arm besides other DOF of hands, waist, and feet. In BHR-6p, there are five DOF in each leg and two passive DOF in each arm which is determined by the simplification of proposed problem and is enough to validate the presented method. The accelerator and gyroscope are set in the trunk, and on the feet, there are two six-dimension force sensors. Elastic cushion protection is set on the hip and the back (seen in Figure 14). Besides, weights for optimal control are as follows: $K_{1} = 1, K_{2} = 0.1, K_{3} = 1, and K_{4} = 1$ .

Figure 14.

BHR-6p and DOF configuration. BHR-6p: BIT Humanoid Robot 6 prototype.

As shown in Figures 15 to 17, three experiments have been performed on the BHR-6p platform: the first experiment is without control; the second one is with control based on the inverted pendulum; and the third one is under the parametric optimal control based on the inverted pendulum with flywheel, combined with the heuristic strategy based on trajectory planning with the best RLL (75%). Considering the destruction caused by the falling collision, the high jump mat is used to prevent the robot from severely hitting the ground. The BHR-6p is standing in front of the high jump mat with an accelerator in the trunk. The sampling period is 0.5 ms. Giving 200 N push force on front trunk, the robot begins to fall back. We analyze the shock acceleration response data recorded by the accelerator in the trunk.

Figure 15.

Falling experiment without control.

Figure 16.

Falling experiment with control on inverted pendulum.

Figure 17.

Falling experiment with the parametric optimal control and heuristic strategy.

Comparing the data of these experiments from Figure 18, the impact acceleration is followed by 7.35, 5.25, and 4.26 g. Obviously, with the presented method, bounce of robot decreases and the acceleration reduces so that the robot will not greatly get impact from collision.

Figure 18.

The acceleration of the falling experiments. The unit of horizontal axis is millisecond, and the unit of vertical axis is gram. The biggest acceleration becomes 4.3 g.

Conclusion

In this article, the law of human falling motion is analyzed and a minimized falling damage method is proposed. On one hand, in the falling phase, the parametric optimal control based on inverted pendulum with flywheel is proposed to accurately calculate the joint angles. On the other hand, in the touchdown phase, heuristic strategy based on trajectory planning of legs with the best RRL is applied to the robot. Experiments have been tested on the BHR-6p, which shows an obvious effect that the acceleration of the trunk reduces from 7.35 g to 4.26 g.

To realize the falling motion while walking will be our next target. Equally important, arms motion will be planed as synergy to enhance the reliability when falling down. When the robot is capable of coping with various kinds of disturbance and falling, the humanoid robots will serve the human society better.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Natural Science Foundation of China under grants 61375103, 61320106012, 61533004, and 61673069 and in part by the “111” project under grant B08043.

References

Hirai

Hirose

Haikawa

. The development of Honda humanoid robot. In: IEEE international conference on robotics and automation, Vol. 2, Leuven, Belgium, 16–20 May 1998, pp. 1321–1326. IEEE Computer Society.

Fujiwara

Kanehiro

Kajita

. UKEMI: falling motion control to minimize damage to biped humanoid robot. In: IEEE/RSJ international conference on intelligent robots and systems, Vol. 3, Lausanne, Switzerland, 30 September–4 October 2002, pp. 2521–2526. IEEE.

Fujiwara

Kanehiro

Kajita

. The first human-size humanoid that can fall over safely and stand-up again. In: IEEE-RSJ international conference on intelligent robots and systems, Vol. 27, Las Vegas, USA, 27–31 October 2003, pp. 1920–1926. IEEE.

Fujiwara

Kanehiro

Kajita

. Safe knee landing of a human-size humanoid robot while falling forward. In: IEEE/RSJ international conference on intelligent robots and systems, Vol. 1, Sendai, Japan, 28 September–2 October 2004, pp. 503–508. IEEE.

Huang

. Bio-inspired falling motion control for a biped humanoid robot. In: IEEE-RAS international conference on humanoid robots, Madrid, Spain, 18–20 November 2014, pp. 850–855. IEEE.

Meng

Chen

. A falling motion control of humanoid robots based on biomechanical evaluation of falling down of humans. In: IEEE-RAS international conference on humanoid robots, Seoul, South Korea, 3–5 November 2015, pp. 441–446. IEEE.

Ishida

Kuroki

Takahashi

. Analysis of motions of a small biped entertainment robot. In: IEEE/RSJ international conference on intelligent robots and systems, Vol. 1, Sendai, Japan, 28 September–2 October 2004, pp. 142–147. IEEE.

Fujiwara

Kajita

Harada

. An optimal planning of falling motions of a humanoid robot. In: IEEE/RSJ international conference on intelligent robots and systems, San Diego, USA, 29 October–2 November 2007, pp. 456–462. IEEE.

Ogata

Terada

Kuniyoshi

. Falling motion control for humanoid robots while walking. In: IEEE-RAS international conference on humanoid robots, Pittsburgh, USA, 29 November–1 December 2007, pp. 306–311. IEEE.

10.

Yun

Goswami

. Hardware experiments of humanoid robot safe fall using Aldebaran NAO. In: IEEE international conference on robotics and automation, Saint Paul, USA, 14–18 May 2012, pp. 71–78. IEEE.

11.

Yun

Goswami

Sakagami

. Safe fall: humanoid robot fall direction change through intelligent stepping and inertia shaping. In: IEEE international conference on robotics and automation, Kobe, Japan, 12–17 May 2009, pp. 781–787. IEEE Computer Society.

12.

Lee

Goswami

. Fall on backpack: damage minimization of humanoid robots by falling on targeted body segments. J Comput Nonlinear Dynam 2013; 8(2): 021005.

13.

Nagarajan

Goswami

. Generalized direction changing fall control of humanoid robots among multiple objects. In: IEEE international conference on robotics and automation, Anchorage, USA, 3–7 May 2010, pp. 3316–3322. IEEE.

14.

Yun

Goswami

. Tripod fall: concept and experiments of a novel approach to humanoid robot fall damage reduction. IEEE international conference on robotics and automation, Hong Kong, China, 31 May–7 June 2014, pp. 2799–2805. IEEE.

15.

Tamagawa

Nakano

Yukimasa

. Precise analysis of head deformation at falling shock. In: IEEE international conference on informatics, electronics and vision, Washington, USA, 15–18 June 2015, pp. 1–5. IEEE Computer Society.

16.

Moeslund

Hilton

Krüger

. A survey of advances in vision-based human motion capture and analysis. Comput Vis Image Underst 2006; 104(2): 90–126. DOI: 10.1016/j.cviu.2006.08.002.

17.

Moeslund

Granum

. A survey of computer vision-based human motion capture. Computer Vis Image Underst 2001; 81(3): 231–268.

18.

Lee

HWJ

Teo

Rehbock

. Control parameterization enhancing technique for optimal discrete-valued control problems. Automatica 1999; 35(8): 1401–1407.

19.

Chen

Huang

. Gait planning of omnidirectional walk on inclined ground for biped robots. IEEE Trans Syst Man Cybern Syst 2016; 46(7): 888–897.

20.

Chen

Zhang

. Bio-inspired control of walking with toe-off, heel-strike and disturbance rejection for a biped robot. IEEE Trans Ind Electron 2017; PP(99): 1–1.

A minimized falling damage method for humanoid robots

Abstract

Keywords

Introduction

Law of human falling motion

The minimized falling damage method

Parametric optimization control in falling phase

Heuristic strategy in touchdown phase

Experiments and results

Conclusion

Footnotes

Declaration of conflicting interests

Funding

References