Three-dimensional walking of a simulated muscle-driven quadruped robot with neuromorphic two-level central pattern generators

Abstract

We aim to design a neuromorphic controller for the locomotion of a quadruped robot with muscle-driven leg mechanisms. To this end, we use a simulated cat model; each leg of the model is equipped with three joints driven by six muscle models incorporating two-joint muscles. For each leg, we use a two-level central pattern generator consisting of a rhythm generation part to produce basic rhythms and a pattern formation part to synergistically activate a different set of muscles in each of the four sequential phases (swing, touchdown, stance, and liftoff). Conventionally, it was difficult for a quadruped model with such realistic neural systems and muscle-driven leg mechanisms to walk even on flat terrain, but because of our improved neural and mechanical components, our quadruped model succeeds in reproducing motoneuron activations and leg trajectories similar to those in cats and achieves stable three-dimensional locomotion at a variety of speeds. Moreover, the quadruped is capable of walking upslope and over irregular terrains and adapting to perturbations, even without adjusting the parameters.

Introduction

The existence of central pattern generators (CPGs) in an animal’s spinal cord and their contribution to generating basic locomotion rhythms, even without afferent sensory feedback and control signals from the brain, are well documented. ¹ Based on the findings, a variety of mathematical CPG models have been proposed for quadrupedal locomotion. ^{2 –8} A CPG, which is a nonlinear oscillator, can autonomously adjust the phase differences between the four legs of a quadruped robot and its duty factor by receiving feedback from sensory information. Therefore, a quadruped robot with a CPG can safely locomote over irregular terrain without explicitly changing its leg trajectory, ⁹ which is a common method for quadruped robots without CPGs. Many mammalian quadruped simulated models ^{10 –13} and robots ^{9,13 –22} with CPG models have thus successfully achieved quadrupedal locomotion. They made significant contributions to recognizing the value of CPG for quadrupedal locomotion; however, the mechanisms and nervous systems of the quadrupeds were fairly simplified to allow them to safely locomote. For examples, each joint was driven not by plural muscles as for animals but by a single mechanical actuator, such as a motor or a bidirectional linear actuator, leading to fewer total actuators. The relatively simple CPG models were therefore sufficient to control all the actuators.

Since muscles have elasticity, which typical mechanical actuators do not have, when a practical artificial muscle actuator is developed in the future, we can expect to improve the energy efficiency and running speed of quadruped robots by exploiting this elasticity. Moreover, if the robot has two-joint muscles, it can further improve its energy efficiency. ²³ However, to allow a quadruped robot with the realistic legged mechanisms driven by plural muscle actuators to achieve safe locomotion in the future, we should design more realistic neural controller capable of synergistically driving its plural muscles by using more realistic CPGs. To do this, we build a simulated cat model by equipping each leg with three joints driven by six muscle models incorporating two-joint muscles (24 muscles in total) controlled by neuromorphic two-level CPGs and allow it to safely locomote in three-dimensional (3-D) space. We model it on a cat because cats are often used for animal testing in locomotion, such as in decerebrated cat walking, ²⁴ and there are many relevant published experimental data available. ^{24 –32}

In legged locomotion of an animal, the phases of the leg movement in a cycle are typically divided into four phases: swing, touchdown, stance, and liftoff, which are sequentially repeated (Figure 1). The leg form in each sequential phase is generated when the different set of multiple muscles are synergistically activated. ²⁵ In particular, there are durations when flexors, extensors, and two-joint muscles are simultaneously activated in each sequential phase, forming a realistic leg trajectory in a cycle. To achieve this animal-like kinematic movement in each leg driven by multiple muscles, the following two conditions should be satisfied:

Figure 1.

A diagram of the four sequential phases of a leg in a cycle. The bold blue lines represent activated muscles, and the thin lines represent relaxing muscles. The different set of multiple muscles are synergistically activated in each sequential phase.

• Phase transition: The four sequential phases must be switched with appropriate timing.

• Synergistic activation of multiple muscles: In each sequential phase, the different set of multiple muscles must be synergistically activated.

In current neuroscience literature, CPG is divided into multiple layers, and the two-level CPG model is proposed with two hierarchical levels of a rhythm generation (RG) part and a pattern formation (PF) part. ³³ RG and PF parts are mainly responsible, respectively, for conditions (A): phase transition and (B): synergistic activation of multiple muscles. Since the CPGs used in the simple quadruped models and robots ^{9 –22} are not explicitly split in the two layers of RG and PF, they have difficulty with being simultaneously responsible for both (A) and (B). We believe a more realistic CPG model is required to satisfy the two conditions and achieve 3-D locomotion. Thus, we propose a cat-like neuromuscular model incorporating two-level CPG enabling 3-D quadrupedal walking.

Some representative studies have achieved realistic animal-like legged locomotion by synergistically driving multiple muscles in each leg in simulation ^{34 –40} and in robots, ^41,42 some of which ^{36,37,39,40,42} use two-level CPGs designed with neural networks consisting of realistic neuron models, similar to our work. However, all of the related works were limited to biped walking on either forelimbs ³⁷ or hind limbs, ^{34,36,38 –42} and their motions were constrained to the sagittal plane, except for Ekeberg and Pearson’s ³⁵ 3-D biped model with two immovable forelegs sliding on the ground surface without friction.

Our aim in this article is to suggest a minimal nervous system to enable a cat-like muscle-driven quadruped model to three-dimensionally walk even in uneven terrain, where stability is crucial. This article’s main contribution is the improvement of existing components and their appropriate integration to enable 3-D locomotion of a muscle-driven quadruped model, which has not yet been achieved even in simulation, let alone in any robot. The improvement and integration in our quadruped model are specified as follows:

We reproduce the synergistic patterns of muscle activity in each sequential phase by assembling the PF part in unique and simple ways to satisfy condition (B) and form the appropriate motions in the four phases.

We set the passive elastic element F_p of each muscle so that it can both roughly control each leg’s trajectory and self-stabilize each support leg without any particular control.

We stabilize rolling motion with a non-preprogrammed walk gait (i.e. the phase differences between the four legs are not explicitly determined by our program, but the gait results in a walk) that is autonomously generated when the phase differences between the four legs are adjusted via leg loading feedback to the RG part, despite the CPG network being hard-wired.

We facilitate the tuning of a large number of parameters through three procedures, including a genetic algorithm (GA).

Each of these features is discussed in the “Discussion” section, but here we briefly explain how the four features are different from the features of existing models. Regarding No. 1, we expand the PF part of a two-level CPG originally proposed by Markin et al., ³³ which was designed to move a single-joint limb. Markin et al. ⁴⁰ also expanded the same original model ³³ in a different way to achieve hind leg locomotion in simulation, but their expanded PF part was so complex that the CPG of each leg had 32 neurons, whereas our CPG has only 12 neurons, because the expanded PF part is simpler. Despite its simplicity, our CPG is capable of robustly adapting to 3-D quadruped locomotion with perturbations (“Robustness test when changing F_p ” section) and over uneven terrain (“Irregular terrain walking” section).

Regarding No. 2, F_p is also used in related works, ^{34 –40} but they never discuss effective setting of the equilibrium point and stiffness in F_p . We demonstrate their importance for stable 3-D quadruped locomotion in simulation, especially in perturbed locomotion (“Robustness test when changing F_p ” and “Irregular terrain walking” sections).

Regarding No. 3, leg loading feedback to the RG part is applied in related works, ^{34 –42} but it is novel that we find that this element causes a non-preprogrammed walk gait in a 3-D environment (“Flat terrain locomotion with afferent feedback” section) and that this gait results in stable locomotion even over uneven terrain (“Irregular terrain walking” section).

Regarding No. 4, our CPG model is simpler than other closely related two-level CPG models, ^36,37,40 but it still has a large number of parameters. Therefore, we propose three procedures, including a GA, which is not used in related works, ^36,37,40 to facilitate the tuning (“Parameter tuning” section). In addition, we test the parameter values on an artificial muscle-driven 3-D quadruped robot and demonstrate that they work relatively well (see section S1 in Supplementary text for the results).

In this article, we present simulation results of successfully achieving stable 3-D quadrupedal walking, in which recorded motoneuron activations and produced leg trajectories are similar to those of a cat walking. Moreover, even without adjusting the parameters, the simulated quadruped is capable of walking upslope and over irregular terrains and adapting to perturbations. It can also safely walk at a variety of speeds by changing a tonic descending input to CPG and a couple of parameters to change muscle strengths. We thus design a biomorphic neural controller for the locomotion of a quadruped model with realistic muscle-driven leg mechanism. We believe the results contribute to proposition of the control method for the locomotion of a biomorphic quadruped robot with animal-like joint configuration and artificial muscle actuators.

Related works with neuromorphic two-level CPG models

We here describe the simulated models and the robot that used two-level CPGs in the aforementioned related works, ^{36,37,39,40,42} which succeeded in producing animal-like realistic stepping motions by controlling the muscle-driven leg mechanism, and explain the advantages of our two-level CPG model.

Several studies (simulations by Hunt et al., ³⁹ Maufroy et al. ^36,37 and Markin et al. ⁴⁰ and a robot by Hunt et al. ⁴² ) have thus focused on the neuromorphic two-level CPG, which was designed with neural networks consisting of realistic neuron models, capable of simultaneously satisfying conditions (A) and (B) to reproduce a realistic leg movement. Maufroy et al.’s ³⁷ simulated model only had forelegs and the others only ^36,39,40,42 had hind legs, and all performed planar locomotion.

Hunt et al.’s ⁴² robot called Puppy had hip, knee, and ankle joints in each leg, and each joint had an antagonistic pair of artificial muscle actuators (without two-joint muscles). Puppy’s CPG models were simplified versions of two-level CPG models proposed by Zhong et al. ⁴³ Muscle length and loading feedback to the CPGs were used. The CPG synergistically activated and drove a different set of artificial muscle actuators in each of the four sequential phases. However, the single CPG was put on each joint and there was no CPG to activate would-be two-joint muscles, which are important for fast and energy-efficient locomotion, as mentioned in the “Introduction” section. The researchers also developed a rat simulation model using the same method to perform hind leg walking. ³⁹

Maufroy et al. ^36,37 and Markin et al. ⁴⁰ used a two-level CPG in a more realistic way for each leg. Maufroy et al. ³⁶ demonstrated the planar walking of cat-like hind legs in simulation. Each leg had three joints driven by seven muscle models. The neural phase generator (NPG) proposed by Wadden and Ekeberg ⁴⁴ and a motor output shaping stage (MOSS) were used as RG and PF parts, respectively. The NPG and MOSS received a variety of sensory feedback of muscle length and loading. However, since their model was very complex (i.e. each leg’s CPG consists of 73 neurons, whereas our CPG has 12 neurons), parameter tuning was a problem, even for hind leg walking. It may be difficult to use their model for a quadruped robot. The researchers also succeeded in biped walking of the forelegs, using the same method. ³⁷

The most similar two-level CPG to ours was Markin et al.’s model. ⁴⁰ Each CPG manipulated three joints driven by nine muscle models in each leg, allowing the simulated hind legs of a cat to walk in the sagittal plane. Similar to our CPG, this two-level CPG was an extended version of a CPG for driving a single-joint limb that was proposed by Markin et al., ³³ as mentioned in the “Introduction” section (see “Design concept of two-level CPG” section for the differences). Each CPG received a variety of sensory feedback on muscle length and loading. They built a finely designed hind leg model based on biological findings because of their stated aim from a physiological standpoint that “The model can be used as a testbed to study spinal control of locomotion in various normal and pathological conditions.” ⁴⁰ Therefore, similarly to Maufroy et al.’s model, ^36,37 Markin et al.’s model ⁴⁰ may be too complex to be used as a locomotion controller for a quadruped robot (i.e. each leg’s CPG consists of 32 neurons, whereas our CPG has 12 neurons).

Methods

Mechanism of a simulated quadruped model

We used a dynamic simulator called Webots, ⁴⁵ which has been used in many locomotion studies. Figure 2 shows the simulated quadruped model, Nyanko, and its diagram. The head, torso, and tail are united in a single rigid body. The lengths of the leg segments were determined by being modeled on a cat in Reighard and Jennings’s work. ²⁶ The model weighs 6.0 kg. Each foreleg has scapula, shoulder, and elbow joints, and each hind leg has hip, knee, and ankle joints. All the joints turn around the pitch axis. Each leg is driven by six muscles as shown in Figure 2(b). The length and fitted position of each muscle were approximately determined based on those of each main muscle in the anatomical figures of cats. ^25,26 Nyanko has neither joints around the roll nor the yaw axis. We found in our test that the six muscles in each leg at least enabled the leg to reproduce the kinematic movements for the four sequential phases. We also plan to apply our method to a quadruped robot with artificial muscle actuators in future studies, so with its mechanical design in mind, we limited the number of muscles so that we could equip the robot with actuators that do not rub against one another. We used a muscle model proposed by Brown et al., ²⁷ similar to related works. ^{33 –38,46} The model was constructed based on soleus muscles, but it could be used for other muscles, according to their findings. ⁴⁷ The output force is

$\begin{array}{c}F=F_{\text{max}}\cdot \left(\lambda \cdot f\left(V\right)\cdot F_l\cdot F_v+F_p\right)\hspace{0.17em}\left[\text{N}\right]\end{array}$

1

Figure 2.

A quadruped model “Nyanko” built in a simulator (a) and its diagram (b).

where $F_{\text{max}}$ is the maximal isotonic force, $f\left(V\right)$ is the motoneuron activity (see equation (10) later), F_l and F_v are length- and velocity-dependent force, respectively, and F_p is a passive component. An animal locomotes by exploiting a large number of muscles, not only major muscles but also tiny muscles, but since in Nyanko all the muscles are reduced to the major six muscles, we regard each of the six muscles as a representative muscle that consists of the corresponding major muscle along with some minor muscles around it. Since we cannot know the value of $F_{\text{max}}$ of the representative muscle based on data from a cat, we set the value of each $F_{\text{max}}$ as shown in Table S1 in the Supplementary tables. We chose these settings as the muscle force required to withstand the maximum loads applied to the feet when our previous simulated quadruped model ran. ¹² Specifically, we put a weight on the torso of standing Nyanko so that the same load was applied to each foot as the load when the previous model was running, and we set $F_{\text{max}}$ as the muscle force required to support this weight. λ is an original parameter to adjust the level of the output from the corresponding motoneuron (see “Adaptation to speed variation” section for details) according to speed, and it is constant at constant speed on flat terrain. The length-dependent force F_l in equation (1) is expressed as

$F_1=\exp \left(-{\left|\left(l_{\text{norm}1}^{2.3}-1\right)/1.26\right|}^{2.0}\right)$

2

where $l_{\text{norm}1}$ indicates a normalized muscle length given by $l_{\text{norm}1}=l/L_{\text{opt}}$ (l is a muscle length, and $L_{\text{opt}}$ is the optimal length of the muscle when F_l = 1). The velocity-dependent force F_v in equation (1) is expressed as

$\begin{array}{c}{F}_v\hspace{0.17em}=\hspace{0.17em}\left\{\begin{array}{cc}\frac{-0.69\hspace{0.17em}-\hspace{0.17em}0.17\hspace{0.17em}\cdot \hspace{0.17em}{v}_{\text{norm1}}}{v_{\text{norm1}}\hspace{0.17em}-\hspace{0.17em}0.69}& \hspace{0.17em}\left(\text{if}\hspace{0.17em}{v}_{\text{norm1}}\hspace{0.17em}<\hspace{0.17em}0\right)\\ \frac{0.18\hspace{0.17em}-\hspace{0.17em}s\left(l_{\text{norm1}}\right)\hspace{0.17em}\cdot \hspace{0.17em}{v}_{\text{norm1}}}{v_{\text{norm1}}\hspace{0.17em}+\hspace{0.17em}0.18}& \hspace{0.17em}\left(\text{otherwise}\right)\end{array}\right.\end{array}$

3

where $v_{\text{norm}1}$ is the muscle velocity normalized by $v_{\text{norm}1}=v/L_v$ , v is the muscle velocity, and when v is positive, the muscle extends. According to Brown et al., ²⁷ L_v was defined as the fascicle length of a muscle when maximal isometric force is observed. However, similar to $F_{\text{max}}$ , since it was impossible to know the value of L_v of the representative muscle from a cat’s data, we set its value to the length of each muscle when Nyanko stood still, as shown in Table S1 (in Supplementary tables). The parameter $s\left(l_{\text{norm}1}\right)=-5.34l_{\text{norm}1}^2+8.41l_{\text{norm}1}-4.7$ . The parameter values in equations (2) and (3), representing their inherent characteristic, were determined according to the muscle model proposed by Brown et al., ²⁷ similar to the related works. ^33,35,46

We used the following equation of passive force F_p proposed by Brown et al. ²⁷ as used in the related works ^33,35,46 except for using the adjustable parameters of $k_{\text{stiff}}$ and $L_{\text{opt}}$ in $l_{\text{norm}1}$

$\begin{array}{c}\begin{array}{cc}{F}_p=& k_{\text{stiff}}\cdot \text{ln}\left(\text{exp}\left(\left(l_{\text{norm1}}-1.4\right)/0.05\right)+1.0\right)\\ & -0.02\cdot \left(\text{exp}\left(-18.7\cdot \left(l_{\text{norm1}}-0.3\right)\right)-1.0\right)\end{array}\end{array}$

4

Each muscle has different values of $k_{\text{stiff}}$ and $L_{\text{opt}}$ . The parameter values in equation (4), representing their inherent characteristic, was determined according to the related works. ^33,35,46 $L_{\text{opt}}$ is a constant parameter, whereas $k_{\text{stiff}}$ can be adjusted according to the speed (see the “Adaptation to speed variation” section for details), but $k_{\text{stiff}}$ is constant at constant speed. Figure 3(a) illustrates equation (4). We see that if $l_{\text{norm}1}>L_{\text{opt}}$ , F_p acts as a passive pull spring to contract the extended muscle toward the length of $L_{\text{opt}}$ , which is close to the natural length of the muscular pull spring. Note that F_p was rarely negative in our simulation. In addition, the stiffness of the muscle is adjusted by $k_{\text{stiff}}$ . The value of $L_{\text{opt}}$ of each muscle was set to a length at which all the muscular springs were maintained in an equilibrium leg position when Nyanko was suspended, as shown in Figure 3(b). Since the value of $L_{\text{opt}}$ of each muscle was set as constant, each leg was potentially controlled to the equilibrium position as the desired position in local and mechanical manners while walking. F_p supplementarily works in addition to the force provided by the motoneuron, as shown in equation (1), and plays an important role in controlling the leg’s trajectory to allow the leg driven by the motoneuron activation to steadily oscillate.

Figure 3.

Each muscle’s passive elastic element and its effect on Nyanko. (a) Graph of the passive elastic element F_p (equation (4)) of each muscle. l _norm1 indicates a normalized muscle length given by l _norm1 = l/L _opt (l is a muscle length, and L _opt is the optimal length of the muscle when F_l = 1). The stiffness of the muscle is adjusted by k _stiff. (b) An example of the equilibrium leg position of Nyanko during locomotion. Each spring shows the passive elastic property of the corresponding muscle.

Neural system

Design concept of two-level CPG

A key problem is the construction of a two-level CPG model for each leg allowing for 3-D quadruped locomotion. Our CPG model extends the two-level CPG model proposed by Markin et al., ³³ which was divided into the RG and PF parts. We call this basic model the Rybak CPG model. Since the Rybak CPG model was designed to move a single-joint limb, as shown in Figure 4(a).

Figure 4.

Three kinds of two-level CPG models. (a) The Rybak CPG model proposed by Markin et al. ³³ to move a single-joint limb. (b) Expanded version of (a) by Markin et al. ⁴⁰ that is designed to move a simulated hind leg model with multiple joints modeled on a cat. (c) Expanded version of (a) by the current authors that is designed to move a simulated cat quadruped model whose leg consists of multiple joints. E, F, and M denote neurons for extension, flexion, and motoneurons (neural populations, to be exact), respectively. PF-{hip, knee, ankle, two-joint} in (b) represent PF circuits for the hip, knee, ankle joint muscles, and the two-joint muscles, respectively. PF_{{sw, td, st, lo}} in (c) represent PF neurons for the swing, touchdown, stance, and liftoff phases, respectively. Excitatory and inhibitory synaptic connections are represented by arrows and small black circles, respectively. For simplicity, we omitted some details from these figures (e.g. feedback and interneurons through which E and F in each model are mutually inhibited). CPG: central pattern generator.

The organization of the PF part to synergistically activate a different set of multiple motoneurons is still unrevealed in biology. To address this issue, Markin et al. ⁴⁰ expanded the PF part of the Rybak CPG model ³³ as shown in Figure 4(b) to drive the hind leg models with multiple joints modeled on a cat. Their improved CPG model has three PF circuits for the three hip, knee, and ankle joints, each of which activates the motoneurons of the muscles to drive the corresponding joint. The model has another PF circuit that activates the two motoneurons to drive the two-joint muscles of a biceps femoris posterior and a rectus femoris, which are partially active during both flexor and extensor phases. Thus, the model has four PF circuits. Each PF circuit has a half-center form consisting of an extensor neuron and a flexor neuron inhibiting each other.

On the other hand, as shown in Figure 4(c), our proposed two-level CPG model uses the four sequential PF neurons PF_sw, PF_td, PF_st, PF_lo to make the phase patterns of swing, touchdown, stance, and liftoff, respectively. Each PF neuron does not form a half-center, but a single neuron, resulting in the four simple PF neurons in the PF part. The PF part in Figure 4(c) has only 20 parameters, whereas that in Figure 4(b) has 74. However, our CPG model was capable of generating realistic motoneuron activation to reproduce the leg movements in the four sequential phases. Other advantages are discussed in the “Discussion” section.

Neuron models and their connection in two-level CPG

Figure 5 shows a detailed diagram of our CPG model. The model of each neuron itself used in our CPG model is similar to the neuron models in the Rybak CPG model. ³³ However, the number of neurons and the neural connections are completely different, as shown in Figure 4(a) and (c). The membrane potential of the extensor and flexor neurons in the RG and PF parts (RG-E, RG-F, PF_sw, PF_td, PF_st, PF_lo in Figure 5) and the motoneurons (Mn₁–Mn₆ in Figure 5) is expressed as

$\begin{array}{c}C\cdot \frac{\mathit{\text{d}}V}{\mathit{\text{d}}t}=-I_{\text{NaP}}-I_{\text{K}}-I_{\text{Leak}}-I_{\text{SynE}}-I_{\text{SynI}}\end{array}$

5

Figure 5.

Detailed diagram of the proposed two-level CPG for each leg. This CPG is divided into the two layers of RG and PF parts. The RG part consists of half-center rhythm generators, including RG-E (RG neuron for extension) and RG-F (RG neuron for flexion). The PF part consists of the four PF neurons, where PF_{{sw, td, st, lo}} are PF neurons for the swing, touchdown, stance, and liftoff phases, respectively. Mn_{1−6} show motoneurons to activate the corresponding muscles. Inab-E shows an excitatory interneuron, and each In shows an inhibitory interneuron. All neurons and interneurons are represented by large and small spheres, respectively. Excitatory and inhibitory synaptic connections are represented by arrows and small circles, respectively. CPG: central pattern generator; RG: rhythm generation; PF: pattern formation.

and the membrane potential of the interneurons (In and Inab-E in Figure 5) is expressed as

$\begin{array}{c}C\cdot \frac{\mathit{\text{d}}V}{\mathit{\text{d}}t}=-I_{\text{Leak}}-I_{\text{SynE}}-I_{\text{SynI}}\end{array}$

6

This activity-based neuron model was also used in the aforementioned related works of Hunt et al. ^39,42 and Markin et al. ⁴⁰ In these equations, $I_{\text{NaP}}$ is a persistent sodium current, $I_{\text{K}}$ is a potassium rectifier current, $I_{\text{Leak}}$ is a leakage current, and $I_{\text{SynE}}$ and $I_{\text{SynI}}$ are excitatory and inhibitory synaptic currents, respectively. C is a neuronal capacitance. V is the average membrane voltage of the neurons in the neural population. $I_{\text{NaP}}$ and $I_{\text{K}}$ are omitted from equation (6). This is based on the Rybak CPG model, ³³ in which Markin et al. omitted $I_{\text{NaP}}$ and $I_{\text{K}}$ from their previous CPG model ⁴⁸ for simplicity, but nevertheless successfully demonstrated that they could create the rhythmic motion of a leg without any problems. The ionic currents are formulated as

$\begin{array}{l}{I}_{\text{NaP}}={\overline{g}}_{\text{NaP}}\cdot m_{\text{NaP}}\cdot h_{\text{NaP}}\cdot \left(V-E_{\text{Na}}\right)\\ I_{\text{K}}={\overline{g}}_{\text{K}}\cdot m_{\text{K}}^4\cdot \left(V-E_{\text{K}}\right)\\ \begin{array}{c}\begin{array}{l}{I}_{\text{Leak}}={\overline{g}}_{\text{Leak}}\cdot \left(V-E_{\text{Leak}}\right)\\ \begin{array}{cc}{I}_{\text{SynE},i}=& {\overline{g}}_{\text{SynE}}\cdot \left(V_i-E_{\text{SynE}}\right)\cdot \left({\sum }_j{a}_{ji}\cdot f\left(V_j\right)+c_i\cdot d+w_{1,i}\cdot {\mathit{\text{feedback}}}_1\right)\end{array}\\ \begin{array}{ccc}{I}_{\text{SynI},i}=& {\overline{g}}_{\text{SynI}}\cdot \left(V_i-E_{\text{SynI}}\right)& \cdot \left({\sum }_j{b}_{ji}\cdot f\left(V_j\right)+w_{2,i}\cdot {\mathit{\text{feedback}}}_2\right.\\ & \left.+{\sum }_k{\alpha }_{ki}\cdot f\left(V_k\right)+{\sum }_k{\beta }_{mi}\cdot f\left(V_m\right)\right)\end{array} \end{array}\end{array}\end{array}$

7

where $E_{\text{{Na, K, Leak, SynE, SynI}}}$ are the corresponding reversal potentials, ${\overline{g}}_{\text{{NaP, K, Leak, SynE, SynI}}}$ are the maximal conductances of the corresponding ionic channels, $a_{ji}$ is the gain of the excitatory synaptic input from neuron j to neuron i, $b_{ji}$ is the gain of the inhibitory input from neuron j to neuron i, c_i is the gain of the tonic descending signal d to neuron i, and $w_{\left\{1,2\right\},i}$ is the gain of afferent feedback ${\mathit{\text{feedback}}}_{\left\{1,2\right\}}$ to neuron i (see “Sensory feedback” section).

In equation (7), the voltage-dependent activation variables for the potassium delayed rectifier and persistent sodium channels are represented as

$\begin{array}{c}\begin{array}{c}{m}_{\text{K}}=1/\left(1+\text{exp}\left(-\left(V+44.5\right)/5\right)\right)\hspace{1em}\text{and}\\ m_{\text{NaP}}=1/\left(1+\text{exp}\left(-\left(V+47.1\right)/3.1\right)\right)\end{array}\end{array}$

8

respectively. The voltage-dependent inactivation variables $h_{\text{NaP}}$ are calculated using the following differential equation

$\begin{array}{c}{\tau }_{h\text{NaP}}\cdot \frac{\mathit{\text{d}}}{\mathit{\text{d}}t}{h}_{\text{NaP}}=h_{\infty \text{NaP}}-h_{\text{NaP}}\end{array}$

9

in which

$\begin{array}{l}{h}_{\infty \text{NaP}}=1/\left(1+\text{exp}\left(\left(V+51\right)/4\right)\right)\\ {\tau }_{h\text{NaP}}={\tau }_{h\text{NaPmax}}/\text{cosh}\left(\left(V+51\right)/8\right)\end{array}$

where ${\tau }_{h\text{NaPmax}}$ is a time constant. Note that the parameter values in equations (8) and (9) cannot be changed because those are values that represent their inherent neural characteristic. The same values are used in the Rybak CPG model. ³³

$f\left(V_j\right)$ in equation (7) represents an average population activity and an output from neuron j, which is defined as the following sigmoid function

$\begin{array}{c}f\left(V\right)=\left\{\begin{array}{cc}1/\left(1+\text{exp}\left(-\left(V-V_{1/2}\right)/k\right)\right)& \left(\text{if}\hspace{0.5em}V\ge V_{\text{th}}\right)\\ 0& \left(\text{otherwise}\right)\end{array}\right.\end{array}$

10

where $V_{1/2}$ is a half-activation voltage, k is a sigmoid gain, and $V_{\text{th}}$ is the threshold of each neuron. ${{\displaystyle \sum }}_k{\alpha }_{ki}\cdot f\left(V_k\right)+{{\displaystyle \sum }}_m{\beta }_{mi}\cdot f\left(V_m\right)$ shows the hard-wired connection from the other CPGs’ neuron to neuron i and creates the inherent interlimb coordination called the CPG network. This hard-wired connection is used only for the mutual inhibitory connections between the RG neurons of the lateral neighboring CPGs, as shown in Figure 6. Thus, a trot is basically generated from this hard-wired network as a basic gait pattern, but this gait would be autonomously modified when feedback to the CPG, such as the leg loading feedback of equation (13), strongly affects its RG.

Figure 6.

Outline of the CPG network. Each dashed line oval indicates a CPG. RG-E and RG-F represent the RG neurons for the extension and flexion, respectively. The small black circles indicate inhibitory synaptic connections. The weights α and β represent the connection strengths between the contralateral neurons and ipsilateral neurons, respectively.

Tables S4, S5, and S6 (in Supplementary tables) show the parameters in equations (5 ) to (10) used for our simulation. The weights ( $a_{ji}$ and $b_{ji}$ in equation (7)) to govern the synaptic connections between the neurons were originally determined to construct our CPG’s particular connection structure shown in Figure 5. Table S7 (in Supplementary tables) shows the optimally tuned values of those weights and the gains c_i and $w_{\left\{1,2\right\},i}$ for stable flat terrain walking (see “Flat terrain locomotion with afferent feedback” section for details). The connection strengths between the PF neurons and the motoneurons are visualized by the thickness and boldness of the connection lines in Figure 5.

Sensory feedback

Ia afferent feedback to extensor motoneurons as a stretch reflex is effective to support weight; ^34,40 thus, we applied the Ia afferent feedback used in Yakovenko et al.’s simulation. ³⁴ Specifically, the following feedback signals from the scapula and hip extensors are input to the corresponding motoneurons when the leg is loaded

$\begin{array}{c}{\mathit{\text{feedback}}}_1\hspace{0.17em}=\hspace{0.17em}\left\{\begin{array}{c}\begin{array}{c}{k}_v\hspace{0.17em}\cdot \hspace{0.17em}{v}_{\text{norm}2}^{0.6}\hspace{0.17em}+\hspace{0.17em}20.0\hspace{0.17em}\cdot \hspace{0.17em}{l}_{\text{norm2}}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\left(\text{if}\hspace{0.17em}{F}_{\mathit{\text{load}}}\hspace{0.17em}>\hspace{0.17em}0.01\hspace{0.17em}\hspace{0.17em}[\text{N]}\right)\end{array}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\left(\text{otherwise}\right)\end{array}\right.\end{array}$

11

where $F_{\mathit{\text{load}}}$ is a load of each leg measured by a force sensor fitted in its paw, k_v is a gain, $l_{\text{norm}2}$ is the muscle length normalized by

$\begin{array}{c}{l}_{\text{norm2}}\hspace{0.17em}=\hspace{0.17em}\left\{\begin{array}{cc}\left(l-L_{\text{th}}\right)/L_{\text{th}}& \left(\text{if}\hspace{0.17em}l\hspace{0.17em}>\hspace{0.17em}{L}_{\text{th}}\right)\\ 0& \hspace{0.17em}\left(\text{otherwise}\right)\end{array}\right. \end{array}$

12

where l is a muscle length, and $L_{\text{th}}$ is the minimal muscle length evoking afferent activation, which is also used in $v_{\text{norm}2}=v/L_{\text{th}}$ . The parameter values in equation (11) was determined according to the related works. ³⁴ The value of each muscle’s $L_{\text{th}}$ is shown in Table S8 (in Supplementary tables). Equation (11) is input to ${\mathit{\text{feedback}}}_1$ in equation (7) only for the scapula and hip extensor motoneurons, as shown in Figure 5. We observed that the addition of the Ia afferent allowed Nyanko to locomote more steadily.

Studies have noted the importance of leg loading feedback to CPG. ^24,28 In our CPG model, the following feedback was input to the RG flexor to make it inhibited (Figure 5), based on Pearson’s reports ²⁴

$\begin{array}{c}{\mathit{\text{feedback}}}_2=k_{\mathit{\text{load}}}\cdot F_{\mathit{\text{load}}}\end{array}$

13

where $F_{\mathit{\text{load}}}$ is a load of each leg measured by a force sensor fitted in its paw, and $k_{\mathit{\text{load}}}$ is a gain. Equation (13) is input to ${\mathit{\text{feedback}}}_2$ in equation (7) only for the RG-F neuron so that it is inhibited when the leg is loaded. This feedback was important in extending the stance phase while the leg was loaded, in initiating the transition from the support (stance) phase to the nonsupport (liftoff) phase in response to a lack of leg loading, and in eventually leading to the autonomous generation of a walking gait.

Parameter tuning

As described in the “Related works with neuromorphic two-level CPG models” section, our CPG model for each leg is simpler, with far fewer parameters than in the most closely related works, ^36,37,40 but it still has a large number of parameters. However, we found that the parameters in each part of our CPG had a particular feature. This could help tune all the parameters. Specifically, the connection parameters between the RG and PF neurons were responsible for making a basic rhythm. The connection parameters between the PF neurons and motoneurons mainly took care of creating the leg trajectories in the four sequential phases. The stiffness parameter $k_{\text{stiff}}$ in the passive force F_p (equation (4)) and the gain $k_{\mathit{\text{load}}}$ of the leg loading feedback (equation (13)) tended to result in more realistic leg trajectories and robust walking over uneven terrain and under perturbations. Based on these particular features of the parameters, we proposed the following three procedures, and eventually succeeded in facilitating the parameter tuning.

We suspended Nyanko with its legs swinging in the air and implemented a rough manual tuning of the parameters of the CPGs and the muscle models so that each leg could reproduce a loose trajectory in each of the four sequential phases and steadily oscillate. Specifically, we first determined the connections between the RG and PF neurons to reproduce the basic rhythm. Secondly, we focused on exploring the connections between the PF neurons and motoneurons to roughly reproduce the synergistic patterns of muscle activity in each of the four sequential phases.

We put Nyanko on the ground and tuned a variety of parameters using a GA so that Nyanko could walk on flat terrain.

We manually modified the parameters so that Nyanko’s motoneuron activation patterns and leg trajectories were close to those of cats.

To alleviate the complexity of tuning so many parameters, we left the majority of the tuning to GA. Specifically, we trained 76 parameters altogether, including the muscle stiffnesses $k_{\text{stiff}}$ in equation (4), the connection strengths of $a_{ji}$ from the PF neurons to the motoneurons in equation (7), and the connection strengths between the CPGs ${\alpha }_{ki}$ and ${\beta }_{mi}$ in equation (7).

We used real-coded GAs with BLX-α. ⁴⁹ The initial population in this GA comprised 300 individuals, which was randomly generated near the values of the parameters tuned in the aforementioned (I). For the population of the next generation, the best two individuals in the current population were kept as elite individuals, and the others were renewed by BLX-α with a roulette wheel selection with a 100% crossover rate between the individuals, a 30% crossover rate of the loci, and a 200% expansion rate of the hyperrectangle. The fitness function is given by

$\begin{array}{c}\begin{array}{c}\text{fitness}\left(t,{\theta }_{\mathit{\text{pitch}}}\right)=F_{\mathit{\text{time}}}\left(t\right)+F_{\mathit{\text{pitch}}}\left({\theta }_{\mathit{\text{pitch}}}\right)\\ F_{\mathit{\text{time}}}\left(t\right)=\{\begin{array}{cc}t& \left(\text{if }t<50\right)\\ 50& \left(\text{otherwise}\right)\end{array}\\ F_{\mathit{\text{pitch}}}\left({\theta }_{\mathit{\text{pitch}}}\right)\hspace{0.17em}=\hspace{0.17em}55\cdot \left(1-1/\left(1\hspace{0.17em}+\hspace{0.17em}{\text{e}}^{-\left({\theta }_{\mathit{\text{pitch}}}-2\right)/0.8}\right)\right)\end{array}\end{array}$

14

where t in seconds is a continuous locomotion duration at speeds of no less than 0.05 m/s, ${\theta }_{\mathit{\text{pitch}}}$ in radians is a mean trunk angle around the pitch axis while locomoting, and $F_{\mathit{\text{time}}}\left(t\right)$ and $F_{\mathit{\text{pitch}}}\left({\theta }_{\mathit{\text{pitch}}}\right)$ are evaluation functions to calculate the fitnesses for locomotion continuability and posture stability, respectively. The longer Nyanko walks, the larger $F_{\mathit{\text{time}}}\left(t\right)$ becomes. $F_{\mathit{\text{time}}}\left(t\right)$ is maximized at 50. The more horizontal the trunk is during locomotion, the larger $F_{\mathit{\text{pitch}}}\left({\theta }_{\mathit{\text{pitch}}}\right)$ is. $F_{\mathit{\text{pitch}}}\left({\theta }_{\mathit{\text{pitch}}}\right)$ is added to omit individuals that allow tilted Nyanko to proceed while dragging the legs or the body.

Results

Suspended locomotion

We allowed suspended Nyanko to locomote in the air to roughly create basic kinematic leg movements for locomotion. We did not use the afferent feedback of equations (11) and (13) in this suspended locomotion.

Initially, to examine whether our two-level CPG could reproduce the four sequential leg motions with only the muscles’ force generated by the CPG outputs, we excluded the muscle passive elastic element F_p in equation (1) and manually tuned the parameters of the CPG ( $a_{ji}$ , $b_{ji}$ , c_i , d, ${\alpha }_{ki}$ , and ${\beta }_{mi}$ in Eq. (7)) and the maximal isotonic max force ( $F_{\text{max}}$ in equation (1)). In particular, we focused on exploring the connections between all the neurons, including the four sequential PF neurons, to roughly reproduce the synergistic patterns of muscle activity in each of the four sequential phases. Thus, we were able to produce each leg’s oscillation, but the center of the oscillation went too far backward and it took too long for the oscillation to converge to the steady state after starting to swing.

Then, applying F_p along with the initial manual tuning of its parameters ( $k_{\text{stiff}}$ and $l_{\text{norm}1}$ in equation (4)) enabled each leg to steadily oscillate around the realistic center of its oscillation that was close to the leg position, as depicted in Figure 3(b). In addition, the hip oscillation converged to the steady state quickly after it started moving. For example, even for the well-trained Nyanko used in the “Flat terrain locomotion with afferent feedback” section, in which all the parameters were ultimately optimally tuned, the legs swung improperly in the air without F_p , as shown in Figure 7(a). However, applying F_p enabled proper swinging, as shown in Figure 7(b). In Figure 7(a), the center of the oscillation of the left hip angle was approximately 160°, which was too far backward, and it took about 8 s for the oscillation to converge to a steady state after the leg had started to swing. By contrast, in Figure 7(b), the center of the oscillation was approximately 70°, which is almost the same as the equilibrium hip angle shown in Figure 3(b), and the hip oscillation converged quickly to a steady state after the leg had started moving. Moreover, it was observed while walking on the ground in the sections “Flat terrain locomotion without afferent feedback,” “Flat terrain locomotion with afferent feedback,” “Adaptation to speed variation,” “Robustness test when changing F_p ,” and “Irregular terrain walking” that when each leg was excessively swung forward or backward by strong motoneuron activation, F_p could curb the excessive oscillation and switch the direction of the leg at appropriate fore and aft hip angles.

Figure 7.

Graphs of the angles of the left hip joint while Nyanko used in the “Flat terrain locomotion with afferent feedback” section was suspended (a) without and (b) with the muscle passive element F_p for swinging the four legs. The origin and orientation of the joint angles are shown in Figure 2. The dashed line shows the center of the oscillation.

This suspended locomotion helped us to roughly design the CPG and muscle models of each leg, especially with respect to the PF part reproducing the leg trajectories in the four sequential phases and F_p making possible the steady leg oscillations. At this stage, since we loosely targeted the steady rhythmic leg movements through the four sequential phases, we did not focus on quality (e.g. the magnitude of leg oscillation, the ratio of time between the four sequential phases, or the optimal value of each $L_{\text{opt}}$ ).

In terms of Nyanko’s gait, a default trot gait emerged in the suspended locomotion via the hard-wired CPG network, which was the mutual inhibitory connections between the lateral neighboring CPGs.

Flat terrain locomotion without afferent feedback

After the suspended locomotion (“Suspended locomotion” section), we allowed Nyanko to locomote on flat terrain by using the controller designed in the suspended locomotion, which did not incorporate the afferent feedback of equations (11) and (13). Then we used real-coded GAs with BLX-α ⁴⁹ to find parameter values with which Nyanko could locomote steadily. We tuned 76 parameters ( $k_{\text{stiff}}$ in equation (4), $a_{ji}$ , ${\alpha }_{ki}$ , and ${\beta }_{mi}$ in equation (7)) with the GA so that Nyanko could locomote for as long as possible; therefore, Nyanko steadily trotted but with quite different leg trajectories and much shorter strides from those of cats.

Next, we allowed Nyanko to have longer strides, but after taking several steps, Nyanko collapsed because Nyanko failed the timing of the transition between the support phase and the nonsupport phase. Specifically, as shown in Figure 8(a), even while the foot touched the ground, RG-F started bursting (A) and forced the support leg to lift, and the foot touched the ground early even though RG-F was still bursting (B). In addition, each support leg did not always trace the same trajectory in every cycle and wobbled. Note that at this stage, neither afferent feedback of equations (11) nor (13) were used.

Figure 8.

Graphs of the RG-F outputs (black solid lines) and footfalls (bold line segments at the bottom) of left hind leg without any afferent feedback (a) and with leg loading feedback to CPG (b). CPG: central pattern generator; RG: rhythm generation.

Flat terrain locomotion with afferent feedback

We applied the leg loading feedback (equation (13)) to the RG part to solve the abovementioned problem in terms of the timing of transition between the support and nonsupport phases. As a result, as shown in Figure 8(b), the feedback prevented RG-F from being activated while the foot touched the ground, and the loaded leg was therefore kept in the support phase for the right amount of time.

In addition, the proper use of each muscle’s passive elastic element F_p was capable of letting the stance leg to be controlled to the equilibrium position shown in Figure 3(b) to some extent as self-stabilization. In particular, properly setting the equilibrium position ( $L_{\text{opt}}$ ) and the stiffness ( $k_{\text{stiff}}$ ) in the stance phase of F_p (equation (4)) solved the problem of each support leg wobbling, observed in the “Flat terrain locomotion without afferent feedback” section.

We observed that adding the Ia afferent feedback of equation (11) allowed the body oscillation to be more stable.

Finally, we manually modified Nyanko’s parameters (d, $a_{ji}$ , ${\alpha }_{ki}$ , and ${\beta }_{mi}$ in equation (7), $k_{\text{stiff}}$ in equation (4), and $k_{\mathit{\text{load}}}$ in equation (13)) so that the motoneuron activations and the leg trajectories were more consistent with those of cats, resulting in Nyanko’s successful steady locomotion on flat terrain. The states of the locomotion at a constant speed of 0.62 m/s are shown in Figures 9 to 11. A movie can be seen in the supplementary video. Figure 9 shows the outputs from RG and PF neurons and the motoneurons compared with cats’ EMG outputs. Figure 10(a) and (b) shows each joint’s angle in the left fore and hind legs in one cycle, respectively, compared with those of a cat (note that the leg trajectories are different, depending on the kind of animal). Figure 10(c) shows the phase plane trajectory of the left hip joint during locomotion. Figure 10(c) indicates that Nyanko steadily and safely walked on flat terrain. Figure 11 shows the load of each leg.

Figure 9.

Outputs from the RG and PF neurons and the motoneurons (Mn) in (a) the left foreleg and (b) the left hind leg while Nyanko walked safely on flat terrain. The enclosed dashed rectangle in the Mn signals indicates one cycle duration, and the corresponding muscles’ EMG activations of a cat in the same duration are shown on the right. The cat data in (a) and (b) are from Cabelguen et al. ²⁹ and Goslow et al. ³⁰ , respectively. RG-F and RG-E represent RG for flexion and RG for extension, respectively. PF_sw, PF_td, PF_st, and PF_lo represent PF for the swing, touchdown, stance, and liftoff phases, respectively. RG: rhythm generation; PF: pattern formation.

Figure 10.

Joint data while walking on flat terrain. Each joint angle of (a) the left foreleg and (b) the left hind leg in one cycle while Nyanko walked safely on flat terrain, compared with those of cats (Meché ³¹ for forelegs, Rasmussen et al. ³² for hind legs) at almost the same speeds. The origin and orientation of each joint angle are shown in Figure 2. The bold lines at the bottom represent the footfalls. (c) A phase plane trajectory of the left hip joint while Nyanko walked safely on flat terrain. The horizontal and vertical axes show, respectively, the angle and angular velocity of the left hip joint. The origin and orientation of the joint angle are shown in Figure 2.

Figure 11.

Load of each leg while Nyanko was safely walking on flat terrain. The four legs lift in the order the left foreleg (A), the right hind leg (B), the right foreleg (C), and the left hind leg (D), which was a lateral sequence walk.

It can be seen in Figure 9(a) and (b) that PF_sw, PF_td, PF_st, and PF_lo properly activated in the swing, touchdown, stance, and liftoff phases, respectively, in both the fore and hind legs. Therefore, the motoneurons of the six muscles synergistically activated in each sequential phase. In particular, there are periods when the flexors, extensors, and two-joint muscles are simultaneously activated in each sequential phase. For example, in the touchdown phase, which is an intermediate phase between the swing and stance phases, while the scapula and elbow flexors were still bursting (A in Figure 9) (i.e. before the scapula extensor started bursting (B)), the shoulder extensor and the two joint muscles for the shoulder and elbow were strongly burst (C). By simultaneously activating the flexors, extensors, and two-joint muscles in each leg, Nyanko enabled the motion of the foot touching forward to the ground while extending the leg. Similarly, in the liftoff phase, while the scapula extensor was still activating (D) (i.e. before the scapula flexor started bursting (E)), the two-joint muscles for the scapula and shoulder and the elbow flexor were strongly activated (F), thus enabling the leg to retract up and backward, and the tip of the leg was prevented from catching the ground at the following swing phase. The same tendencies of the activations can be seen in the hind leg (Figure 9(b)). In Figures 9(a) and (b), the enclosed dashed rectangle in the left graph indicates one cycle duration, and the corresponding muscles’ activations of a cat in the same duration are shown on the right. The formats of the cat data of (a) and (b) are different because we were unable to find data of a cat’s muscles corresponding to all the muscles used in Nyanko from biological references. There is a certain amount of variation in the data for cats in biological papers, especially in the magnitude of EMG activations, because of measurement error and individual differences. Therefore, we only compare the timing of the EMG activations between the data for Nyanko and cats. It can be observed in Figures 9(a) and (b) that each muscle in Nyanko and a cat is activated and deactivated at similar times in one cycle. The features of Nyanko’s and the cat data are similar in each leg, even though Nyanko did not copy all the muscles in a real cat.

The leg trajectories in Figures 10(a) and (b) are fairly close to those of cats. There was a slight difference between the data of Nyanko’s and the cats. The speeds of the cats and Nyanko were almost the same, but the cyclic duration of Nyanko (0.46 s) was a little shorter than that of the cats (0.64 s). In other words, Nyanko lacked speed in relation to the short cycle compared to real cats. We suggest that one of the reasons is that there are fewer degrees of freedom in Nyanko than there are in a cat because Nyanko does not have hand, toe, and pelvis segments, the spine is rigid and does not bend, and the scapulae do not move linearly. Specifically, in cats, the feet touch the ground before the end of the cycle (A in Figure 10(a) and (b)) because the hand and toe segments continue touching the ground until the end of the cycle. However, Nyanko’s legs without the hand and toe segments left the ground earlier (B), and Nyanko must have slightly lost propulsion.

For Nyanko without leg loading feedback in the “Suspended locomotion” and “Flat terrain locomotion without afferent feedback” sections, a trot gait was originally produced by the hard-wired CPG network, which is the inherent connections between CPGs. In the simulation in this section, leg loading feedback differentiated the timing of the foot lifting from the ground between the two legs of each diagonal pair, which would be in-phase while trotting. This differentiation led to the phase differences between the two legs of each diagonal pair, autonomously resulting in a non-preprogrammed walk gait, as shown in Figure 11. Specifically, the left foreleg (A) lifted earlier than the right hind leg (B) in the first diagonal pair, and the right foreleg (C) lifted earlier than the left hind leg (D) in the second diagonal pair. This was because the body swayed sideways at low speeds, due to longer stance durations, than during trotting at high speeds. This ended up autonomously causing a lateral sequence walk, in which the four legs lifted in the order left foreleg (A), right hind leg (B), right foreleg (C), and left hind leg (D), as shown in Figure 11. In other words, the autonomous gait generation of a walk caused by leg loading feedback seems to have eventually stabilized the locomotion with long stride.

Tables S2–8 (in Supplementary tables) show the values of the optimally tuned parameters for the stable walking. λ in equation (1) was 1.0 in this section’s locomotion.

Adaptation to speed variation

The tonic descending signal to the CPG (d in equation (7)) was constant while locomoting at the constant speed in the “Flat terrain locomotion with afferent feedback” section, but it is known that changing the signal allows an animal to vary the speed. ¹ We tested at what range of speeds Nyanko can safely locomote either by only changing the descending signal d or by changing it along with other parameters.

Both results are shown in Figure 12. First, we changed the descending signal d around a value of 2.05, which was the value used in the “Flat terrain locomotion with afferent feedback” section, at a gradient of 0.083 per second, as shown in the dashed line in Figure 12(a). In the trial, Nyanko was capable of walking at speeds of between 0.55 and 0.72 m/s, as shown by the solid line. The figure indicates that the descending signal is constant at the beginning and at the end because Nyanko was unable to stand with the wider variation of the signal.

Figure 12.

Speeds when Nyanko successfully walked (a) by only changing the descending signal to CPG (d in equation (7)) and (b) by adjusting the parameters (λ in equation (1) and k _stiff in equation (4)) determining muscle strength in addition to the descending signal. The dashed lines represent the descending signals d. The solid lines represent walking speeds. CPG: central pattern generator.

Next, we empirically adjusted the parameters (λ in equation (1) to adjust the level of the output $f\left(V\right)$ in equation (10) from the corresponding motoneuron and $k_{\text{stiff}}$ in equation (4) to adjust the muscle stiffness) in some of the muscles according to the variation of the descending signal d at the same gradient of 0.083 per second, as shown by the dashed line in Figure 12(b). It is expected that the amplitude of a typical CPG increases to speed up only by increasing d, but the amplitude of our CPG tended to decrease slightly as d increased, similarly to the Rybak model. ⁵⁰ Therefore, we should supplement motoneuron output by increasing λ to speed up, instead. Therefore, Nyanko safely walked at a wider range between 0.34 and 0.79 m/s, as shown by the solid line. When Nyanko walked at speeds of 0.34–0.79 m/s, the Froude number ( $Fr=v^2/\left(gl\right)$ , where l is the leg length (0.2 for Nyanko), g is the acceleration due to gravity, and v is the velocity) was calculated as approximately 0.06–0.3. These values seem fair because of a biological reports ⁵¹ that mentioned that an animal’s gait transition from walking to trotting occurred when $Fr$ was 0.3–0.5. Therefore, Nyanko covered nearly the whole range of speeds at which an animal generally walks. We expect that Nyanko could locomote in different gaits at a wider range of speeds if we tune more types of the parameters more carefully.

Robustness test when changing F_p

Since we found in the “Flat terrain locomotion with afferent feedback” section that the passive elastic element F_p of each muscle played an important role in self-stabilizing the support leg, we decided to perform a simulation comparison to specify the effect. We prepared two Nyanko’s with low and high levels of F_p (i.e. in each muscle, $k_{\text{stiff}}$ (equation (4)) in the low-level F_p was 20 times smaller than that in the high-level F_p , which had the same value (Table S1) as used in the “Flat terrain locomotion with afferent feedback” section) in the scapula and hip extensors, responsible for balancing the support legs, and we investigated the limits of both the angle of upslope that each can walk and a perturbation where each can stand. In addition, we allowed Nyanko to walk over simple irregular terrains. All of the movies can be seen in the supplementary video. The values of all the parameters except for F_p were the same as in the “Flat terrain locomotion with afferent feedback” section and were constant in each simulation.

In the upslope walking simulations, Nyanko with low F_p could only walk up a slope of up to 1°; whereas, steady walking was observed in Nyanko with high F_p up to 9°. When Nyanko with a high F_p almost fell backward, the elastic force of F_p in the scapula and hip extensors mechanically and passively returned the support legs to the equilibrium positions. Although all the parameters were the same as when Nyanko walked on flat terrain, while locomoting upslope, the locomotion cycle was somewhat longer, and the gait was a trot at steep slope angles as observed in the supplementary video. The trot was observed because there were few differences of leg loads between the two legs of each diagonal pair through less body oscillation while locomoting upslope, and eventually each diagonal pair of legs became in-phase. The autonomous variation of Nyanko’s gait according to its locomotion states because of leg loading feedback is a main feature of our quadruped model.

For robustness tests against perturbations, we observed Nyanko’s stability when an object hit either the front end or the right side of its body. Specifically, using a pendulum with the object suspended by a 0.5 m massless string, we naturally released the object when the string was horizontal and hit Nyanko with the object. While Nyanko with a low F_p fell over when we hit a 0.5 kg object to its front end and a 0.3 kg object to its side, Nyanko with a high F_p could adapt to up to a 2.0 kg object hitting its front end and up to a 1.0 kg hitting its side.

Based on these results, F_p managed the stable locomotion well because the support legs self-stabilized around the equilibrium position (Figure 3(b)) and were capable of propping the body up in a sturdy way.

Irregular terrain walking

Walking over a step

Figure 13 shows the phase plane trajectories of the left hip joint when Nyanko walked over a step 30 mm high, 500 mm wide, and 600 mm deep. The time was 0 s at the moment the first front leg landed on the step. It took Nyanko approximately 2 s to walk over the step. The model wobbled somewhat 5–10 s after passing the step, but gradually became stable, as shown in the trajectory for 10–15 s. Nyanko was not able to walk over a step higher than 35 mm because of its foot tripping over the side of the step during the swing phase. No matter how wide and deep the step was, however, Nyanko succeeded.

Figure 13.

Phase plane trajectories of the left hip joint after Nyanko walked over a step (30 mm high, 500 mm wide, and 600 mm deep). The horizontal and vertical axes show the angle and angular velocity of the left hip joint, respectively. The origin and orientation of the joint angle are shown in Figure 2. The red, green and blue lines show trajectories in 0–5 s, 5–10 s, and 10–15 s, respectively. The trajectory was disturbed while walking over the step (the red line), but it immediately returned close to the stable track (the green) and finally returned much closer to the stable track (the blue line).

Walking with the left feet landing on ground of a higher level

Figure 14(A) shows how the body tilted around the roll axis when Nyanko walked with its left feet landing on a step 10 mm high (the body was tilted rightward (positive) after 3 s) after walking on the flat ground for the first 3 s. Figure 14(B) and (C) show its footfalls while it was walking on the flat terrain and then with the left feet landing the step, respectively. Each of (a), (b), (c), and (d) in Figure 14(B) and (C) represents the length of the period of each footfall in (B), for the purposes of comparing (C) with (B). While Nyanko was walking with its left feet landing on the step, as shown in Figure 14(C), the stance durations of both right legs were longer and those of both left legs were shorter than when it was walking on the flat ground (Figure 14(B)). This result showed that the stance duration of each leg on the side toward which the body tilted in the lateral plane was extended, and the reverse adaptation happened on the other side, because of the leg loading feedback, and the model eventually successfully avoided falling rightward. In addition, the gait was close to a pace while walking with the left feet were landing the step, as shown in Figure 14(C). Thus the gait was autonomously adjusted according to the walking terrain.

Figure 14.

Graphs when Nyanko walked with its left feet landing on a step 10 mm high. (A) How the body tilted around the roll axis (positive when tilting rightward). (B) and (C) Its footfalls it was walking on the flat terrain and then with the left feet landing the step, respectively.

We conducted the uneven-terrain locomotion because we wanted to show that Nyanko had an autonomous adaptability to a certain level of unevenness in the terrain without changing any parameters. However, note that our proposed controller is not sufficient to adapt to rougher terrains. For example, when Nyanko walks over terrain where there are a number of randomly placed relatively high steps, we will need to add a control that can adjust the trajectory of the tip height of each leg to maintain the body height, so that the loading will not be biased toward any particular leg.

Discussion

The following four key features enable the quadruped model to locomote in a 3-D environment.

A simple configuration of the PF part enabled the proper leg trajectories in the four sequential phases of each leg

Markin et al. ⁴⁰ developed the PF part divided into four half-center PF circuits (i.e. three respective PF circuits to activate the muscles of hip, knee, and ankle joints and another PF circuit to activate the two-joint muscles) as shown in Figure 4(b) and achieved realistic leg movements on each leg. On the other hand, as shown in Figure 4(c), we designed a PF part that has four single PF neurons (neural populations, to be exact) to make the four sequential phase patterns of swing, touchdown, stance, and liftoff, respectively. Despite the simple configuration, precise leg movements in the four sequential phases were performed.

In addition, since each PF neuron is responsible for the individual sequential phase unlike each PF circuit of Markin et al.’s, ⁴⁰ we expect that it would be easy to induce transition to each phase by adding feedback to the corresponding neuron in the future. For example, if the length-dependent Ia feedback from the scapula/hip flexor excites the PF_lo neuron when the scapula/hip is extended too far backward, the feedback could directly and quickly induce the transition from the stance phase to the liftoff phase. This could be a big advantage if we want to add sensory feedback in the future.

Moreover, since each PF neuron inhibits the other PF neurons via the corresponding interneurons in our PF configuration, as shown in Figure 5 (e.g. PF_lo inhibits PF_st, PF_td inhibits PF_st, PF_sw inhibits PF_lo and PF_st, and PF_st inhibits PF_lo, PF_sw, and PF_td), we can avoid erratic simultaneous activations of the PF neurons such as PF_sw activating in the middle of the stance phase. This could contribute particularly to irregular terrain locomotion, because random and irregular disturbances tend to be fed back to the CPG via sensory feedback, and the CPG is easily disturbed.

1. The loose trajectory control of each leg using the preferable setting of the passive elastic element of F_p

The related works of bipedal locomotion ^{34 –42} mainly suggested the importance of weight bearing and the timing of transition between extensor and flexor activities through afferent feedback, but the passive elastic elements of muscles such as F_p were arbitrarily determined and their importance was hardly noted. By contrast, we found that since more stability was necessary in 3-D quadrupedal locomotion, leg trajectory control to some extent was vital. Specifically, the control was made possible by appropriately setting the equilibrium length of each muscle $L_{\text{opt}}$ (fixed all the time) of F_p making the equilibrium leg positions as shown in Figure 3(b), and the stiffness $k_{\text{stiff}}$ of F_p in each muscle, which varied between the excitations of RG-F and RG-E. As shown in the demonstration in the “Suspended locomotion” section, the existence of F_p enabled the prevention of overswinging in each leg and allowed the steady oscillation of each leg in the air around the fixed center of the leg’s oscillation, as shown in Figure 7(b). In addition, F_p contributed the self-stability of each support leg and had a huge effect on the robustness in walking up the slope, against the perturbations, and over irregular terrains as well as flat terrain locomotion. The passive elastic element F_p in the stance phase enhanced the stability as self-stabilization. Wagner and Blickhan ⁵² proved in a musculoskeletal model that the intrinsic mechanical properties of muscles can stabilize the oscillatory movement without reflexive changes in activation, called self-stabilization. Kubow and Full ⁵³ also reported that the self-stabilization played an important role in the stability of hexapod locomotion. Leg trajectory control is possible by controlling each joint with joint angle feedback, but in this study, simply exploiting the existing muscle feature F_p facilitated the required leg trajectory control.

1. Stabilization of rolling motion by a non-preprogrammed walk gait autonomously generated via leg loading feedback

The related works ^{34 –42} demonstrated that leg loading feedback played a key role in adjusting the stance duration by contributing to stability in the sagittal plane. In addition, the leg loading feedback to CPG played a pivotal role in stabilizing the balance in the lateral plane in our 3-D locomotion. Specifically, the feedback extended the stance duration of each leg on the side toward which the body tilted in the lateral plane and prevented Nyanko from falling toward that side even during irregular terrain walking, as described in the “Walking with the left feet landing on ground of a higher level” section, even though Nyanko has no joints around the roll axis. In addition, the feedback of different loads between the legs to each CPG modified the default trot, which was the basic gait originally generated by the hard-wired CPG network, and therefore, a walk gait which was not preprogrammed spontaneously emerged (i.e. the phase differences between the four legs in this gait were not explicitly determined by our program, but emerged on their own). In other words, the autonomous gait generation of a walk-through leg loading feedback caused stability in the lateral plane as well as in the sagittal plane, resulting in steady 3-D quadrupedal locomotion. Similarly, in our previous simulation work, ¹² a walk gait was autonomously generated because of the foot loading feedback, but the quadruped model’s nervous system and mechanism were too simple and its motion was limited in the sagittal plane. In this article, we demonstrated in the 3-D quadruped model with more realistic nervous system and mechanism that the autonomous generation of a walk gait stabilized locomotion.

1. Facilitating the tuning of a large number of parameters through three procedures including GA.

We may be able to synergistically drive multiple muscles in the four sequential phases for walking and running by using a simpler controller with fewer parameters. However, if we want to allow our quadruped to achieve a higher level of locomotion and behaviors that real animals can perform in future studies, it is reasonable to build a realistic controller. However, the more realistically we design our model, the more parameters the model will have and the more painstaking the parameter tuning will be. The three closely related works ^36,37,40 even manually tuned the parameters. Maufroy et al. ³⁶ pointed out the difficulty in the parameter tuning of their two-level CPGs, and Markin et al. ⁴⁰ mentioned that special care was taken to tune the model parameters. Our CPG model was simpler than the closely related two-level CPG models, ^36,37,40 but it still had a large number of parameters. However, we proposed three procedures that included a GA, to facilitate the tuning (“Parameter tuning” section).

Conclusion

In this article, we aimed to demonstrate 3-D quadruped locomotion in simulation and designed each leg’s two-level CPG model consisting of the RG and PF parts, which can synergistically drive the multiple muscles of each leg. It was generally difficult for a quadruped model with realistic neural systems and muscle-driven leg mechanisms to walk even on flat terrain, but we successfully demonstrated stable 3-D quadruped locomotion, and based on our results, we suggested four important factors for stable 3-D quadruped locomotion: (1) design the simple PF part to reproduce the realistic trajectory of each leg; (2) set the passive elastic element of each muscle to roughly control each leg’s trajectory and to stabilize each stance leg as self-stabilization; (3) stabilize the rolling motion with a non-preprogrammed walk gait; and (4) facilitate the tuning of a large number of parameters through three procedures including GA. In our simulation results with 3-D four-legged locomotion, the leg trajectories and the motoneuron activities were very close to those recorded in cats even compared to the biped locomotion in related works. In addition, our quadruped model was able to change speed and showed robustness when it walked upslope and over irregular terrains and received perturbations.

We are testing whether our proposed controller works on a prototype cat-like muscle driven quadruped robot (see in the section S1 in Supplementary text). We demonstrate that the robot is capable of safely trotting at speeds of 0.91–1.25 m/s on the treadmill. Therefore, we consider that our proposed controller is promising enough for a muscle-driven quadruped robot.

Supplemental material

Supplemental Material, Supplementary_tables - Three-dimensional walking of a simulated muscle-driven quadruped robot with neuromorphic two-level central pattern generators

Supplemental Material, Supplementary_tables for Three-dimensional walking of a simulated muscle-driven quadruped robot with neuromorphic two-level central pattern generators by Yasushi Habu, Keiichiro Uta and Yasuhiro Fukuoka in International Journal of Advanced Robotic Systems

Supplemental material

Supplemental Material, Supplementary_text - Three-dimensional walking of a simulated muscle-driven quadruped robot with neuromorphic two-level central pattern generators

Supplemental Material, Supplementary_text for Three-dimensional walking of a simulated muscle-driven quadruped robot with neuromorphic two-level central pattern generators by Yasushi Habu, Keiichiro Uta and Yasuhiro Fukuoka in International Journal of Advanced Robotic Systems

Supplemental material

Supplemental Material, Title_and_description_for_supplemental_materials - Three-dimensional walking of a simulated muscle-driven quadruped robot with neuromorphic two-level central pattern generators

Supplemental Material, Title_and_description_for_supplemental_materials for Three-dimensional walking of a simulated muscle-driven quadruped robot with neuromorphic two-level central pattern generators by Yasushi Habu, Keiichiro Uta and Yasuhiro Fukuoka in International Journal of Advanced Robotic Systems