Kinematic-based locomotion mode recognition for power augmentation exoskeleton

Abstract

This article presents a kinematic-based method for locomotion mode recognition, for use in the control of an exoskeleton for power augmentation, to implement natural and smooth locomotion transition. The difference in vertical foot position between a foot already in contact with ground and a foot newly in contact with the ground was calculated via kinematics for the entire exoskeleton and used to identify the locomotion mode with other sensor data including data on the knee joint angle and inclination of the thigh, shank, and foot. Locomotion on five different types of terrain—level-ground walking, stair ascent, stair descent, ramp ascent, and ramp descent—were identified using two-layer decision tree classes. An updating process is proposed to improve identification of the transition and accuracy using the foot inclination at the mid-stance. An average identification accuracy of more than 99% was achieved in experiments with eight subjects for single terrains (no terrain transitions) and hybrid terrains. The experimental results show that the proposed method can achieve high accuracy without significant misrecognition and minimize the delay in locomotion mode recognition of the exoskeleton.

Introduction

An exoskeleton is a mechanical system with active actuators to augment human power. The concept of this type of powered system is that the human provides intelligent control of the exoskeleton while the exoskeleton actuators provide most of the strength necessary for walking. Intensive development of exoskeletons has been accomplished in recent years for purposes of power augmentation, assistance, and rehabilitation. ^{1 –4} Exoskeletons can be employed in many fields, such as the military, construction, rescue teams, and assistance to people with disabilities. In the military, they are commonly used to help soldiers carry heavy loads for long periods of time. In civilian applications, similar exoskeletons are used to help firefighters and other rescue workers function in dangerous environments. The rehabilitation and medical fields are other prime areas for exoskeleton technology: they can be used to help disabled people regain the ability to walk.

A large number of research studies have been devoted to implementation of suitable human–robot interaction methods to control exoskeleton systems. These methods include virtual joint torque, ⁵ variable impedance, ⁶ direct fore feedback, ⁷ and bio-signal-based control. ⁸ To achieve stable and natural walking, the locomotion modes to be used for different types of terrain, such as level ground (LG), stairs, and ramps, need to be identified as quickly as possible when the foot contacts the ground. Instantaneous recognition of a terrain permits active adaptation to terrains, and locomotion transitions enhance safety for the exoskeleton and make walking more natural. Thus, it is important to the performance of augmentation exoskeletons to be able to identify different terrains without delay.

Various methods for locomotion mode recognition with high accuracy and minimal delay have been proposed. Wang et al. ⁹ presented a wearable plantar pressure measurement system for locomotion mode recognition based on the ground reaction force (GRF). This system has been implemented with four force sensors and used with lower-limb amputees. Although the GRF is easy to determine, it is difficult to detect the locomotion mode robustly during the swing phase using this system. Chen et al. ¹⁰ proposed a wearable capacitive sensing system for recognizing locomotion modes. This is inconvenient, however, in that the wearable capacitive sensing system must be attached directly to the skin. Li and Hsiao-Wecksler ¹¹ proposed the use of an inertial measurement unit (IMU) to identify LG walking and stair ascent (SA)/stair descent (SD) by means of a threshold method. The foot position as estimated from accelerometer data and the foot inclination angle at full contact with the ground is used to recognize the gait mode. This approach has the advantage of using fewer sensors and requiring less computation, but there is a delay of one step in identifying the transition. This is a major disadvantage in achieving stable walking with an exoskeleton. Varol et al. ¹² proposed an activity mode intent recognition framework for a powered lower-limb prosthesis to detect standing, sitting, and walking. In this framework, a Gaussian mixture model with linear discriminant analysis (LDA) dimension reduction was used as a classifier. The researchers have not yet demonstrated that the framework has the ability to detect activities associated with stairs and ramps. Chen et al. ¹³ used two foot pressure insoles, three IMUs, and LDA to identify six locomotion modes, that is, SA, SD, ramp ascent (RA), ramp descent (RD), LG, and standing. Yuan et al. ¹⁴ proposed a system for fuzzy-logic-based terrain identification with an IMU and a force-sensitive resistor for use by transtibial amputees. Although this method exhibits high accuracy despite using minimal sensors on various terrains, identification delay of the wearer’s transitions is relatively long. Additionally, the modulation constant of the terrain transition constraint is determined to improve the accuracy. Long et al. ¹⁵ proposed a support vector machine (SVM) optimized by particle swarm optimization to identify different locomotion modes for a rehabilitation robotic exoskeleton. This system has the disadvantage of long mode identification delay, which causes lagging in the exoskeleton control. Furthermore, there is significant misrecognition in this method with regard to ascending and descending motions. Several other research studies have been performed to improve locomotion mode recognition using electromyography (EMG) signals for powered lower-limb prostheses, including those by Au et al., ¹⁶ Huang et al., ^17,18 Young et al., ^19,20 and Joshi and Hahn. ²¹ An EMG-based recognition system is a reasonable solution for an amputee, but there are inherent drawbacks to using EMG, such as the discomfort of sensor attachment and the fatigue produced. Although several approaches to locomotion mode recognition for an augmentation exoskeleton have been proposed, it remains an unsolved problem.

Motivated by this challenge, this study was conducted to develop a method for no significant misrecognition, minimal delay, and high accuracy locomotion mode recognition. The proposed kinematic-based method uses leg kinematic information and foot sensor data from angle sensors, along with an IMU, a load-cell-based foot sensor, and decision trees, to identify the following basic terrain types encountered in activities of daily living: LG, SA/SD, and RA/RD. After detecting the gait phase using GRF, reference coordinates such as the heel-fixed coordinate and toe-fixed coordinate are used to calculate the vertical position of the strike foot (heel strike or toe strike) by kinematics for the whole exoskeleton. Compared to prosthesis, ^14,17,18 consist only of partial joints, the exoskeleton system in this article is totally linked by mechanical structures. Using this kinematic information, the exoskeleton posture can be estimated and used to recognize the locomotion mode. Based on this important information used to classify LG and ascending/descending walking, we can identify the locomotion mode with minimal identification delay. Sensor signals from the exoskeleton and the vertical position are used with the decision tree to identify the locomotion mode. Among the advantages of a decision tree are that it is simple, easy to understand and interpret, and robust. ^{22 –27} In addition, the accuracy of the locomotion mode determination is updated using the foot inclination at the mid-stance. The proposed method is a very simple and intuitive way to improve locomotion mode identification accuracy and minimize the associated delay. The effectiveness of the proposed method was demonstrated by an experiment with eight subjects and single and hybrid locomotion modes.

The remainder of this article is organized as follows. In “Hydraulic lower extremity exoskeleton system” section, the exoskeleton’s data collection system is introduced. The results of analysis of sensor data from locomotion on different types of terrain are presented in the “Sensor data analysis on locomotion” section. “Kinematic-based locomotion mode recognition method” section describes the proposed kinematic-based recognition method. “Experiment protocol and results” section presents the experiment conducted according to the proposed approach and the results obtained. Finally, conclusions are presented in the last section.

Hydraulic lower extremity exoskeleton system

Mechanical part and actuator

Figure 1 shows the exoskeleton system used in this study. The system consists of hydraulic power units (HPUs), a battery, a controller, a sensor signal processor (SSP), a hydraulic actuator, joint modules, and links. The battery, main controller, load, and two HPUs for actuating each leg are installed on the backpack frame. SSPs for acquiring signals from sensors, such as encoders, load cells, and IMUs, are mounted on a pelvic module. This robot has 4 active joints for the sagittal plane (one degree of freedom (DOF) for each of the hip and knee joints) and 10 passive joints (one DOF for the roll and yaw motion of each hip and three DOFs for each ankle). The robot’s range of motion (ROM) is determined from human gait analysis of LG, SA/SD, and RA/RD. The details of the ROM are given in Table 1. The robot link is also designed to have sufficient adjustable length for people from 170 cm to 185 cm in height (with a nominal height of 175 cm). To avoid harming the operator, the exoskeleton joint’s operating range is designed to be smaller than the human joint range and is controlled using a mechanical stopper.

Figure 1.

Mechanical part and joint: (a) hip joint, (b) knee joint, (c) ankle joint, and (d) foot sensor module.

Table 1.

Range of motion of the exoskeleton.

Joint	Axis	Type	Range
Hip	Pitch (Extension/Flexion)	Active	−20° to 100°
	Roll (Adduction/Abduction)	Passive	−20° to 45°
	Yaw (Internal rotation/External rotation)	Passive	−45° to 45°
Knee	Pitch (Extension/Flexion)	Active	0° to 125°
Ankle	Pitch (Plantar flexion/Dorsiflexion)	Passive	−50° to 50°
	Roll (Inversion/Eversion)	Passive	−10° to 15°
	Yaw (International rotation/External rotation)	Passive	−18.5° to 18.5°

As Figure 1(a) shows, the hip joint module consists of passive joints on the roll and yaw axes of each leg, active joints on the pitch axis of each leg, hydraulic actuators, and encoders. The length is adjusted to correspond to the wearer’s thigh length. There is also an adjustment device for changing the shank link length. A strap worn by the wearer is attached to the thigh link to tighten the attachment (Figure 1(b)). The ankle joint module consists of passive joints and the foot structure. The passive joints include the roll, pitch, and yaw axes of each leg and are designed as ball-and-socket-type joints. This type of joint can provide the wearer with comfort in walking in various environments, as shown in Figure 1(c). Four load cells are located on the upper plate for detection of the gait phase. In addition, a flexible panel made of MC Nylon is installed between the front and rear plate for the comfort of foot during bending while walking, as shown in Figure 1(d). The HPU capacity is determined on the basis of human walking data (joint angle and torque) obtained at 6 km/h with a 50-kg payload on LG and during stair ascending/descending and ramp ascending/descending. The hydraulic cylinders were designed to be double-acting at the hip pitch joint and single-acting at the knee pitch joint. The hip and knee cylinders were designed for maximum thrusts of 5 and 2 kN, respectively, and for maximum piston strokes of 72 and 112 mm, respectively.

Sensors and controller

Various types of sensors are used to measure robot joint angles, actuator forces, orientation of link, and GRFs. Four small magnetic encoder sensors (RMB20, RLS, Komenda, Slovenia) are used to measure the joint angles (hip and knee pitch angles) of the robot. Five IMUs (MTi 1-series, Xsens, Enschede, Netherlands) are used to measure the orientation of the body, thigh, and foot. Kalman filtering is applied to address the problem of signal drift and output accurate data. In this system, an important issue is achieving a lightweight exoskeleton design. Thus, angular sensors are not embedded in the passive joints, such as the hip roll and yaw axis of the hip and ankle joints. Joint angle estimation is performed using a direction cosine matrix (DCM) with IMU data for calculation of the kinematics of the robot. ²⁸ In the case of the hip joint, the roll motion of the joint is estimated, but yaw motion is not used, because IMUs yield unreliable yaw data and because observations of the characteristics of human gait have shown that joint yaw motion is negligible. The hip roll motion is calculated by DCM conversion between an IMU installed on the backpack and an IMU installed on the thigh link. The ankle joint angles are calculated by DCM conversion between an IMU installed on the thigh and an IMU installed on the foot link, as well as the angle of the knee joint. In-line load cells (Model 31, Honeywell, Golden Valley, MN, USA) are equipped on each cylinder tube end to detect the actuator force. Foot sensors consisting of four one-axis (normal force) load cells (Model 13, Honeywell, Golden Valley, MN, USA) are used to detect the gait phase. The position of each load cell is chosen based on analysis of the GRF for level walking: hallux (toe), first metatarsal (inside), fourth metatarsal (outside), and heel, as in the study by Kong and Tomizuka, ²⁹ as shown in Figure 1(d). A two-mode gait phase, such as stance and swing, is detected by the threshold method, using the sum of the load cell measurements. In addition, the gait status as a whole, for example, left single stance, right single stance, or double stance (DS), is calculated from the two foot phases. The sensor signal data are transferred by three SSPs to the main controller after low-pass filtering. A high-performance processor (Zynq Dual Cortex-A9, Xilinx, San Jose, CA, USA) with a field-programmable gate array is used as the main controller for locomotion control of the exoskeleton. A data sampling rate of 1 kHz was used for communication and control.

Sensor data analysis on locomotion

The locomotion mode must be recognized correctly to achieve stable walking of a human–robot integrated exoskeleton on various types of terrain. In the case of an exoskeleton for power augmentation, however, it is difficult to identify the locomotion mode because of the large motion variability generated by the load condition and the interface between the human and the robot and because of the variability of gait characteristics. Figure 2 shows the characteristics of each sensor, demonstrated using locomotion data obtained from eight healthy male subjects on LG, stair, and ramp terrains. Although ascending and descending motion can be classified a little more clearly, it is difficult to recognize stair and ramp motion from data associated with the instant of contact, as shown in Figure 2(a) to (d). Locomotion mode identification can be successfully performed using the foot inclination at the mid-stance phase, as shown in Figure 2(d). To control an augmentation exoskeleton stably, the locomotion mode needs to be recognized as soon as the foot makes contact with the ground. The standard deviations of the sensor data for the eight subjects during a gait cycle are presented in Table 2. As the table shows, the sensor values have large standard deviations, depending on the operation mode. This makes locomotion mode classification is difficult. Thus, to identify the locomotion mode more accurately, we propose a kinematic-based method that is constrained with respect to the sensors.

Figure 2.

Sensor variability (mean-1) for a gait cycle from heel strike of right leg: (a) hip angle, (b) knee angle, (c) thigh pitch angle, and (d) foot pitch angle.

Table 2.

Maximum variation of sensor data during a cycle of gait (standard deviation (°)).

	LG	SA	SD	RA	RD
Hip joint (R)	6.5	27.5	10.7	18.1	7.5
Knee joint (R)	10.8	30.0	24.0	15.9	20.9
Thigh IMU pitch (R)	9.7	24.3	9.1	16.4	7.0
Foot IMU pitch (R)	12.0	11.0	16.7	17.4	18.9
Hip joint (L)	7.6	8.0	8.2	9.3	7.8
Knee joint (L)	12.0	15.3	27.0	15.5	19.1
Thigh IMU pitch (L)	8.9	7.2	7.7	8.8	7.1
Foot IMU pitch (L)	11.6	14.1	11.3	13.5	19.8

LG: level ground; SA: stair ascent; SD: stair descent; RA: ramp ascent; RD: ramp descent; IMU: inertial measurement unit; R: right leg; L: left leg.

Kinematic-based locomotion mode recognition method

As explained in the previous section, the motion of a human–robot integrated exoskeleton has a larger intra-subject variability and inter-subject variability than a prosthesis or powered orthosis because the mechanical structure of the exoskeleton is totally linked and constrained. Thus, the joint motion of this system tends to vary according to the interface conditions for connection (e.g. the link strap, backpack strap, and waist belt), load conditions, and terrain. This suggests that it is difficult to identify the locomotion mode for an augmentation exoskeleton. That is why reducing the delay associated with locomotion mode recognition for a power augmentation robot is a technical challenge and why a robust method to identify the mode is needed. In other words, the locomotion mode should be detected as soon as the foot contacts the ground. In the case of a prosthesis, in contrast, some locomotion delay is acceptable because of the slower motion and lighter payload involved and the balancing role of the sound limbs. We propose a kinematic-based method for instantaneous and robust locomotion mode recognition for a power augmentation robot. The proposed approach consists of coordinate selection using the GRF, foot position calculation based on exoskeleton kinematics, and locomotion mode recognition using a decision tree method.

Figure 3 illustrates the overall process involved in the proposed method. First, sensor data are acquired from the robot. Using these data, pivot coordinates for kinematics calculation are obtained. Second, the difference in the vertical positions of the leading strike foot and the trailing fixed foot is calculated using the kinematic relationship. Third, a decision tree method is used to identify the locomotion mode from the sensor data measured for the exoskeleton and the vertical foot positions. Last, the locomotion mode is updated using the foot inclination at the mid-stance to improve the accuracy of the transition motion. The mode identification is dependent on the gait phase. That is, the locomotion mode identified during the stance phase is maintained during the subsequent swing phase and renewed when the other foot becomes the striking foot.

Figure 3.

Overall process for proposed recognition method.

Coordinate selection

Human gait is a very complicated and coordinated series of movements. ^30,31 Walking is divided into two main phases: the stance phase and the swing phase. The stance phase is the weight-bearing portion of each gait cycle. It is initiated by heel strike and ends with the toe of the same foot pushing off the ground. The swing phase is initiated when the toe pushes off and ends with the heel strike of the other foot. During a normal gait cycle, approximately 60% of the time is spent in stance and 40% in swing. The reciprocal action of the two limbs is timed to balance their weight-bearing responsibilities during a period of DS (i.e. when both feet are in contact with the ground) and usually involves the initial and terminal 10% intervals of stance, as shown in Figure 4(a). In addition, three functional rockers exist in the stance phase to ensure smooth walking: heel, ankle, and forefoot rocker mechanisms, as shown in Figure 4(b). When the kinematics of gait motion are calculated, this mechanism must be considered to improve the accuracy and natural motion of the motion. In our approach, just two fixed coordinates (equation (1)), such as the toe and heel coordinates, are used for simplicity. Using the GRF and the foot inclination angle (θ_f) measured from both foot sensor modules, as shown in Figure 5(a) and (b), total gait modes, such as the heel-fixed coordinate and toe-fixed coordinate, are estimated, as shown in Figure 5(c).

Figure 4.

Reference pivot coordinate system: (a) proposed method and (b) human walking.

Figure 5.

Result of total gait mode: (a) GRF, (b) foot inclination, and (c) total gait mode. GRF: ground reaction force.

Total gait mode 1 is the heel-fixed coordinate on the right leg (RHF), mode 2 is the toe-fixed coordinate on the right leg (RTF), mode 3 is the heel-fixed coordinate on the left leg (LHF), and mode 4 is the toe-fixed coordinate on the left leg (LTF), which can be derived as follows

$\text{Total gait mode}=\{\begin{array}{c}1:\text{RHF},\text{ for }{F}_{\text{heel},R}\ge F_{\text{toe},R},{\text{ GRF}}_{\text{sum,}R}\ge \lambda ,{\theta }_{f,L}\ge \beta \\ 2:\text{RTF},\text{ for }{F}_{\text{toe},R}\ge F_{\text{heel,}R},{\text{ GRF}}_{\text{sum,}R}\ge \lambda \\ \begin{array}{c}3:\text{LHF},\text{ for }{F}_{\text{heel},L}\ge F_{\text{toe},R},{\text{ GRF}}_{\text{sum,}L}\ge \lambda ,{\theta }_{f,R}\ge \beta \\ 4:\text{LTF},\text{ for }{F}_{\text{toe,}L}\ge F_{\text{heel,}R},GR{F}_{\text{sum,}L}\ge \lambda \end{array}\end{array}$

1

where λ is the threshold used to determine swing-stance phase and β is the threshold angle used to detect the heel-off of the opposite leg. Through walking experiments, λ was set to 7 kgf and β was set to 9°.

Foot position calculation

To improve the robustness of the pose estimation for the robot, we propose a kinematic-based movement recognition method. The kinematic model for a lower-limb exoskeleton with 14 DOFs in total is shown in Figure 6. To calculate the link position, a Denavit–Hartenberg (D-H) model with seven DOFs for each leg is considered. ^{28 –32} Table 3 shows the corresponding D-H parameters for RHF gait mode. In D-H models, the homogeneous transformation matrix (A) from the (i − 1)th to the ith coordinate is as follows

${{}_{}{}^{i-1}A}_i=T\left(z_{i-1},{\theta }_i\right)T\left(z_{i-1},d_i\right)T\left(x_i,a_i\right)T\left(x_i,{\alpha }_i\right)$

2

Figure 6.

Coordinate change for continuous walking.

Table 3.

D-H parameters for RHF gait mode.

Joint	${\alpha }_i \left(\text{rad}\right)$	$a_i\left(\text{mm}\right)$	${\theta }_i \left(\text{rad}\right)$	$d_i \left(\text{mm}\right)$	Link parameters	Transformation description
1	pi/2	0	pi/2+ ${\text{θ}}_{{\text{fp}}_{\text{r}}}$	0		From base coordinate to heel coordinate (stance leg)
11	0	−l1f	0	−l2f	l1f = 120.0 mm, l2f = 67.4 mm	From heel coordinate to ankle joint coordinate (stance leg)
2	−pi/2	0	${\text{θ}}_{{\text{ar}}_{\text{r}}}$	0		From ankle joint coordinate to ankle roll coordinate (stance leg)
3	−pi/2	0	pi/2+ ${\text{θ}}_{{\text{ap}}_{\text{r}}}$	0		From ankle roll coordinate to ankle pitch coordinate (stance leg)
4	pi/2	0	${\text{θ}}_{{\text{ay}}_{\text{r}}}$	lsh	lsh = 400.0 mm	From ankle pitch coordinate to ankle yaw coordinate (stance leg)
5	−pi/2	0	${\text{θ}}_{{\text{kp}}_{\text{r}}}$	0		From ankle yaw coordinate to knee pitch coordinate (stance leg)
6	−pi/2	0	pi/2+ ${\text{θ}}_{{\text{hy}}_{\text{r}}}$	l1th	l1th = 365.6 mm	From knee pitch coordinate to hip yaw coordinate (stance leg)
7	−pi/2	-l2th	pi/2+ ${\text{θ}}_{{\text{hr}}_{\text{r}}}$	0	−l2th = 34.0 mm	From hip yaw coordinate to hip roll coordinate (stance leg)
8	0	l1p	pi/2+ ${\text{θ}}_{{\text{hp}}_{\text{r}}}$	0	l1p = 174.5 mm	From hip roll coordinate to hip pitch coordinate (stance leg)
81	0	0	pi/2	l2p	l2p = 216.0 mm	From hip pitch coordinate to pelvic coordinate (stance leg)
82	pi/2	0	0	l2p	l2p = 216.0 mm	From pelvic coordinate to body coordinate
83	−pi/2	0	0	−l1p	l1p = 174.5 mm	From body coordinate to pelvic coordinate (swing leg)
9	−pi/2	-l2th	${\text{θ}}_{{\text{hp}}_{\text{l}}}$	0	l2th= 34.0 mm	From pelvic coordinate to hip pitch coordinate (swing leg)
10	pi/2	0	pi/2+ ${\text{θ}}_{{\text{hr}}_{\text{l}}}$	0		From hip pitch coordinate to hip roll coordinate (swing leg)
11	−pi/2	0	pi/2+ ${\text{θ}}_{{\text{hy}}_{\text{l}}}$	l1th	l1th = 365.6 mm	From hip roll coordinate to hip yaw coordinate (swing leg)
12	pi/2	0	${\text{θ}}_{{\text{kp}}_{\text{l}}}$	0		From hip yaw coordinate to knee pitch coordinate (swing leg)
13	−pi/2	0	${\text{θ}}_{{\text{kp}}_{\text{l}}}$	−lsh	lsh = 400.0 mm	From knee pitch coordinate to ankle yaw coordinate (swing leg)
14	pi/2	0	pi/2+ ${\text{θ}}_{{\text{ap}}_{\text{l}}}$	0		From ankle yaw coordinate to ankle pitch coordinate (swing leg)
15	0	0	${\text{θ}}_{{\text{ar}}_{\text{l}}}$	0		From ankle pitch coordinate to ankle roll coordinate (swing leg)

D-H: Denavit–Hartenberg; RHF: heel-fixed coordinate on the right leg.

where

$T\left(Z_{\text{i}-1},{\theta }_i\right)=\left[\begin{array}{cccc}\text{cos}{\theta }_i& -\text{sin}{\theta }_i& 0& 0\\ \text{sin}{\theta }_i& \text{cos}{\theta }_i& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right],T\left(Z_{\text{i}-1},d_i\right)=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& d_i\\ 0& 0& 0& 1\end{array}\right],T\left(x_{\text{i}},a_i\right)=\left[\begin{array}{cccc}1& 0& 0& a_i\\ 0& 1& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right],T\left(x_i,{\alpha }_i\right)=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& \text{cos}{\alpha }_i& -\text{sin}{\alpha }_i& 0\\ 0& \text{sin}{\alpha }_i& \text{cos}{\alpha }_i& 0\\ 0& 0& 0& 1\end{array}\right]$

The result of the calculation is

${}_{}{}^{i-1}A{}_{}^{}=\left[\begin{array}{cccc}\text{cos}{\theta }_i& -\text{cos}{\alpha }_i\text{sin}{\theta }_i& \text{sin}{\alpha }_i\text{sin}{\theta }_i& a_i\text{cos}{\theta }_i\\ \text{sin}{\theta }_i& \text{cos}{\alpha }_i\text{cos}{\theta }_i& -\text{sin}{\alpha }_i\text{cos}{\theta }_i& a_i\text{sin}{\theta }_i\\ 0& \text{sin}{\alpha }_i& \text{cos}{\alpha }_i& d_i\\ 0& 0& 0& 1\end{array}\right]$

3

The homogeneous transformation matrix from each ith link to the base link, [0], can then be represented as

${{}_{}{}^{0}A}_i={{}_{}{}^{0}A}_1{{}_{}{}^{1}A}_2{{}_{}{}^{2}A}_3\cdots {{}_{}{}^{i-1}A}_i$

4

The transformation for foot position calculation is as follows

${{}_{}{}^{\text{ref}}A}_{{\text{front foot}}_R}={{}_{}{}^{{\mathit{\text{fixed coord}}}_L}A}_1{{}_{}{}^{1}A}_2{{}_{}{}^{2}A}_3\cdots {{}_{}{}^{{\mathrm{ankle}}_R}A}_{{\text{front foot}}_R}$

5

${{}_{}{}^{\text{ref}}A}_{{\text{rear foot}}_R}={{}_{}{}^{{\text{fixed coord}}_L}A}_1{{}_{}{}^{1}A}_2{{}_{}{}^{2}A}_3\cdots {{}_{}{}^{{\text{ankle}}_R}A}_{{\text{rear foot}}_R}$

6

where the base coordinate is changed according to the total gait mode. The difference in the vertical foot position is calculated as follows, using the current foot position and the previous foot position

$D{Z}_{\text{Pos}}\left(n\right)=Z_{\text{Pos}}\left(n\right)-Z_{\text{Pos}}\left(n-1\right)$

7

where $Z_{\text{Pos}}\left(n-1\right)\text{ and }{Z}_{\text{Pos}}\left(n\right)$ are the previous and current vertical foot position at initial contact, respectively. $D{Z}_{\text{Pos}}\left(n\right)$ is the difference in the vertical foot position which is calculated when the contact foot is changed, that is, the total gait mode is changed from 1, 2 to 3, 4 and vice versa. The simulation results of the trace of the position of the foot front and rear points for a single step on various terrains are illustrated in Figure 7. Note that the final position is distinguished by the relative position. In the case of ascending motion, the final foot position is higher than the LG, whereas in the case of descending motion, the final foot position is below the LG. This is a simple and intuitive concept to use to classify ascending/descending motion. This additional information plays an important role in identifying the locomotion mode instantaneously when the foot comes into contact with the ground. The foot position is summarized in Table 4. The results are as would be expected and are used to identify the locomotion mode without delay.

Figure 7.

Trace of foot position for a step.

Table 4.

Results of the difference in foot position for a step.

Terrain	Front foot position (mm)	Rear foot position (mm)
LG	−9.4	−8.4
SA	128.3	136.4
SD	−163.6	−154.7
RA	232.4	161.0
RD	−234.2	−144.8

LG: level ground; SA: stair ascent; SD: stair descent; RA: ramp ascent; RD: ramp descent.

Locomotion mode recognition by decision tree method

Using the kinematic information and sensor data measured for the exoskeleton, locomotion mode recognition is performed using a decision tree method that is robust and simple to understand and interpret. The method is illustrated in Figure 8. First, the foot gait phase (swing and stance) and total gait mode (RHF, RTF, LHF, and LTF) are detected from the GRF. Then, the foot position is calculated from the kinematics of the exoskeleton. After integrating data measurements and calculating vertical foot positions from the kinematic information, the locomotion mode is identified using the decision tree method. A two-layer decision tree is used to improve the recognition accuracy. The first layer is non-level layer that is applied when the difference in the vertical foot position between the foot of the trailing leg and the foot of the leading leg is larger than a specific threshold. The probability of LG may be nearly zero in this area. Ascending or descending motion is classified using a decision tree class, independent of the LG data. The next layer is a complex-level layer that applies in the region in which the difference in the vertical foot position between the trailing-leg foot and the leading-leg foot is within the range of LG. The probabilities of ascending or descending motion and LG are mixed in this layer. Hence, a decision tree class with ascending/descending and LG is used to identify the locomotion mode, as shown in Figure 8.

Figure 8.

Algorithm for locomotion recognition.

The combined signal is expressed as follows

$X=[D{Z}_{\text{Pos}},{\theta }_{ST},{\theta }_{SS},{\theta }_{SF},{\theta }_{SKA},{\theta }_{OT},{\theta }_{OS},{\theta }_{OF},{\theta }_{OKA}]$

8

where ${\theta }_{ST}, {\theta }_{SS}, {\theta }_{SF},\text{ and }{\theta }_{SKA}$ are the thigh, shank, and foot inclinations measured by IMUs and the knee pitch angle measured by the encoder in a newly striking leg and ${\theta }_{OT}, {\theta }_{OS}, {\theta }_{OF},\text{ and }{\theta }_{OKA}$ are the thigh, shank, and foot inclinations measured by IMUs and the knee pitch angle measured by an encoder in contact with the leg, as shown in Figure 8. δ and ε are thresholds for distinguishing between the complex-level layer and the non-level layer. In this study, δ was set to 100 mm and ε was set to −120 mm

$\text{Decision tree}=\{\begin{array}{c}\text{Non-level layer}:DT1,\text{ for }D{Z}_{\text{Pos}}\left(n\right)\ge \delta \\ \text{Non-level layer}:DT2,\text{ for }D{Z}_{\text{Pos}}\left(n\right)\le \epsilon \\ \begin{array}{c}\text{Complex-level layer}:DT3,\text{ for }0\le D{Z}_{\text{Pos}}\left(n\right)<\delta \\ \text{Complex-level layer}:DT4,\text{ for }\epsilon \le D{Z}_{\text{Pos}}\left(n\right)<0\end{array}\end{array}$

9

In our approach, the hip encoder signal is excluded because the base value of the signal changes depending on the load condition. For instance, a human’s upper body tends to bow as a backpack load increases. The classification and regression tree (CART, Breiman et al. ²² ) method used is a decision tree method, which partitions the predictor space into rectangular regions. Using the classregtree function in MATLAB, the CART algorithm is applied to calculate the decision tree classes. In this function, the Gini index is used as the error index that is needed to identify the best value for classification

$\text{Gini diversity index}\left(\text{GDI}\right)=1-{\displaystyle \sum }_i{p}^2\left(i\right)$

10

where the sum is calculated over the i classes at the node and p(i) is the observed fraction of classes of class i that reach the node. A node with just one class has a Gini index of 0; otherwise, the Gini index is positive. The Gini index is thus a measure of node impurity.

The results of the classification for a specific signal using the decision tree are also shown in Figure 9. The decision tree class RA-SA and decision tree RD-SD is the classification class for the non-level layer. The rest of the class, such as decision tree LG-RA-SA and decision tree LG-RD-SD, is included in the complex-level layer. As Figure 9 shows, the number of leaf nodes of the complex-level layer is larger than that of the non-level layer because LG is included. The decision tree classification can be implemented easily for real-time application using “if-then” functions.

Figure 9.

Results of decision tree: DZ_Pos is the difference of the vertical foot position; ${\theta }_{ST},{\theta }_{SS}\text{, }{\theta }_{SF}, \text{ and }{\theta }_{SKA}$ are thigh, shank, and foot inclination and knee pitch angle for newly striking leg; and ${\theta }_{OT},{\theta }_{OS}\text{, }{\theta }_{OF}\text{, }\text{ and }{\theta }_{OKA}$ are thigh, shank, and foot inclination and knee pitch angle for opposite leg as shown in Figure 8.

Updating of locomotion mode

In our approach, gait-phase-dependent classification is performed: The locomotion mode is determined at the initial contact and maintained until the other foot makes contact with the ground. This suggests that there is no way to improve the accuracy if the locomotion mode is misidentified at initial contact. Therefore, the locomotion mode is updated using the foot inclination at the mid-stance, when both feet are in contact with the ground, as shown in Figure 10

$\text{Update}=\{\begin{array}{c}\begin{array}{c}\text{LG},\text{ for}-{\varphi }_{\text{LG}}<{\theta }_f<{\varphi }_{\text{LG}}\text{ and }-\rho <D{Z}_{\text{Pos}}<\rho \\ \text{SA},\text{ for }-{\varphi }_{\text{LG}}<{\theta }_f<{\varphi }_{\text{LG}}\text{ and RA }\end{array}\\ \text{SD}, \text{for} -{\varphi }_{\text{LG}}<{\theta }_f<{\varphi }_{\text{LG}} \text{and RD} \\ \begin{array}{c}\text{RA},\text{ for }{\theta }_f<{\varphi }_{\text{RA}}\text{ and SA }\\ \text{RD},\text{ for }{\theta }_f>{\varphi }_{\text{RD}}\text{ and SD }\end{array}\end{array}$

11

Figure 10.

Updating of locomotion: (a) transition from RA to LG and (b) transition from ramp descent to LG. RA: ramp ascent; LG: level ground.

where ${\varphi }_{\text{LG}},{\varphi }_{\text{RA}},\text{ and }{\varphi }_{\text{RD}}$ represent the threshold angles for level walking, ramp ascending, and ramp descending, respectively. ρ denotes a threshold displacement to detect level walking. In this study, ${\varphi }_{\text{LG}},{\varphi }_{\text{RA}},{\varphi }_{\text{RD}}$ , and ρ are set to 3°, −10°, 14°, and 40 mm from analyzing experimental data, respectively.

Experiment protocol and results

Experiment protocol

To assess the performance of the proposed method, eight healthy male subjects participated in a set of experiments. Those eight subjects had an average height of 1.75 ± 0.05 m, an average weight of 76.63 ± 7.58 kg, and an average age of 34.63 ± 3.53 years. Before the experiments, the exoskeleton system was tested to ensure that it worked well. All of the experiments in locomotion mode recognition were conducted without actuator action. The experiments consisted of two types of tests. First, the subjects were required to walk on a single terrain, such as LG (using a treadmill) or SA/SD or RA/RD, as shown in Figure 11. Each experiment included five trials for each terrain and subject. The experiment data for level walking were measured with each subject walking on a treadmill at 4 km/h speed.

Figure 11.

Experiment photographs for single terrain.

Second, the subjects were required to perform locomotion on hybrid terrains: LG, SA, LG, RD, LG, RA, LG, SD, and LG, as shown in Figure 12. Parameters such as threshold values and decision tree classes for locomotion mode recognition were derived using the experimental data. For performance validation of multiple wearers on a hybrid terrain, leave-one-out cross-validation was applied by excluding each subject data set in the training to obtain decision tree classes.

Figure 12.

Experiment schematic for hybrid terrains.

Performance evaluation

A confusion matrix to describe the error distribution between terrains was used to make reasonable evaluations of recognition performance. Each column of the matrix represents the instances in a predicted terrain, and each row represents the instances in an actual terrain. The matrix is defined as follows ^14,15

$C=\left(\begin{array}{cccc}{c}_{11}& c_{12}& \cdots & c_{15}\\ c_{21}& c_{22}& \cdots & c_{25}\\ ⋮& ⋮& \ddots & ⋮\\ c_{51}& c_{52}& \cdots & c_{55}\end{array}\right)$

12

Each element of the confusion matrix is defined as follows

$c_{ij}=\frac{n_{ij}}{{\sum }_{j=1}^n{n}_{ij}}\times 100\text{\%}$

13

where n_ij is the number of events of the ith terrain being recognized as the jth terrain. The denominator is the total number of events for the ith terrains. To quantify the overall accuracy of the classification, an average identification accuracy (AIA), defined as follows, is used to evaluate the accuracy of the classifier

$\text{AIA}=\frac{N_{\text{correct}}}{N_{\text{total}}}\times 100\text{\%}$

14

where N_correct is the number of correct identification and N_total is the total number of test events.

Results analysis and discussion

Using the proposed method, locomotion mode recognition on a single terrain (LG walking, SA/SD, and RA/RD), not including transition motion, was performed. The AIA (equation (14)) was 99.15%. The confusion matrices are shown in Figure 13(a). The elements of the confusion matrices (equation (12)) are mean and standard deviations for the eight subjects.

Figure 13.

Results of recognition accuracy: (a) single terrain, (b) hybrid terrain without update, and (c) hybrid terrain with update.

As Figure 13(a) shows, the maximum identification accuracy (100%) was achieved for the LG and SD modes, and the minimum identification accuracy (96.81%) was achieved for the RD mode. The reason for this is that the RD locomotion mode was misrecognized as the LG and SD mode because of the small differences in the vertical foot position at the end of the RD in some cases. In addition, tests in locomotion mode recognition on a hybrid terrain were conducted. The subjects moved through all of the terrains as shown in Figure 12. The AIA for the hybrid terrains was 93.05%. The confusion matrix is shown in Figure 13(b). The rate misrecognition was greater than that for a single terrain because transition between modes of locomotion was included. The result shows that the maximum identification accuracy (99.6%) was achieved for the RA mode and the minimum identification accuracy (74.12%) was obtained for stair descending. Large identification errors occur mainly between the RD and the SD and between the RA and the SA at the transition stage. Example results for one subject for locomotion mode classification for the hybrid terrain are shown in Figure 14(a). The first plot in this graph shows the result for the difference in the vertical position as calculated from the kinematics of the robot. This result shows that the difference in the foot vertical position changes according to the terrain type. The second plot shows the gait mode results for the right, left, and total gait modes determined using equation (11). The locomotion mode identification results are shown in the last plot. For this subject, all of the locomotion modes were identified perfectly using the proposed method. In the case of the hybrid terrain, the transition motion is included. So, without an updating process, some transition modes are misidentified, as shown in Figure 14(b). For example, the transition from RA to LG or from RD to LG is recognized as RA or RD at the initial contact, respectively. Updating the locomotion mode using the foot inclination at the mid-stance, as shown in Figure 10, plays an important role in improving the accuracy. After updating the recognition process, we were able to improve the accuracy of the locomotion mode identification, as shown in Figure 14(b). The AIA with updating was 99.07%. The confusion matrix is shown in Figure 13(c). In this update, there is 0–80 ms time delay (Δt) in the estimation of the mid-stance for updating and this small delay comprises 0–5.13% of the gait cycle. Note that the significant misrecognition that caused unstable walking when the system was controlled via locomotion mode was not observed in confusion matrix results, unlike in some previous research. ^15,18 For example, ascending motion such as stair and RA was not recognized as descending motion mode such as stair and RD. This suggests that we can be assured that the exoskeleton can be operated to produce similar motions at least, which is an important safety concern in using the exoskeleton for human–robot interaction.

Figure 14.

Results of hybrid terrain locomotion: (a) mode recognition results and (b) updating locomotion mode.

In summary, the experimental results show that the proposed approach is effective in locomotion mode identification for an augmentation exoskeleton on hybrid terrain. These results show that the proposed method is effective in terms of high identification accuracy (Table 5), minimal misrecognition, and small delay in identifying the locomotion mode in comparison to previous research. ^14,15 The identification delay that causes increased wearer’s muscle fatigue is minimized because identification is performed instantaneously when the foot contacts the ground. We also achieve minimal misrecognition on integrating kinematic relation with sensor data and applying two-layered decision tree method. In addition, the sensor installation involved is easy because of the types of sensors mounted on the exoskeleton, for example, encoders, IMUs, and load cells (in contrast to EMG sensors). Notwithstanding the above advantages, the proposed method does not consider walking speed variations and atypical terrain that need to be considered for practical application. Furthermore, it is necessary to analyze the effect of motion recognition on locomotion control of exoskeleton robot through real-time implementation.

Table 5.

Results of average identification accuracy.

	Single terrain	Hybrid terrains (without update)	Hybrid terrains (with update)
AIA	99.15%	93.05%	99.07%

AIA: average identification accuracy.

Conclusions

In this article, we presented a kinematic-based method to recognize the locomotion mode of an augmentation exoskeleton. Using differences in the vertical foot position and sensor data measured from the exoskeleton, five representative types of terrain encountered in daily life (LG, SA, SD, RA, and RD) can be identified with a decision tree method. This approach achieves a performance improvement to identify the locomotion mode at the initial contact in comparison to the delayed locomotion mode recognition achieved in previous studies. In addition, updating the locomotion mode using the foot inclination at the mid-stance was found to improve the recognition of the transition mode. The AIA achieved for eight subjects was 99.15% for a single terrain and 99.07% for a hybrid terrain. The experimental results show that the proposed method can achieve high accuracy without significant misrecognition and minimize the delay for locomotion mode recognition of the exoskeleton. Future work will focus on integrating this locomotion recognition method into the locomotion control architecture to achieve smooth and natural movement of an augmentation exoskeleton.