Sage Journals: Discover world-class research

Abstract

Sit-to-stand transfer is a very common and critical movement of daily life in elderly individuals, especially independent elderly individuals. However, most assistive robots do not have a sit-to-stand transfer function. In this article, a multi-fuzzy Sarsa learning-based sit-to-stand motion control method for walking-support assistive robot was proposed. First, the mechanical design of walking-support assistive and sit-to-stand transfer motion control problems were introduced. Then, the fuzzy Sarsa learning method, which is a model-free algorithm, was used to design the motion control algorithm for the human–robot system. To realize natural and intuitive sit-to-stand transfer movement for a human–robot system, the interactive force between the robot and human and the error position between the real-time center of mass and reference center of mass were state variables of the proposed fuzzy Sarsa learning-based sit-to-stand motion control algorithm. Considering the computing efficiency of the controller, a multi-fuzzy Sarsa learning -based motion control algorithm was developed to realize natural sit-to-stand transfer motion. Finally, the experimental results verify the effectiveness of the proposed algorithm.

Keywords

Sit-to-stand transfer walking-support robot fuzzy Sarsa learning the center of mass

Notation

$M (q)$ : inertia matrix of the human body model

$C (q, \dot{q})$ : Coriolis effects matrix of the human body model

$G (q)$ : gravitational forces vector of the human body model

q: joint angle of the user

$τ_{robot}$ : driving torque of the walking-support assistive robot

$τ_{hum}$ : active joint driving torque of the user

d: disturbance of the dynamic model

m_i : the mass of the user’s segment

${\hat{p}}_{i}$ : the mass position of the user’s segment

$p_{com}$ : the center of mass (COM)

s_t : state variable at instant t of RL algorithm

a_t : the selected action at instant t of RL algorithm

$p (s_{t}, a_{t}, s_{t + 1})$ : transition probability of state transformation

r_t : the reward at instant t of RL algorithm

Q: action value of RL algorithm

$γ$ : discount factor

α: learning rate

n: the number of state variable

m: the number of fuzzy rule

x_i : $i - t h$ input state vectors of FSL algorithm

c_t : system output of the FSL algorithm at instant t

$μ_{i} (x_{j})$ : normalized firing strength of the $i^{th}$ rule for state x_j

R: the rules of the T-S system

$ω^{i j}$ : approximate value of the $j^{th}$ candidate action in the $i^{th}$ rule

$L_{i}^{j}$ : linguistic term for variable x_j under rule R_i

T: temperature factor

o: the index of the optimal action

F_c : interactive force of the human–robot hybrid system

$e_{x}, e_{z}$ : trajectory errors of the user’s COM in the horizontal direction and vertical direction

$v_{x}, v_{z}$ : velocities of linear actuator and mobile base of the robot in the horizontal direction and vertical direction

Introduction

Due to technological advancements and extended human longevity, over the last 60 years, the rate of aging has accelerated markedly. Moreover, elderly nursing services spend too many health-care resources and money to meet elderly people’s needs for rehabilitation and independent living. Therefore, developing auxiliary support equipment that provides physical support and recovery and allows the elderly’s to walk and perform daily activities is urgently required.¹ Due to cognitive disorders and lower limb dysfunction, standing up and sitting down are prevalent and critical activities of daily life in elderly individuals, especially independent elderly individuals. Falls often occur during walking, standing up, and sitting down.² Hence, the sit-to-stand (STS) transition is considered critical to an individual’s quality of life and functional independence.³ Thus, there is a great need for walking-assist robotic systems to help elderly or disabled people stand up independently.

Currently, walking-assist robotic systems with STS transfer functions can be categorized into two types: exoskeleton robots and walking vehicle robots.⁴ The exoskeleton robots can help the use to stand up and sit down. However, few walking vehicle robots with STS transfer functions have been developed. Mederic et al.⁵ designed an assist device that provides physical support to elderly people during walking and STS transfer. Chuy et al.⁶ developed a robotic walking support system called “Walking Helper II.” Omar et al.⁷ described a mobility assisting device that helps patients who do not have enough physical strength in their lower limbs during STS transfer.

The control theories for walking-assist robotic STS transfer can be grouped into three categories.⁸ (1) Motion control is the first category. Mederic et al.⁵ described the trajectory generation problem of assistive device handles with interpolating cubic splines. Then, by guiding the angle and height of the support plate, the assistive device followed the desired trajectory. Omar et al.⁷ proposed a robot motion control algorithm that follows the natural pattern of human motion during STS transfer and provides assistance force to the user’s shoulder. (2) Force control is the second category. Mederic et al.⁹ calculated the zero moment point of the user and controlled the interaction force between the robot and the user to maintain balance during STS transfer. (3) Switching control is the third category. Different control methods were used in the different phases of STS transfer.¹⁰ In the lifting body phase, a damping control method was implemented with a force reference, and in other stages of STS transfer, compliant impedance control was used in other stages of STS transfer.

In fact, elderly or disabled people are encouraged to practice walking and STS transfer to prevent the degeneration of residual mobility abilities. Therefore, the motion control algorithm for assistive robots should consider the user’s motion intention and give a degree of assistance to the user rather than replace the user’s motion capability completely. To encourage the user to practice daily activities as much as possible, the coupled human–robot system should behave naturally and intuitively. Hence, the assistive robot should have the ability to understand and imitate human behavior during STS transfer. However, establishing an exact model for the coupled human–robot system is difficult because accurate active joint torque values are difficult to obtain. In addition, time-varying uncertainties will be raised from the coupled human–robot system and environmental interactions.

The good performance of traditional adaptive control methods rely on an exact system model.^11,12 Hence, model-free adaptive control techniques are feasible for these problems. Jabbari et al.¹³ proposed a neural network-based trajectory tracking control method for robotic exoskeletons, and Hussain et al.¹⁴ designed an adaptive impedance controller for a robotic orthosis. To solve the structure and parameters uncertainty and disturbance problem of upper limb exoskeleton, Li et al.¹⁵ utilized adaptive fuzzy approximators to estimate the dynamical uncertainties of the human–robot system, and an iterative learning scheme was employed to compensate for unknown time-varying periodic disturbances.

Over the last several years, reinforcement learning (RL) has catched the research attention in terms of exoskeletons and walking-assist devices.¹⁶ Huang et al.¹⁷ designed a hierarchical interactive learning (HIL)-based control strategy to learn piecewise linear models of lower limb exoskeletons. High-level motion learning is dynamic movement primitives combined with locally weighted regression, and the lower-level motion learning method is RL. In control paradigms, the RL-based interactive learning control method can reduce the complexity of the coupled human–robot system, adapt to uncertainty and address varying interaction dynamics problems.¹⁸ RL-based motion control methods are mainly used in the trajectory tracking motion control of exoskeletons and walking-assist devices and rarely involve STS transfer. Therefore, in this article, RL method was used in STS transfer for walking-support assistive robot. The serious drawback of RL is the dimension explosion problem, which is the result of the large dimension of the discrete state–action pairs. To solve this problem, fuzzy RL (FRL) is usually used.¹⁹ Fuzzy Sarsa learning (FSL) is a well-known RL method that has a convergence theorem. Compared to fuzzy Q learning, FSL has a significantly higher learning speed and action quality.¹⁹ Therefore, we develop the FSL-based STS motion control for the walking-support assistive robots.

The main contributions of the article include the following:

Propose the RSL-based motion control method for the STS motion control of walking-support assistive robots;

Decrease the dimension of state–action pairs by using two FSL-based STS motion controllers to control the linear actuator and mobile base of the robot;

Apply the proposed method in practice for the STS motion control of walking-support assistive robots.

In this article, a multi-FSL-based STS motion control method for walking-support assistive robot is proposed. First, the walking-support assistive robots is introduced. Two value indexes of the STS problem for walking-support assistance are proposed. Then, the FSL algorithm is described. To realize the natural and intuitive STS transfer movement, two FSL-based motion control algorithms are designed. Finally, some experiments were conducted to validate the effectiveness of the proposed multi-FSL-based STS motion control algorithm.

Walking-support assistive robot

Mechanics of the walking-support assistive robot

An overview of the walking-support assistive robot is described in Figure 1. It consists of a mobile base, an STS support subsystem, and a human–robot interactive (HRI) system. The mobile base consists of two passive casters and two differential driving wheels. The two passive casters are mounted under the controller, and the two driving wheels are installed on the extended base, as shown in Figure 1. The extended base extends the robot’s support area and provides relatively safe and comfortable support. The STS support subsystem contains a linear actuator and a support frame. The linear actuator is the power source of the STS support subsystem, which helps the user realize STS transfer movement. To increase the support area and improve stability and safety, the support frame has two jointed supporting bars, as shown in Figure 1.

Figure 1.

The walking-support assistive robot. (a) The side view of the robot and (b) the front view of the robot.

The HRI system includes two force-sensing resistance (FSR)-based HRI subsystems and wearable sensor (WS)-based HRI subsystem.

(i) FSR-based HRI subsystem

Two FSR sensors are installed underneath the two support plates, as shown in Figure 2, to detect the interactive forces between the user and the robot, which reflects the user’s intention of standing up.

(ii) WS-based HRI subsystem

To realize a natural and intuitive STS transfer movement, identifying user body parts is critical. The triple inverted pendulum has been widely used as a simplified biomechanical model of the human body in the STS transfer literatures.^10,20 In this article, we also used a triple inverted pendulum to represent the human body, as shown in Figure 3. To obtain the real-time position information of the human body, five wearable sensor units were attached on the user’s waist, thighs, and shanks, as shown in Figure 4. A wearable sensor unit consists of a triaxial magnetometer, a triaxial accelerometer, and a triaxial gyroscope for measuring the acceleration and angular velocity along three orthogonal axes simultaneously. The triaxial accelerometer and magnetometer of each sensor unit are integrated on a chip (LSM303). The product model of the triaxial gyroscope is MPU3050. An STM32F020 Micro Control Unit (MCU) is used to collect data for all sensor units, which detect the position of the user’s trunk, thighs, and shanks; the details can be seen in the literature.²¹

Figure 2.

The FSR-based human–robot interactive subsystem. FSR: force-sensing resistance.

Figure 3.

The triple inverted pendulum model of the human body.

Figure 4.

Wearable sensors–based human–robot interactive subsystem.

Dynamic model of human body model

The walking-support assistive robot and human consist of a coupled human–robot system, which is shown in Figure 3. The dynamic model of the human body model can be expressed as follows

M (q) \ddot{q} + C (q, \dot{q}) \dot{q} + G (q) = τ_{robot} + τ_{hum} + d

However, obtaining accurate driving torque $τ_{hum}$ of active joints is difficult. Then, in this article, the RL method, which is a model-free algorithm, was used to design the control law of the walking-support assistive robot.

STS transfer problem for walking-support assistive robot

STS is a common movement but difficult movement of daily life in the elderly, as humans require a high amount of energy and joint force to stand up. However, it is dangerous for the elderly to stand up without any auxiliary instruments. In the process of STS transfer movement, safety and compliance are most important for the elderly. To ensure the user’s safety, the robot should maintain human balance and postural stability during STS transfer.

In consideration of the requirements of natural and intuitive STS transfer, the STS transfer motion control for the robot should consider the following two points.

(i) Center of mass (COM): The COM is widely used to evaluate human balance²² and represents the balance point of an object’s mass. When the system is balanced around its COM, then it is in a state of equilibrium. In this article, the COM of the user can be calculated by the following equation

p_{com} = \frac{\sum_{i = 1}^{5} m_{i} {\hat{p}}_{i}}{\sum_{i = 1}^{5} m_{i}}

where m_i and ${\hat{p}}_{i}$ are the mass and mass positions of the human segment. The trajectory of the human’s COM during the STS transfer movement is shown in Figure 5.

Figure 5.

The trajectory of the human’s COM during the STS transfer movement. COM: center of mass; STS: sit-to-stand.

(ii) Interactive force: The interactive forces between the robot and the user express the user’s motion intention. At the beginning of the STS transfer, a large value of interactive forces means that the user wants to stand up. In the process of STS, the interactive forces decrease slowly. When user stands almost straight, the interactive forces approach zero at the end of the STS transfer.

Fuzzy Sarsa learning

The human user and walking-support assistive robot constitute a coupled human–robot system, and establishing an accurate mathematical model for this system is difficult. RL is an online model-free incremental learning technique that is widely used in coupled human–robot system.^23,24 However, the large dimensionality of discrete state–action pairs will cause the curse of dimensionality. Thus, in this article, FRL was used in the motion control of walking-support assistive robots because it can overcome the curse of dimensionality problem.²⁵

Before designing the robot motion control algorithm, a brief explanation of the RL and Sarsa learning (SL) algorithms is introduced in this section.

Reinforcement learning

In RL, at each time step t, the agent observes the current state s_t of the environment. Then, the agent executes action a_t from the set of actions A under the action selection policy $π$ and obtains reward r_t . Consequently, the state transfers to $s_{t + 1}$ with transition probability $p (s_{t}, a_{t}, s_{t + 1})$ . Then, the agent selects action $a_{t + 1}$ and obtains reward $r_{t + 1}$ . The interactive cycle between an agent and the environment will eventually converge to the best result. The schematic diagram of RL is shown in Figure 6.

Figure 6.

The schematic diagram of RL. RL: reinforcement learning.

The value of action a in state s under policy $π$ can be estimated according to the following formula

Q^{π} (s, a) = E_{π} {Σ_{k = 0}^{\infty} γ^{k} r_{t + k + 1} s_{t} = s}

where $γ$ is the discount factor, with $0 \leq γ \leq 1$ .

The most popular action policies include the $ε$ -greedy action selection method and Soft-Max method. The $ε$ -greedy action selection method selects the action (or one of the actions) with the highest estimated action value Q. The Soft-Max method ranks and weights all actions with their value estimates Q, and then the action with the highest selection probability is selected.

Sarsa learning

SL is a well-known RL methods that is an online model-free learning technique. SL has been widely applied to a variety of problems in robotics. Herein, the SL algorithm is used in the motion control of STS transfer for walking-support assistive robots.

In the SL algorithm, the future reward is the actual reward for executing real action. Therefore, the value of action $Q (s, a)$ in state s is updated with the value of action and the current policy using equation (4)

\begin{array}{l} Q (s_{t}, a_{t}) \leftarrow Q (s_{t}, a_{t}) + α_{t} [r_{t + 1} + γ Q (s_{t + 1}, a_{t + 1}) \\ - Q (s_{t}, a_{t})] \end{array}

where $r_{t + 1}$ denotes the immediate reward received from the environment after applying action a_t in state s_t .

Fuzzy Sarsa learning

FSL is an extension of SL. A zero-order T-S fuzzy system is used to approximate the function of a continuous state and action space. In the FSL algorithm, the input state vector $s = {x_{1}, x_{2}, \cdot \cdot \cdot, x_{n}}$ , n is the number of input state vectors. For the STS motion control of the human–robot system, the input state variable x ₁ is the trajectory error of the user’s COM and desired COM and x ₂ is the HRI force, as shown in Figure 7. The schematic diagram of FSL is shown in Figure 7. In this figure, the rule firing strengths of each state variable are generated by a Gaussian membership function, and the normalized firing strength of $i^{th}$ rule for state x_j is $μ_{i} (x_{j})$ . In each rule, the possible discrete control action set is $c = {u_{1}, u_{2}, \cdot \cdot \cdot, u_{m}}$ , which consists of the velocity of the linear actuator. The $R_{i} (i \in N)$ rules of the T-S system is as follows

Figure 7.

The whole process of the FSL algorithm. FSL: fuzzy Sarsa learning.

\begin{array}{r} R_{i} : I f x_{1} i s L_{i}^{1}, and \cdot \cdot \cdot, and x_{n} is L_{i}^{n}, \\ then u = u^{i 1} with ω^{i 1} \\ or u = u^{i 2} with ω^{i 2} \\ \cdot \cdot \cdot \\ or u = u^{i m} with ω^{i m} \end{array}

where $ω^{i j}$ is the approximate value of $j^{th}$ candidate action in $i^{th}$ rule and $L_{i}^{j}$ is the linguistic term for variable x_j under rule R_i .

Then, the system output at instant t is calculated as follows

c_{t} (s_{t}) = \sum_{i = 1}^{N} μ_{i} (s_{t}) u^{i o}

where o is the index of optimal action and $u^{i o}$ is the optimal action in rule R_i according to the following Softmax policy:

p (u^{i j}) = \frac{exp (μ_{i} ω^{i j} / T)}{Σ_{k = 1}^{m} exp (μ_{i} ω^{i j} / T)}

The action value function is computed by the following equation

Q_{t} (s_{t}, c_{t}) = \sum_{i = 1}^{N} μ_{i} (s_{t}) ω_{t}^{i o}

where $ω_{t}^{i o}$ is the weight parameter of optimal action in rule $R i$ .

The system executes the inferred optimal action c_t at instant t. Then the system transfers to the next state at instant $t + 1$ and generates a reinforcement signal ‘r‘. This signal r is used to calculate the approximate action value error $Δ Q_{t} (s_{t}, c_{t})$ as follows

Δ Q_{t} (s_{t}, c_{t}) = r (s_{t}, c_{t}) + γ Q_{t} (s_{t + 1}, c_{t + 1}) - Q_{t} (s_{t}, c_{t})

The weight value $ω (s_{i}, u_{j})$ of the $i^{th}$ rule is updated by the following equation

w_{t + 1}^{i j} = w_{t}^{i j} + α Δ Q_{t} (s_{t}, c_{t}) μ_{i} (s_{t})

If j is the index of the selected action weight, then $Δ Q_{t} (s_{t}, c_{t}) μ_{i} (s_{t}) = 0$ .

The whole process of FSL is summarized in Table 1.

Table 1.

The process of FSL algorithm.

Input: s_t

Output: c_t

1. Get the observe state

s_{t + 1}

and reward value

r_{t + 1}

2. Choose an action for each rule by equation (7).

3. Calculate the system output action

c_{t + 1}

and Q_t according to equations (6) and (8), respectively.

4. Calculate the

Δ Q_{t}

and

ω

according to equations (9) and (10), respectively.

5. Calculate the new approximate

Q_{t + 1}

using equation (8).

6. Send the system output c_t to the robot controller.

7. t = t + 1, return to step 1.

FSL: fuzzy Sarsa learning.

Multi-FSL-based STS motion control for the walking-support assistive robot

Elderly individuals are encouraged to practice STS transfer movements to prevent the degeneration of residual mobility ability. Hence, a walking-support assistive robot is designed to help the user stand up in this article. Actually, the robot should assist humans in standing up independently rather than pulling them up. That is, the motion controller of the robot should take the user’s motion intention fully into account, behave naturally and intuitively, and follow human STS movements. Then, the input state variables are the interactive force F_c in the coupled human–robot system and the trajectory errors $e = {e_{x}, e_{z}}$ of the user’s COM in the horizontal direction and vertical direction. The system outputs are the velocities $v = {v_{x}, v_{z}}$ of the linear actuator and mobile base of the robot in the horizontal direction and vertical direction, respectively. Then, the control object of the proposed algorithm obtains the optimal velocities of the linear actuator and mobile base of the robot, which helps the user naturally and intuitively realize STS transfer movements

e_{x} = x_{com} - x_{rcom}, e_{z} = z_{com} - z_{rcom}

where $x_{com}$ and $x_{rcom}$ are the actual position and reference position of the COM in the horizontal direction and $z_{com}$ and $z_{rcom}$ are the actual position and reference position of the COM in the vertical direction.

To decrease the dimensionality of state–action pairs, two FSL-based STS motion controllers were used in this article to control the linear actuator and mobile base of the robot, as shown in Figure 8. The FSL-based motion controller 1 was used to generate the control command of the linear actuator. The input state variables were F_c and e_z , and the output was v_z . FSL-based motion controller 2 was used to generate the control command of the mobile base in the horizontal direction. The input state variables were F_c and e_x , and the output was v_x . And F_c and e_z are uniformly partitioned by five fuzzy sets, e_x is uniformly partitioned by three fuzzy sets. The defined Gaussian membership functions for the three inputs are shown in Figure 9.

Figure 8.

The multi-FSL-based STS motion control for the walking-support assistive robot. FSL: fuzzy Sarsa learning; STS: sit-to-stand.

Figure 9.

The membership degree of three input state variables.

The output of FSL-based motion controller 1 determines the motion velocity of the linear actuator. The candidate actions set is $U = {0,0.01,0.03,0.07,0.1,0.12,1.15} m/s$ . The reward function is defined as follows

r {(t)}_{1} = {\begin{array}{l} 1, & e_{z} = 0 \\ 0.7, & e_{z} \leq 10 cm, {\dot{e}}_{z} < 0 \\ 0.3, & 10 cm < e_{z}, {\dot{e}}_{z} < 0 \\ 0, & {\dot{e}}_{z} > 0 \end{array}

The output of FSL-based motion controller 2 determines the motion velocity of the mobile base. The candidate actions set is $U = {- 0.01,0,0.005,0.01,0.02,0.03,0.04} m/s$ . The reward function is defined as follows

r {(t)}_{2} = {\begin{array}{l} 1, & e_{x} = 0 \\ 0.5, & e_{x} \leq 10 cm, {\dot{e}}_{x} < 0 \\ 0, & {\dot{e}}_{x} > 0 \end{array}

Experimental evaluation

In this section, some experiments were conducted to verify the proposed multi-FSL-based STS motion control algorithm for walking-support assistive robots. First, the process of the experiment was introduced. Then, to verify the effectiveness of the proposed algorithm, some STS experiments with walking-support assistive robots were performed and a discussion was provided after the experiments.

Experiment process

The experiments were conducted by one subject (aged 33, height 160 cm, weight 52 kg). According to the method of calculating the mass of human segments,²⁶ m_i is shown in Table 2. In the experiment, the subject was asked to wear wearable sensor units, as shown in Figure 4. Then the subject grabbed the handles and sat on the chair, as shown in Figure 10. The walking-support assistive robot detected the interactive force and the robot started to help the user stand up. The process of STS with the walking-support assistive robot is delineated as in Figure 10.

Table 2.

The parameters of subject.

Parameter	Weight (kg)	Description
m ₁	29.1	Mass of HAT
m ₂	2.24	Masses of arms
m ₃	24.62	Mass of torso-head
m ₄	7.39	Masses of thighs
m ₅	3.04	Masses of shanks

Figure 10.

The process of STS with walking-support assistive robot. STS: sit-to-stand.

For all state-action pairs in the proposed algorithm, $ω (s, c)$ is initialized to zero. The parameters of the proposed FSL algorithm are listed in Table 3.

Table 3.

The parameters of FSL algorithm.

$γ$	α	T	$Δ$ t(s)
0.9	0.05	0.95	0.05

FSL: fuzzy Sarsa learning.

Before the experiments, the subject was asked to stand up with wearable sensors 30 times. The wearable sensors detected the motion positions of the user’s two legs. Then, the reference trajectory-fitted curve of the user’s COM can be calculated, as shown in Figure 11.

Figure 11.

The reference trajectory of user’s COM during the process of STS. COM: center of mass; STS: sit-to-stand.

Training of weight values

The output action of the FSL algorithm is the weighted sum of the selected actions in the rules. Therefore, in the FSL algorithm, $ω (s, c)$ is critical. However, the FSL algorithm needs many trials to obtain the optimal output action. Then, it will spend considerable time conducting the experiments with the walking-support assistive robot. To decrease the time of the experiment, a simulated experiment was implemented before the experiments with the robot to train $ω (s, c)$ .

To obtain input–output training data of the FSL algorithm, the robot and subject implemented an STS transfer trial to gather the interactive force and COM data of the coupled human–robot system. Then, according to the dynamic model equation (1), $ω (s, c)$ of the candidate actions in each rule was trained by the proposed method 1000 times. Finally, we conducted the experiments with a walking-support assistive robot. The details of the experimental results and discussion are explained in the next section.

Experimental results and discussion

In this section, we evaluate the proposed algorithms by three measures:

(i) Trial time is the time of a complete STS trial;

(ii) MCE is the mean COM error, thus representing the average error between the user’s actual COM trajectory and the reference trajectory

MCE = \bar{e_{x} + e_{z}} = \frac{\sum_{i = 1}^{N} (e_{x} (n)) + \sum_{i = 1}^{N} (e_{z} (n))}{N}

where N is the sampling times in each trial, $e_{x} (n)$ and $e_{z} (n)$ are the trajectory errors of the user’s COM in the horizontal direction and vertical direction on the $n^{th}$ sampling time in a trial.

(iii) MIF is the mean of the interactive force, thus representing the average interactive force in a trial.

MIF = \bar{F_{c}} = \frac{\sum_{i = 1}^{N} (F_{c} (n)}{N}

where $F_{c} (n)$ is the interactive force on the n-th sample time and $x_{rcom} (n)$ is the desired position of COM in the X-axis on the $n^{th}$ sampling time in a trial.

The experiments included 30 trials. Table 4 shows the experimental results. The average trial time was computed by taking the average of 30 STS trials. As seen in Table 4, after 30 trails, all three measures significantly outperformed those of the first trial, with the Trial time, MCE, and MIF decreasing by 16.8%, 64.5%, and 27.2% compared with the first trial.

Table 4.

Experiment results for STS.

Trial	Trial time	MCE	MIF
1	12.5	26.64	37.7
30	10.4	9.44	27.44

STS: sit-to stand; MCE: mean COM error; MIF: mean of the interactive force.

The purpose of this robot is to allow robots to assist user to achieve STS transfer and make the user feel that the robot is following user’s standing movements rather than just supporting the user to stand up. That is, the robot needs to assist the user’s actual COM closer to the reference COM. And according to the results of the subject questionnaire survey, the subject does felt that the robot was following her STS transfer movement rather than simply supporting the user standing up. In other words, the robot realizes a more comfortable STS transfer. Therefore, the interactive force was less than that in the first trial. Figure 12 shows the curve of the three measurement indexes.

Figure 12.

The curve of three measures index in the experiments.

Figures 13 and 14 show the experimental results in the last trial. The STS transfer movement can be divided into several phases, namely, preparation, rising, and stabilization.²⁷ Based on Figure 14, the preparation stage is in the time interval ( $0 \leq t < 4.5$ )[s], the rising phase is in the time interval ( $4.5 \leq t < 7.4$ )[s], and the stabilization phase is for $t \geq 7.4$ )[s]. In the preparation phase, the error of the COM in the horizontal direction e_x does not change much, and the velocity v_x is almost zero. The interactive force was very large, as the user was ready to stand. In the lifting phase, the user left the chair and the position of the COM in the vertical direction increased rapidly; then, v_x also increased with e_x . In the stabilization phase, the position of the COM no longer changes, and v_x is almost zero. The linear actuator ascended with the whole process of STS transfer, as shown in Figure 13.

Figure 13.

The experiment results in last trial. (a) The velocities of the robot in vertical direction and horizontal direction. (b) The height that the linear actuator rise and the interactive force during STS transfer movement.

Figure 14.

The trajectory of user’s COM during the last trial of STS. COM: center of mass; STS: sit-to-stand.

Comparative experiment

To verify the effectiveness of the proposed STS transfer motion control algorithm, the motion control algorithm in the literature.²⁸ was compared with the proposed algorithm in this article. The robot in the literature²⁸ is the same as our walking-support assistive robot. Autonomous control uses the voltage across the FSR as the control signal, while posture control uses the attitude angles in the literature.²⁸ The three indexes (IMF and Trial time) of the two motion control algorithms are shown in Table 1. According to Table 5, the proposed method has better performance than in the literature.²⁸

Table 5.

Comparative experiment results for STS.

	Trial time (s)	MIF
STS method²⁸	12.7	36.25 N
Multi-FSL-based STS	10.4	27.44 N

STS: sit-to-stand; MIF: mean of the interactive force; FSL: fuzzy Sarsa learning.

Conclusion

A multi-FSL-based STS motion control for walking-support assistive robot was introduced in this article. According to the requirements for natural and intuitive STS transfer movement, a multi-FSL-based STS motion control algorithm for walking-support assistive robot was proposed. The mechanism of walking-support assistive and STS transfer problem were briefly introduced. Considering the difficulty in obtaining accurate mathematical model for the coupled human–robot system, FSL method was used to control the robot. To increase the efficiency of FSL algorithm, two FSL-based controllers were implemented to control linear actuator and mobile base, respectively. At last, walking-support assistive robot experiments have proved the validity of the proposed algorithm.

The presented control scheme for walking-support assistive robot was only evaluated on healthy subjects. And the robot needs time and experimentations to learn the user’s STS transfer movement. The elderly may not be able to conduct too many trials. To establish the effect of multi-FSL-based STS motion control, elderly individual trials are necessary and are currently undergoing as a part of this research.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was funded by the National Natural Science Foundation Youth Fund of China Grant Numbers 61803286 and 61906139 and the Nature Science Foundation (Youth fund) of Hubei province Grant Number 2018CFB163.

ORCID iD

Xu Wenxia

References

Bedaf

Gelderblom

De Witte

Overview and categorization of robots supporting independent living of elderly people: what activities do they support and how far have they developed. Assis Technol 2015; 27(2): 88–100.

Rubenstein

. Falls in older people: epidemiology, risk factors and strategies for prevention. Age Ageing 2006; 35(Suppl 2): ii37–ii41.

Ploutz-Snyder

Manini

Ploutz-Snyder

, et al. Functionally relevant thresholds of quadriceps femoris strength. J Gerontol A 2002; 57(4): B144–B152.

Zhao

Yang

Okoye

, et al. Walking assist robot: a novel non-contact abnormal gait recognition approach based on extended set membership filter. IEEE Access 2019; 7: 76741–76753.

Mederic

Pasqui

Plumet

, et al. Sit to stand transfer assisting by an intelligent walking-aid. In: Climbing and Walking Robots 2005, Berlin, Heidelberg: Springer; pp. 1127–1135.

Chuy

Hirata

Wang

Z D

, et al. Approach in assisting a sit-to-stand movement using robotic walking support system. In: 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, Beijing, China, 9 October–15 October 2006, paper no. IROS 2006, pp. 4343–4348. IEEE.

Omar

Ahmed

, et al. Development of parallel manipulator sit to stand assistive device for elderly people. In: 2013 IEEE Workshop on Advanced Robotics and its Social Impacts, Tokyo, Japan, 7 November–9 November 2013, paper no. ARSO, pp. 27–32. IEEE.

Geravand

Korondi

Werner

, et al. Human sit-to-stand transfer modeling towards intuitive and biologically-inspired robot assistance. Auto Robot 2017; 41(3): 575–592.

Mederic

Pasqui

Plumet

, et al. Elderly people sit to stand transfer experimental analysis. In: International Conference on Climbing and Walking Robots, Berlin, Heidelberg, 13 September–15 September 2005, paper no. CLAWAR 2005, pp. 953–960. Springer.

10.

Chugo

Morita

Sakaida

, et al. A robotic walker for standing assistance with realtime estimation of patients load. In Proceding of 12th IEEE International Workshop on Advanced Motion Control, Sarajevo, Bosnia-Herzegovina, 25 March–27 March 2012, paper no. AMC 2012, pp. 1–6. IEEE.

11.

Huang

Fukuda

, et al. A disturbance observer based sliding mode control for a class of underactuated robotic system with mismatched uncertainties. IEEE Trans Auto Cont 2019; 64(6): 2480–2487.

12.

Huang

Cao

Wang

. Adaptive proxy-based sliding mode control for a class of second-order nonlinear systems and its application to pneumatic muscle actuators. ISA Transction, 2020.

13.

Jabbari Asl

Narikiyo

Kawanishi

. Neural network based bounded control of robotic exoskeletons without velocity measurements. Control Eng Pract 2018; 80: 94–104.

14.

Hussain

Xie

Jamwal

. Adaptive impedance control of a robotic orthosis for gait rehabilitation. IEEE Trans Cyber 2013; 43(3): 1025–1034.

15.

, et al. Fuzzy approximation-based adaptive backstepping control of an exoskeleton for human upper limbs. IEEE Trans Fuzzy Sys 2015; 23(3): 555–566.

16.

Huang

Wang

, et al. Reinforcement learning-based shared control for walking-aid robot and its experimental. Adv Robot 2015; 29(22): 1463–1481.

17.

Huang

Cheng

Guo

, et al. Hierarchical interactive learning for a human-powered augmentation lower exoskeleton. 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden, 2016, pp. 257–263.

18.

Khan

Tufai

Shah

, et al. Reinforcement learning based compliance control of a robotic walk assist device. Adv Robot 2019; 33(24): 1–12.

19.

Deng

. Online tuning of fuzzy inference systems using dynamic fuzzyQ-learning. IEEE Trans Sys Man Cyber Part B 2004; 34: 1478–1489.

20.

Zijlstra

Bisseling

Schlumbohm

, et al. A body-fixedsensor-based analysis of power during sit-to-stand movements. Gait Posture 2010; 31: 272–278.

21.

Huang

Mohammed

, et al. Posture estimation and human support using wearable sensors and walking-aid robot. Robot Auto Sys 2014; 73: 24–43.

22.

Motlagh

Arshi

. Symmetry comparison between sacrum and center of mass during walking. In: Institution of Mechanical Engineers Part H-Journal of Engineering in Medicine, 1 July 2016, paper no. 230(7), pp. 682–689. SAGE.

23.

Howard

Braun

Vijayakumar

. Transferring human impedance behavior to heterogeneous variable impedance actuators. IEEE Trans Robot 2013; 29: 847–862.

24.

Kronander

Billard

. Online learning of varying stiffness through physical human-robot interaction. In: 2012 IEEE International Conference on Robotics and Automation, Saint Paul, MN, USA, 2012, pp. 1842–1849.

25.

Hirasawa

. Continuous probabilistic model building genetic networkprogramming using reinforcement learning. Appl Soft Comput 2015; 27: 457–467.

26.

Leva

. Adjustments to zatsiorsky-seluyanov’s segment inertia parameters. J Biomech 1996; 29(8): 1223–1230.

27.

Lindemann

Claus

Stuber

, et al. Measuing power during the sit-to-stand transfer. Eur J Appl Physiol 2003; 89(5): 466–470.

28.

Yang

Zhou

Huang

, et al. Aided sit to stand transfer by assistive robot and wearable sensors. In: 2017 2nd International Conference on Advanced Robotics and Mechatronics, Hefei, China, 27 August–31 August 2017, paper no. ICARM 2017, pp. 239–244. IEEE.

Multi-fuzzy Sarsa learning-based sit-to-stand motion control for walking-support assistive robot

Abstract

Keywords

Notation

Introduction

Walking-support assistive robot

Mechanics of the walking-support assistive robot

Dynamic model of human body model

STS transfer problem for walking-support assistive robot

Fuzzy Sarsa learning

Reinforcement learning

Sarsa learning

Fuzzy Sarsa learning

Multi-FSL-based STS motion control for the walking-support assistive robot

Experimental evaluation

Experiment process

Training of weight values

Experimental results and discussion

Comparative experiment

Conclusion

Footnotes

Declaration of conflicting interests

Funding

ORCID iD

References