A novel approach for robot-assisted upper-limb rehabilitation

Abstract

This study presented a novel control approach for rehabilitation robotic system using the hybrid system theory and the subject’s bio-damping and bio-stiffness parameters. Resistance training was selected as a paradigm. The proposed control architecture incorporated the physical therapist’s behavior intervention, the stroke survivor’s muscle strength changes, and the robotic device’s motor control into a unified framework. The main focuses of this research were to (i) automatically monitor the subject’s muscle strength changes using the online identified bio-damping/stiffness parameters; (ii) make decisions on the modification of the desired resistive force so as to coincide with the subject’s muscle strength changes; and (iii) generate accommodating plans when the safety-related issues such as spasticity and the abnormal robotic working states happen during the execution of training tasks. A Barrett WAM compliant manipulator-based resistance training system and two experiments including four scenarios were developed to verify the proposed approach. Experimental results with healthy subjects showed that the hybrid system–based control architecture could administrate the subject’s muscle strength changes and the robotic device’s interventions in an automated and safe manner.

Introduction

Patients suffering from strokes need to receive intensive rehabilitation training to improve range of motion, movement coordination, and muscle strength. These therapy programs are traditionally conducted by physical therapists. However, the efficacy of therapist-centered rehabilitation therapy often relies on the therapist’s subjective clinical experience. In recent years, there is an increasing interest in using robotic devices to help providing rehabilitation training because these devices can provide a variety of highly repetitive movements and training protocols for stroke patients. ¹

Human–robot interactive control approaches, that specify how rehabilitation robotic devices adaptively interact with stroke survivors in an automated and safe manner, are one of the major challenges in developing robot-assisted training system. The control strategies explored in some early rehabilitation robotic devices, such as the proportional integral derivative controller for MIME, ² the impedance controller for MIT-Manus, ³ and the admittance controller for GENTLE/s, ⁴ are mainly concentrated on providing constant assistance while not integrating stroke patients’ feedback into the control loop. It is a common hypothesis in the field of robot-assisted rehabilitation that the control approaches, that can close the loop via the patients and further adapt robotic devices’ assistance to the stroke patients’ progress, will be more efficient. ⁵ This issue is typically addressed using assist-as-needed ^6,7 or user-cooperative control strategies. ^8,9 In the study by Hussain et al., ⁶ an adaptive seamless assist-as-needed control scheme is developed for the robotic gait training, which learns in real time the disability level of human subjects based on the trajectory tracking errors and adapts the robotic assistance accordingly. Riener et al. ⁸ presents a patient-cooperative strategy for robotic gait training, and results with healthy subjects show that subjects train more actively and only get support as much as needed. Nonetheless, the potential issue of these methods is that they focus on low-level robotic motor controllers. The training tasks update, the safety-related issues (e.g. spasticity and twitch) monitoring, the robotic working states (e.g. joint torque, voltage, workload, and end-effector velocity) detection, the therapeutic progress assessment, and the decision-making behaviors are all administered by physical therapists.

In the last few years, hierarchical supervisory control strategies have been developed. These approaches incorporate the training tasks update and the physical therapist’s behavior decision into the low-level robotic motor controller. Denève et al. ¹⁰ merge a high-level sequential controller into a robotic-assisted upper limb rehabilitation system, by which three different low-level control laws for passive, active, and resistance modes can be switched. In the study by Varol et al., ¹¹ a three-level hierarchical supervisory control architecture is proposed, which consists of the lowest level robotic joint torque controller, the middle-level torque references generator, and the high-level intent recognizer. Fuzzy-based hierarchical supervisory control strategies are also presented in our previous studies. ^12,13 Especially in the study, ¹³ a high-level safety supervisory controller is designed to monitor spasticity-related issues. Great improvements have been made in the current hierarchical supervisory control architecture for incorporating therapists’ behavior-decision experiences. Unfortunately, these supervisory control methods are statically designed concerning some predefined situations, and the absence of dynamic mechanism makes them incapable of dealing with the extended/unexpected events and the complex training tasks coordination. Besides, few rehabilitation robotic control system design takes the safety-related issues into consideration.

In this article, we present a hybrid system–based control architecture using resistance training as a paradigm, which can incorporate the physical therapist’s behavior intervention, the stroke survivor’s muscle strength progress, and the robotic device’s motor control into a unified framework. In fact, over the years, the hybrid systems framework has been effectively used in many fields to model and analyze their performances, such as the power systems, ¹⁴ the communication networks, ¹⁵ and the coordinated control of assistive robotic devices for complex tasks. ¹⁶

The primary focus of this article is to (i) automatically monitor the impaired limb’s muscle strength progress using the online identified bio-impedance changes; (ii) make decisions on the modification of the desired resistive force so as to coincide with the impaired limb’s muscle strength changes; and (iii) be aware of robotic working state/safety-related issues during the execution of training task and to generate accommodating plans when such events happen. The remainder of this article is organized as follows: “Methods” section presents the experimental setup and protocols, bio-impedance parameters identification, and controller development. “Results” section details the results of the proposed control strategy. Some discussions and conclusions are given in sections “Discussions” and “Conclusions”.

Methods

Experimental setup

The rehabilitation robotic system for upper limb muscle strength training used in the trials, shown in Figure 1(a), consisted of a Barrett WAM™ manipulator, a three dimensional (3-D) force sensor, and an external PC offered by Barrett Technology. ¹⁷ The standard WAM™ is a four degree of freedom highly dexterous, back-drivable manipulator. Its human-like kinematics and high back drivability enable inherent force-control, haptic interaction, and rehabilitation application. In order to record the force interaction between the impaired limb and the WAM end-effector, a 3-D force sensor was designed and attached to the end-effector. Figure 1(b) and (c) shows the mechanical structure, appearance, and strain gauges distribution of the 3-D force sensor. Force data measured from the sensor must be transformed from the WAM tool frame into its world frame. The graphical user interface developed using Linux/GDK technology, shown in Figure 1(d), was used to display the actual training trajectory when the patient moved his arm in the XOZ vertical plane (O-XYZ coordinates shown in Figure 1(a)), where no reference trajectories were predefined except for several via points. The external PC, running with the Ubuntu Linux system and the Xenomai real-time module, was responsible for executing the control loop and sending high-level commands to the WAM-aided rehabilitation training system. Real-time communication between the external PC and motor Pucks™ was conducted via a high-speed controller area network bus.

Figure 1.

The Barrett WAM™ rehabilitation robotic system for upper limb muscle strength training and its attachments. (a) The Barrett WAM™ rehabilitation robotic system, (b) mechanical structure and appearance of the 3-D force sensor, (c) distribution of 16 strain gauges on the cross beam, and (d) the graphical user interface for muscle training in XOZ plane.

Experimental protocol

Individuals were asked to seat in a height adjustable chair in front of a computer screen. The WAM robot, under full gravity compensation, was restricted in the XOZ vertical plane with control threads. The training subjects were asked to grasp the WAM handle and to move the robot from the starting point to the end point by performing lift-and-reach exercises. The desired resistive force generated from the proposed controller was exerted on the WAM end-effector and constantly against the direction of the desired movement through the interface function defined in the WAM control loop. When the subjects arrived at the end point position, the resistive force was removed till the start point in the next training trial was detected. Before the beginning of the training session, participants were presented with five trials in order to acquaint them with the hardware and user interface. The experimental procedure was based on training blocks, and each block was consisted of five training trials.

Two experiments including four scenarios were designed. In the first experiment, two scenarios (scenario 1 and scenario 2) were developed. The scenario 1 was carried out to verify that how the proposed controller makes automated modifications on the desired resistive force by monitoring the gradual increment of the training subject’s muscle strength. The scenario 2, which simulated the spasticity case by sharply increasing the subject’s muscle strength, was developed to demonstrate the controller’s capability in dealing with the safety-related events. In the second experiment, another two scenarios (scenario 3 and scenario 4) were designed. Apart from the scenario of monitoring the gradual decrement of the subject’s muscle strength (scenario 3), a typical suspending scenario inspired by ¹⁶ (scenario 4), where the subject was asked to press the pause/stop shortcut key, was described to show that how the proposed controller modifies the training tasks in a safe manner. This pause scenario often occurs in clinical rehabilitation therapy when the stroke patients have discomfort or fatigue and want to have a rest.

The bio-impedance parameters identification

As mentioned in the studies by Tung Fai and Wilson ¹⁸ and Zhang et al., ¹⁹ the human arm’s bio-impedance parameters, bio-damping, and bio-stiffness, can reflect the arm’s muscle strength changes with high sensitivity. Based on our previous work in the study by Xu et al., ²⁰ the subject arm’s bio-impedance characteristic was modeled as the following linear time-variant system

$f_e(t)=b_e(t)\delta \dot{x}(t)+k_e(t)\delta x(t)$

1

where f_e denoted the force interaction between the robot and the impaired limb, $\delta x=x-x_e$ was the displacement between the actual and equilibrium positions of the impaired limb, $\delta \dot{x}$ was the velocity vector, and b_e, k_e were the impaired limb’s bio-damping/stiffness.

Using the bilinear transformation in the study by Mallapragada et al., ²¹ equation (1) was reorganized in the discrete-time domain as

$f_e(z)=\left[b_e(z)\left(\frac{2}{T}\right)\left(\frac{1-z^{-1}}{1+z^{-1}}\right)+k_e(z)\right]\delta x(z)$

2

where T was the sampling period and z⁻¹ was one shift step in the time domain. Using equation (2), the continuous linear model was discretized to provide a model for linear estimation

$f_e(k)+f_e(k-1)=a_1(k)\delta x(k)+a_2(k)\delta x(k-1)$

3

where

$a_1(k)=b_e(k)\left(\frac{2}{T}\right)+k_e(k),a_2(k)=k_e(k)-b_e(k)\left(\frac{2}{T}\right)$

Furthermore, equation (3) was represented as a regression vector, as follows

$\begin{array}{l}y[k]=h^{\text{T}}[k]\theta [k]\\ \theta [k]={[a_1(k),a_2(k)]}^{\text{T}}h[k]={[\delta x(k),\delta x(k-1)]}^{\text{T}}\\ y[k]=f_e(k)+f_e(k-1)\end{array}$

4

where θ[k] was the estimated parameter vector and h[k] and y[k] represented the input and output regression vectors, respectively.

According to the regressive least square (RLS) method with forgetting factor, we had

$\theta [k]=\theta [k-1]+\frac{P[k-1]h[k]\left(y[k]-h^{\text{T}}[k]\theta [k-1]\right)}{\lambda +h^{\text{T}}[k]P[k-1]h[k]}$

5

where λ was the constant forgetting factor, which influenced the weight given to the earlier data relative to the newly acquired data. The covariance matrix P[k] was calculated as

$P[k]=\frac{1}{\lambda }\left(P[k-1]-\frac{P[k-1]h[k]h^{\text{T}}[k]P[k-1]}{\lambda +h^{\text{T}}[k]P[k-1]h[k]}\right)$

6

In the study by Mallapragada et al., ²¹ the estimated results using the above RLS method with forgetting factor are still not satisfactory, especially for the damping coefficient, although the given environment parameters, stiffness, and damping are constant. In clinical rehabilitation, the impaired limb’s dynamics vary randomly and its bio-impedance parameters are time variant throughout the whole rehabilitation process. It is suggested that the estimation of the impaired limb’s bio-impedance parameters using RLS method with an adaptive forgetting factor is superior to the one with a fixed forgetting factor. The forgetting factor was adapted as

$\begin{array}{l}\lambda [k]=1-\left[1-\frac{h^T(k-l-1)P[k-1]h(k-l-1)}{1+h^T(k-l-1)P[k-1]h(k-l-1)}\right]\\ \hspace{1em}\hspace{1em}\hspace{1em}\times \left[\frac{{\left(y[k]-h^{\text{T}}[k]\theta [k-1]\right)}^2}{R}\right]\end{array}$

7

where l and R were the forgetting step and parameter, respectively. In addition, the maximum and minimum forgetting factors λ_max, λ_min were set for preventing unfortunate disturbances. After the vector θ was estimated, the subject arm’s bio-damping ${\widehat{b}}_e$ and bio-stiffness ${\widehat{k}}_e$ were obtained from equation (3)

${\widehat{b}}_e(k)=T\left(\frac{a_1(k)-a_2(k)}{4}\right),\hspace{1em}{\widehat{k}}_e(k)=\frac{a_1(k)+a_2(k)}{2}$

8

Controller design using hybrid system theory

The main idea of the proposed controller was to develop a control architecture that can incorporate the physical therapist’s monitoring and behavior intervention, the stroke survivor’s muscle strength assessment, and the robotic device’s motor control into a unified framework. Nevertheless, the behaviors of the physical therapist’s monitoring and decision-making are discrete, while the stroke survivor’s muscle strength changes and the robotic device’s exerted force and working parameters are continuous. They cannot share the same control space. To administrate the interaction between the stroke survivor and the robotic device in an automated and safe manner, it is necessary to design an interface so that the transitions between the discrete and continuous control can be established.

Hybrid system theory, ²² which can incorporate the discrete dynamic system and the continuous dynamic system into a unified framework, offers mathematical tools to develop our controller. Figure 2 shows the proposed hybrid system–based control architecture. It consisted of a high-level discrete event dynamic system (DEDS) (“controller”), an “interface,” and a low-level continuous variable dynamic system (CVDS) (“plant”). The high-level DEDS ideally playing the role of the physical therapist was designed to make decisions on the modifications of the desired resistive force by monitoring the stroke survivor’s muscle strength progress and to generate accommodating plans when abnormal robotic working states or safety-related events happened. The low-level CVDS was developed to online estimate the subject’s bio-impedance parameter changes and to achieve accurate desired force tracking control between the robotic device and the training subject. The continuous/discrete part in the interface was to accept plant inputs and to generate discrete events that the DEDS monitoring module needed, while the discrete/continuous part was to receive discrete outputs from the controller and to generate control commands sending to the robotic device force tracking controller.

Figure 2.

The proposed controller architecture for robot-aided muscle strength training using hybrid system theory.

The DEDS and discrete/continuous interface

The DEDS and the discrete/continuous interface were mathematically represented by $G_D=\left(Q,\sigma ,C,f_D,\delta \right)$ according to the hybrid system theory. Here, Q was the set of discrete controller states, σ was the set of plant events generated from the continuous states transition, C was the set of controller outputs via the discrete/continuous interface, $f_D:Q\times \sigma \to Q$ was the discrete control state transition function, and δ : Q → C was the discrete/continuous interface transition function, which converted the discrete control commands into the continuous control outputs. The control outputs, also called plant inputs, were then sent to the low-level force regulating and tracking controller. According to the resistance training protocols, Tables 1 and 2 present the lists of the discrete control states and control vectors definitions, respectively.

Table 1.

Lists of discrete control states definitions.

Control states	Description
Q 1	Initial state: robot-assisted muscle strength training with initial resistive force
Q 2	Increasing state: robot-assisted muscle strength training with increased resistive force
Q 3	Decreasing state: robot-assisted muscle strength training with decreased resistive force
Q 4	Holding state: robot-assisted muscle strength training with the current resistive force
Q 5	Stopping state: robot-assisted muscle strength training system stopped working
Q 6	Resuming state: robot-assisted muscle strength training system resumed working with the resistive force at the previous state

Table 2.

Lists of control events definitions.

Control vector	Description
c 1	Initialized the resistive force exerted on the robotic end-effector
c 2	Increased the resistive force exerted on the robotic end-effector
c 3	Decreased the resistive force exerted on the robotic end-effector
c 4	Held the current resistive force exerted on the robotic end-effector
c 5	Removed the resistive force exerted on the robotic end-effector and saved the training task parameters
c 6	Resumed the robotic training system and exerted the same resistive force as the previous state on the robotic end-effector

The CVDS and continuous/discrete interface

According to the hybrid system theory, the CVDS and the continuous/discrete interface were mathematically described by $G_{\text{C}}\text{=}\left(X,C,\sigma ,f_{\text{C}}\text{,ψ}\right)$ . Here, X was the set of continuous variable states, C was the set of controller outputs via the discrete/continuous interface, σ was the set of plant events generated from the continuous states transition, $f_C:X\times C\to X$ was the continuous variable state transition function; $\text{ψ}:X\times X\to \sigma$ was the continuous/discrete interface transition function. The continuous variable states were separated based on the predefined hypersurfaces. The hypersurfaces were often decided by considering the training tasks requirements and the stroke survivor’s clinical comfort and safety.

In this research, the hypersurfaces were defined based on the following two considerations: (i) the muscle strength of the subject to perform the resistance training tasks and (ii) the capability of the rehabilitation robotic system to ensure the safety of the subject during the task execution. The former can be well evaluated by computing the changes of the subject’s bio-impedance averages, and the latter may be achieved by regulating smooth motion transitions when monitoring the abnormal robotic working parameters or the safety-related issues like spasticity. When the hypersurfaces were crossed, plant events occurred.

The subject arm’s bio-damping and bio-stiffness parameters averages in each training trial and block were calculated as follows

${\overline{\widehat{b}}}_{e\_\text{trial}}=\frac{t_s}{t_f-t_{\text{i}}}\cdot {\displaystyle \sum _{t_{\text{i}}}^{t_f}{\widehat{b}}_e\left(t\right)},\hspace{1em}{\overline{\widehat{k}}}_{e\_\text{trial}}=\frac{t_s}{t_f-t_{\text{i}}}\cdot {\displaystyle \sum _{t_{\text{i}}}^{t_f}{\widehat{k}}_e\left(t\right)}$

9

${\overline{\widehat{b}}}_{e\_\text{block}}=\frac{1}{N}{\displaystyle \sum }_{i=1}^N{({\overline{\widehat{b}}}_{e\_\text{trial}})}_i,{\overline{\widehat{k}}}_{e\_\text{block}}=\frac{1}{N}{\displaystyle \sum }_{i=1}^N{({\overline{\widehat{k}}}_{e\_\text{trial}})}_i$

10

$\begin{array}{l}\Delta {\overline{\widehat{b}}}_{e\_\text{trial}}\text{=}{\overline{\widehat{b}}}_{e\_\text{trial}}\left(j\right)-{\overline{\widehat{b}}}_{e\_\text{trial}}\left(j-\text{1}\right)\\ \Delta {\overline{\widehat{k}}}_{e\_\text{trial}}\text{=}{\overline{\widehat{k}}}_{e\_\text{trial}}\left(j\right)-{\overline{\widehat{k}}}_{e\_\text{trial}}\left(j-\text{1}\right)\end{array}$

11

$\begin{array}{l}\Delta {\overline{\widehat{b}}}_{e\_\text{block}}\text{=}{\overline{\widehat{b}}}_{e\_\text{block}}\left(k\right)-{\overline{\widehat{b}}}_{e\_\text{block}}\left(k-\text{1}\right)\\ \Delta {\overline{\widehat{k}}}_{e\_\text{block}}\text{=}{\overline{\widehat{k}}}_{e\_\text{block}}\left(k\right)-{\overline{\widehat{k}}}_{e\_\text{block}}\left(k-\text{1}\right)\end{array}$

12

where ${\widehat{b}}_e\left(t\right)$ and ${\widehat{k}}_e\left(t\right)$ were the instantaneous estimated values of the bio-damping and bio-stiffness parameters; t_i, t_f, and t_s were the starting time, final time, and sample time in each trial, respectively; N is the number of trials in each training block; ${\overline{\widehat{b}}}_{e\_\text{trial}}$ , ${\overline{\widehat{b}}}_{e\_\text{block}}\text{,}{\overline{\widehat{k}}}_{e\_\text{trial}}$ and ${\overline{\widehat{k}}}_{e\_\text{block}}$ denoted the subject’s bio-damping/stiffness averages in each training trial and block; $\Delta {\overline{\widehat{b}}}_{e\_\text{trial}}$ , $\Delta {\overline{\widehat{b}}}_{e\_\text{block}},\Delta {\overline{\widehat{k}}}_{e\_\text{trial}}$ and $\Delta {\overline{\widehat{k}}}_{e\_\text{block}}$ were the changes of the subject arm’s bio-impedance averages in each training trial and block; and j and k denoted the training trial and block, respectively. The changes of the subject arm’s bio-impedance averages in each training trial and block were used to monitor the sharp and gradual muscle strength progress. Table 3 shows the lists of the hypersurfaces and plant events definitions. Based on the definitions of the discrete control states in the DEDS and the plant events in the CVDS, we may further develop the behavior-decision rules for the high-level discrete event controller. At any given time, only one control state is active. The next discrete control state was activated based on the current discrete control state and the associated plant event. Table 4 presents the behavior-decision rules lists for the high-level discrete event controller. According to the above discrete control rules description, Figure 3 shows the high-level control state transitions diagram using hybrid automation.

Table 3.

Lists of hypersurfaces and plant events definitions.

Plant events definitions	Hypersurfaces definitions
σ 1	$\Delta {\overline{\widehat{b}}}_{e\_\text{block}}<0$ and $\Delta {\overline{\widehat{k}}}_{e\_\text{block}}<0$ and $\Delta {\overline{\widehat{b}}}_{e\_\text{block}}$ and $\Delta {\overline{\widehat{k}}}_{e\_\text{block}}$ were the changes of the subject’s bio-impedance averages in each training block, respectively
The exerted resistive force was bigger than the subject’s muscle strength endurance
σ 2	$0<\Delta {\overline{\widehat{b}}}_{e\_\text{block}}<\Delta {\overline{\widehat{b}}}_{e\_\text{low}}\hspace{0.17em}\text{or}\hspace{0.17em}0<\Delta {\overline{\widehat{k}}}_{e\_\text{block}}<\Delta {\overline{\widehat{k}}}_{e\_\text{low}}$ , $\Delta {\overline{\widehat{b}}}_{e\_\text{low}}$ and $\Delta {\overline{\widehat{k}}}_{e\_\text{low}}$ were the low thresholds for the $\Delta {\overline{\widehat{b}}}_{e\_\text{block}}$ and $\Delta {\overline{\widehat{k}}}_{e\_\text{block}}$ , respectively
The exerted resistive force was fitted for the subject’s muscle strength endurance
σ 3	$\Delta {\overline{\widehat{b}}}_{e\_\text{low}}\le \Delta {\overline{\widehat{b}}}_{e\_\text{block}}\le \Delta {\overline{\widehat{b}}}_{e\_\text{upper}}$ and $\Delta {\overline{\widehat{k}}}_{e\_\text{low}}\le \Delta {\overline{\widehat{k}}}_{e\_\text{block}}\le \Delta {\overline{\widehat{k}}}_{e\_\text{upper}}$ , $\Delta {\overline{\widehat{b}}}_{e\_\text{upper}}$ and $\Delta {\overline{\widehat{k}}}_{e\_\text{upper}}$ were the upper thresholds for the $\Delta {\overline{\widehat{b}}}_{e\_\text{block}}$ and $\Delta {\overline{\widehat{k}}}_{e\_\text{block}}$ , respectively
The exerted resistive force was smaller than the subject’s muscle strength endurance
σ 4	$u_r>u_{r\_\text{limit}}\hspace{0.17em}\hspace{0.17em}\text{or}\hspace{0.17em}\hspace{0.17em}{v}_r>v_{r\_\text{limit}}\hspace{0.17em}\hspace{0.17em}\text{or}\hspace{0.17em}\hspace{0.17em}{t}_r>t_{r\_\text{limit}}$ , $u_{r\_\text{limit}},v_{r\_\text{limit}}$ and $t_{r\_\text{limit}}$ were the safety-related limits specified for the robotic system working voltage, end-effector velocity, and joint torque, respectively
The abnormal robotic system working states
σ 5	$\Delta {\overline{\widehat{b}}}_{e\_\text{trail}}>\Delta {\overline{\widehat{b}}}_{e\_\text{limit}}$ and $\Delta {\overline{\widehat{k}}}_{e\_\text{trial}}>\Delta {\overline{\widehat{k}}}_{e\_\text{limit}}$ , $\Delta {\overline{\widehat{b}}}_{e\_\text{limit}}$ and $\Delta {\overline{\widehat{k}}}_{e\_\text{limit}}$ are the safety-related limits specified for the $\Delta {\overline{\widehat{b}}}_{e\_\text{trial}}$ and $\Delta {\overline{\widehat{k}}}_{e\_\text{trial}}$ , respectively
The safety-related issues just like spasticity
σ 6	The subject pressed pause/stop button for having a rest after several training trials or finished the training tasks
The robotic system suspended working
σ 7	The robotic system resumed working
The robotic system resumed working
σ 71	The robotic system resumed working with the same resistive force as the one at the previous initial state
The robotic system resumed working at initial state
σ 72	The robotic system resumed working with the same resistive force as the one at the previous increasing state
The robotic system resumed working at increasing state
σ 73	The robotic system resumed working with the same resistive force as the one at the previous decreasing state
The robotic system resumed working at decreasing state
σ 74	The robotic system resumed working with the same resistive force as the one at the previous holding state
The robotic system resumed working at holding state

Table 4.

Lists of the discrete control rules description.

No.	Description
1	When monitoring the plant event σ1, the discrete control state Q3 was activated and the control vector c3 was generated to decrease the resistive force
2	When monitoring the plant event σ2, the discrete control state Q4 was activated and the control vector c4 was generated to hold the current resistive force
3	When monitoring the plant event σ3, the discrete control state Q2 was activated and the control vector c2 was generated to increase the resistive force
4	When monitoring the plant events σ4, σ5, and σ6, the discrete control state Q5 was activated and the control vector c5 was generated to shut off the rehabilitation robotic system and to save the current training task parameters
5	When monitoring the plant event σ7, the discrete control state Q6 was activated and the control vector c6 was generated to turn on the rehabilitation robotic system. The exerted resistive force is further determined by the events σ71, σ72, σ73, and σ74

Figure 3.

The control states transitions based on hybrid automation.

The commands of modifying the desired resistive force generated from the high-level discrete event controller were sent to the low-level force regulating and tracking controller. In order to regulate the resistive force in a smooth and safe manner, a piecewise S-shape curve transition method ²³ was adopted instead of directly giving a step-like force inputs which may give rise to unsafe tremble. When the desired resistive was smoothly regulated, it was then sent to the tracking controller to achieve accurate resistive force tracking. The following proportional (P)/derivative (D) force tracking controller presented in our previous work ¹⁷ was designed

${\tau }_F=K_{\text{P}}(f_{\text{d}}-f_{\text{act}})+K_D\frac{\text{d}(f_{\text{d}}-f_{\text{act}})}{\text{dt}}$

13

where K_P and K_D were the PD force tracking controller gains and f_d and f_act were the desired and actual resistive force, respectively. τ _F was the robot joint torque obtained from the above PD force tracking controller.

Safety

Ensuring the stroke subject’s safety is a very important issue when designing the WAM-assisted rehabilitation training system. The WAM safety systems were consisted of the safety board and the safety supervisory control threads developed in the current research, which were used to monitor the joint torques and velocities, communication between the computer and the Pucks™, voltage levels to the WAM, and other safety states including E-STOP, IDLE, and ACTIVATED. The safety systems will register a heartbeat fault and shut down the WAM manipulator, when the joint torques, end-effector velocities, and working voltages exceed the predefined thresholds or any motor controllers fail to issue communication.

Results

The subject arm’s bio-impedance identification

To demonstrate the fact that the bio-stiffness and bio-damping parameters can be used to show the subject arm’s muscle strength change, two experimental scenarios with a healthy subject were developed. The first is to perform the resistance exercises many times, and the other is to simulate a spasticity-like situation by asking the training subject to intentionally apply force against the direction of the desired resistive force. Using the trial and error method, the parameters x_e, T, λ_max, λ_min, l, R, θ₀, P₀ were set as 0.576, 0.05, 0.99, 0.1, 5, 20, [1E−6, 1E−6], [2.5E+6, 2E+6; 1.5E+6, 1E+6], respectively.

Figure 4(a) describes the estimation results of the subject’s bio-damping and bio-stiffness parameters under the first scenario. As can be seen from the profiles, the training subject’s bio-impedance parameters can sensitively reflect the subject’s muscle strength changes. Figure 4(b) shows the subject’s bio-impedance parameter changes when the subject intentionally applied force. It is obvious that the estimated subject arm’s bio-damping and bio-stiffness show a substantial increase at the moment when the arm generates intentionally increased muscle force. Both the estimated results agree with the fact that the arm’s bio-stiffness and bio-damping parameters show high sensitivity and consistency to the subject arm’s muscle strength change using the proposed RLS method with an adaptive forgetting factor.

Figure 4.

Estimation of the subject arm’s bio-impedance parameters with two experimental scenarios. (a) The subject’s regular resistance exercise and (b) the subject’s intentionally applied force.

The hybrid system–based controller regulation

To demonstrate the feasibility of the proposed hybrid system–based control architecture in generating accommodating regulations in an automated and safe manner. The performances from one healthy subject were selected as representative examples. Following the physical therapist’s advice, the low and upper thresholds for the changes of the training subject’s bio-impedance averages, $\Delta {\overline{\widehat{b}}}_{e\_\text{low}}$ , $\Delta {\overline{\widehat{b}}}_{e\_\text{upper}}$ , $\Delta {\overline{\widehat{k}}}_{e\_\text{low}}$ , and $\Delta {\overline{\widehat{k}}}_{e\_\text{upper}}$ , were set as 10 N s/m, 20 N s/m, 50 N/m, and 70 N/m, respectively. The safety-related limits for the changes of the subject’s bio-impedance averages, $\Delta {\overline{\widehat{b}}}_{e\_\text{limit}}$ and $\Delta {\overline{\widehat{k}}}_{e\_\text{limit}}$ , were set as 45 N s/m and 100 N/m, respectively. The plant events and the discrete control states, except for the emergent safety-related events, would not be activated until the changes of the subject’s bio-impedance averages in one block exceed the predefined thresholds. Once the increasing or decreasing control state was activated, 5 N resistive force would be generated. According to DeLorme’s muscle strength training program, ¹² the recruited healthy subject was firstly asked to perform 10-repetition maximum (10RM) resistance exercises, and 70%–80% of 10RM was set as the subject’s initial resistive force. The 30N initial resistive force was set. According to the WAM manipulator’s specifications, the safety-related limits for the robotic system working voltage ( $u_{r\_\text{limit}}$ ), end-effector velocity ( $v_{r\_\text{limit}}$ ), and joint torque ( $T_{r\_\text{limit}}$ ) were specified as 55 V, 1.5 m/s and 9000, respectively. The low-level PD force tracking controller gain were set as K_P = 23 and K_D = 85.

In the first case, the hybrid system–based controller regulation results concerning a healthy subject’s muscle strength improvements and spasticity-like safety-related issue, shown in Figure 5(a) to (c), were representatively analyzed. When the training task began, the initial state Q₁ was activated (Figure 5(b)), and the 30 N initial resistive force was exerted on the robotic end-effector. At the end of the fifth training trial, the changes of the subject’s bio-impedance averages in the first block met requirements $0<\Delta {\overline{\widehat{b}}}_{e\_\text{block}}<\Delta {\overline{\widehat{b}}}_{e\_\text{low}}$ or $0<\Delta {\overline{\widehat{k}}}_{e\_\text{block}}<\Delta {\overline{\widehat{k}}}_{e\_\text{low}}$ (Figure 5 (a)), and thus σ₂ was generated, then the holding state Q₄ became active and the control vector of holding the current desired resistive force c₄ was sent to the low-level force regulating and tracking controller. In the second training block (scenario 1), the changes of the training subject’s bio-impedance averages met the σ₃ hypersurface definition, $\Delta {\overline{\widehat{b}}}_{e\_\text{low}}\le \Delta {\overline{\widehat{b}}}_{e\_\text{block}}\le \Delta {\overline{\widehat{b}}}_{e\_\text{upper}}$ and $\Delta {\overline{\widehat{k}}}_{e\_\text{low}}\le \Delta {\overline{\widehat{k}}}_{e\_\text{block}}\le \Delta {\overline{\widehat{k}}}_{e\_\text{upper}}$ , then the increasing state Q₂ became active and the control vector c₂ of increasing the current resistive force from 30 N to 35 N was generated. To ensure the subject’s safety and comfort, the 5 N resistive force increment was smoothly exerted using the S-shape transition method (Figure 5(c), left) instead of directly giving a step-like force inputs (A–A′).

Figure 5.

The hybrid system–based controller regulation results concerning the subject’s muscle strength increment and spasticity-like safety event. (a) The changes of the training subject’s bio-damping and bio-stiffness averages, (b) the plant events and the discrete control states activation, and (c) the desired resistive force regulating and tracking controller.

The increased resistive force was held in the subsequent training trials until the changes of the subject’s bio-impedance averages increased substantially in the 23th training trial (scenario 2), where the subject was asked to intentionally increase his muscle force to simulate a spasticity situation. In this scenario, the changes of the subject’s bio-impedance averages, $\Delta {\overline{\widehat{b}}}_{e\_\text{trial}}$ and $\Delta {\overline{\widehat{b}}}_{e\_\text{trial}}$ , exceeded the predefined safety limits $\Delta {\overline{\widehat{b}}}_{e\_\text{limit}}$ and $\Delta {\overline{\widehat{k}}}_{e\_\text{limit}}$ , and the safety-related event σ₆ was triggered. According to the decision-control rules shown in Figure 3, the stopping state Q₅ was activated and the control vector c₅ was generated to shut off the rehabilitation robotic system. Similarly, the 35 N resistive force exerted on the WAM end-effector cannot be canceled in a step way (B′–B), which could cause a sudden stop and should be removed in a smooth manner. The same switching mechanism as the desired resistive force regulation in the former scenario was achieved. Nevertheless, the S-shape acceleration and deceleration parameters were differently designed based on the step amplitude. It can be further seen from Figure 5(c), the actual resistive force has a good tracking with the desired resistive force although a certain force tracking errors still exist.

In the second case, the training profile between the 60th and the 85th training trials was extracted, shown in Figure 6(a) to (c). At the end of the 60th training trial, the muscle strength increasing event σ₃ was triggered (Figure 6(b)), then the increasing control state Q₂ was activated, and the desired resistive force from 40 N to 45 N was smoothly achieved. The training subject performed resistance exercises in the subsequent training block with the increased 45 N resistive force. Nonetheless, possibly due to fatigue, the subject’s muscle strength cannot well sustain the exerted resistive force, which can be seen from the changes of the subject’s bio-impedance averages between the 65th and the 70th training trials, $\Delta {\overline{\widehat{b}}}_{e\_\text{block}}<0$ and $\Delta {\overline{\widehat{k}}}_{e\_\text{block}}<0$ (Figure 6(a)). At the end of the 70th training trial (scenario 3), the muscle strength decreasing event σ₃ was generated, and the decreasing control state Q₃ was activated. The 5 N resistive force decrement was obtained from the control vector c₃ (Figure 6(c), left). To avoid an instantaneous step force (C–C′), the same S-shape transition as the one in the 5 N resistive force increment transition was set.

Figure 6.

The hybrid system–based controller regulation results concerning the subject’s muscle strength decrement and suspending event. (a) The changes of the training subject’s bio-damping and bio-stiffness averages, (b) the plant events and the discrete control states activation, and (c) the desired resistive force regulating and tracking controller.

When the subject wanted to have a rest and pressed the pause shortcut key “/” (predefined in the WAM control threads) at the 72th training trial (scenario 4, the point D in Figure 6(c), right), a suspending event σ₆ was triggered and the stopping state Q₅ became active. The commands of canceling the current resistive force exerted on the robotic end-effector, saving the current training task parameters, and shutting off the rehabilitation robotic system c₅ were then sent to the low-level desired resistive force regulating and tracking controller. In order to achieve a smooth null actual resistive force and prevent the training subject from substantial force shock (D–D′), the S-shape desired resistive force trajectory (D–E) was planned using the abovementioned transition method. After pausing for a while, the subject wanted to continue the resistance exercise and pressed the resume shortcut key “\” at the point E′, the resuming event σ₇ was generated and the resuming control state Q₆ was activated. The control vector of turning on the rehabilitation robotic system, resuming the saved training task parameters, and exerting the previous resistive force on the WAM end-effector, c₆, was given to the low-level desired resistive force regulating and tracking controller. Then the 40 N desired resistive force at the decreasing state and before pressing the pause shortcut key was resumed. The resuming event σ₇₃ was triggered to activate the control state Q₃. And thus the training subject could continue the resistance exercise with the same resistive force as before. Likewise, a smooth resistive force trajectory from zero to 40 N (E′–F′) was generated and good actual force tracking was achieved. The 40 N resistive force was held till the next training block. At the end of the 80th trial, the changes of the subject’s bio-impedance averages met the σ₃ activation definition $\Delta {\overline{\widehat{b}}}_{e\_\text{low}}\le \Delta {\overline{\widehat{b}}}_{e\_\text{block}}\le \Delta {\overline{\widehat{b}}}_{e\_\text{upper}}$ and $\Delta {\overline{\widehat{k}}}_{e\_\text{low}}\le \Delta {\overline{\widehat{k}}}_{e\_\text{block}}\le \Delta {\overline{\widehat{k}}}_{e\_\text{upper}}$ again, and the increasing control state Q₂ was switched to generate the 5 N resistive force increment control vector. It can be seen from Figure 6(c), right, that if the proposed hybrid system–based controller does not exist, the subject could not resume the training exercise with the same task parameter as before. Besides, when the subject continues his resistance training after pausing a while, the desired resistive force would not be automatically modified to follow the trajectory D–E–E′ –F′, while be directly exerted on the WAM end-effector with non-zero resistive force at the point F. This is an unsafe rehabilitation training condition and will cause a second injury especially for the stroke survivors.

Discussions

A novel control architecture for rehabilitation robotic training systems using the hybrid system theory was proposed in this research. With the presented control architecture, a paradigm study on progressive resistance training using the subject’s online estimated bio-damping and bio-stiffness changes was conducted. Preliminary results confirm the efficacy of the proposed strategy.

The advantages of the hybrid system–based control architecture in the robot-assisted rehabilitation training were summarized as the following two aspects. On the one hand, the proposed control architecture incorporated the physical therapist’s monitoring and decision-making, the stroke survivor’s biomechanical feedback, and the robotic device’s motor intervention into a unified framework. To some extent, it played the physical therapist’s roles on the therapeutic progress assessment, the behaviors decision-making, and safety-related issues monitoring. On the other hand, the hybrid system–based control architecture is open and flexible. The discrete control states and plant events may be flexibly extended or modified according to the new task requirements. Its cost of extending and modifying is far below some of the intelligent cybernetics-based control methods. In our previous work, ¹² an adaptive hierarchical control framework, integrating a high-level resistive force-based fuzzy supervisory controller and a low-level resistive force triggered fuzzy regulator, is developed. The disadvantage of this control method is that if the training program makes a change or a new task is to perform, the fuzzy control rules and the membership functions will have to reprogram. More importantly, if safety-related events are generated, the only emergency way is to shut down the power supply button, but sometimes this is inconvenient or unachievable for the severe stroke survivors.

It must be noted that this is a preliminary pilot study of such a control architecture. The current research mainly focuses on the interactive control between the training subject’s biomechanical feedback such as the bio-impedance changes and the robotic device’s intervention. In the future, the stroke survivor’s physiological or even psychological feedback should also be considered.

Conclusions

In this article, a paradigm research on robot-assisted resistance training, using the hybrid system modeling technique and the subject’s bio-damping and bio-stiffness estimation, was carried out. Experimental results with healthy subjects conducted on the Barrett WAM^TM manipulator verified that (1) the subject arm’s muscle strength improvements were well evaluated by the online estimated bio-damping and bio-stiffness changes; (2) the modification of the desired resistive force generated from the discrete event controller was well matched to the subject’s muscle strength progress; and (3) the proposed hybrid system–based control architecture was capable of automatically generating accommodating plans when monitoring the safety-related issues such as spasticity, and the abnormal robotic working states happen.