Sage Journals: Discover world-class research

Abstract

Space robot teleoperation system with haptic device has two operation modes: offline teleoperation mode and online teleoperation mode. High acceleration and jerk produced by the teleoperation system may degrade the performance of trajectory tracking and cause large residual vibration. In this article, human hand is treated as a kind of sensor, and two smoothers based on the standard Kalman smoother have been proposed to limit the acceleration and jerk in joint space in a certain range. On the one hand, the offline smoother for offline teleoperation mode makes use of a smoothing algorithm proposed in the literature, which is based on the interior point techniques. On the other hand, the online smoother for online teleoperation mode is proposed and named as reduced-Q method, which is a fixed-lag smoother with adjustable measurement noise. Experimental results have shown that the methods can constrain the acceleration and jerk of the trajectories in joint space within the specified range approximately and can improve the tracking accuracy without causing too many deformations. Thus, the psychology burden of operator will be alleviated.

Keywords

Teleoperation space robot haptic interface Kalman smoother

Introduction

The space robot has been an important research direction in the past few decades.¹ However, it is almost impossible to realize a perfect autonomous space robot due to the limitations of the current technologies. Thus, the teleoperation system, which can assist human to teleoperate a semiautonomous space robot, is still a very important research topic.^2

–6 The teleoperation system has two modes by using haptic device: offline teleoperation mode and online teleoperation mode. In online mode, the operator controls the space robot based on the vision feedback directly. When there are few barriers nearby the operating path, the online teleoperation mode should be used to guarantee real-time operations. Otherwise, the offline teleoperation mode is used instead. Specifically, the operator controls the virtual model of the space robot to complete the mission and then sends the data to the real space robot after the rationality has been validated.

On account of the increasing demand that space robots are more lightweight, both link and joint flexibility effects become the main limitation for the performance of trajectory tracking.⁷ The high acceleration and jerk can lower the precision of trajectory tracking and cause large residual vibration.⁸ Moreover, when undertaking the teleoperation tasks, different operators have different proficiencies and cannot sense the acceleration and jerk calculated by the inverse kinematics quantitatively. The time delay of space robot teleoperation system is so large that the effect of current manipulation on the space robot is difficult to predict. Therefore, setting the limitations of acceleration and jerk is very important in both online and offline work mode of space robot teleoperation system. However, a brutal saturation on the planning data can cause a negligible deformation of the planning path. Interpolation methods such as cubic polynomial, quintic polynomial, and spline functions are widely used to deal with the robot motion planning with acceleration and jerk bounded problem.^9
–11 Interpolation methods are not quite fit for the trajectory planning of teleoperation system of space robot, since that planning is time-consuming and the trajectory generated also has a large deviation compared with anticipated trajectory. Some researchers propose the acceleration feedback control to improve the trajectory tracking performance.^12,13 For online teleoperation mode, very few papers take into account the acceleration and jerk’s characteristics simultaneously. Since it is difficult to obtain the accurate dynamic model of the manipulator, some manipulators may do not have current loop. Another noteworthy topic is how to improve the measurement precision of the haptic device effectively.¹⁴ And these methods do not take into account the constraints of kinematics of slave robot (space robot). Additional time delay method is first proposed in the teleoperation experiment of ETS-7, and it can be used to promote the safety of collision detection and reedit the command data stored in command buffer.¹⁵ However, the time delay in the experiment is up to 10 s, and the deformation comparing with the original trajectory is too large. Despite the effort devoted at solving the constrained trajectory planning, at the best of the author’s knowledge, none of the literatures address the online smoothing methods for teleoperation system which can satisfy the limitation of the slave robot in joint space.

When designing a teleoperation system, we can take the operator’s hand as a kind of sensor under the assumption that the haptic device has high precision. The expected trajectory with acceleration and jerk bounded approximately of slave robot in the joint space is treated as states with zero progress noises. And this kinds of limitation are known as soft constraints. Thus, filter or smoother can be used to smooth the data to be sent. Kalman filtering is a very effective and practical means of separating signal from noise, which is known to be the optimal estimator for linear systems corrupted by white noises.¹⁶ However, in the practical application, the noises are not white mainly because the state equation cannot be modeled precisely. Thus, the incorporation of equality or inequality constraints of states can improve the performance of the Kalman filter. There are many examples of state-constrained systems in engineering applications including navigation, robotics, and target tracking.^17

–20 And the system model in this article is a typical target tracking problem. The main methods of Kalman filter with states constraints are described by Simon.²¹ The basic idea of which is using the constrained numerical optimization to incorporate the equality or inequality constraints into the frame of Kalman filter. Kalman smoother has a better performance compared with Kalman filter. It can be classified into three categories: fixed-interval smoother, fixed-point smoother, and fixed-lag smoother.²² Thus, smoothing methods with state constraints can be considered as suitable approaches for the teleoperation systems.

In this article, two smoothers are presented to smooth the data to be sent. On the one hand, the inequality-constrained fixed-interval smoother proposed by Bell et al.²³ is used as offline smoother for the offline teleoperation mode. On the other hand, a fixed-lag Kalman smoother by using the additional time delay is proposed as online smoother for the online teleoperation mode, which can adjust smoothness online according to different operators and kinestate constraints of the slave robot.

The rest of this article is organized as follows. In “Problem statement” section, the problem statement is described. The offline smoother with soft constraints is discussed in “Offline smoothing with interior point techniques” section. “Online smoothing with additional time delay and reduced-Q method” section describes the method of online smoothing with soft constraints. Experiments and discussions are presented in “Experiment” section, followed by concluding remarks in “Conclusion” section.

Problem statement

During teleoperation tasks, the operator cannot sense the acceleration and jerk of slave robot in the joint space quantitatively. Even though some teleoperation system provides real-time display of kinestate, operators have to pay more attention to the current operation tasks. Different operators have different proficiency, and even skilled operators cannot guarantee that all operations are smooth enough. From the perspective of signal processing application, human’s hand can be taken as a kind of sensor, and the expected trajectory can be taken as ideal state with zero progress noise. Thus, smoother can be used to get rid of the operation deviation.

The smoother will have an effect in any position of the data flow, but we expect to limit acceleration and jerk calculated by the inverse kinematics of slave robot. Therefore, the data can be smoothed before sending to the slave robot, as shown in Figure 1.

Figure 1.

The data flow of teleoperation system.

The smoother will smooth each joint target points separately. And jounce, the derivative of jerk, will be taken as the progress noise. Thus, it is a target tracking problem. Then, a four-state target tracking system model of Kalman smoothing in continuous time can be expressed as

\frac{d}{d t} [\begin{matrix} q_{i} \\ {\dot{q}}_{i} \\ {\ddot{q}}_{i} \\ {\overset{⃛}{q}}_{i} \end{matrix}] = [\begin{matrix} 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \end{matrix}] [\begin{matrix} q_{i} \\ {\dot{q}}_{i} \\ {\ddot{q}}_{i} \\ {\overset{⃛}{q}}_{i} \end{matrix}] + [\begin{matrix} 0 \\ 0 \\ 0 \\ 1 \end{matrix}] w_{i} (t)

z_{i} = q_{i} + v_{i} (t)

where q_i and z_i denote the position and measurement at time t of joint i. And w_i(t) and v_i(t) denote the process noise and measurement noise, respectively. Then the discrete model can be given as follows

x_{i, k + 1} = Φ x_{i, k} + w_{i, k}

z_{i, k} = H x_{i, k} + v_{i, k}

x_{i, k} = [\begin{matrix} q_{i} \\ {\dot{q}}_{i} \\ {\ddot{q}}_{i} \\ {\overset{⃛}{q}}_{i} \end{matrix}], H = [\begin{matrix} 1 & 0 & 0 & 0 \end{matrix}]

Φ = [\begin{matrix} 1 & Δ t & \frac{Δ t^{2}}{2} & \frac{Δ t^{3}}{6} \\ 0 & 1 & Δ t & \frac{Δ t^{2}}{2} \\ 0 & 0 & 1 & Δ t \\ 0 & 0 & 0 & 1 \end{matrix}], Q_{i, k} = σ_{i, k} [\begin{matrix} \frac{Δ t^{7}}{252} & \frac{Δ t^{6}}{72} & \frac{Δ t^{5}}{30} & \frac{Δ t^{4}}{24} \\ \frac{Δ t^{6}}{72} & \frac{Δ t^{5}}{20} & \frac{Δ t^{4}}{8} & \frac{Δ t^{3}}{6} \\ \frac{Δ t^{5}}{30} & \frac{Δ t^{4}}{8} & \frac{Δ t^{3}}{3} & \frac{Δ t^{2}}{2} \\ \frac{Δ t^{4}}{24} & \frac{Δ t^{3}}{6} & \frac{Δ t^{2}}{2} & Δ t \end{matrix}], R_{i, k} = r_{i, k}

where k is the time step, x_i,k is the states, y_i,k is the measurement, w_{i, k} is the process noise, v_i,k is the measurement noise, Φ is the state transition matrix, H is measurement matrix, Q_i,k is the covariance matrix for the process noise, R_i,k is the covariance for the measurement noise, σ_i,k is the covariance for w_i(t), r_i,k is the covariance for v_i(t), and Δt is the sample interval. The equation (6) is the application foundation of Kalman smoother, for they are the modes of online and offline smoother.

The derivation of the standard Kalman smoother assumes that the both process noise and measurement noise are Gaussian white noise, but the process noise is colored noise in our application. There are several methods for dealing with colored process noise such as state augmentation and shaping filter. Prerequisite for application of these methods is mathematical modeling for the colored noise. However, the movement of operator’s hand mainly depends on the operation tasks and operator’s decision, so it cannot be modeled accurately. In order to improve smoother performance in this circumstances, the fading-memory method or just increasing process noise can be used directly to give more credence to the measurement in the practical application of Kalman smoother.²⁴

Meanwhile, operators also want to constrain the acceleration and jerk within certain limits as much as possible

| {\ddot{q}}_{i, k} | ≲ {acc}_{i, max}, | {\overset{⃛}{q}}_{i, k} | ≲ {jerk}_{i, max}

where acc_{i, max} and jerk_{i, max} are the limits of acceleration and jerk of the slave robot. Thus, it can be seen that the problem is a linear system model with linear soft inequality constraints according to formulas (6) and (7).

According to the operating modes of our space robot teleoperation system, two smoothing methods are presented in the following two sections.

Offline smoothing with interior point techniques

On the one hand, the time window of space robot teleoperation is short, so the offline trajectory planning should be efficient. On the other hand, the reasonableness of the planned trajectory should also be verified, which includes the collision detection, singularity detection, and rationality of planned kinestate for the slave robot. In the offline teleoperation mode, operator controls the virtual model by using the haptic device to generate the trajectory according to the task requires at first. Then, the generated trajectory will be smoothed, and the reasonableness of track will be verified. Finally, the planning data will be sent to the space robot. When smoothing the trajectory, a kind of fixed-interval smoother with inequality constraints can be used to limit its acceleration and jerk.

For the linear system model with linear inequality constraints, there are four main modifications of Kalman filter or smoother²¹: estimate projection, gain projection, probability density function truncation, and interior point likelihood maximization (IPLM). These methods are mainly dealing with the following quadratic programming problem within the framework of Kalman filter

{\tilde{x}}_{i, k}^{+} = arg min_{x} {(x - {\hat{x}}_{i, k}^{+})}^{T} W (x - {\hat{x}}_{i, k}^{+})

Such that

b_{i, k} + d b_{i, k} \cdot x_{i, k} \leq 0

where ${\hat{x}}_{i, k}^{+}$ is the a posteriori estimation of x_i,k , b_i,k is the limitation, and db_i,k is the sign matrix. IPLM proposed by Bell et al.²³ has a higher computational efficiency compared with other optimization algorithms, so it is chosen as offline smoothing algorithm. When applying the algorithm, the very important thing is how to select the constraints correctly. It is designed to limit the acceleration and jerk only. However, there is potential association between states. If the acceleration and jerk is presented by their difference expressions as follows

{\ddot{q}}_{i, k} = \frac{q_{i, k} - 2 q_{i, k - 1} + q_{i, k - 2}}{Δ t^{2}}

{\overset{⃛}{q}}_{i, k} = \frac{q_{i, k} - 3 q_{i, k - 1} + 3 q_{i, k - 2} - q_{i, k - 3}}{Δ t^{3}}

Such that

| {\ddot{q}}_{i, k} | \leq {acc}_{max}, | {\overset{⃛}{q}}_{i, k} | \leq {jerk}_{max}

This gives that

{\begin{matrix} - {acc}_{max} \cdot Δ t^{2} - q_{i, k - 2} + 2 q_{i, k - 1} \leq q_{i, k} \leq {acc}_{max} \cdot t^{2} - q_{i, k - 2} + 2 q_{i, k - 1} \\ - {jerk}_{max} \cdot Δ t^{3} + q_{i, k - 3} - 3 q_{i, k - 2} + 3 q_{i, k - 1} \leq q_{i, k} \leq {jerk}_{max} \cdot Δ t^{3} + q_{i, k - 3} - 3 q_{i, k - 2} + 3 q_{i, k - 1} \end{matrix}

Which means that

{\begin{matrix} l b_{i,1} = max (- {acc}_{max} \cdot t^{2} - q_{i, k - 2} + 2 q_{i, k - 1}, - {jerk}_{max} \cdot Δ t^{3} + q_{i, k - 3} - 3 q_{i, k - 2} + 3 q_{i, k - 1}) \\ u b_{i,1} = min ({acc}_{max} \cdot t^{2} - q_{i, k - 2} + 2 q_{i, k - 1}, {jerk}_{max} \cdot Δ t^{3} + q_{i, k - 3} - 3 q_{i, k - 2} + 3 q_{i, k - 1}) \\ l b_{i,1} \leq q_{i, k} \leq u b_{i,1} \end{matrix}

The limitation of joint velocity can be calculated similarly

{\begin{matrix} l b_{i,2} = max (- {acc}_{max} \cdot Δ t + {\dot{q}}_{i, k - 1}, - {jerk}_{max} \cdot Δ t^{2} - {\dot{q}}_{i, k - 2} + 2 {\dot{q}}_{i, k - 1}) \\ u b_{i,2} = min ({acc}_{max} \cdot Δ t + {\dot{q}}_{i, k - 1}, {jerk}_{max} \cdot Δ t^{2} - {\dot{q}}_{i, k - 2} + 2 {\dot{q}}_{i, k - 1}) \\ l b_{i,2} \leq {\dot{q}}_{i, k} \leq u b_{i,2} \end{matrix}

Thus, in order to meet (12), the constraints (14) and (15) must also be satisfied. However, since all points should be dynamically adjusted simultaneously during smoothing processing, equations (14) and (15) cannot be satisfied when using fixed-interval smoother. Only equation (11) can be used to constrain the process states, and the effect of that is equal to the constraints(7) which is soft constraints. The constraint (11) can be modified in the form of equation (9)

\underset{b_{i, k}}{\underset{︸}{[\begin{matrix} - {acc}_{max} \\ - {acc}_{max} \\ - {jerk}_{max} \\ - {jerk}_{max} \end{matrix}]}} + \underset{d b_{i, k}}{\underset{︸}{[\begin{matrix} 0 & 0 & 1 & 0 \\ 0 & 0 & - 1 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & - 1 \end{matrix}]}} \cdot \underset{x_{i, k}}{\underset{︸}{[\begin{matrix} q_{i, k} \\ {\dot{q}}_{i, k} \\ {\ddot{q}}_{i, k} \\ {\overset{⃛}{q}}_{i, k} \end{matrix}]}} \leq [\begin{matrix} 0 \\ 0 \\ 0 \\ 0 \end{matrix}]

When the constraints are replaced, the smoother with inequality constraints will be degenerated into the standard fixed-interval smoother.

Online smoothing with additional time delay and reduced-Q method

In the mode of online teleoperation, a 0.75 s additional time delay is added to guarantee safety. The additional time delay is composed of a first-in first-out (FIFO) queue. The command buffer and the smoother are placed between the task plan manager and the telemetry and telecommand subsystem (TM/TC). The data flow of teleoperation system is shown in Figure 2.

Figure 2.

The data flow of teleoperation system.

When collision of the three-dimensional (3-D) pre-simulator is detected, the buffer will be filled with the latest data immediately, as shown in Figure 3.

Figure 3.

The response of the buffer with collision detection.

The additional time delay can ensure that the actual space robot keeps a certain distance away when the pre-simulator collides with the bounding box.

The model with dashed lines represents the actual space robot, and the model with solid lines represents the pre-simulator. The configuration of the pre-simulator shown in Figure 4 is near the mechanical limits on joints. So it is difficult to find the right direction in which operator can move the robot away without the additional time delay.

Figure 4.

The effect of collision detection with additional time delay.

In the meantime, the data stored in the buffer can be used to construct a fixed-lag Kalman smoother. Considering the numerical stability, Biswas–Mahalanobis fixed-lag smoother (BMFLS) is used as the smoother.²⁵ The data in the FIFO buffer are used to define a superstate vector as follows

X_{i, k}^{BMFLS} = [\begin{matrix} q_{i, k} \\ q_{i, k - 1} \\ q_{i, k - 2} \\ q_{i, k - 3} \end{matrix}]

The corresponding state transition and measurement matrixes are

Φ^{BMFLS} = [\begin{matrix} Φ & 0 & 0 & 0 \\ I & 0 & 0 & 0 \\ 0 & I & 0 & 0 \\ 0 & 0 & I & 0 \end{matrix}], H^{BMFLS} = [\begin{matrix} H & 0 & 0 & 0 \end{matrix}]

And the covariance matrixes for process noise and measurement noise are

Q_{i, k}^{BMFLS} = [\begin{matrix} Q_{i, k} & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{matrix}], R_{i, k}^{BMFLS} = R_{i, k}

In fact, the implementation equations for the BMFLS are the same as those for the standard Kalman filter by using the parameters above. If viewed as sensors with different precisions, different operators mean different initial values of process noise covariance. On the other hand, the time window of space robot teleoperation system is short, so the dexterity of the operators is considered as constant, which means the covariance for process noise can be considered to be unknown constant value. Thus, the appropriate covariance can be estimated according to the constraints during the operation. In the practical application of Kalman smoother, the covariance σ_k can be used to adjust the smoothness of the trajectory, because the smoothness of trajectory will increase when the corresponding σ_k is small.

Thus, an online adjustment method for the covariance σ_k is proposed. First, the position q_i,k of joint i at time step k is smoothed to get ${\hat{q}}_{i, k}$ , and the acceleration ${\ddot{q}}_{i, k}$ and jerk ${\overset{⃛}{q}}_{i, k}$ are calculated by using ${{\hat{q}}_{i, k} \dots {\hat{q}}_{i, k - 3}}$ . Second, if the acceleration and jerk exceed the upper limit, ${\hat{σ}}_{m, k}$ will be given by subtracting Δσ from σ_k. Third, BMFLS is used to calculate the acceleration ${\ddot{q}}_{m, k}'$ and jerk ${\overset{⃛}{q}}_{m, k}'$ again by using the data of first 10 s, and the constraints are judged again. The loop will be terminated when the constraints satisfied or the lower limit σ_min of σ_k is reached. The method proposed here will be called the reduced-Q method, as shown in Algorithm 1.

The σ_k recalculated will be used in the next cycle, and it is lower bounded by σ_min, so it can just satisfy a kind of soft constraints. σ_k will converge to a smaller value after every adjustment, so the trajectory will more smooth in the next operation. Δσ determines the speed of convergence, and a smaller value for it is preferred in accordance with the processor power to reduce the deformation caused by over smoothing.

Experiment

Environment of experiment

The onboard system consists of three subsystems, including robotic manipulator system, free-flying platform, and microtarget system, and the on-ground system consists of TM/TC and teleoperation system. The TM/TC is the data exchange center for onboard system and teleoperation system. Communication mode between onboard system and on-ground is the satellite communication. The relationship is shown in Figure 5.

Figure 5.

The relationship of onboard system and on-ground system.

The simulator of space robot is consisted of a 6-DOF manipulator and planar air bearing. The configuration of the manipulator can be found in the literature.^26,27 Table 1 lists the nominal robot link parameters of the robot.

Table 1.

The nominal link parameters in Denavit-Hartenberg (DH) model for the 6-DOF manipulator.

DH joint	a (mm)	α (rad)	d (mm)	θ (rad)
1	0	−π/2	0	−π/2
2	990	0	0	0
3	0	−π/2	0	−π/2
4	0	−π/2	−780	0
5	0	−π/2	0	0
6	0	0	372	0

DH: Denavit-Hartenberg Matrix.

The manipulator can just move in the two-dimensional plane, so only three joints can be rotated. The 3-D pre-simulator is used to realize the collision detection, and solid-frame model and wire-frame model represent current planning position and position received from space robot. Offline teleoperation mode, online teleoperation mode, and their smoothing are realized by task plan manager. The Virtuose 6D35-45 (Haption Virtuose 6D35-45 technical specifications) is used as the haptic device. Its operational workspace in translation is 450 mm and minimum position resolution is 0.006 mm.And task plan manager receives the position data from haptic device at 100 Hz. API Tracker 3 (Haption Virtuose 6D35-45 technical specifications) is used as a track recorder for the movement of the manipulator’s end effector. Its measurement accuracy is 0.001 mm and measurement frequency is 100 Hz. System structure of ground verification of the space robot teleoperation is shown in Figure 6.

Figure 6.

The system structure of ground verification.

The experimental platform is shown in Figure 7.

Figure 7.

The experimental platform of space robot teleoperation system.

Firstly, the operator controls the manipulator to move along a trajectory. Then, the offline-constrained Kalman smoother (Offline-CKS) based on IPLM and the online-constrained Kalman smoother (Online-CKS) based on BMFLS with reduced-Q method are used to smooth the trajectory. Meanwhile, the fixed-interval Kalman smoother, BMFLS, and Kalman filter are used as comparative method. Communication frequency between task plan manager and central controller of space robot is 4 Hz, so Δt is set to 0.25 s. The maximum acceleration (acc_max) and maximum jerk (jerk_max) of each joint should be determined by the dynamic simulation of robot manipulator and the output power of the motors and then verified by the experiment on the planar air bearing. In this experiment, the two parameters are given according to the calibration, which are ${acc}_{max} = 4$ °/s² and ${jerk}_{max} = 8$ °/s³. Most textbooks and papers are focus on the theoretical derivation and simulation experiment about Kalman filter, and the noise is the Gaussian white noise whose initial values of covariance are given as the known quantity. However, in the practical engineering, the initial value of covariance can be determined by the given parameter of sensors. In this research, the sensor is the operator’s hand, and the initial value is determined by the experience operator. The process noise is generated by the operating error and operating intention, so it should be determined by the operator’s debugging, and we choose σ_k = 25. The accuracy of measuring instrument is relatively high, so the measurement noise is set as r_k = 0.01. The standard Kalman filter can also be used to smooth the trajectory with constraints by adjusting the process noise covariance. However, Kalman filter can only adjust the current point, so the covariance for the measurement noise must be turned down as much as possible. Thus, the process noise covariance of the Kalman filter can be chosen as σ_k = 1.

Results of experiment and discussion

The smoothed joint position, velocity, acceleration, and jerk of the trajectory in joint space are shown in Figures 8 and 9. It can be seen that the proposed methods can constrain the acceleration and jerk within the specified range approximately without causing too many deformations on trajectory.The tail end tracks of original data and smoothed data and the dynamic responses of the space robot simulator by these different approaches are shown in Figures 10 and 11.

Figure 8.

Joint position and velocity of the planning data. The deformations of smoothed joint position and velocity compared with the original are all very little except Kalman filtering.

Figure 9.

The acceleration and jerk of the planning data. The Offline-CKS and Online-CKS can constrain the acceleration and jerk within the limitation approximately. These two methods both achieve the expected result. Offline-CKS: offline-constrained Kalman smoother; Online-CKS: online-constrained Kalman smoother.

Figure 10.

The tail end tracks of manipulator. The deformations of the end track are all small except Kalman filtering.

Figure 11.

Dynamic responses of the space robot simulator. All these smoothing methods can increase the accuracy of dynamic response of the manipulator. And the effect of Offline-CKS is the most obvious. Offline-CKS: offline-constrained Kalman smoother.

Let x_ORG (i), y_ORG (i), x_SMT (i), and y_SMT (i) denote the Cartesian coordinate of the original and smoothed data at time i. And let the corresponding Cartesian coordinate measured by API denote as $x_{ORG}^{API} (i), y_{ORG}^{API} (i), x_{SMT}^{API} (i), and y_{SMT}^{API} (i)$ . We define erro_mean and erro_max as mean error and maximum error of deformation

{erro}_{mean} = Mean ({\sqrt{{(x_{ORG} (i) - x_{SMT} (i))}^{2} + {(y_{ORG} (i) - y_{SMT} (i))}^{2}}})

{erro}_{max} = Max ({\sqrt{{(x_{ORG} (i) - x_{SMT} (i))}^{2} + {(y_{ORG} (i) - y_{SMT} (i))}^{2}}})

Let e_ORG (i) and e_SMT (i) denote the response error of the original and smoothed data at time i

e_{ORG} (i) = \sqrt{{(x_{ORG} (i) - x_{ORG}^{API} (i))}^{2} + {(y_{ORG} (i) - y_{ORG}^{API} (i))}^{2}}

P is defined as improved ratio of dynamic track performance to describe the dynamic track performance improvement with different smoothing methods

P = \frac{max ({e_{ORG} (i)}) - max ({e_{SMT} (i)})}{max ({e_{ORG} (i)})}

The error of deformation and performance improvement is shown in Table 2.

Table 2.

Deformation and performance improvement.

Methods	erro_mean (mm)	erro_max (mm)	P(%)
Offline-CKS	1.516	6.687	38.0135
Offline-KS	1.466	5.636	33.0412
Online-CKS	2.645	10.469	33.4177
BMFLS	1.154	5.213	29.1131
Kalman filter	4.894	13.904	8.4476

Offline-CKS: offline-constrained Kalman smoother; Online-CKS: online-constrained Kalman smoother; Offline-KS: fixed-interval Kalman smoother; BMFLS: Biswas–Mahalanobis fixed-lag smoother.

The high acceleration and jerk can lower the precision of trajectory tracking, and all these methods can improve the dynamic track performance of the manipulator by reducing the acceleration and jerk as shown in Table 2 and Figure 11. Since only the constrained smoothing methods can limit the acceleration and jerk in the specified range, they have a better performance than the others. Moreover, the deformation error is expected to be as small as possible. The expected trajectory is taken as ideal states with zero progress noises in this article. Thus, from the optimal state estimation theory point of view, more information may improve the effect of estimation in most cases. As shown in the table, the deformation of Offline-CKS is smaller than Online-CKS. Because Offline-CKS can reedit more points than Online-CKS. Standard Kalman filter, as a comparison method, exhibits the worst performance in terms.

Algorithm 1. Reduced-Q method
Step 1: ${\hat{q}}_{i, k} \leftarrow BMFLS (q_{i, k}, σ_{k})$ Step 2: ${{\ddot{q}}_{i, k}, {\overset{⃛}{q}}_{i, k}} \leftarrow calculate a c c and j e r k by using {{\hat{q}}_{i, k}, \dots {\hat{q}}_{i, k - 3}}$ Step 3: $i f ({\ddot{q}}_{i, k} > {acc}_{max}) \| \| ({\overset{⃛}{q}}_{i, k} > {jerk}_{max}) then$ Step 4: $for all joints m do$ Step 5: ${\hat{σ}}_{m, k} \leftarrow σ_{k}$ Step 6: $while {\hat{σ}}_{m, k} > σ_{min} do$ Step 7: ${\hat{σ}}_{m, k} \leftarrow {\hat{σ}}_{m, k} - Δ σ$ Step 8: ${{\hat{q}}_{m, k}^{'} \dots {\hat{q}}_{m, k - 40}^{'}} \leftarrow BMFLS ({{\hat{q}}_{m, k}^{'} \dots {\hat{q}}_{m, k - 40}^{'}}, {\hat{σ}}_{m, k})$ Step 9: ${{\ddot{q}}_{m, k}^{'}, {\overset{⃛}{q}}_{m, k}^{'}} \leftarrow calculate acc and jerk by using {{\hat{q}}_{m, k}^{'} \dots {\hat{q}}_{m, k - 40}^{'}}$ Step 10: $if ({\ddot{q}}_{m, k}^{'} < {acc}_{max}) and ({\overset{⃛}{q}}_{m, k}^{'} < {jerk}_{max}) then$ Step 11: break Step 12: end if Step 13: end while Step 14: $end for$ Step 15: $σ_{k} \leftarrow min ({\hat{σ}}_{m, k})$ Step 16: end if

Algorithm 1. Reduced-Q method

Step 1: ${\hat{q}}_{i, k} \leftarrow BMFLS (q_{i, k}, σ_{k})$

Step 2: ${{\ddot{q}}_{i, k}, {\overset{⃛}{q}}_{i, k}} \leftarrow calculate a c c and j e r k by using {{\hat{q}}_{i, k}, \dots {\hat{q}}_{i, k - 3}}$

Step 3: $i f ({\ddot{q}}_{i, k} > {acc}_{max}) | | ({\overset{⃛}{q}}_{i, k} > {jerk}_{max}) then$

Step 4: $for all joints m do$

Step 5: ${\hat{σ}}_{m, k} \leftarrow σ_{k}$

Step 6: $while {\hat{σ}}_{m, k} > σ_{min} do$

Step 7: ${\hat{σ}}_{m, k} \leftarrow {\hat{σ}}_{m, k} - Δ σ$

Step 8: ${{\hat{q}}_{m, k}^{'} \dots {\hat{q}}_{m, k - 40}^{'}} \leftarrow BMFLS ({{\hat{q}}_{m, k}^{'} \dots {\hat{q}}_{m, k - 40}^{'}}, {\hat{σ}}_{m, k})$

Step 9: ${{\ddot{q}}_{m, k}^{'}, {\overset{⃛}{q}}_{m, k}^{'}} \leftarrow calculate acc and jerk by using {{\hat{q}}_{m, k}^{'} \dots {\hat{q}}_{m, k - 40}^{'}}$

Step 10: $if ({\ddot{q}}_{m, k}^{'} < {acc}_{max}) and ({\overset{⃛}{q}}_{m, k}^{'} < {jerk}_{max}) then$

Step 11: break

Step 12: end if

Step 13: end while

Step 14: $end for$

Step 15: $σ_{k} \leftarrow min ({\hat{σ}}_{m, k})$

Step 16: end if

Conclusion

When a space robot remote controlled by teleoperation system with haptic device, limitation of acceleration and jerk can improve precision of trajectory tracking and lower residual vibration. How to constrain the acceleration and jerk within certain limits is still an open research area facing many technical challenges. In this article, a new approach has been proposed based on the standard Kalman smoother. The operator hand was taken as a kind of position sensor, and the expected trajectories were taken as ideal states. According to the different requirements and characteristics of two operation modes of teleoperation task, offline and online smoother with soft inequality constraints had been built. IPLM was used as the offline smoother, and the reduced-Q method was used as the online smoother. The experimental results had shown that both of the two smoother can constrain the acceleration and jerk efficiently. Thus, the requirements on the precise of human operation can be reduced substantially and the mental stress of operator can also be relieved. Furthermore, the methods can also be applied in the teleoperation system of other kinds of robot, such as industrial robot and medical robot.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Stoll

Letschnik

Walter

. On-orbit servicing. IEEE Robot Autom Mag 2009; 16(4): 29–33.

Oda

Kibe

Yamagata

. Ets-vii, space robot in-orbit experiment satellite. In: Proceedings., 1996 IEEE international conference on robotics and automation, 1996 (ed George Lee

C.S.

), Minneapolis, Minnesota, 22–28 April 1996, Vol. 1, pp. 739–744. Minneapolis, Minnesota: IEEE.

Matsumoto

Wakabayashi

Penin

. Teleoperation control of ETS-7 robot arm for on-orbit truss construction. In: Artificial Intelligence, Robotics and Automation in Space (ed Michael

Perry

), ESTEC, Noordwijk, the Netherlands, 1–3 June 1999, Vol. 440, pp. 313. Noordwijk, Netherlands: European Space Agency : Contact ESA Publications Division, ESTEC.

Yoon

Tsumaki

Uchiyama

. Model-based robot teleoperation with haptic interface. Ind Robot Int J 2003; 30(6): 584–591.

Imaida

Yokokohji

Doi

. Ground-space bilateral teleoperation of ETS-vii robot arm by direct bilateral coupling under 7-s time delay condition. IEEE Trans Robot Autom 2004; 20(3): 499–511.

Wang

Liu

. A ground-based validation system of teleoperation for a space robot. Int J Adv Robot Syst 2012; 9(4): 1–9.

Flores-Abad

Pham

. A review of space robotics technologies for on-orbit servicing. Prog Aerosp Sci 2014; 68: 1–26.

Ulrich

Sasiadek

. Trajectory tracking control of flexible-joint space manipulators. Can Aeronaut Space J 2012; 58(1): 47–59.

Macfarlane

Croft

. Jerk-bounded manipulator trajectory planning: design for real-time applications. IEEE Trans Robot Autom 2003; 19(1): 42–52.

10.

Xia

Jiang

Liu

. A novel hybrid safety-control strategy for a manipulator. Int J Adv Robot Syst 2014; 11(4): 1–10.

11.

Jahanpour

Alizadeh

. A novel acc-jerk-limited nurbs interpolation enhanced with an optimized s-shaped quintic feedrate scheduling scheme. Int J Adv Manuf Technol 2015; 77(9–12): 1889–1905.

12.

Johansson

Yin

. Acceleration feedback control for nonlinear teleoperation systems with time delays. Int J Control 2015; 88(3): 507–516.

13.

Smith

Jensfelt

. A predictor for operator input for time-delayed teleoperation. Mechatronics 2010; 20(7): 778–786.

14.

Zhou

Shen

Petriu

. Linear velocity and acceleration estimation of 3 DOF haptic interfaces. In: IEEE international workshop on haptic audio visual environments and games, 2008. HAVE 2008, Ottawa, ON, Canada, 18–19 October 2008, pp. 137–142. Ottawa: IEEE.

15.

Wakabayashi

Matsumoto

. Results of the ETS-7 direct teleoperation experiment using additional time delay method. In: Proceeding of the 6th international symposium on artificial intelligence and robotics and automation in space i-SAIRAS, Quebec, Canada, 18–22 June 2001, pp. 1–8. Quebec, Canada.

16.

Kalman

. A new approach to linear filtering and prediction problems. J Basic Eng 1960; 82(1): 35–45.

17.

Alouani

Blair

. Use of a kinematic constraint in tracking constant speed, maneuvering targets: Proceedings of the IEEE conference on decision and control, 1991, Vol. 2, 1991, pp. 2055–2058.

18.

Wang

Chiang

Chang

. Filtering method for nonlinear systems with constraints. Control Theory Appl IEE Proc 2002; 149(6): 525–531.

19.

Simon

Chia

. Kalman filtering with state equality constraints. IEEE Trans Aerosp Electron Syst 2002; 38(1): 128–136.

20.

Pillonetto

Aravkin

Carpin

. The unconstrained and inequality constrained moving horizon approach to robot localization. In: IEEE/RSJ international conference on intelligent robots and systems (ed Luo

Ren C.

), Taipei, Taiwan, 18–22 October 2010, pp. 3830–3835. Taipei International Convention Center.

21.

Simon

. Kalman filtering with state constraints: a survey of linear and nonlinear algorithms. IET Control Theory Appl 2010; 4(8): 1303–1318.

22.

Brown

Hwang

PYC

. Introduction to random signals and applied kalman filtering: with matlab exercises and solutions. In: Brown

(ed) Introduction to random signals and applied Kalman filtering: with MATLAB exercises and solutions, Hwang, Patrick YC, Vol. 1. New York: Wiley, 1997.

23.

Bell

Burke

Pillonetto

. An inequality constrained nonlinear Kalman–Bucy smoother by interior point likelihood maximization. Automatica 2009; 45(1): 25–33.

24.

Simon

. Optimal state estimation: Kalman, H infinity, and nonlinear approaches. Hoboken, New Jersey: John Wiley & Sons, 2006.

25.

Grewal

Andrews

. Kalman Filtering: Theory and Practice Using Matlab. Hoboken, New Jersey: John Wiley & Sons, Inc., 2001.

26.

Liang

Xue

. A chinese small intelligent space robotic system for on-orbit servicing. In: 2006 IEEE/RSJ international conference on intelligent robots and systems, 2006, pp. 4602–4607. IEEE.

27.

Liang

. A modelling and simulation system of space robot for capturing non-cooperative target. Math Comput Model Dyn Syst 2009; 15(4): 371–393.

Kalman smoothing with soft inequality constraints for space robot teleoperation

Abstract

Keywords

Introduction

Problem statement

Offline smoothing with interior point techniques

Online smoothing with additional time delay and reduced-Q method

Experiment

Environment of experiment

Results of experiment and discussion

Conclusion

Footnotes

Declaration of conflicting interests

Funding

References