Model-mediated teleoperation with improved stability

Abstract

Model-mediated teleoperation has been developed to improve both transparency and stability in teleoperation. It uses local model of remote environment to provide non-delayed force feedback rather than using delayed force signals from slave side and thus is robust to arbitrary time delay. However, updating parameters in the local model may cause sudden force change during the operation. Meanwhile, the undesirable deep penetration or overlarge contact force may occur on the slave side due to the modeling error. Both of them will jeopardize the system stability. In this article, we propose a novel force-based model updating algorithm, which restrains the abrupt force caused by parameter updating. The update efficiency has been greatly improved by comparing with the existing solution; meanwhile, it ensures a stable human–machine interaction at the same time. Then, a new adaptive impedance controller that restricts both overlarge force and penetration is introduced. The obtained results on a one-degree of freedom contact experiment with a delay of 5 s demonstrate the superiority of proposed approaches in comparison with state-of-the-art methods.

Keywords

Teleoperation model-mediated teleoperation impedance control

Introduction

Robotic teleoperation system enables operators get a presence perception of interaction with remote environment where human presence is hazardous or costly. To this end, one critical step is the force feedback that gives the operator the perception of the force exerted by the slave manipulator.¹ However, the force feedback could be out of phase compared to operators command due to the delay in telecommunication system and it has been recognized for many years that even a small time delay in the control loop may jeopardize the stability and performance. Although the stability can be achieved by more conservative control strategy, for example, passivity-based schemes, the transparency will be significantly decreased.² Considerable effort has been made to deal with the trade-off between stability and transparency. The mainstream techniques adopted include four-channel architecture,^3,4 wave variable method,^5,6 and sliding mode control.^7,8 Despite these new improvements, there are still many tasks, which require a high degree of transparency under long time delay. But it is hardly to be realized by bidirectional teleoperation.⁹

Furthermore, while many state-of-the-art researches assume a constant time delay, the time-varying delay and packet loss problems are particular challenges for both stability and transparency.¹⁰ Therefore, alternative approaches that are less dependent on a high-performance communication network are becoming more and more desirable.

Model-mediated teleoperation (MMT) was proposed to improve both stability and transparency of teleoperation system.¹¹ The typical architecture of MMT is illustrated in Figure 1. On the master side, a model representing both geometric properties and contact dynamic properties of remote environment is built to be as accurate as possible. The user hand motion information is send to remote robot via a force-feedback device. Force feedback is provided based on the local virtual model of remote environment under negligible delay. On the slave side, the robot tracks the incoming commands while simultaneously collects sensor data (e.g. force, position, and image) for online estimating parameters of the model of the environment that it contacts. Rather than sending slave sensory data to the master side, estimated model parameters are transmitted back to master side under communication delay. Then, the parameters of the model on master side were updated according to the received parameters.

Figure 1.

Structure of the MMT. MMT: model-mediated teleoperation.

The steady state in MMT is defined as the state that the estimated model parameters converge and the model mismatch error between the estimated model and real environment is neglectable. A transition state in MMT refers to a period that a model mismatch between local model and actual environment is existing.¹²

Using estimated a model of remote environment, which include both geometric properties and contact dynamics, the operator could interact with local model rather than remote environment and thus could percept non-delayed force feedback. Theoretically, the teleoperation system can obtain ideal transparency in the presence of arbitrary time delay if the local model is identical with real remote environment. Different from conventional bilateral teleoperation algorithms, the MMT leads to the two subsystems and its stability condition is easy to meet. The study by Xu et al.¹² provides more details about MMT.

The application and extension of MMT have attracted much attention and its most problems are at least partly solved. MMT has been adopted in multi-operator multi-robot systems by the coordination of the master devices using one centralized variable position-based admittance controller.¹³ Based on the traditional position and force sensors, vision sensor is also used to strengthen the prediction ability of MMT systems.¹⁴ Interaction with moving object that is much more complicated than fixed object was realized, while the modeling of general moving object is still difficult.¹⁵ For better overall performance, other teleoperation techniques such as virtual fixture were actively incorporated in MMT systems.¹⁶ Some promising variants of MMT were also developed.^17,18

Two key components of MMT are the estimation and obtaining the model of the remote environment and designing the slave side controller.¹⁹ Multiple degrees of freedom mass-spring models and finite element models are highly accurate for simulation of deformable object. But they are computationally prohibitive for teleoperation system.²⁰ The main environment models used in robotics research are spring model, linear Kelvin–Voigt model, and nonlinear Hunt–Crossley model, which is the result of a balance between complexity and accuracy. All of them enable user to percept the virtual object’s mechanical impedance to some extent, which is necessary for the haptic rendering on master side.²¹

Meanwhile, it is usually difficult to model the remote environment accurately before operation tasks or even after. In order to obtain better approximation or keep pace with the changing environments, the parameters describing the local model are continuously estimated on slave side and transferred back to the master, which is referred to online parameter estimation. Different techniques in environment modeling and online parameter estimation had been extensively studied and compared.^22
–24

Then, the local model on master side is updated based on the estimated parameters, which is known as model updating. Special attention must be paid in this process as improper update schemes may lead to undesired motion and instability. When the data received from slave side trigger a sharp change of the model parameters, such as the stiffness and environment location, the force feedback may be changed accordingly. It is difficult for users to stabilize the master controller if the force changed suddenly. Thus, the abrupt changing force may disrupt the user’s operation and cause unsafety issue on slave side because the abnormal command also is transmitted to slave. The negative impact of the abrupt change of model parameter is referred to as “model jump effect.”²⁵ Different schemes were adopted to deal with this problem, but these approaches delayed the updating of model parameters or changed them in a limited rate in order to maintain the stability or passivity of the system.^26,27 There is an alternative approach, the user is informed by a small force impulse when model updating is needed. Then, the user must move the master end to a safe area to enable the update of the model parameters. This approach will interrupt the operation frequently.

The transition state of model updating can be defined as the period when the force rendered by the master device is different from the original one that is computed based on the received parameters directly. To provide force feedback accurately, the transition period should be as short as possible. However, in order to obtain a stable haptic rendering, a trade-off of between smooth force rendering and quick transition is preferred.

Another challenge of MMT is the slave controller. While tracking the delayed command from the master side, the command is often built on inaccurate information due to the error between local model and real environment. Two state-of-the-art control approaches for slave control in MMT are the switching position control/force control method and the relative tracking method.¹²

In switching position/force control approach, the slave is position controlled in free space and force controlled while in contact with the environment. Thus, the slave is controlled to maintain same contact force as applied on the master side. However, while the stiffness of the remote environment is smaller than that of the local model, the slave end-effector will penetrate into the environment more than that on the master side. The overlarge penetration may cause damages to the environment.

The relative tracking method proposed in the study by Winck and Okamura¹⁹ is a modified PD controller. Instead of tracking the absolute position on master side, the relative tracking method takes the mismatch position into account. Environment model information is also sent to the slave side. The aim of relative tracking is to make the slave to track the master position relative to the model and thus maintains the same penetration for the slave side and master side. Even though the relative tracking avoids the undesired penetration on the slave side, it is not capable to deal with the stiffness mismatch between the local model and remote environment. Once the stiffness mismatch exsits, the contact force is deviating and the overlarge force will jeopardize the operation safety.

In this article, we propose a simplified MMT system and address the stability problem of MMT by introducing a new model updating approach and an adaptive impedance controller. The proposed model updating approach limited the ‘abnormal’ force changing and ensured stable human–machine interaction on master side. The proposed controller is kind of a supervisory controller. In addition to position commands, the corresponding local model information is also sent over communication link to the slave side. The slave controller interprets the operator’s intention on the basis of corresponding master state. After the comparison between the expected model and the real environment, it carries out command from master in a conservative way to prevent from both overlarge force and penetration. The proposed impedance control approach enables the controller to adjust its behavior smoothly without switching among different states and maintains a compliant contact with the environment. Eventually, the proposed controller can limit overlarge force and penetration in the presence of both position uncertainty and stiffness uncertainty in the transition state. Meanwhile, accurate tracking of both position and force is achieved in the steady state, that is, the system tracking capability is not compromised evidently. Note that the accurate tracking of both position and force in steady state does not mean the controller is able to control the force and position simultaneously, it is the visualized description of the result that both force and position on slave side are following that of the master side.

Furthermore, we made some discussion on the limitation of existing position-dependent environment model²⁸ and proposed to use interpolation mechanism to estimate environment impedance in an unmeasured position prior to the contact happened. By incorporating the prior knowledge and available vision information, the method is expected to shorten the transition state in MMT and accelerate the parameter estimation.

The article is organized as follows. The environment modeling and identification are presented in section “Environment modeling and identification.” Section “Environment model updating algorithm” presents the proposed model updating strategy. The adaptive impedance controller is presented in section “Adaptive impedance controller.” Experimental setup and results are described in “Experiments” section. Section “Discussion and conclusions” provides a discussion about the extension of the environment model in MMT and concludes the article with a summary and outlook.

Environment modeling and identification

Environment modeling

We first have a quick review and discussion on several mainstream contact dynamic model in MMT. Geometry parameters of environment are out of the scope as they are normally not included in the environment model.¹² The identification of geometric profile can be seen in the literature.^14,29
–31

The spring model is the simplest contact force model. Though different models had shown better consistency with real environment, it is still one of the dominate approach in related research.^29,32,33 It denotes a proportion relationship between the penetration and contact force. The only parameter is the stiffness K

F_{e} = {\begin{array}{l} K δ x & δ x > 0 \\ 0 & δ x < 0 \end{array}

where δx represents the penetration depth into the environment and K denotes the environment stiffness.

Another common environment contact dynamic model for teleoperation is the Kelvin–Voigt model, which incorporates the dynamics of a linear damper-spring system

F_{e} = {\begin{array}{l} K δ x + B \dot{δ} x & δ x > 0 \\ 0 & δ x < 0 \end{array}

where B denotes the environment damping. While the inertia term may be included in popular second-order model, it is often neglected as the environment is usually stationary or quasi-static for most applications.

Nonlinear model has been shown to have better agreement with the real dynamic behavior of physical environment. The Hunt–Crossley model is the most popular nonlinear model for contact dynamics in teleoperation

F_{e} = {\begin{array}{l} K δ x^{n} + B δ x^{n} \dot{δ} x & δ x > 0 \\ 0 & δ x < 0 \end{array}

where n is a constant typically lies between 1 and 2. The complexity and accuracy of the above-mentioned three models increases in turn and the selection of modeling approach requires a appropriate trade-off in each context.

Although the Hunt–Crossley model is nonlinear, a single-stage method is proposed to linearized it by taking the natural logarithm of both sides of the model for δx > 0.³⁴ Consequently, the Hunt–Crossley model can be identified using common methods. But even so, the Hunt–Crossley model is still have difficulties for practical application. The applicability of estimation method is limited as its validity and consistency requirement, for example, low damped environment and low operation speed. More importantly, experiments in the studies by Haddadi and Hashtrudi-Zaad²⁴ and Achhammer et al.³⁵ show much slower converge speed comparing to Kelvin–Voigt model, which probably caused by additional exponential parameter n.

For the Kelvin–Voigt model, experiment results in the studies by Achhammer et al.³⁵ and Yamamoto et al.³⁶ show that the B hardly converges and it may fluctuate significantly while the K has good convergence. While the Kelvin–Voigt model obtains slight smaller force error at the cost of complicity comparing to spring model, the identified damping character is useless for most controllers. Moreover, as slight difference in haptic feedback signals is not perceivable for human operator, it is hard for the operator to find this minor improvement. The threshold of human perceptual discrimination for haptic signals is referred to as just noticeable difference (JND). As reported in the study by Hirche and Buss³⁷, the JND of force is approximately 10%. In contrast, the various adaptive robot controllers of usually accords with the environment stiffness and have acceptable accuracy. The simplicity of spring model also facilitates the estimation process and makes it a more practical choice. This partly explains the popularity of simple spring model that is also used in our experiments.

Model identification

A number of model identification method were investigated for estimating the model parameters.^23,38 The self-disturbing recursive least squares (SPRLS) is a common method used in recent studies as it can be immune to noise and track variable environment at the same time.^23,24,35 The update equations of SPRLS can be written as follows

\begin{array}{l} {\hat{θ}}_{k} = {\hat{θ}}_{k - 1} + K_{k} (y_{k} - ϕ_{k}^{T} {\hat{θ}}_{k - 1}) \\ K_{k} = P_{k - 1} ϕ_{k} {(1 + ϕ_{k}^{T} P_{k - 1} ϕ_{k})}^{- 1} \\ P_{k} = (I - K_{k} ϕ_{k}^{T}) P_{k - 1} + β N I N T (γ {\hat{e}}_{k - 1}^{2}) I \end{array}

where P_k is the covariance matrix at time instant; k, $\hat{θ}$ , and y are the vector of estimated dynamic parameters and system output, respectively; ϕ consisting of input variables; β and γ are designed constants that determine the sensitivity and gain; I is the identity matrix of the same size as the matrix P; and $\hat{e}$ is the estimation error calculated by $\hat{e} = y - \hat{y}$ . The NINT function is a round off operator

N I N T (γ {\hat{e}}_{k - 1}^{2}) = {\begin{array}{l} γ {\hat{e}}_{k - 1}^{2} & γ {\hat{e}}_{k - 1}^{2} \geq 0.5 \\ 0 & γ {\hat{e}}_{k - 1}^{2} < 0.5 \end{array}

When the error is lower than the maximum error bound determined by γ, the self-disturbing term is equal to zero and it is identical to the regular exponentially weighted recursive least squares algorithm that have good convergence character. Otherwise the P_k is increased according to the error and the sensitivity gain β, which means the latest data are endowed with larger weight in the estimation process. Thus, the SPRLS can be immune to noise and have great ability of tracking variable parameters as well. One limitation of SPRLS is that its performance heavily depends on appropriate value of the designed parameters γ and β. Although the SPRLS had been used in many studies, guide or discussion on the selection of design parameters is missing.

In practice, prediction error is usually caused by the mismatch between the real contact dynamics and the model characteristic instead of the force measurement noise. Such a mismatch causes inevitable error between the calculated force and measured force even with the optimal parameters. γ determines a minimum error level is to be considered as signal of changed environment dynamics. Thus, the threshold determined by γ is tuned to be slight larger than the maximum error caused by the mismatch and β is tuned according to the scale of parameters in our experiment.

Environment model updating algorithm

Stable force rendering on the master side is vital for MMT. However, the model updating during the operation task may induce unstable force rendering. For example, when the salve end just made contact with a object, which is stiffer than that in local model, the stiffness in local model should be updated as soon as the real stiffness is received so that it could reflect the real environment property correctly. However, the force feedback is rendered continuous and relies on the stiffness. The update of stiffness will cause larger force increment in short time and such discontinuity of the force feedback would hazard the system stability.

Parameter-based model updating

Gradual-update scheme has been adopted in MMT. With the gradual-update strategy, the model parameters were changed in a fixed change rate or in a fixed time period in some cases. The update law for position updating can be written as below while the stiffness updating is similar

x_{e, t + 1}^{m} = {\begin{array}{l} x_{e, t}^{m} + v_{e}^{m} Δ T & x_{e, t + 1} > x_{e, t}^{m} + v_{e}^{m} Δ T \\ x_{e, t + 1} & else \\ x_{e, t}^{m} - v_{e}^{m} Δ T & x_{e, t + 1} < x_{e, t}^{m} - v_{e}^{m} Δ T \end{array}

where x_e is the received environment position, $x_{e}^{m}$ is the environment position in local model, and $v_{e}^{m}$ is the change rate of $x_{e}^{m}$ . Though it can mitigate the model jump effectively, stability is not ensured theoretically. More importantly, a fixed maximum parameter change rate does not confine the bound of the force variation. In the contacting stage, the force feedback on master side is computed as

f_{m} = k_{m} (x_{m} - x_{e}^{m})

where f_m , x_m , and k_m represent the force feedback, position, and stiffness on master side, respectively. The force variation induced by parameter v_f can be describe as

v_{f} = {\dot{k}}_{m} \frac{\partial f_{m}}{\partial k} + {\dot{x}}_{e}^{m} \frac{\partial f_{m}}{\partial x_{e}^{m}} = v_{k}^{m} (x_{m} - x_{e}^{m}) - v_{e}^{m} k_{m}

where $v_{k}^{m}$ is the change rate of k_m . As shown in equation (8), the force variation rate caused by parameter change, which affects users’ experience and system stability directly, is also affected by current contact state. As a result, the fixed parameter change rates have to be low to avoid large force variation rate in some cases. Thus, this approach is not efficient enough.

There are two passivity-based model update strategies that ensure system passivity. The one in the study by Mitra et al.²⁶ was a basic approach that delayed the parameter changes until the update introduces no energy increment. The newer one proposed in the study by Xu et al.²⁷ used an adaptive virtual damper to dissipate the energy generate of the model parameters, and thus it was more efficient than the former one. For spring model in one-degree of freedom (1-DOF), the passivity condition for separate stiffness updating and position updating was given as

{\dot{k}}_{t + 1} = \frac{Δ k}{Δ T} \leq 2 (α + b_{d e} - \frac{k_{t} Δ T}{2}) \cdot {(\frac{{\dot{x}}_{t}}{x_{t + 1}})}^{2}

\begin{array}{l} \frac{2 Δ T {\dot{x}}_{t}}{k_{t}} [k_{t} (x_{m, t} - x_{e, t}^{m} + (α + b_{d e}) {\dot{x}}_{t}] + {(x_{m, t} - x_{e, t}^{m})}^{2} \\ \geq (x_{m, t + 1} - x_{e, t}^{m} - Δ x_{e, t + 1}^{m}) \end{array}

where $Δ x_{e, t + 1}^{m} = x_{e, t + 1}^{m} - x_{e, t}^{m}$ , b_de is the device damping, α is the given adaptive damping, k denotes the stiffness of local model, and the subscript of m is omitted for clarity. In practice, the adaptive damping often needs to be large or even infinite. To avoid a damping value exceeding the capability of device, α was set with a upper bound.

While the passivity-based methods guarantee the system passivity, the impedance of human arm has not been considered. Human arm contributes to system stability by adjusting itself. Small energy generation is most likely to be dissipated by human arm and the whole system including the human operator is still stable. Therefore, a update strategy that strictly complies with the passivity condition is too conservative in application. However, this is still an open issue as it is so difficult to take human arm damping into design of passivity-based methods (the damping property of human arm is too complicated to be modeled). Another important defect of passivity-based methods is that it cannot deal with the condition that the system energy is decreasing. These weaknesses of gradual-update scheme and passivity-based methods motivate us to find a better approach and the proposed force-based model updating method is introduced in next section.

Force-based model updating

To the best of our knowledge, all existing approaches aiming to solve the effect of model jump, control the variation of the force feedback in an indirect way, that is, they control the change of model parameters, and thus limit the abrupt change of force. However, we do not have to control the model parameter in the transition state, as the output that user can perceived the force feedback is the force rendered by the force-feedback device. In other words, the key novel idea is that we can deal with the model jump effect by directly controlling the rendered force instead of updating the parameters step by step. In the model updating process, the goal is to make the force feedback from the master device to be identical with that obtained the computation result with latest parameters as soon as possible. The only constraint is to avoid abrupt force change and make it easy for human to handle. Both the goal and constraint are about the force rather than parameters. Thus, implementing model updating in the level of force instead of model parameters is a more reasonable solution, which can be referred as force-based scheme. By modifying the force instead of model parameters on which the force is based, we are able to avoid the detour and redundancy and the model updating could be much more efficient.

The difference of parameter-based model updating and force-based model updating is shown in Figure 2.

Figure 2.

Parameter-based and force-based model update schemes.

For now, the challenge becomes how to develop such a model updating algorithm at the level of force rather than model parameters. An intuitional approach is to limit the change rate of the force feedback to avoid the strike of “model jump effect.” We can constrain the force change rate to a safe range with smaller value than a threshold to ensure stable interaction. However, according to our experiment, a smaller force change rate can bring significant negative effect on user’s operation while a larger force change rate does not. An intuitive example can make it more clear. It is easy for the operator to decrease the contact force toward a virtual wall from 5 N to 1 N in approximately 0.2 s with a neglectable movement. But if the user is holding still with a force at 5 N, and then let the force decreases to 1 N in 0.2 s without any warning in advance, the user’s hand will move forward suddenly with a feeling of “missed step.” Note that the force change rate in the two cases is the same. We can conclude that an appropriate force change rate depends on the specific conditions and thus is difficult to set. Instead of force change rate, we need a new criterion that helps us define a safe range for stable interaction.

Once human is involved on the master side, whenever the user executes a motion or hold still, a force is expect roughly and unconsciously due to human’s physical intuition. It is assumed that when a movement is executed, both remaining unchanged or changing according the displacement and the stiffness of the environment are acceptable and easy to handle for users. Meanwhile, the force should not be larger or smaller than these two. Such an assumption has been validated through experiments and further explanation can be found in last section. The range between these two values will be used as a safe range in the design of force-based methods.

The aim of model updating is to make the force rendered by the force-feedback device f_m equal to the predicted force f_p , which calculated by the latest model parameters. Thus, it is preferred that f_m changes in the direction that reduce the error between f_m and f_p , the opposite direction is undesired. Thus, a value within the aforementioned range can be set as the expected force f_ex . So the force f_p calculated with the latest parameters transmitted from the slave side can be divided into two parts, the expected force f_ex and the abrupt force f_ab . By limiting f_ab , the force feedback is close to the range that is easily to be handled by operators and thus the stable interaction is ensured.

Based on the foregoing discussion, the update scheme of the force feedback f_m on master side is defined as follows. First, the predicted force f_p is calculated

f_{p, t} = {\begin{array}{l} k_{e, t}^{m} (x_{m, t}^{m} - x_{e, t}^{m}) & x_{m, t}^{m} - x_{e, t}^{m} > 0 \\ 0 & x_{m, t}^{m} - x_{e, t}^{m} \leq 0 \end{array}

Then, the maximum and minimum expected force $f_{e x}^{max}$ and $f_{e x}^{min}$ are defined based on the f_m , $k_{e}^{m}$ , and the displacement in the last sampling period

f_{e x}^{min} = min (f_{m, t - 1}, f_{m, t - 1} + δ x_{t} k_{e, t - 1}^{m})

f_{e x}^{max} = max (f_{m, t - 1}, f_{m, t - 1} + δ x_{t} k_{e, t - 1}^{m})

Then, the selected expected force is obtained as

f_{e x, t} = {\begin{array}{l} f_{e x}^{max} & f_{p, t} > f_{e x}^{max} \\ f_{p, t} & f_{e x}^{min} \leq & f_{p, t} \leq f_{e x}^{max} \\ f_{e x}^{min} & f_{p, t} < f_{e x}^{min} \end{array}

The force to be rendered by the force-feedback device can be obtained

f_{m, t} = f_{e x, t} + SAT (f_{a b, t})

where rendered force f_ab,t = f_p,t − f_ex,t and SAT() is a saturation function defined as

SAT (f_{a b}) = {\begin{array}{l} f_{m}^{c} & f_{a b} > f_{m}^{c} \\ f_{a b} & - f_{m}^{c} < f_{a b} < f_{m}^{c} \\ - f_{m}^{c} & f_{a b} < - f_{m}^{c} \end{array}

where $f_{m}^{c}$ is the value of allowed abrupt force in a sampling period.

The rendered force can be divided into two parts: one is the reasonable force expected by the user f_ex and the other is the abnormal part. Note that in the steady state where no model updating is needed, f_ex is exactly equal to f_p and thus we obtain f_m = f_p , which means the force render is accurate. In the transition state when f_m is not equal to f_p , the f_m is close to f_ex . Stable interaction is ensured since f_ex is an expected force that can be handled effortlessly. Moreover, via the selection of f_ex , the f_m is approaching to the f_p as soon as possible. Once the f_m is equal to f_p , the modeling updating is finished and normal force rendering is also achieved with the given force updating scheme.

Adaptive impedance controller

Slave controller in MMT

Transition states occur irregularly during teleoperation due to inaccurate environment estimation or environment changes. In this case, the command made on master side is based on inaccurate state information, which may result in undesirable and risky deep penetration or large contact force on slave side. The main challenge of slave controller in MMT is to carry out the master command under an uncertain or unknown environment meanwhile maintains the stability of the teleoperation system.

Figure 3 shows model mismatch issue during the transition state. Both position mismatch and stiffness mismatch may cause overlarge contact force using conventional position control approach. Fortunately, the model mismatch problem is only partly solved, either by switching position/force control or by relative tracking method.

Figure 3.

Model mismatch in MMT. MMT: model-mediated teleoperation

Switching position/force control is the most frequently used slave controller in MMT. The slave switches to force control mode when the measured contact force exceeds a preset threshold. In the force control mode, the contact force is effectively controlled and thus the overlarge force is avoided in both cases of position mismatch and stiffness mismatch. However, if the stiffness of remote environment is smaller than that of the local model, in order to apply the same contact force, the slave will penetrate to the contact object large than the master side even though the larger penetration is probably not the user’s intention, that is,. once the stiffness mismatch exist, potential overlarge penetration may occurred and damage the contact object.

Relative tracking had the same framework of PD control but modified it based on the relative positions and velocities. The original PD controller for position tracking is

F_{I m} = k_{p s} (x_{m} - x_{s}) + k_{d s} (v_{m} - v_{s})

where F_Im is the force applied to the slave and k_ps and k_ds are the PD gains. Relative tracking method modified it to

\begin{array}{l} F_{I m} = k_{p s} ((x_{m} - x_{e}^{m}) - (x_{s} - x_{e})) \\ + k_{d s} ((v_{m} - v_{e}^{m}) - (v_{s} - v_{e})) \end{array}

where $x_{e}^{m}$ and x_e are the position of local model and the real environment of master and slave, respectively. $v_{e}^{m}$ and v_e denote the velocities.

With such a modification, relative tracking method greatly improved the position tracking performance in the case of position mismatch. However, it cannot solve the stiffness mismatch issue. Considering the stiffness mismatch exists, for example, when the slave is to contact an object, which is stiffer than what supposed on master side. There would be no position mismatch, for example, $x_{e}^{m} = x_{e}$ and $v_{e}^{m} = v_{e}$ . Then, the relative tracking degenerates to standard PD control that tracks the position on master side. In the transition state, the position controlled slave follows the master position and the resulting contact force is larger than it on the master side due to larger stiffness. Such a larger force may also damage the object.

Adaptive impedance controller

As discussed earlier, either one of the two slave controller cannot address the potential stability issues in transition state. A possible solution suggest in the study by Xu et al.¹² is using a hybrid control scheme and the slave do not execute any motion commands whenever the contact force or the penetration has reached the values as them on the master side. However, it has not been verified that the frequently interruption and switching operation would degrade of execution efficiency significantly.

Inspired by the wide used impedance controller in compliant control and its advantages, we figure out an adaptive impedance controller for the slave control in MMT. The original idea of impedance control is to control force and position simultaneously and it indirectly controls a dynamic relationship between these two, that is, using reference inputs of both desired force and position is allowed. But the application of impedance controller usually involves only one input, while the other is neglected or adjusted adaptively.^39,40 One possible reason is using two reference input requires the command conforming to environment dynamics, which is not possible in most cases.

However, the local model in MMT enables the operator to give both force and position input at the same time. More importantly, they are reproducible on slave side and useful for slave controller to understand the operator’s intention. In the steady state, the rendered force on master side is calculated by the accurate estimating of remote environment dynamics. This means both the rendered force and its corresponding position conform to real environment and can be reproduced on slave side simultaneously. Although minor error is inevitable due to the deviation of local model, the residual error is no more than the modeling error. As far as we know, this is the first work that leverages the inherent interaction between impedance controller and MMT. In comparison with switching position/force control, our approach can realize compliant contact while avoiding the chattering problem. In addition, the adaptive impedance controller’s behavior is more similar to the human arm that is identified as an impedance module in some teleoperation frameworks.^9,41

Consider the low inertia and frictionless nature of common haptic devices on master side, for example, Geomagic Touch, together with the limited velocity and acceleration in teleoperation, the rendered force output by haptic device has very little difference with the operator applied force. Therefore, it is feasible to use the rendered force on master side as the desired force on slave side.

The overview of proposed controller is shown in Figure 4. In the proposed controller, both master position and rendered force are transmitted to slave side as the reference inputs of the slave impedance controller. The design parameters in controller are adaptively adjusted.

Figure 4.

Structure of the controller.

For the impedance controller, the desired impedance model (in the case of 1-DOF) is specified as

M_{d} \ddot{x} + B_{d} (\dot{x} - {\dot{x}}_{d}) + K_{d} (x - x_{d}) = e_{f}

\ddot{x} = M_{d}^{- 1} [e_{f} - B_{d} (\dot{x} - {\dot{x}}_{d}) - K_{d} (x - x_{d})]

where M_d , B_d , and K_d are the desired inertia, damping, and stiffness, respectively; e_f = f_e − f_d ; and x_d is the desired position, f_d is the desired force, x and f_e are the actual position and actual force; $\dot{x}$ and $\ddot{x}$ denote the actual velocity and acceleration.

When the slave is moving in free space, both f_e and f_d are zero, and the impedance controller is equivalent to a position controller. With a fixed high desired stiffness K_f , that is, high proportional gain, accurate position tracking could be achieved.

In the case that the slave is contacting with environment, we first assumed a static environment for the simplicity. It is also not considered the direction motion tangent to the environment surface, which means no sliding. The method can be easily extended to these cases, with a modification of parameter when estimating the stiffness and penetration at the same time.⁴² Let x_s and x_m denote the penetration on slave and master sides, respectively. For an environment represented by spring model, f_e can be approximate by

f_{e} = K_{s} x_{s}

where K_s denotes the stiffness of environment. With SPRLS and spring model, K_s can be obtained shortly after the contact occurred and as the estimation of K_s is on slave side and thus is immune to communication delay. f_d is the rendered force on master side

f_{d} = K_{m} x_{m}

At the equilibrium point, we obtain

K_{d} (x_{m} - x_{s}) = f_{e} - f_{d}

Plugging equations (21) and (22) to equation (23), we obtain

x_{s} = \frac{K_{m} + K_{d}}{K_{s} + K_{d}} x_{m}

For both position mismatch and stiffness mismatch, we expect the following relative bounded of f_e and x_s for safe operation

\frac{f_{e}}{f_{d}} \leq α_{F}

\frac{x_{s}}{x_{m}} \leq α_{P}

where α_F and α_P are constant larger than 1, respectively, denoting the acceptable ratios of actual contact force to desired force and actual penetration to desired penetration.

For the restriction of force, using equations (21) to (23) and (25), the following condition can be obtained

{\begin{array}{l} K_{d} \leq \frac{(α_{F} - 1) K_{s} K_{m}}{K_{s} - α_{F} K_{m}} & K_{s} \geq α_{F} K_{m} \\ K_{d} \geq \frac{(α_{F} - 1) K_{s} K_{m}}{K_{s} - α_{F} K_{m}} & K_{s} < α_{F} K_{m} \end{array}

Similarly, for the restriction of penetration, the following condition can be obtained

K_{d} \geq \frac{K_{m} - α_{P} K_{s}}{α_{P} - 1}

Note that ∀ K_d > 0 meet equation (27) when K_s < α_FK_m and ∀K_d > 0 meet equation (28) when K_s > K_m /α_P . In addition to restriction of overlarge force and penetration, the impedance controller’s performance can be improved by adjusting the parameters based on environment stiffness.⁴³ To this end, defines Kd = F(K_s ) where F(K_s ) is set toward best controller’s performance without the consideration of potential overlarge force or penetration issue. Generally, the rule of K_d in order to meet equations (27) and (28) can be determined as following

K_{d} = {\begin{array}{l} max (F (K_{s}), \frac{(α_{F} - 1) K_{s} K_{m}}{K_{s} - α_{F} K_{m}}) & K_{s} \leq K_{m} / α_{P} \\ F (K_{s}) & else \\ min (F (K_{s}), \frac{K_{m} - α_{P} K_{s}}{α_{P} - 1}) & K_{s} \geq α_{F} K_{m} \end{array}

With regard to an intuitional explanation, larger K_d can be seen as more weight on position tracking. Emphasis on position tracking is a conservative approach when the environment is softer than estimated (as an extreme case, the master is contacting while the slave is not). Similarly, smaller K_d means force tracking is preferred when the environment is stiffer than estimated (such as the slave end is contacting while the master end is not).

Once the K_d is determined, M_d and B_d can be selected on the basis of both K_d and K_s to shape the closed-loop transient behavior. The damping ratio ξ_d is given by

ξ_{d} = \frac{B_{d}}{2 \sqrt{M_{d} (K_{d} + K_{s})}}

To overcome the contact instability problem, critical damping or over-damped is preferred.

The modeling error can be denoted as e_m = K_mx_m − K_sx_m . Then, eqaution (23) can be transformed as following by substituting f_e = K_sx_s and f_d = K_sx_m + e_m into it

K_{d} (x_{m} - x_{s}) = K_{s} x_{s} - (K_{s} x_{m} + e_{m})

Consequently, we obtain the only equilibrium point

x_{s} - x_{m} = \frac{e_{m}}{K_{s} + K_{d}}

Since |e_m | is lower bounded, it can be easily obtained that |x_s − x_m | is upper bounded as long as K_s + K_d is lower bounded. K_d is set to a high value when K_s is small and |e_m | is relatively small (usually less than 0.1f_e ). It is safe to assume that the tracking error |x_s − x_m | is acceptable for most teleoperation tasks. It is worth noting that the f_d and x_d are the input of the impedance controller, the tracking ability of system does not depend on the adaptive parameter K_d . In fact, when the modeling error is neglectable, x_s will converge to x_d no matter how K_d changes by the estimated environment stiffness.

Note that the position tracking error are based on the equilibrium point of contact, the penetration or contact force may exceed the bounds due to abrupt change of environment, modeling error, dynamic error, and so on. However, as shown in the “Experiments” section, the overall restriction is effective.

Experiments

Comparison of model updating

Stiffness updating only

A simple experiment setup is used for comparison of model updating approach. Operator contacts with a virtual floor with Geomagic Touch as the force-feedback device on master side and only vertical motion is considered. A spring model is used as local model and thus the force feedback is computed as equation (1), while δx is derived by the endpoint position of Geomagic Touch and the position of virtual floor. The stiffness and position of virtual floor was updated regularly. The stiffness of virtual floor was set as 1100 N/m initially and switches between 100 N/m and 1100 N/m every 10 s. The operator was told to make contact with the virtual floor in a static or fluctuate manner. No model updating scheme is used to serve as a benchmark. As can be noted in Figure 5, the operator was unable to keep the handle still when the stiffness changes. The operator pushed the handle deeper accidentally (seeing position increase in the figure) when the stiffness decreases abruptly at time t = 10 s. The handle was pushed up (seeing position decrease in the figure) accidentally and had an overshoot when the force rendered on device increases suddenly at time t = 20 s. The maximum value of most force rendered is below 4 N. Even so, the abrupt change of model parameters imposed a significant negative effect to the normal operation.

Figure 5.

Force and position without model updating.

Three model updating methods were tested in the same condition. The stiffness change rate in gradual-update scheme and $f_{m}^{c}$ in force-based method were set to 1000 N/m·s and 0.002 N, respectively, by manual tuning so that they can just be handled the operator. In passivity-based method,²⁷ b_de was set to zero as the device damping is small and unfixed when α was 5 N·s/m.

Figure 6 demonstrates the results of passivity-based method, the force changed very smooth at time t = 20 s. However, at time t = 10 s, the force diminished suddenly and caused the operator push down the handle in about 0.1 s accidentally because the decreased stiffness lead to decreased energy and the algorithm does nothing through the variation. This indicates that passivity-based methods should be used combining with other methods to deal with the system energy decrease. Figure 7 shows the change of stiffness during the operation. The stiffness changes from 1100 N/m to 100 N/m at time t = 10 s directly and varies from 100 N/m to 1100 N/m adaptively according to the state of the contact. When the penetration decreases to smaller value after time t = 25 s, the bigger stiffness changes rate is allowed and the update process still takes more than 6 s. The result of gradual-update scheme is shown in Figure 8, the force f_m changed smooth at time t = 10 s and t = 20 s. The master handle is pushed down gently at time t = 10 s due to the decrease of force and then be handled quickly as the operator noticed the decrease of force. No position overshoot happened when increased force increase at time t = 20 s.

Figure 6.

Force and position with passivity-based model updating.

Figure 7.

Stiffness change in passivity-based model updating.

Figure 8.

Force and position with gradual-update scheme.

The result of force-based updating is shown in Figure 9, which is similar to gradual-update method and the position fluctuation is even smaller. The negative effect of model jump is effectively avoided.

Figure 9.

Force and position with force-based model updating.

Both position and stiffness updating

To evaluate the force-based method thoroughly, another experiment with simultaneous position and stiffness updating is conducted. Passivity-based method is not compared as no existed solution with both location and position updating there is. The stiffness switches between 1100 N/m and 100 N/m every 10 s, the position of virtual floor switches between 0 mm and 20 mm every 5 s. The initial value for stiffness and position is 1100 N/m and 20 mm, respectively. The force and position profiles are shown in Figures 10 and 11. As can be noted in the enlarged view, after the parameter updating at time t = 5 s, f_m changed toward to f_p directly in force-based method, while it fluctuated in graduate-update method. Note that in the transition state started from t = 5 s, f_p is always bigger than f_m which means the fluctuation of f_m is unnecessary and inefficient. This is also how the force-based method be more efficient than the parameter-based methods. As a result, the force-based method takes only 2.52 s for model updating while the gradual-update scheme takes 5 s.

Figure 10.

Force and position with gradual-update scheme, while both location and stiffness are updated.

Figure 11.

Force and position with force-based model updating, while both location and stiffness are updated.

The abrupt force during the operation of two methods is compared and shown in Figures 12 and 13. The maximum and average values of f_ab for gradual-updating scheme are 0.062 N and 0.009 N, respectively. In contrast, the maximum and average values of f_ab for force-based methods are both only 0.002 N. Smaller f_ab indicates that force-based method may provides more comfortable interaction.

Figure 12.

Abrupt force profile with gradual-update scheme.

Figure 13.

Abrupt force profile with force-based model updating.

The results of gradual-update scheme and force-based approach are summarized in Table 1. It shows clearly that the proposed approach provides a much better model updating performance with high efficiency and smooth user experience.

Table 1.

Comparison of model updating algorithms.

Algorithm	Time used	f_ab (N)
Algorithm	(s)	Max	Mean
Gradual-update	5.0	0.062	0.009
Force-based	2.52	0.002	0.002

Comparison of slave controller

Setups

To evaluate the proposed controller, we used a MMT system as shown in Figure 14 with a Geomagic Touch as the master haptic device and a 3-DOF translation parallel manipulator as the slave. An ATI mini 40 force sensor (ATI Industrial Automation, Inc., NC, USA) was mounted at the end-effector of the slave robot to measure the contact force. Currently, for simple, only 1-DOF motion in vertical direction was allowed.

Figure 14.

Setup of the teleoperation system.

For online parameter estimation, spring model was used as the local model and SPRLS is utilized. The online estimation process was implemented at the slave side by the force sensor data. Specifically, the estimation result is updated by equations (4) and (5) at position measurement frequency 170 Hz. The threshold for detection of contact was set as 0.05 N. The place where a contact happened was considered as the position of environment, therefore δx can be easily obtained. The estimated environment stiffness and position were sent back to the master side and the force feedback is computed using equation. (1). It should be noted that the environment position and stiffness used for force computation came from the slave side under time delay. When accurate position and stiffness is obtained, a non-delayed accurate force feedback will be provided to users. One practical trick used is that, when the slave reaches the environment according to the environment and no contact is detected, the slaves current position is sent back to replace the location of environment. In this way, the known information that the environment location is lower than the current location is utilized, and the modeling error is minimized.

For the proposed controller, position-based impedance control is used to carry out the command of impedance model. M_d is set to 2 Kg and B_d is set according to equation (30), while ξ_d is set to 1 for better transient behavior. F(K_s ) is defined as

F (K_{s}) = {\begin{array}{l} λ {(\frac{ψ - K_{s}}{ψ})}^{2} & K_{s} \leq ψ \\ 0 & K_{s} > ψ \end{array}

where ψ and λ are constant setting before the operation tasks. Specifically, ψ, λ, and K_f were set to 8000, 5000, and 5000 N/m, respectively. Both α_P and α_F set to 1.3, which means 30% larger force and penetration are expected. The other parameter settings are invariant for all the following experiments except the time delay. To validate the robustness of the proposed approach, a delay of 2 s was given in the comparison experiments and 5 s time delay was used in the continuous contact experiments.

As shown in Figure 15, different contact materials (environments) were used for the experiment. They included a rubber pad, a foam, and a metal cover board. The stiffnesses of the three materials are approximately 400, 6000, and 19 000 N/m, respectively.

Figure 15.

Contact materials in experiments.

The proposed controller was first validated in comparison with the switching position/force control method¹² and the relative tracking method.¹⁹ Then, it was evaluated in a continuous contact experiment with changing environment (both stiffness error and position error exist). For the convenience of observation, all curves plotted in the figure are translated by 2 s along the time axis to counteract the forward time delay.

Potential of existing approaches

The experiments were designed for detecting the potential of existing control approaches. Both forward and backward communication time delay were set to be 2 s and the feedback from the slave side was blocked to maintain a transition state. The initial stiffness in local model was set as 2000 N/m in every trial. Comparing with the environment model on master side, foam is a lower stiffness environment, while rubber pad and metal cover board have higher stiffness.

In the first set of experiments, the slave was in contact with a rubber pad using relative tracking, which is a harder environment than expected. As shown in Figure 16, while the position tracking is accurate, the slave contact force is much larger than master due to unexpected larger stiffness. The overlarge force is unpredictable for operator and may damage the remote environment and the stability of the system. The results using the proposed controller in the same condition are shown in Figure 17. It is verified that the controller effectively restricted the unexpected over large force when it encountered an unexpected high environment stiffness.

Figure 16.

Force and position with relative tracking (hard environment).

Figure 17.

Force and position with adaptive impedance controller (hard environment).

Another experiment is to contact with foam, which is a softer environment than expected, using switching position/force controller. The results are shown in Figure 18. The force controller drives the slave to a position higher than master obviously which will cause large and dangerous penetration in order to obtain desired contact force. In contrast, as shown in Figure 19, the overlarge penetration may be avoided using our impedance controller. The comparison results are summarized in Table 2. As could be expected, during the transition state when the local model on master side exists error relative to the real environment on slave side, the proposed impedance controller can ensure task be carrying out by a conservative way. The slave contact force and penetration were restricted to a safe range when the stiffness does not match, which is impossible for other approaches.

Figure 18.

Force and position with switching position/force controller (soft environment).

Figure 19.

Force and position with an adaptive impedance controller (soft environment).

Table 2.

Comparison of slave controller.

Algorithm	Force overshoot	Position overshoot
Adaptive impedance	0.7 N	0.4 mm
Force/position	N/A	5.7 mm
Relative tracking	9.5 N	N/A

Continuous contact experiment

To validate the proposed controller thoroughly, we conducted the experiments on changing environment in which both position and stiffness of the object are varying and unknown by the operator in advance. The initial stiffness in local model is still 2000 N/m. Different from former experiment sets, the master received the estimated parameters from slave side and updated the local model for accurate approximation. Thus, the operation will entry to the steady state from transition state during the operation. To mediate the model jump effect while updating parameters, the force-based model updating is utilized and the $f_{m}^{c}$ is set to 0.002 N.

The operator was allowed to contact with different environment. The order of contact material is rubber pad, foam, double rubber pads, and metal cover board. The full process of the experiment can be seen in the Online Supplementary Video. The result of online parameter estimation is shown in Figure 20. The initial stiffness value was 2000 N/m and converges to corresponding value once the slave is in contacting with a new environment. Four steady stages at 7000, 400, 5000, and 19,000 N/m represent four different material stiffness, respectively. The converge of stiffness estimating takes only a few seconds and no undesired overshoot happened. Besides, the estimation algorithm can distinguish the difference between the rubber pad and double rubber pad apparently, which further suggests the accuracy of the parameter estimation algorithm. Overall, the online stiffness estimation employed in the system worked effectively. On the other hand, the quick variation of stiffness at such a magnitude challenges the model updating on the master side. It will be shown soon that the proposed model updating algorithm’s performance is satisfactory as expected.

Figure 20.

Result of online parameter estimation.

The position tracking and force-feedback results are shown in Figure 21 and the enlarged views of four stages are shown in Figures 22 to 25.

Figure 21.

Continuous experiment of adaptive controller.

Figure 22.

Continuous experiment of adaptive controller: contact stage with rubber pad.

Figure 23.

Continuous experiment of adaptive controller: contact stage with foam.

Figure 24.

Continuous experiment of adaptive controller: contact stage with double rubber pads.

Figure 25.

Continuous experiment of adaptive controller: contact stage with metal cover board.

Since position and stiffness of all four different environments are unknown to user, there are position error and stiffness error at the initial phase of the each contact. As a result, both accurate position tracking and force feedback may not be achievable and proper action must be done to avoid overlarge force or penetration. For example, as shown in Figure 22, the slave tracking the master before time t = 8 s. Because the position of environment model on master side is lower than the real one, the master kept moving downward without force feedback. While the slave was in contact with the environment but the master did not, the impedance parameters of the controller were adjusted in order to avoid overlarge contact force so the slave did not go on tracking the master. Meanwhile, the new environment position was transmitted back to the master. After the command based on corrected environment model is sent to the slave, both position tracking on slave side and force prediction on master side are achieved. The impedance control scheme adopted in the controller provides a high degree of robustness to robot unmodeled dynamics and the unknown environments. As a result, which also demonstrated in the plots, the controller not only effectively avoided overlarge contact force and penetration but also guaranteed the smooth trajectories.

It is worth noting that the master position changed smoothly when the model parameter abruptly at time t = 10, 55, 112, and 175 s. The operator also can feel smooth force changing during the whole experiments, which validated the effectiveness of the model updating strategy.

As shown in Figure 25, when the position of local model on master side is higher than the real one, the operator feels like the slave is in contact but it is actually not, then the controller will drive the slave to slightly surpass the position of master. It can help the slave “found” that the real environment position is farther than expected. As a result, the slave is able to keep moving until it makes contact with the environment and the environment position in local model becomes correct.

The point is that both overlarge force and penetration are effectively avoided under unknown environment and large time delay, which is impossible for conventional bilateral teleoperation approaches. The residual tracking error comes from the dynamic error and modeling error. The tracking error, when contacting with metal cover board, is minimum in the four cases due to high stiffness and small modeling error.

Discussion and conclusions

In this section, we offer some discussion about the proposed methods and position-dependent environment dynamics model, which is necessary for the application of MMT at a higher level.

Discussion on proposed methods

For the force-based model updating algorithm, it is believed to be more effective than existing parameter-based approaches. Actually, the goal of the parameter updating in MMT is to maintain the stable human–machine interaction and make the rendered force equal to the force calculated via latest model parameters as soon as possible rather than update the model parameters step by step. While updating the parameters step by step is surely feasible and useful, which also have been used wildly, the force-based method can make full use of the feasible space to accomplish its task. Thus, model force updating is a more reasonable and more effective solution in comparison with the parameter-based methods. Note that the same allowed abrupt force $f_{m}^{c}$ may mean different parameter updating rate under different conditions. For example, when the penetration is small even lager stiffness variation causes limited force change, thus the model updating will be finished quickly. Meanwhile, small stiffness change may cause large force change and the model updating will be finished slowly. In other words, while $f_{m}^{c}$ is fixed to ensure stable interaction, the update rate is adaptive.

For the slave controller, the primary limitation is that either position or force cannot be controlled precisely. It is also worth to point out that, for the proposed approach, obtaining ideal tracking in steady state requires environment can be represented by spring model. As shown in Figures 22 and 25, the strong nonlinear dynamics of foam lead to apparent larger tracking error. Modification toward nonlinear contact model needs to be investigated, thought it may lead to a complicated boundary condition. Besides, as can be noticed in Figure 22, the slave position tends to be lower than the master side. The reason of this trend possibly is that the slave is designed to be controlled by a position controller based on a simple PID controller without gravity compensation.

Another possible concern is the stiffness of the robot. It is well-known that stiffness is important to any robotic system’s performance.⁴⁴ In a MMT system with the proposed controller, the geometry change of the robot means that the estimated environment stiffness is coupled with the stiffness of the environment and slave manipulator, so the modeling accuracy may degrades. In addition, the stiffness of end-effector force sensors also contributes to the error. Fortunately, even though the stiffness properties of the slave manipulator cannot be neglected. We can easily found that since the environment is modeled to include the stiffness of both environment and slave manipulator, the force tracking does not degrade like the position tracking, which is comforting for most force-feedback teleoperation tasks.

The stiffness analysis of a robotic system is a general issue in robot control rather than a teleopration problem. However, a complete analysis that takes the slave manipulator’s stiffness performance into consideration should be done in future work due to their importance for a successful robotic systems.

Position-dependent environment dynamics model

Different from typical simplified task in the experiment that has only one contact point, in real complex tasks, environment model must be position-dependent so as to match the position-dependent nature of environment impedance. Position-dependent model was proposed and has shown its advantages comparing with time-based approach.²⁸ The workspace is quantized into discrete components and each of them represents a small area in the slave robot’s workspace. The estimated parameters are stored in the data node corresponding to the current tip position of the robot. Thus, prestored estimated parameters can provide prior knowledge when the robot back to the same position to shorten transition state.

However, it is still an open issue. In current position-dependent model, only parameters in measured position are updated while others remain the initial value. Considering the large amount of data nodes, only a few part of information in the model can be updated using the direct measurement. The model works only when the new contact point is the exactly one of the previous contact points. While the operator on the master side may be able to estimate the impedance in an unmeasured position empirically and manual adjustment by operator leads to significant decrease in operation efficiency. Meanwhile, making use of more available information to estimate the parameters before the real contact happened plays an important role as it determines the difference between local model and remote environment at current position. If the estimation is accurate, there will be no transition state in current contact point. In addition, unnecessary dramatic changes between updated points and the others will be introduced if the estimated parameters differ greatly from the initial value.

To solve this problem, a mechanism to estimate the environment model parameters in unmeasured position is needed. Meanwhile, its computation complexity must be limited so that it can keep compatible with high refresh rate.

Considering the physical characteristic of common environment surface, the parameters are continuous in most areas even in the presence of dramatic changes at the boundaries, which can be either visible or invisible. The environment impedance in one position is usually similar or even identical to the impedance in near position when the material in these positions is same. In addition, teleoperation system usually has a camera and the vision information is valuable for estimating parameters of contact dynamics model because the images show which part surface of environment are in the same appearances. Same appearances mean possible same material and thus similar dynamic characteristic. The perceivable boundary may mean different object or material surface, which has different or unrelated impedance characteristics. Note that estimating environment impedance by vision and measured sample is in accord with human common sense. It is reasonable to assume that the near area has the similar impedance.

In order to estimating environment parameters before contact, we propose a method that consists by two steps. First, get the image of the environment surface and extract the contour. Then divides the position-dependent model into many regions using the extracted contour. For the accuracy, the division can be done by operator manually. The expense on this process is acceptable because it is only a one-time effort. Second, every time a new estimated result is given, updating the data node corresponding to the current position. Then using nearest neighbor interpolation to obtain the value of all unmeasured data nodes.

Although the interpolation does not guarantee the accuracy as there is no any direct information, it is difficult to prove its effectiveness theoretically as we cannot obtain mathematical formulation of the unknown real environment. It is believed that this method is effective in most cases based on the discussion before. An improved estimation of environment impedance not only shortens the transition state but also accelerates the parameter estimation process and thus improves the impedance controller’s performance.²⁸ In addition, the improved model can be used to display a continuous impedance map of the environment to give the operator a visible description of environment dynamics. This is particular valuable for palpation or damage detection tasks. A similar application was investigated in the study by Yamamoto et al.²²

Conclusions

As an alternative to bilateral force-feedback teleoperation, MMT is developed to enable efficient operation especially under large time delay. In this article, we put forward a force-based model updating algorithm and an adaptive impedance controller to improve the stability of MMT.

We firstly proposed a new force-based approach to solve the model jump effect in MMT. The force calculated by the latest model parameters is set as the target, and it makes the rendered force to approach the target as soon as possible. By dividing the rendered force into normal part and abnormal part, then the abnormal part is limited to ensure stable interaction and the normal part is retained to respond the normal operation. By accomplishing the model updating more effectively, the force-based methods are able to significantly improve the updating efficiency compared to the existing solutions while ensuring a stable human-machine interaction.

On the slave side, motivated by the potential risk of current slave controllers, an adaptive impedance controller was introduced. We leveraged the specialty of MMT that both force and position can be acquired for slave controller, which give two reference input to the controller. Meanwhile, by comparing the real environment stiffness with the one in the prediction model on the master side, the impedance parameters were adjusted automatically. As a result shown in both theory and experiment, when there are errors between the prediction model and real environment, the slave executes the task in a conservative way that emphasis on force or position accordingly to avoid undesired large penetration or contact force. Together with the switching-free characteristic, the proposed controller is deemed to provide better stability and safety for MMT. We implemented the controllers in a 1-DOF experiment system. Experiments results on different environment material verified the superiority of the proposed methods.

Although the results in the experiment is encouraging, it is worth noting that obtaining ideal tracking and force feedback under large time delay requires more accurate environment model. In addition to the environment dynamics model, the geometry model of environment is also needed and vital for the application of MMT.

Future work consists in the extension of proposed controller to a multiple degree of freedom tasks and the experiments with improved position-dependent environment model. Furthermore, a comprehensive and theoretical stability analysis is needed.

Supplemental Material

Supplemental Material, Demonstration_Video - Model-mediated teleoperation with improved stability

Supplemental Material, Demonstration_Video for Model-mediated teleoperation with improved stability by Jingzhou Song, Yukun Ding, Zhihao Shang and Ji Liang in International Journal of Advanced Robotic Systems

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The project was supported by the Open Research Fund of Key Laboratory of Space Utilization, Chinese Academy of Science (no. LSU-2016-05-2).

Supplemental material

Supplementary material for this article is available online.

References

Becquet

. Teleoperation: numerical simulation and experimental validation, vol. 4. Dordrecht: Springer Science & Business Media, 2012.

Lawrence

. Stability and transparency in bilateral teleoperation. IEEE Trans Robot Autom 1993; 9(5): 624–637.

Rebelo

Schiele

. Time domain passivity controller for 4-channel time-delay bilateral teleoperation. IEEE Trans Haptics 2015; 8(1): 79–89.

Sun

Naghdy

. Wave-variable-based passivity control of four-channel nonlinear bilateral teleoperation system under time delays. IEEE/ASME Trans Mechatron 2016; 21(1): 238–253.

Niemeyer

Slotine

. Stable adaptive teleoperation. IEEE J Ocean Eng 1991; 16(1): 152–162.

Kawashima

. Achieving stable tracking in wave-variable-based teleoperation. IEEE/ASME Trans Mechatron 2014; 19(5): 1574–1582.

Hace

Franc

Jezernik

. Sliding mode control for scaled bilateral teleoperation. In: IECON 2011-37th annual conference on IEEE industrial electronics society, Melbourne, Australia, 7–10 November 2011, pp. 3430–3435. IEEE.

Gonzalez

Salas-Peña

DeLeon-Morales

. Observer-based integral sliding mode approach for bilateral teleoperation with unknown time delay. Automatika J Control Meas Electron Comput Commun 2016; 57(3): 749–760.

Ajoudani

Tsagarakis

NG,

Bicchi

. Tele-impedance: teleoperation with impedance regulation using a body-machine interface. Int J Rob Res 2012; 31(13): 1642–1656. DOI: 0278364912464668.

10.

Pan

YJ,

Hilliard

. Bilateral teleoperation with time-varying delay: a communication channel passification approach. IEEE/ASME Trans Mechatron 2013; 18(4): 1431–1434.

11.

Mitra

Niemeyer

. Model-mediated telemanipulation. Int J Rob Res 2008; 27(2): 253–262.

12.

Cizmeci

Schuwerk

. Model-mediated teleoperation: toward stable and transparent teleoperation systems. IEEE Access 2016; 4: 425–449.

13.

Passenberg

Peer

Buss

. Model-mediated teleoperation for multi-operator multi-robot systems. In: 2010 IEEE/RSJ international conference on intelligent robots and systems (IROS), Taipei, Taiwan, 18–22 October 2010, pp. 4263–4268. IEEE.

14.

Willaert

Bohg

Van Brussel

. Towards multi-dof model mediated teleoperation: using vision to augment feedback. In: 2012 IEEE international workshop on haptic audio visual environments and games (HAVE), Munich, Germany, 8–9 October 2012, pp. 25–31. IEEE.

15.

Winck

Sketch

Hawkes

. Time-delayed teleoperation for interaction with moving objects in space. In: 2014 IEEE international conference on robotics and automation (ICRA), Hongkong, China, 31 May–7 June 2014, pp. 5952–5958. IEEE.

16.

Xia

Léonard

Kandaswamy

. Model-based telerobotic control with virtual fixtures for satellite servicing tasks. In: 2013 IEEE international conference on robotics and automation (ICRA), Karlsruhe, Germany, 6–10 May 2013, pp. 1479–1484. IEEE.

17.

Smisek

van Paassen

RM,

Schiele

. Naturally-transitioning rate-to-force controller robust to time delay by model-mediated teleoperation. In: 2015 IEEE international conference on systems, man, and cybernetics (SMC), Hong Kong, China, 9–12 October 2015, pp. 3066–3071. IEEE.

18.

Feth

Peer

Buss

. Incorporating human haptic interaction models into teleoperation systems. In: 2010 IEEE/RSJ international conference on intelligent robots and systems (IROS), Taipei, Taiwan, 18–22 October 2010, pp. 4257–4262. IEEE.

19.

Winck

Okamura

. Model-mediated teleoperation with predictive models and relative tracking. In: ASME 2013 dynamic systems and control conference, Palo Alto, United States, 21–23 October 2013, pp. V002T26A003–V002T26A003. American Society of Mechanical Engineers.

20.

Lloyd

Székely

Harders

. Identification of spring parameters for deformable object simulation. IEEE Trans Vis Comput Graphics 2007; 13(5): 1081–1094.

21.

Mihelj

Podobnik

. Haptics for virtual reality and teleoperation, vol. 67. Dordrecht: Springer Science & Business Media, 2012.

22.

Yamamoto

Vagvolgyi

Balaji

. Tissue property estimation and graphical display for teleoperated robot-assisted surgery. In: IEEE international conference on robotics and automation, 2009. ICRA’09, Kobe, Japan, 12–17 May 2009, pp. 4239–4245. IEEE.

23.

Haddadi

Hashtrudi-Zaad

. Online contact impedance identification for robotic systems. In: 2008 IEEE/RSJ international conference on intelligent robots and systems, Nice, France, 22–26 September 2008, pp. 974–980. IEEE/RSJ.

24.

Haddadi

Hashtrudi-Zaad

. Real-time identification of hunt–crossley dynamic models of contact environments. IEEE Trans Robot 2012; 28(3): 555–566.

25.

Willaert

Van Brussel

Niemeyer

. Stability of model-mediated teleoperation: discussion and experiments. In: International conference on human haptic sensing and touch enabled computer applications, vol. 1. Springer: Springer Berlin Hedidelberg, 2012, pp. 625–636.

26.

Mitra

Gentry

Niemeyer

. User perception and preference in model mediated telemanipulation. In: Second joint eurohaptics conference and symposium on haptic interfaces for virtual environment and teleoperator systems (WHC’07), Tsukaba, Japan, 22–24 March 2007, pp. 268–273. IEEE.

27.

Schuwerk

Steinbach

. Passivity-based model updating for model-mediated teleoperation. In: 2015 IEEE international conference on multimedia & expo workshops (ICMEW), Turin, Italy, 29 June–3 July 2015, pp. 1–6. IEEE.

28.

Love

Book

. Force reflecting teleoperation with adaptive impedance control. IEEE Trans Syst Man Cybern Syst 2004; 34(1): 159–165.

29.

Cizmeci

Al-Nuaimi

. Point cloud-based model-mediated teleoperation with dynamic and perception-based model updating. IEEE Trans Instrum Meas 2014; 63(11): 2558–2569.

30.

Mobasser

Hashtrudi-Zaad

. Predictive teleoperation using laser rangefinder. In: 2006 Canadian conference on electrical and computer engineering, Ottawa, Canada, 7–10 May 2006, pp. 1279–1282. IEEE.

31.

Huijun

Aiguo

. Virtual-environment modeling and correction for force-reflecting teleoperation with time delay. IEEE Trans Ind Electron 2007; 54(2): 1227–1233.

32.

Kammerl

Chaudhari

. Hybrid signal-based and geometry-based prediction for haptic data reduction. In: 2011 IEEE international workshop on haptic audio visual environments and games (HAVE), Qinhuangdao, China, 14–17 October 2011, pp. 68–73. IEEE.

33.

Verscheure

Swevers

Bruyninckx

. On-line identification of contact dynamics in the presence of geometric uncertainties. In: IEEE international conference on robotics and automation, Pasadena, United States, 19–23 May 2008, pp. 851–856. ICRA 2008 IEEE.

34.

Haddadi

Hashtrudi-Zaad

. A new method for online parameter estimation of hunt-crossley environment dynamic models. In: 2008 IEEE/RSJ international conference on intelligent robots and systems, Nice, France, 22–26 September 2008, pp. 981–986. IEEE.

35.

Achhammer

Weber

Peer

. Improvement of model-mediated teleoperation using a new hybrid environment estimation technique. In: 2010 IEEE international conference on robotics and automation (ICRA), Anchorage, United States, 3–8 May 2010, pp. 5358–5363. IEEE.

36.

Yamamoto

Bernhardt

Peer

. Techniques for environment parameter estimation during telemanipulation. In: 2008 2nd IEEE RAS & EMBS international conference on biomedical robotics and biomechatronics, Scottsdale, United states, 19–22 October 2008, pp. 217–223. IEEE.

37.

Hirche

Buss

. Human-oriented control for haptic teleoperation. Proc IEEE 2012; 100(3): 623–647.

38.

Erickson

Weber

Sharf

. Contact stiffness and damping estimation for robotic systems. Int J Rob Res 2003; 22(1): 41–57.

39.

Jung

Hsia

TC,

Bonitz

. Force tracking impedance control of robot manipulators under unknown environment. IEEE Trans Control Syst Technol 2004; 12(3): 474–483.

40.

Stanczyk

Peer

Buss

. Development of a high-performance haptic telemanipulation system with dissimilar kinematics. Adv Rob 2006; 20(11): 1303–1320.

41.

Hogan

. Controlling impedance at the man/machine interface. In: 1989 IEEE international conference on robotics and automation, 1989. Proceedings, Scottsdale, United States, 14–19 May 1989, pp. 1626–1631. IEEE.

42.

Tzafestas

Velanas

Fakiridis

. Adaptive impedance control in haptic teleoperation to improve transparency under time-delay. In: IEEE international conference on robotics and automation, 2008, Pasadena, United States, 19–23 May 2008, pp. 212–219. ICRA 2008, IEEE.

43.

Love

Book

. Environment estimation for enhanced impedance control. In: 1995 IEEE international conference on robotics and automation, 1995. Proceedings, vol. 2. Nagoya, Japan, 21–27 May 1995, pp. 1854–1859. IEEE.

44.

Carbone

. Stiffness analysis and experimental validation of robotic systems. Front Mech Eng 2011; 6(2): 182–196.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

58.07 MB