Motion mapping of the heterogeneous master–slave system for intuitive telemanipulation

Abstract

Heterogeneous master–slave robots are widely used as both the assistive robot system for the elder/disabled people and the teleoperation robot system in dangerous environments. For the sake of intuitive teleoperation, motion mapping should be applied to keep the slave arm possessing the similar configuration of the master arm in the whole course of telemanipulation. We propose a motion mapping on the joint analog level based on unit dual quaternions (UDQs) for the telecontrol of a heterogeneous slave robot arm via the motion of human master arm. First, we analogize the links of the slave robot to the links of the human master arm, and accordingly group the slave arm joints into “shoulder,” “elbow,” and “wrist” joint analog sets. Then, we capture the motion of the human master arm via a wearable motion capture system and take it as the motion order of the slave arm. Finally, we solve the motion of slave robot arm’s joint via UDQ-based 1-, 2-, and 3-Degree of freedom (DOF) orientation error UDQ decomposition algorithms that are suit for solving a typical class of robotic slave arms. As a result, the operator can manipulate the heterogeneous robot system intuitively and conveniently, and the fatigue degree and error rate of the user can be significantly reduced, and thus the safety of teleoperation in an unstructured and constrained environment can be significantly improved.

Keywords

Motion mapping heterogeneous master–slave system telemanipulation unit dual quaternion

Introduction

Telemanipulation robots are widely used as both the assistive robot system for the elder/disabled people (e.g. wheelchair-mounted robotic arm) and the teleoperation robot system in dangerous or hazardous environments (radiation, explosive, chemical, search and rescue, etc.). The teleoperation environments are usually unstructured and constrained and the teleoperation tasks are usually complex, composite, and unpredictable; thus, the advanced autonomous robot cannot accomplish these tasks in the current stage and a human operator is needed in the control loop using his intelligence to control the robot accomplishing some special tasks.

Ideally, the master and the slave arms should have similar kinematical structure (e.g. the NASA Robonaut 2) so that the arm motion of the teleoperator can be directly adopted to control the manipulation of anthropomorphic robotic manipulator using a joint-to-joint mapping methodology,^1,2 where the teleoperation of the human is intuitive and convenient. It means that a very complex and elaborated mechanical structure of the slave robot arm is needed and the robot system may become very expensive. Therefore, the master–slave telemanipulation robot systems are usually heterogeneous instead of homogeneous, and motion mapping algorithms from the master arm to the slave arm are needed.

The widely used motion mapping algorithm is a trajectory-concerned method that adopts a forward–inverse kinematics approach. The human operator issues a motion command via a mouse, keyboard, or joystick, which maps the pose (position and orientation) of the end effector of the slave arm in its workspace, and the joints’ angle values of the slave arm are calculated via its inverse kinematics (IK). This method cannot take both the position of the end effector and the orientation of the links of the robot arm into consideration simultaneously because of the heterogeneous structure between the master and slave arms.^3

–8 As a result, the telerobot system may have the following drawbacks: (1) the teleoperation is unintuitive, and extensive training is needed for the operator to correctly operate the system; (2) the operator may become error-prone under heavy work pressure, especially when the teleoperation robot system is working in an unstructured and constrained environment with many obstacles.

Casper et al. studied the World Trade Center robot-assisted rescue response and emphasized that the human–robot interaction is a key component of a successful rescue system.⁹ A good telemanipulation robot system should be able to enable the operator controlling the slave manipulator intuitively and conveniently so as to alleviate the physiological and psychological fatigue of the teleoperator.

For the sake of achieving intuitive teleoperation, we should concern the similarity of the configuration between the master and the slave arms instead of the position of the end effector of the slave arm in its workspace. In other words, the intuitive teleoperation system should be an orientation-concerned system instead of a trajectory-concerned one. An orientation-concerned approach is important for keeping the similarity of configurations between the heterogeneous master and the slave arm so that the operator can control the motion of the slave arm intuitively and conveniently. As a result, the operator needs only concern the higher level tasks and thus can improve the quality of telemanipulation.

For this propose, we studied the orientation-concerned motion mapping approach for heterogeneous telemanipulation system. We adopt unit dual quaternion (UDQ), which is computationally effective and singularity free^10
–12 and thus be regarded as the best formulation method of both the forward kinematics (FK) and the IK for serial robot manipulators¹³ to describe the kinematics of the slave robot arm. We suggest that our approach can be applied to a wide range of slave robotic arms for intuitive telemanipulation. The detailed composition of the heterogeneous telemanipulation robot system as well as the analog-based joint set of the slave arm is presented in the “Heterogeneous telemanipulation system” section. The orientation-concerned FK and IK approach based on UDQ for motion mapping of the slave arm is presented in the “UDQ-based kinematics for motion mapping” section. Simulation results are presented in the “Simulation with a UR5 model” section. Conclusions are drawn and future works are also discussed in the “Conclusions” section.

Heterogeneous telemanipulation system

The heterogeneous telemanipulation system studied in this article is shown in Figure 1. The master arm is the teleoperator’s right arm because the most intuitive and habitual way to perform an operation for a person is using his own hand directly, and on the slave side, there is a robot with dual arms. Here we take UR5 from Universal Robots as an example, but as shown in this article, any robotic arms that are similar to the human arm in structure to a certain extent can be adopted as the slave arms and controlled with our approach. The telemanipulation order of the master arm is obtained via a wearable motion capture system, which can get the orientation data of the links of the master arm (the upper arm, forearm, and hand) simultaneously.¹⁴ These data are expressed in UDQ and serve as the reference input of the slave arm.

Figure 1.

The constitution of the heterogeneous telemanipulation system.

As illustrated in Figure 2, the slave arm UR5 is taken as an analogy to the human operator’s arm with links of “the upper arm” ( ${x_{u}, y_{u}, z_{u}}$ ), “the forearm” ( ${x_{f}, y_{f}, z_{f}}$ ), and “the hand” ( ${x_{h}, y_{h}, z_{h}}$ ), and its joints are accordingly divided into groups of “the shoulder” (joints 1 and 2), “the elbow” (joint 3), and “the wrist” (joints 4, 5, and 6), respectively. For every given configuration of the human master arm, we compute the corresponding angle of every joint set via UDQ-based IK of the slave arm, and thus achieve the similar operation between the master and the slave arms.

Figure 2.

An illustration of the analog structure of UR5.

UDQ-based kinematics for motion mapping

As shown in Figure 1, the main links of the slave arm are expected with the same orientation of the corresponding links of the master arm for the sake of intuitive telemanipulation, and the angle of the joints that realize the expected orientation of the slave arm is computed using the IK of the slave arm. Then, these angle values are applied to the slave robotic arm as reference input to drive it performing the expected operations.

UDQ-based FK of the slave arm

UDQ can be used to represent any rigid transformation (i.e. translation and rotation) and thus is particularly suitable for the orientation-concerned intuitive telemanipulation applications. We adopt the UDQ-based method proposed in Özgür and Mezouar¹⁰ to formulate the FK of the slave robotic arm UR5. Usually only two coordinate frames are needed for an n-DOF serial manipulator in UDQ-based kinematics. In this study, we attach three right-handed coordinate frames ${u}, {f}, and {h}$ to the upper arm, forearm, and hand analogs of the slave arm, respectively (Figure 2), and describe the orientations of these links with UDQ representation because we concerned all the orientations of the upper arm, forearm, and hand of the slave arm.

We define the home configuration ${\hat{θ}}_{0}$ of the slave arm as shown in Figure 2 and denote the current value of the joints J₁ to J₆ of the salve arm as

\hat{θ} = [{\hat{θ}}_{1}, {\hat{θ}}_{2}, {\hat{θ}}_{3}, \dots, {\hat{θ}}_{6}]^{T}

A UDQ ${\hat{δ}}_{i}$ that displaces the slave robot arm about the ith joint’s screw axis while the rest of joints are locked can be expressed as

{\hat{δ}}_{i} = exp (\frac{{\hat{θ}}_{i}}{2} {\hat{s}}_{i 0}) = cos (\frac{{\hat{θ}}_{i}}{2}) + {\hat{s}}_{i 0} sin (\frac{{\hat{θ}}_{i}}{2})

The dual angle ${\hat{θ}}_{i}$ is a relative joint displacement with respect to the home joint position. Since all the joints of the slave robot arm are revolute, then ${\hat{θ}}_{i} = Δ θ_{i}$ .

The unit dual vector ${\hat{s}}_{i 0}$ represents the joint screw axis calculated at home configuration in terms of the Plücker line coordinates:

\begin{array}{l} {\hat{s}}_{i 0} = l_{i 0} + ε m_{i 0} \\ m_{i 0} = p_{i 0} \times l_{i 0} \end{array}

where l _i0 is the unit vector of the direction of the joint axis, ε² = 0, ε ≠ 0, is the dual factor, m _i0 is the moment vector of the joint axis about the origin of the home frame, and p _i0 is a position vector from the origin of the home frame to any point lying on the joint axis.

Then, for any deviation from the home configuration ${\hat{θ}}_{0}$ , the poses of the links we concerned (the upper arm, forearm, and hand analogs) of the robot arm can be calculated by multiplying all the UDQs of successive joint displacements from the base to the concerned ith link, given ${\hat{δ}}_{i}$ as the UDQ:

{}^{0}{\hat{x}}_{l} = {}^{0}{\hat{δ}}_{1} {}^{0}{\hat{δ}}_{2} \dots {}^{0}{\hat{δ}}_{j}

Where ${}^{0}{\hat{x}}_{l}$ is the new pose of the concerned links with respect to the base frame {0}, l = u, f, h is the concerned links of the upper arm, forearm, and hand, respectively, and j is the closest joint on the base side of the concerned link.

When we get the UDQ pose (position and orientation) of a concerned link, we can represent it as

{}^{0}{\hat{x}}_{l} = cos (\frac{{\hat{θ}}_{l}}{2}) + {\hat{s}}_{l 0} sin (\frac{{\hat{θ}}_{l}}{2})

where ${\hat{θ}}_{l}$ and ${\hat{s}}_{l 0}$ are the dual angle and the unit dual vector of a directed three-dimensional (3D) line passing through the axle center of the link. ${θ, d, l, m}$ are screw displacement parameters that describe the position and orientation of the link in the 3D space with respect to the base frame. θ is the rotation angle around the screw axis, d is the translation along the same screw axis, l is the unit direction vector of this screw axis, and m is the moment vector of this axis computed with respect to the origin of the home frame of the robot arm.

The UDQ pose can also be rewritten with the known screw displacement parameters as

{}^{0}{\hat{x}}_{l} = q_{R} + ε q_{T}

where q _R is a unit quaternion for rotation and q _T is a quaternion for translation. They can be computed with the known screw displacement parameters

\begin{array}{l} q_{R} = (cos (\frac{θ}{2}), l_{l 0} sin (\frac{θ}{2})) \\ q_{T} = (- \frac{d}{2} sin (\frac{θ}{2}), l_{l 0} \frac{d}{2} cos (\frac{θ}{2}) + m_{l 0} sin (\frac{θ}{2})) \end{array}

By this way, we can easily get the position and orientation information of the concerned link in the world (base) coordinate system. This UDQ-based representation is compact, computationally efficient, and singularity free.¹²

UDQ-based IK for the typical joint analogs

Usually the motion mapping is achieved by solving the configuration of the slave arm from the pose information of the end effector using the Paden–Kahan subproblem,^15,16 which is a trajectory-concerned method and the motion of the slave arm may not be intuitive for the human operator. We suggested that the human operator will feel intuitive in the teleoperation only if the slave arm possesses the similar configuration of the master arm in the whole course of telemanipulation. In other words, the upper arm, forearm, and hand of the slave arm are expected to have the same orientation of the counterpart of the master arm, so we should study an orientation-concerned method for motion mapping.

The structural difference between the human master arm and the slave robot arm must be taken into consideration when we draw an analog between them. For the human master arm, both the shoulder and the wrist joints can be viewed as ball joints, while the upper arm is simply the link between the shoulder joint and the elbow joint, as well as the forearm is the link between the elbow joint and the wrist joint. However, the joints of the robot arm usually have a serial layout instead of a ball joint. We intuitively draw an analogy that the upper arm and the forearm of the slave arm UR5 are the links of line CB instead of line AB as shown in Figure 3, and the joint set of shoulder, elbow, and wrist analogs are illustrated in Figure 2 and described in the “Heterogeneous telemanipulation system” section.

Figure 3.

The analog links of the upper arm and forearm of UR5 (left for the upper arm and right for the forearm).

In the orientation-concerned method, we first draw an analogy between the human master arm and the slave robot arm, and then solve the motion mapping between the corresponding analog body parts (i.e. the upper arm, forearm, and hand) using the solutions for the 1-, 2-, and 3-DOF analogs as described in the following.

Because of the structural difference between the human master arm and the slave robot arm, it is difficult or even impossible to realize the same position and orientation between the master and the slave arms simultaneously. For the sake of orientation-concerned intuitive teleoperation, we will ignore the position information and only realize the same orientation of the links of the slave arm for a given pose UDQ of the counterpart of the master arm, which can be done by setting the dual part of the given pose UDQ to zero quaternion. It will not affect the result of the orientation-concerned motion mapping of the slave arm.

The IK solution of the 1-DOF joint analog

The elbow analog is usually a 1-DOF joint.

Given the pose of the upper arm, suppose that the current and the desired orientations of the forearm in UDQ are ${\hat{x}}_{c}$ and ${\hat{x}}_{d}$ , respectively, then we can compute the error UDQ ${\hat{x}}_{e}$ between the current and the desired orientation:

{\hat{x}}_{e} = {\hat{x}}_{c} {\hat{x}}_{d}^{*}

θ = 2 arccos (s_{R})

where s _R is the scale part of the quaternion for rotation $q_{R} = (s_{R}, V_{R})$ of the error UDQ ${\hat{x}}_{e}$ , and ${\hat{x}}_{d}^{*}$ is the classical quaternion conjugate of ${\hat{x}}_{d}$ .¹⁰

Since there is only 1-DOF of motion that results in the difference between the initial and target orientations of the forearm, θ should be exactly the revolute angle of the elbow analog that drives the forearm to its target orientation.

The IK solution of the 2-DOF joint analog

Most shoulder analogs of the robot arm have a 2-DOF structure, in which the axes of the two revolute joints are perpendicular to each other (Figures 2 and 5(a)).

Given the current orientation ${\hat{x}}_{c}$ and the desired orientation ${\hat{x}}_{d}$ of the upper arm in UDQ with the pose of the trunk fixed, we can solve the virtual revolute axis l and the desired angle θ rotating around the virtual axis using formulas (9) and (10).

As illuminated in Figure 4, since the vectors of the 2-DOF joint analog, namely, l ₁ and l ₂, are perpendiculars that intersect at a point, and a rotation movement can be depicted as a vector with its direction representing the rotation axis and its length representing the rotation angle, we can decompose the virtual rotation to the actual rotate joint:

l = \frac{V_{R}}{sin (θ / 2)}

θ_{i} = \frac{l \cdot l_{i}}{| l | | l_{i} |} θ (i = 1, 2)

where l ₁ and l ₂ are the direction vectors of the axes of joints 1 and 2, respectively, at the current time, and θ₁ and θ₂ are the angular increments of joints 1 and 2, respectively, that realize the desired orientation of the slave upper arm.

Figure 4.

The decomposition of virtual revolution to the two perpendicular joints of the 2-DOF joint analog.

The IK solution of the 3-DOF joint analog

Most wrist analogs and some shoulder analogs possess a 3-DOF structure. We study the 3-DOF analog that possesses the following two structure characteristics:

The joints #1 and #2 as well as the joints #2 and #3 in the 3-DOF analog are two pairs of perpendicular joints.

The axis of joint #3 coincides with the axle of the adjacent link.

The aforementioned 3-DOF analog is a typical mechanical structure widely adopted in robotic arms. For example, the 3-DOF shoulder and/or wrist analogs of the assistive robotics such as the Kinova Jaco2 and the UR5, as well as the space manipulators such as the Canadarm2 of the Canadian Space Agency and the European Robotic Arm of the European Space Agency belong to this kind of joint analog (Figure 5).

Figure 5.

Typical 2- and 3-DOF joint analogs of robot arms. (a) The 2- and 3-DOF joint analogs of UR5. (b) The 3-DOF joint analog of Kinova Jaco2.

As illuminated in Figure 6, we solve the virtual revolute axis l and its desired rotation angle θ according to the error UDQ ${\hat{x}}_{e}$ between the current orientation ${\hat{x}}_{c}$ and the desired orientation ${\hat{x}}_{d}$ of the concerned link of the robot arm using formulas (8), (9), and (10). We at the same time get the orientation and the length of l as well as the current orientation information of the axes of joints $J_{i}, (i = 1, 2, 3)$ . First, we can compute the projection of l on the axis of J ₂ to get l ₂ and the projection of l on a plane to get l ₁₃. That plane is perpendicular to the axis of J ₂ and determined by two vectors that parallel to the axes of J ₁ and J ₃, which form a skew coordinate system. Then, we decompose l ₁₃ to the coordinate axes of the skew coordinate system to get l ₁ and l ₃. As a result, the computed component of vectors l ₁, l ₂, and l ₃ is the desired rotation of J ₁, J ₂, and J ₃, respectively, in the 3-DOF joint analog to achieve the given orientation of the concerned link of the slave robotic arm.

Figure 6.

The decomposition of virtual revolution to the real joints of the 3-DOF joint analog.

UDQ-based motion mapping for telemanipulation

In the telemanipulation system as shown in Figure 1, the teleoperation motion is captured at the master side and sent to the slave side in every sampling period. Given the orientation data of every link (the upper arm, forearm, and hand) of the master arm, we can compute the corresponding joint angles of the joint sets that analogize to “the shoulder,” “the elbow,” and “the wrist” of the slave robot arm. These joint angles are sequentially solved using the IK model of the salve arm as shown in Figure 7. By making the links (successively from the upper arm, to the forearm, then to the hand) of the slave arm possessing the same orientation as the corresponding links of the master arm, we can achieve the similar configuration between the master and the slave arms. Different from the conventional FK–IK approach, our approach solves the IK problem on the analog joint subset instead of the whole robot arm. As a result, our approach is simple and can achieve the intuitive and convenient operation of the telerobot system.

Figure 7.

The flowchart of orientation-concerned motion mapping approach.

For some repetitive and fixed routines of manipulation, we need not let the operator repetitively giving the motion order at the master side. When we solve the angle increments of the joints of “the shoulder,” “the elbow,” and “the wrist” that will achieve the desired configuration of the slave manipulator at some crucial point, we can drive the corresponding links of the robot arm from their current pose to their desired pose through linear interpolation according to the angular velocity requirement of the corresponding joints:

θ_{i} = θ_{0} + \sum_{i = 1}^{k} Δ t \cdot ω_{i}

where θ_i and ω_i are the angle value and the angular velocity of the concerned joint at the ith time step, respectively, θ₀ is the initial angle value of the joint, and Δt is the time interval of a step and the robot is supposed to move to its desired orientation in k time steps.

By this way, we can further alleviate the burden of the teleoperator and thus improve the efficiency of the telemanipulation system in critical situations.

Simulation with a UR5 model

We validate our approach via simulations on a UR5 model. First, we randomly generate the desired orientation of the upper arm, forearm, and hand of the slave robotic arm in UDQ, which are supposed as the equivalents of the orientation data of the operator’s counterparts obtained via the wearable motion capture system when he moves his arm in his habitual workspace. Then, we calculate the motion of the joint sets corresponding to the shoulder, the elbow, and the wrist sequentially using the UDQ-based IK algorithms. Finally, we calculate the real orientations of the shoulder, the elbow, and the wrist analogs of the slave robotic arm using its UDQ-based FK, and then compute the difference between the resulted orientations of the master and those of the slave arms. The simulation results validated that our approach can definitely drive the heterogeneous slave robotic arm(s) to the desired configurations.

The establishment of the UR5 model

We draw an analogy between the UR5 robotic arm and the human arm to apply the UDQ-based motion mapping algorithm. In our scheme (Figures 2 and 3), joints 1 and 2 correspond to the shoulder, joint 3 corresponds to the elbow, joints 4, 5, and 6 correspond to wrist, and the two longer links correspond to the upper arm and forearm, respectively. According to the home configuration as shown in Figure 2, we get the unit vectors of all joint axes as shown in Table 1, and select six points of p _i0 on the corresponding direction vectors of l _i0 as shown in Table 2, where $l_{i}, (i = 1 - 7)$ , are defined in Figure 2 and their values are 500, 240, 109, 425, 392, 95, and 82 mm, respectively.

Table 1.

The unit vectors of the six joint axes of UR5.

Joint axis	Unit vector	Joint axis	Unit vector
l ₁₀	$[\frac{\sqrt{2}}{2}, 0, - \frac{\sqrt{2}}{2}]$	l ₂₀	$[\frac{\sqrt{2}}{2}, 0, \frac{\sqrt{2}}{2}]$
l ₃₀	$[\frac{\sqrt{2}}{2}, 0, \frac{\sqrt{2}}{2}]$	l ₄₀	$[\frac{\sqrt{2}}{2}, 0, \frac{\sqrt{2}}{2}]$
l ₅₀	$[\frac{\sqrt{2}}{2}, 0, - \frac{\sqrt{2}}{2}]$	l ₆₀	$[\frac{\sqrt{2}}{2}, 0, \frac{\sqrt{2}}{2}]$

Table 2.

The selected points on the direction of unit vectors of the six joint axes.

Point	Coordinates
p ₁₀	$[l_{2}, 0, l_{1}]$
p ₂₀	$[l_{2} - \frac{\sqrt{2}}{2} l_{3}, 0, l_{1} - \frac{\sqrt{2}}{2} l_{3}]$
p ₃₀	$[l_{2} + \frac{\sqrt{2}}{2} (l_{4} - l_{3}), 0, l_{1} - \frac{\sqrt{2}}{2} (l_{3} + l_{4})]$
p ₄₀	$[l_{2} + \frac{\sqrt{2}}{2} (l_{4} + l_{5}), 0, l_{1} - \frac{\sqrt{2}}{2} (l_{4} + l_{5})]$
p ₅₀	$[l_{2} + \frac{\sqrt{2}}{2} (l_{4} + l_{5} - l_{3}), 0, l_{1} - \frac{\sqrt{2}}{2} (l_{3} + l_{4} + l_{5})]$
p ₆₀	$[l_{2} + \frac{\sqrt{2}}{2} (l_{4} + l_{5} + l_{6} - l_{3}), 0, l_{1} - \frac{\sqrt{2}}{2} (l_{3} + l_{4} + l_{5} + l_{6})]$

The moment vectors $m_{i 0}, (i = 1 - 6)$ of all joint axes are computed according to formula (3), and then both the FK and IK models of the slave robot manipulator can be obtained using the methods stated in the “UDQ-based kinematics for motion mapping” section.

The simulation results of 1-DOF elbow analog

Elbow is a typical example of 1-DOF joint analog. In all of the joints (shoulder, elbow, and wrist) of human arm, elbow has the least DOF, but takes the most quantity of motion and exerts the greatest strength. Given the orientation of the upper arm, the orientation of the forearm is decided by only the angle value of the elbow joint (joint 3). Thus, we can easily get the IK solution of the elbow joint analog of the UR5 using the current and the target orientation information of the forearm.

Experimental results of 50 random trials are presented in Figure 8, in which the black thin line indicates the 50 random value of the elbow joint, and the red dashed line and the blue thick line indicate the target and the solved value of the elbow joint, respectively. From Figure 8, we can learn that the desired orientation of the forearm can be correctly achieved using our method in every of the 50 random simulations. The maximum error between the desired angle and the solved angle of joint 3 is less than 0.5°, which is not a perceivable error for a human teleoperator.

Figure 8.

The simulation results of IK solution for the 1-DOF elbow analog. (a) Initial, desired, and final angles of joint 3 in 50 random trials. (b) Errors of joint 3 in 50 random trials.

The simulation results of 2-DOF shoulder analog

The combination of joints 1 and 2 of UR5 forms a typical 2-DOF shoulder analog. Given the current and the target orientation of the upper arm, we can get the IK solution of the shoulder joint analog of the UR5 by applying the vector decomposition method to the virtual rotation axis as proposed in the “UDQ-based kinematics for motion mapping” section.

Again, 50 random trials are performed and the results are presented in Figure 9 with the similar legends of Figure 8. For the 50 pairs of the randomly generated initial and desired angular values for both joints 1 and 2, there are 35 and 33 times the error bigger than 1° for joints 1 and joint 2, respectively. Obviously, the result is unsatisfactory with the biggest error of 61.3707° for joint 1 and 36.6768° for joint 2.

Figure 9.

The results of IK solution for the 2-DOF analog. (a) Initial, desired, and final angles of joint 1 in 50 random trials. (b) Initial, desired, and final angles of joint 2 in 50 random trials (c) Errors of joints 1 and 2 in 50 random trials without iteration.

We iteratively apply the vector decomposition method to the top 10 biggest errors of Figure 9(c) and get the results of Figure 10: We successfully solve the IK solutions in no more than three iterations, and the biggest errors are 0.4394° for joint 1 and 0.2911° for joint 2, which is an ideal result of telemanipulation for the human operator.

Figure 10.

The top 10 largest errors in Figure 9 approaching to target with iterations. (a) The convergence of the error of joint 1. (b) The convergence of the error of joint 2.

The simulation results of 3-DOF wrist analog

The structure of joints 4, 5, and 6 of UR5 forms a typical 3-DOF wrist analog. As shown in Figure 6, we solve the IK problem of the robot hand of UR5 by applying the vector decomposition method of 3-DOF joint analog to the virtual rotation vector l , and compute the needed rotation angle of joints 4, 5, and 6 to achieve the desired orientation of the robotic hand of UR5.

We randomly generate a set of values [55.35, 67.84, −24.34] for joints 4, 5, and 6 as the target configuration of the robotic hand with both the shoulder and the elbow joints locked and randomly generate another 10 sets of values for joints 4, 5, and 6 as their initial value of angle. We apply the vector decomposition algorithm of 3-DOF analog iteratively and present the experiment results in Figure 11, from which we can learn that the IK solutions of the wrist analog can be correctly solved within six iterations. The maximum error between the desired angle and the actual angle of joints 4, 5, and 6 are less than 1°, and this result is sufficient for any intuitive telemanipulation application.

Figure 11.

The simulation results of 3-DOF wrist analog with iterations. (a) The convergence of the error of joint 4. (b) The convergence of the error of joint 5. (c) The convergence of the error of joint 6.

We also apply the proposed IK solution algorithms of the 1-, 2-, and 3-DOF joint analogs to the Kinova Jaco2 robotic arm and get a similar result. Thus, the proposed algorithms are suit for solving the IK problem on the joint analog level for a typical class of robotic arms, no matter the robot manipulator is more anthropomorphic like Jaco2 or less anthropomorphic like UR5.

We randomly set both the initial and target configurations of the robotic arm in the simulation, and therefore get a big error between the initial and target orientations. In the real applications, the error between the target and the current orientations of the robotic arm is much small because the sampling time interval is short and the velocity of the robotic arm is limited. Since the proposed algorithm can efficiently solve the IK problem of joint analog in several iterations for a big orientation error, it should be able to perform much better for a much small orientation error for the real application.

Conclusions

It is very important for the human operator to perform telemanipulation conveniently and intuitively, especially when the teleoperation task is complicated and the working environment of the slave arm is constrained. We interpret the intuitive teleoperation as the similar configuration between the master and the slave arms, and thus propose a UDQ-based motion mapping approach for orientation-concerned telemanipulation of heterogeneous master–slave telerobot system. Our approach takes the main links of the slave arm analogous to “the upper arm,” “the forearm,” and “the hand” and takes the corresponding joint sets of the slave arm analogous to “the shoulder,” “the elbow,” and “the wrist,” respectively. It applies the UDQ-based motion mapping approach to solve the IK problem of the upper arm, forearm, and hand analogs sequentially, and as a result keeps the link analogs of the slave arm the same orientations of the according parts of the master human arm.

Because of the heterogeneous structure between the master and slave arms, we cannot guarantee both the same pose of end effector and the same configuration of the total arm between the master and the slave side simultaneously. Our method keeps the similarity of the configuration between the master arm and the slave arm for the sake of intuitive and convenient teleoperation and lets the position of the end effector of the slave arm relying on the visual feedback and the control of the human in the loop. This approach possesses the following advantages: the algorithm is concise, simple, and effective; the solving process of IK is much simplified because it is solved on the joint analog instead of the total robotic arm; and the telemanipulation is consequently intuitive and easy to realize.

In future research, we will further improve our approach and deploy it on a physical telemanipulation robotic system. Our work may pave a road to control a telerobotic system by different operators as well as to control a variable slave manipulator by the same operator intuitively and conveniently. As a result, the fatigue degree and error rate of the user can be remarkably reduced, and the safety of teleoperation in an unstructured and constrained environment can be significantly improved.

Footnotes

Author note

All authors are also affiliated to the Beijing Key Lab of the Computational Intelligence and Intelligent System, Beijing, China.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study has been supported by the National Natural Science Foundation of China, grant no. 61673003.

References

Liarokapis

Artemiadis

Kyriakopoulos

. Mapping human to robot motion with functional anthropomorphism for teleoperation and telemanipulation with robot arm hand systems. In: Proceedings of the 2013 IEEE/RSJ international conference on intelligent robots and systems (IROS) (ed Nancy

Amato

), Tokyo, Japan, 3–7 November 2013, pp. 2075. IEEE.

Liarokapis

Artemiadis

Kyriakopoulos

. Functional anthropomorphism for human to robot motion mapping. In: Proceedings of the 21st IEEE international symposium on robot and human interactive communication (RO-MAN), Paris, France, 9–13 September 2012, pp. 31–36. IEEE.

Pierce

Kuchenbechker

. A data-driven method for determining natural human-robot motion mappings in teleoperation. In: Proceedings of the fourth IEEE RAS/EMBS international conference on biomedical robotics and biomechatronics (eds Jaydev

P. Desai

Louis Phee

Soo Jay

Loredana

Zollo

), Rome, Italy, 24–27 June 2012, pp.169–176. IEEE.

Park

Her

. Whole-body multi-modal semi-autonomous teleoperation of mobile manipulator systems. In: Proceedings of 2015 IEEE international conference on robotics and automation, Seattle, Washington, USA, 26–30 May 2015, pp.164–170. IEEE.

Anvari

. Robotics in general surgery. Part XV The future of robotic platforms. New York: Springer, 2014, pp. 485–497.

Mavridis

Pierris

Gallina

. Subjective difficulty and indicators of performance of joystick-based robot arm teleoperation with auditory feedback. In: Proceedings of the 2015 international conference on advanced robotics (eds Sinan

Kalkan

Uluç

Saranl

) Istanbul, Turkey, 27–31 July 2015, pp. 91–98. IEEE.

Atoelen

De tejada

Huete

. Distributed and adaptive shared control systems. IEEE Robot Autom Mag 2015; 22(4): 137–146.

Kim

Lee

. Excavator tele-operation system using a human arm. Autom Constr 2009; 18: 173–182.

Casper

Murphy

. Human–Robot interactions during the robot-assisted urban search and rescue response at the world trade center. IEEE Trans Syst Man Cybern B: Cybern 2003; 33(3): 367–385.

10.

Özgür

Mezouar

. Kinematic modeling and control of a robot arm using unit dual quaternions. Robot Auton Syst 2016; 77: 66–73.

11.

Singh

Singla

. Kinematic modeling of robotic manipulators. In: Proceedings of the national academy of sciences, India section A: physical sciences 2017; 87(3): 303–319.

12.

Sahu

Biswal

Subudhi

. A novel method for representing robot kinematics using quaternion theory. In: IEEE conference on computational intelligence, control and computer vision in robotics & automation, 2008, pp.76–82. Available at: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.459.8578&rep=rep1&type=pdf

13.

Sariyildiz

Cakiray

Temeltas

. A comparative study of three inverse kinematic methods of serial industrial robot manipulators in the screw theory framework. Int J Adv Robot Syst 2011; 8(5): 9–24.

14.

Zuo

Gong

. Operator attitude algorithm for tele-robotic nursing system (in Chinese). Acta Autom Sinica, 2016; 42(12): 1839–1848.

15.

Paden

. Kinematics and control robot manipulators. PhD Thesis Department of Electrical Engineering and Computer Science, University of California, Berkeley, 1986.

16.

Tan

Xiao

. Extension of the second Paden-Kahan sub-problem and its application in the inverse kinematics of a manipulator. In: IEEE conference on robotics, automation and mechatronics (eds Liang

Yan

Arthur

Tay

), Chengdu, China, 21–24 September 2008, pp.379–384. IEEE.