Sage Journals: Discover world-class research

Abstract

Soft actuators are used in bilateral systems as slave-side actuators. To broaden the applications and usage of these soft actuators in teleoperated applications required high accuracy and control. This article focuses on the improvement and control of the soft actuators operated through the Internet using a glove-based gesture control system. A hand-worn sensor glove (a glove with flex sensors and accelerometers) is used to detect hand gestures (specifically bending angle and force generated in a finger while gripping a spherical object) of an operator. The operator’s finger movements are mimicked in a remote location, through the Internet, in a soft actuator. The bending angles of the human finger have converted into the pressure variations inside the actuator using a linear calibration technique. A vertically mounted actuator and a spherical object are used to demonstrate the gripping action. The main problems that occurred during the controlling of this setup are noise and delays. Electronic noise (line noise and circuit noise), mechanical noise (vibration, nonuniformities in actuator fabrication, wear, and tear), elastic effect (energy absorbed during the state transformation), communication delays (delays occurred due to geography, telecommunication infrastructure, and round trip transmission), and other noises (other environmental effects) degraded the performance. This article considered a Brownian motion model, an additive Gaussian noise model, and a Kalman filter to solve these problems. The experimentations are performed in three different locations (to demonstrate the teleoperation) and the recorded improvement in the performance is approximately 17%.

Keywords

Soft robotics bilateral systems teleoperation Kalman filter communication delays gesture control

Introduction

Teleoperation is one of the main features that many researchers are looked into embed in their robotic designs. Various systems have been built with this feature.^1–4 The robots with teleoperation are extremely useful in remote monitoring and control applications. These systems are used in surveillance,⁵ telemedicine,⁶ and other applications.^7,8 Telerobotic concepts are applied in the teleoperation of soft robotics (soft material robots) to expand the range of applications.⁹ A typical teleoperation system is controlled by a remote-operator that connects to an electromechanical system or a software system from a long distance. The connection could be either wired or wireless. These systems are built with simplex and duplex communication systems. One of the key problems in teleoperation is communication delay. The communication delay is occurred due to many reasons. Telecommunication infrastructure, weather conditions, processing delays, and geographical conditions are few factors that directly affect communication delays. Researchers have investigated various techniques to curtail these problems in teleoperation systems.¹⁰

In teleoperated soft actuators, elastic saturation delay is one of the key issues that negatively affect the real-time performance of the system.⁹ Even though, with well-established real-time communication, elastic saturation required special consideration. The soft actuators use the energy of differential pressure input to generate movements. In these systems, the pressure difference is analogues to desired movement commands (e.g. bending angle) that receive through the Internet. The early experimentations found that transmission delays and disturbances significantly degrade the performance of teleoperated systems.⁹ The use of soft actuators in surgical applications (e.g. STIFF-FLOP: Stiffness Controllable Flexible and Learnable Manipulator for Surgical Operation) is demonstrated in controlled environments.¹¹ These systems can perform medical-grade accuracy, precision, and repeatability in surgical tasks. However, the gesture-controlled teleoperation and Internet-based communication degrade the overall performance due to time lag, delay, and disturbances. Data replication, compression, bandwidth allocation, and visualization are used as software solutions to solve these problems. Moreover, improvements in communication protocols and telecommunication infrastructure can significantly improve the performance of these systems. The use of nonlinear components—such as pneumatics and elastic actuators—is the main problems that significantly degrade the overall performance of these systems. Therefore, this article introduces a Kalman filter (KF) based with a random additive Gaussian noise model in the slave side (soft actuator) to enhance the performance. This article is organized as follows: section “Literature review” introduces the state of the art of teleoperation systems; section “Methodology” explains the methodology; section “Mathematical model” derives the noise model and KF model; section “Experimental setup” elaborates the experimental design; section “Results” presents the results; and section “Conclusion” explains the conclusion and future directions.

Literature review

The bilateral control approach is used to operate robotics systems from a distance to control and manipulate objects.¹² Precise control of an actuator in bilateral systems is important to manipulate tools and control objects in highly sensitive applications such as surgical operations.¹³ Commercially available surgical robot systems (e.g. DaVinci)¹⁴ and research-purpose-surgical-robots (e.g. MiroSurge)¹⁵ are tested with different techniques to achieve better performance in control. Ferraguti et al.¹³ introduced a technique with passive virtual fixtures to achieve accurate and safe trajectory generation. These systems are treated as either constant time-delayed systems¹⁶ or time-varying delayed systems¹⁷ in different applications, and various techniques such as Smith predictor method,¹⁸ sliding mode control,¹⁷ and observer-based techniques¹⁹ are tested as controllers. However, most of the studies have used electromechanical actuators in the bilateral controlling system.

Teleoperation is an important feature that is used in bilateral systems. This feature helps the operator to control electromechanical systems from long distances.²⁰ Highly toxic chemical operations, radiation-related activities, surveillance tasks, and disaster management systems are handled using these types of systems.^21–23 However, most of these teleoperated systems have used rigid electromechanical systems. In these traditional slave mechanisms, the mismatch in compliance between the end effector (e.g. electrotechnical gripper) and the object is caused by many problems. These problems are caused to reduce the compatibility of these systems in high sensitivity applications.²⁴ Therefore, the use of these systems with humans (interaction with live human’s human, tissues, and organs), animal (farming applications), and plant (agricultural robots: plucking fruits) based tasks is complex and limited. In order to solve these issues, this article introduced a soft actuator to interact with the external environments.

The main issues in teleoperation are communication media, sensors, actuators, and time delay problems.²⁵ The control perspective provides two main aspects to consider in the designing process. Stability and telepresence are important to the design, which will improve the flexibility.²⁵ The closed-loop system is stable irrespective of the operator’s behavior. In this application, the human operator should receive the required sensory information from the remote location to execute the actions. The experimentation of the presented article is not considered an alternative strategy such as a two-port network approach, scattering approach, wave variable approach, or four-port approach mentioned in the literature to establish stability in telepresence. The main goal of the presented article is to investigate the current issues in the teleoperated soft actuator system and introduce a computationally efficient technique to improve the delays and disturbances. However, the telepresence is provided to the human operator, using a live stream camera in the remote location through the same network.

The use of a soft actuator introduced control and delay problems to the system. The fully realized dynamic models are not investigated for these types of actuators. Therefore, the model-based closed-loop controlling of this actuator is hard to accomplish.²⁶ The actuator uses pneumatics to control its position. This is another disadvantage, because, in pneumatics, signal speed is slow, the fluid is compressible (air is a compressible fluid), leaks may cause pressure drops, and switching-based control is complex.²⁷ Another main issue is the use of the elastic material as the material of fabrication. A portion of the energy is absorbed by the actuator during its state transition. This creates a delay because the actuator required a certain time to reach its equilibrium after a change in pressure.⁹

KF-based systems are widely applied in robotics. In the traditional robot design practice, KF is used in trajectory controlling, localization,²⁸ and vision.²⁹ The performance of these systems is significantly improved after applying KF. Li and Hannaford³⁰ introduced the unscented KF approach to improving the cable-driven surgical instrumentation. The system improved its performance in the teleoperation process after applying KF.

Methodology

This section presents the design and implementation of the bilateral system and its significant components. The section introduces the designing, fabrication, and function of each component.

Overall design

Figure 1 illustrates the overall design of the system. The human operator receives visual feedback from a camera in the remote location. The operator moves the fingertip in the hand he wears a data glove (DG) to grip the object. The computer processes the bending angles/force received from the flex sensors and converted into required pressure values. These values are communicated over the Internet through Message Queuing Telemetry Transport (MQTT) server to the remotely located pneumatic control circuit. Finally, the soft actuator mimics the human finger motion by changing its internal pressure difference. The movements and forces in the soft actuator are recorded using integrated force and flex sensors. MQTT protocol is used to establish duplex communication. The communication platform is expected to deliver the data packets that are carrying control signals. The system’s control lag is occurred because it depends on external factors such as telecommunication infrastructure, the elasticity of the actuator, and other issues.

Figure 1.

Block diagram of the teleoperated bilateral system. The fingertip motion of the human operator is translated into the bending angle of the soft actuator. The finger movements are tracked using a data glove and communicated to the slave side through the Internet.

DG

The DG is constructed using a regular fabric glove. The sensors are attached to the glove to detect motions of the fingers and the palm. GY-521 IMU (inertial measurement unit) with a built-in gyroscope and accelerometer is mounted to the glove. This is used to track the accelerations and the orientations of the hand. A Raspberry Pi module is used to collect the data. GY-521 IMU has a range of 255 to −255°s⁻¹ and required to calibrate to record angular velocities and directions. Figure 2 illustrates the configuration, setup, and working envelope of the DG. Five 4.5-inch flex sensors are attached along with the fingers to detect the bending angle. The flex sensor is a resistive, carbon-based, and printed substrate that provides a change in resistance in bending movements. The resistance of this sensor depends on the curvature, a smaller radius curvature will generate a higher resistance. The spatial movements are detected using two 2-mega pixels 8 mm lens high definition (HD) cameras. The motion track and average pixel tracking algorithms are used to generate the displacement data. Therefore, the cameras are used to identify the gloves’ relative motion using five luminous yellow color patches attached to vital locations that are important to identify the motion.

Figure 2.

The design and the working envelope of the data glove (DG). The DG has been designed to track the stretch and clench of the fingers by its flex sensors. The relative motion of the hand is measured through cameras and accelerometers.

Soft actuator

The soft actuator is the most important part of this teleoperated bilateral system study. The actuator is used to mimic the human operator’s finger movements during the gripping action. The actuator is designed and evaluated using SolidWorks and Abaqus software for computer-aided design (CAD) modeling and finite element analysis (FEA). The SolidWorks design of the actuator is imported in Abaqus software to study the behavior of the actuator. This step is used to identify the working pressure range of the actuator, the range of the bending angle, soft actuator trajectory, and gripping force (maximum payload). Neo Hookean hyper-plastic model with 8873 tetrahedral quadric elements and nodes is used in the analysis. These elements are formulated in three-dimensional (3D) space with three degrees of freedom per node. The translation is considered in 3D space (X-, T-, and Z-axes). The material properties used are as follows: Hardness Shore A (ISO 868) is 28, tensile strength (ISO 37) is 6.5 N mm⁻², viscosity at 23 °C (ISO 3219) is 10,000 MPa, density at 23 °C is 1.01 g cm⁻³, tear strength (ASTM D 624 B) is >30 N mm⁻¹, and elongation at break (ISO 37) is 700%. Figure 3 shows the behavior of the actuator around the spherical object.

Figure 3.

Simulation of the gripping action. The finite element model shows the behavior of the soft actuator due to its internal pressure difference. The internal pressure difference is caused to bend the actuator around the spherical object.

The validated soft actuator model is used to implement two-part molds of the actuator. The top part is a membrane with cavities and the bottom part consists of a strain-limiting layer. Molds are 3D printed using polylactic acid (PLA) at 200 °C extruder temperature and bed temperature of 60 °C using a Thrimana 3D printer. ELASTOSIL^® M 4601 A/B RTV-2, two-part elastomer compound in 9:1 mixed ratio is used to fabricate the actuator. The 9:1 ratio mixture is mixed using a centrifugal mixture, poured into the molds, treated in a vacuum chamber, and cured in 80 °C in an oven. The two parts are separately removed from the molds and bond together to form the final structure using the same method. The main steps of fabrications of soft actuators are explained in Figure 4.

Figure 4.

The fabrication procedure of the soft actuator: (a) polylactide 3D printed molds, (b) elastomer material is mixed with a ratio of 9:1, (c) stir mixing of the material using an automated mixture, (d) the mixture is poured into the molds, (e) the extra material is removed from the molds, (f) the molds are placed in the vacuum chamber to extract the air bubbles, (g) the molds are heat treated to cure the material inside the oven, (h) the two parts (top and bottom layers of the actuator) are combined using the same approach, and (i) the final product (soft/flexible actuator).

Pneumatics control

The bending angle and the trajectory of the soft actuator are controlled using the pressure difference in internal cavities. This pressure difference in the internal cavities (see Figure 4: steps (h) and (i)) causes to expand the cavity structure. The forces generated during this process cause to bend the entire actuator due to the continuous strain limiting the bottom layer. Therefore, to control the bending angle of the actuator, a specifically designed pneumatic control system is used. The airflow is supplied using an air tank running through a miniature diaphragm pneumatic pump. The electro-pneumatic solenoid valves are used to control the airflow. A metal–oxide–semiconductor field-effect transistor (MOSFET) driver module is used to switch the valves according to the desired digital signal generated by the controller.

Teleoperation

The teleoperation is established over the Internet. MQTT server is used to establish communication between the master and the slave. The main intention of using MQTT is to reduce communication delays. MQTT shows a low data loss in transmission (see Table 1). Table 1 highlights the publication of IBM, Hypertext Transfer Protocol Secure (HTTPS), and MQTT protocol characteristics and transmission ability.

Table 1.

The details of the Hypertext Transfer Protocol Secure (HTTPS) and Message Queuing Telemetry Transport (MQTT) protocols via 3G and Wi-Fi connection.^19,20

Characteristics	3G		Wi-Fi
Characteristics	HTTPS	MQTT	HTTPS	MQTT
Receive messages
Messages/h	1708	160,278	3628	263,314
% Battery/h	18.43	16.13	3.45	4.23
% Battery/messages	0.01709	0.0001	0.00095	0.00002
Messages received (note the losses)	240/1024	1024/1024	524/1024	1024/1024
Send messages
Messages/h	1926	21,685	5229	23,184
% Battery/h	18.79	17.8	5.44	0.00016
% Battery/messages	0.00975	0.00082	0.00104

HTTPS: Hypertext Transfer Protocol Secure; MQTT: Message Queuing Telemetry Transport.

RTT (round-trip time) delay is mainly used in MQTT serves to determine the packet transmission delay. RTT is defined as the length of time taken to send a data packet and the length of time taken to acknowledge the received data packet. The delay between two network nodes is often asymmetric. However, this article has assumed that 50% of the RTT value is the average of forwarding and reverse delays. Therefore, this delay has been used as an approximation to the one-way delay (OWD).

Equation (1) is the probability mass function (PMF) of a random variable that can be expressed as a function of the network delay distribution. PMF is an absolute error (E) between smaller of the forward value and reverse value of OWDs, and their average (i.e. RTT/2). “a” and “b” are the forward and reverse edges, respectively.²¹ The accuracy of the estimation depends on the nature of delay distribution in both directions. If these delays in both directions become more symmetric, the accuracy increases. However, end-to-end delay and RTT delays can be calculated as equations (2) and (3)

\Pr (E = x) = \sum_{i = 1}^{\infty} f_{i} (a) \cdot f_{i} (b), x = 0

(1)

d_{end \cdot end} = N [d_{trans} + d_{prop} + d_{proc} + d_{queue}]

(2)

where d_end·end is the end-to-end delay, d_trans is the transmission delay, d_prop is the propagation delay, d_proc is the processing delay, d_queue is the queuing delay, and N is the number of links (number of routers − 1)

\begin{matrix} RTT delay time = average of forward delays \\ + average of reverse delays \end{matrix}

(3)

Mathematical model

This section explains the noise model, KF, and the delay calculation equation and techniques.

Noise

The noise $u (t)$ is the input to the deterministic plant, which gives an output $y (t)$ . A white noise signal $e (t)$ enters to the stochastic process and generates the additive noise signal $n (t)$ . The final measurement of $z (t)$ is a combination of all these factors. Analysis of the stochastic system required to extract the deterministic system from the measurement. This experimentation consists of a teleoperated gesture-controlled soft actuator. The experimentation shows that the noise model in this system should not be identical to Gaussian white noise, but this assumption of Gaussian noise had provided improved results as presented in section “Results.”

KF model

To minimize the elastic and pneumatic disturbances, this article introduces a KF with an optimal estimation to smooth the response. The present technique used stochastic difference equations as shown in equations (4) and (5)

x_{k} = A x_{k - 1} + B u_{k - 1} + w

(4)

y_{k} = C x_{k} + v

(5)

where $x = nx 1$ is the state vector, $y = mx 1$ is the measurement vector, $w = nx 1$ is the process noise system disturbance, and $n = mx 1$ is the measurement noise. Assuming the process and measurement noises to be white noise with probability density functions as, $p (w) N (0, Q)$ and $p (v) N (0, R)$ , where Q and R represent the process noise covariance and measurement noise covariance, respectively. Equations (4) and (5) show the model of the system. The constants are taken as $Q = 0.07$ —process noise and $R = 0.6304$ —sensor variations. The variation of DG is calculated as 0.6304 from the standard deviation of the data generated during experimentation. In this model, we have not considered the input. Therefore, B matrix is considered to be zero.

State and covariation predictions are

\hat{x} = A x_{k - 1}

(6)

P_{k} = A P_{k - 1} A^{T} + Q

(7)

The Kalman gain update

K_{k} = P_{k} H^{T} {(H P_{k} H^{T} + R)}^{- 1}

(8)

State estimation

{\hat{x}}_{k} = {\hat{x}}_{k} + K_{k} (Z_{k} - H {\hat{x}}_{k})

(9)

New covariance

P_{k} = (I - K_{k} H) P_{k}

(10)

where $Z_{k}$ is the 1 × 1 matrix including one observation values from DG (flex sensor/bending angle) and k is the instance of the system form number of k observations. Figure 5 shows the overall block diagram of the KF embed soft actuator system. KF has been established on the slave side before the pneumatic controller. The KF uses the bending angle feedback as the estimation.

Figure 5.

The block diagram of the implemented Kalman filter of the teleoperated system.

Total delay and disturbance calculation

Total time delay (TTD) is considered as shown in equation (11). Figure 6 shows that TTD is defined as the time difference between the desired signal (DG: recorded from the flex sensor placed on DG) and the output signal (soft actuator: recorded from the flex sensor placed on the soft actuator)

T_{TD} = T_{C} + T_{E} + μ

(11)

where T_TD is the total time delay, T_C is the delay in communication, T_E is the elastic saturation delays, and µ is the noises and other delays.

This time delay exists in both angle and force transmission. The disturbance is measured in terms of root-mean-square error (RMSE)

RMSE = \sqrt{\frac{1}{N} \sum_{n = 1}^{N} {(θ_{SA} [n] - θ_{DG} [n])}^{2}}

(12)

where θ_SA is the angle generated in soft actuator, θ_DG is the angle generated in data glove, and n is the sample number.

This experimentation is considered the delays and noise to be zero (µ = 0) and communication delay as symmetric in transmission and receiving (end to end = RTT/2). The simplified version of equation (11) is as equation (13)

T_{TD} \approx T_{C} + T_{E}

(13)

Therefore, T_TG is the total delay time (time in DG in certain angle − time in the soft actuator, in the same angle), T_C is the communication delay, that is, RTT/2, and $T_{E} / {T_{E}}^{'}$ is calculated from the finite element method (FEM) for the same pressure input or from the previous experiment.⁹

Figure 6.

The approach of the total time delay (TTD) calculation. The difference between data glove and the soft actuator is considered as TTD.

Experimental setup

Figure 7 shows the experimental setup of the system (the setup is similar to the schematic diagram explained in section “Methodology”). The master side DG is used to measure the finger movements of the operator during gripping action. The recorded data are transferred using the MQTT server to the slave side that is located remotely. A soft actuator is placed in the remote location that is used to mimic the operator’s finger movements to grip an identically similar spherical object. The bending angles of the remotely placed soft actuator are measured using a flex sensor and recorded on a computer. The experiments are carried out in three geographically distinct locations (shown in Figure 8), GL-J, GL-A, and GL-K that are respectively 402, 200, and 133 km apart from the base station. The remote server that is used for this experimentation is located 16,142 km away from the base station and the remote locations.

Figure 7.

The experimentation setup. (a) The data glove is used to track the finger movements of the human operator who grips the spherical object. (b) The remote login of the operator. The soft actuator is controlled from a remote location. (c) The actuator setup that is used to record the motion of the actuator.

Figure 8.

Map of the geographical location and their distances from the base station (based on Google Maps).

Results

The experiments were carried out with KF and without KF. Figure 9 shows a typical response of the soft actuator with KF and without KF. A similar procedure was used in all three distinct locations. Figure 10(a) shows the error variation with KF and without KF, and Figure 10(b) shows the time-varying Kalman gain of the system. The recorded RMSE before KF was 6.6944%, and after KF, it was improved to 5.4186%. Therefore, this technique has been able to reduce the error by 19.06% by introducing the KF. The experimentation also considered the time delay and its effects on the enhancement of soft actuator response. Table 2 illustrates the recorded RTT delay from each location. Table 3 presents the time delay effects due to the disturbances in each location before KF and after KF.

Figure 9.

An example of a recoded response. This illustrates a typical control signal, before KF, after KF, and ideal responses, during an experimentation cycle.

Figure 10.

The responses and the Kalman filter gain plot generated during an experimentation cycle: (a) before KF, after KF (w.r.t. control signal) versus time plot and (b) Kalman gain versus time plot.

Table 2.

The recorded RTT delays from the three distinct locations measured with respect to the central station.

Distance	Average RTT (ms)	Average end to end	$T_{TD}$ average (ms)
16,142 + GJ-J	710.2	355.1 (54.2%)	655
16,142 + GJ-A	688.23	344.11 (39.7%)	867
16,142 + GJ-K	712.51	356.25 (26.7%)	1333

RTT: round-trip time.

Table 3.

The recorded and calculated delay values.

Location	$T_{E}$ average (ms)	$T_{E}^{'}$ average (ms)	$μ = δ T$ average (ms)	$T_{TD}$ average (ms)	$T_{TD}$ average (ms) after KF
GJ-J	300 (48.5%)	281	19 (2.9%)	655, STD: 1.53	507, STD: 1.48
GJ-A	523 (60.3%)	518	5 (0.5%)	867, STD: 1.41	798, STD: 1.53
GJ-K	794 (59.5%)	767	27 (2.1%)	1333, STD: 1.65	1274, STD 1.61

$T_{E}$ : experimental elastic saturation delay; $T_{E}^{'}$ : FEA-based elastic saturation delay; $μ$ : noises and other delays; $T_{TD}$ : total time delay; STD: standard deviation.

Conclusion

Time delays, noise, and disturbances in bilateral systems with DG and soft actuators had negatively affected the establishment of a real-time, accurate, and repeatable control system. This article had investigated the delays and disturbances that occurred in these systems during remote teleoperation. The soft actuator showed a lag and sudden fluctuations during the normal control procedure. The investigation of this article had revealed the main problems that cause these issues are communication protocols, communication infrastructure, geographical/weather conditions, elastic saturation of the actuator, and problems in mechanical/electronic components. Most of these factors are hard to improve and expensive. However, the work presented in this article had investigated the potential use of a KF to solve these problems. The presented technique had reduced the unexpected fluctuations in the soft actuator, RMSE, and time lag. The KF smoothened the response of the soft actuator and reduced the RMSE of the system by approximately 15% (average of all three locations). Also, the delay time between the desired output and the actual output had been reduced by approximately 17% (average of all three locations). This system can be further improved by introducing a state estimator to the controller. Therefore, future research will be focused on implementing a dynamic model-based state estimation system for this application.

Footnotes

Acknowledgements

The authors thank all engineering trainees at the Soft Robotics Research Group of The Open University of Sri Lanka and also like to express their gratitude for WSO2 Pvt. Ltd, senior software engineer Eng. Charitha Gunathilaka for providing WSO2 server facilities.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was funded by the university research grant of The Open University of Sri Lanka.

ORCID iD

Palpolage Don Shehan Hiroshan Gunawardane

References

Levinson

. Teleoperation system and method for trajectory modification of autonomous vehicles. Patent US20200004241A1, USA, 2020.

Huang

Pham

Wang

, et al. A case study in human-robot collaboration in the disassembly of press-fitted components. Proc IMechE, Part B: J Engineering Manufacture 2020; 234(3): 654–664.

Gharaybeh

Chizeck

Stewart

. Telerobotic control in virtual reality. In: Proceedings of the oceans 2019 MTS/IEEE Seattle, Seattle, WA, 27–31 October 2019. New York: IEEE.

, et al. Motor imagery based continuous teleoperation robot control with tactile feedback. Electronics 2020; 9(1): 174.

Laso

Brosset

Giraud

M-A

. Defining role-based access control for a secure platform of unmanned surface vehicle fleets. In: Proceedings of the OCEANS 2019—Marseille, Marseille, 17–20 June 2019. New York: IEEE.

Karopadi

. Telemedicine system for remote monitoring and managing patients undergoing peritoneal dialysis and methods employed thereof. Patent US20200020445A1, USA, 2020.

Krishna Sharma

Woungang

Anpalagan

, et al. Towards tactile Internet in beyond 5G era: recent advances, current issues and future directions. IEEE Access 2020; 8: 56948–56991.

García

Cano

Contreras

. Digital retrofit: a first step toward the adoption of Industry 4.0 to the manufacturing systems of small and medium-sized enterprises. Proc IMechE, Part B: J Engineering Manufacture 2020; 234: 1156–1169.

Gunawardane

PDSH

Pallewela

REA

Medagedara

. Tele-operable controlling system for hand gesture controlled soft robot actuator. In: Proceedings of the 2019 2nd IEEE international conference on soft robotics (RoboSoft), Seoul, Korea, 14–18 April 2019. New York: IEEE.

10.

Shen

Pan

Y-J

. Improving tracking performance of nonlinear uncertain bilateral teleoperation systems with time-varying delays and disturbances. IEEE/ASME T Mech 2020; 25: 1171–1181.

11.

Brancadoro

Manti

Grani

, et al. Toward a variable stiffness surgical manipulator based on fiber jamming transition. Front Robot AI 2019; 6: 12.

12.

Anderson

Spong

. Bilateral control of teleoperators with time delay. IEEE T Autom Contr 1989; 34(5): 494–501.

13.

Ferraguti

Preda

Bonfè

, et al. Bilateral teleoperation of a dual arms surgical robot with passive virtual fixtures generation. In: Proceedings of the 2015 IEEE/RSJ international conference on intelligent robots and systems (IROS), Hamburg, 28 September–2 October 2015. New York: IEEE.

14.

Intutive (Da Vinci Robot), https://www.intuitive.com/

15.

Tobergte

Konietschke

Hirzinger

. Planning and control of a teleoperation system for research in minimally invasive robotic surgery. In: Proceedings of the 2009 IEEE international conference on robotics and automation, Kobe, Japan, 12–17 May 2009. New York: IEEE.

16.

Lee

Spong

. Passive bilateral teleoperation with constant time delay. IEEE T Robot 2006; 22(2): 269–281.

17.

Cho

Park

Kim

, et al. Sliding-mode-based impedance controller for bilateral teleoperation under varying time-delay. In: Proceedings of the IEEE international conference on robotics and automation, Seoul, South Korea, 21–26 May 2001. New York: IEEE.

18.

Ganjefar

Momeni

Janabi-Sharifi

. Teleoperation systems design using augmented wave-variables and Smith predictor method for reducing time-delay effects. In: Proceedings of the IEEE international symposium on intelligent control, Vancouver, BC, Canada, 30 October 2002. New York: IEEE.

19.

García-Valdovinos

L-G

Parra-Vega

Arteaga

. Observer-based sliding mode impedance control of bilateral teleoperation under constant unknown time delay. Robot Auto Syst 2007; 55(8): 609–617.

20.

Bejczy

Kim

Venema

. The phantom robot: predictive displays for teleoperation with time delay. In: Proceedings of the IEEE international conference on robotics and automation, Cincinnati, OH, 13–18 May 2002. New York: IEEE.

21.

Shukl

Karki

. Application of robotics in offshore oil and gas industry: a review: part II. Robot Auto Syst 2016; 75(B): 508–524.

22.

Mueller

Van Susante

. A review of extra-terrestrial mining robot concepts. https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20120008777.pdf

23.

Tsitsimpelis

Taylor

Lennox

, et al. A review of ground-based robotic systems for the characterization of nuclear environments. Prog Nucl Energ 2019; 111: 109–124.

24.

Albu-Schaffer

Eiberger

Grebenstein

, et al. Soft robotics. IEEE Robot Autom Mag 2008; 15(3): 20–30.

25.

Lichiardopo

. A survey on teleoperation. Eindhoven: Dynamics and Control Group, Technische Universiteit Eindhoven, 2007, https://pure.tue.nl/ws/files/4419568/656592.pdf

26.

Lee

Kim

, et al. Soft robot review. Int J Contr Autom Syst 2017; 15(1): 3–15.

27.

Barber

. Pneumatic handbook. Amsterdam: Elsevier, 1997.

28.

Roumeliotis

Bekey

. Collective localization: a distributed Kalman filter approach to localization of groups of mobile robots. In: Proceedings 2000 ICRA: millennium conference: IEEE international conference on robotics and automation, San Francisco, CA, 24–28 April 2000. New York: IEEE.

29.

Chen

. Kalman filter for robot vision: a survey. IEEE T Ind Electr 2012; 59(11): 4409–4420.

30.

Hannaford

. Gaussian process regression for sensorless grip force estimation of cable-driven elongated surgical instruments. IEEE Robot Autom Lett 2017; 2(3): 1312–1319.

Additive Gaussian noise model and Kalman filter to enhance controllability of gesture-controlled teleoperated soft actuators