Minimally invasive surgery requires the inclusion of cable-driven manipulators, due to the advantages in terms of lower inertia, compact design, and remote actuation. However, the nonlinear factors underlying in a cable-driven system bring a challenge for the motion control of the end-effectors of a surgical robot. Further, conventional data-driven approaches for cable-driven systems fail to jointly consider local and temporally sequential information in control signals. To this end, we propose a motion-estimation approach for cable-driven end-effectors, using 1D-convolution and recurrent neural networks with attention, taking into account local and sequential information in modelling control series. Then, we perform cross-domain experiments on our Hefei University of Technology’s Cable-Driven End-effector’s Motion (HFUT-CDEM) dataset, with the experimental results showing that the proposed approach achieves better performance, compared with conventional regression approaches for sequential-data modelling.
GuoYYangYFengM, et al. Review on development status and key technologies of surgical robots. In: Proceedings of the 2020 IEEE international conference on mechatronics and automation (ICMA), Virtual Event, pp.105–110. New York: IEEE.
2.
WangZBaoSWangD, et al. Design of a novel flexible robotic laparoscope using a two degrees-of-freedom cable-driven continuum mechanism with major arc notches. J Mech Robot2023; 15(6): 064502.
3.
HaghighipanahMMiyasakaMHannafordB.Utilizing elasticity of cable-driven surgical robot to estimate cable tension and external force. IEEE Robot Autom Lett2017; 2(3): 1593–1600.
4.
OmisoreOMHanSXiongJ, et al. A review on flexible robotic systems for minimally invasive surgery. IEEE Trans Syst Man Cybern Syst2022; 52(1): 631–644.
5.
WangWWangJChenC, et al. Design of position estimator for rope driven micromanipulator of surgical robot based on parameter autonomous selection model. J Mech Robot2024; 16(4): 041005.
6.
Correa-GallegoCDinkelspielHESulimanoffI, et al. Minimally-invasive vs open pancreaticoduodenectomy: systematic review and meta-analysis. J Am Coll Surg2014; 218(1): 129–139.
7.
SuHMarianiAOvurSE, et al. Toward teaching by demonstration for robot-assisted minimally invasive surgery. IEEE Trans Autom Sci Eng2021; 18(2): 484–494.
8.
PengHYangXSuY, et al. Real-time data driven precision estimator for RAVEN-II surgical robot end effector position. In: Proceedings of the IEEE international conference on robotics and automation (ICRA), Virtual Event, pp.350–356. New York: IEEE.
9.
ThaiMTPhanPTHoangTT, et al. Advanced intelligent systems for surgical robotics. Adv Intell Syst2020; 2(8): 1900138.
10.
HaghighipanahMMiyasakaMLiY, et al. Unscented Kalman filter and 3D vision to improve cable driven surgical robot joint angle estimation. In: Proceedings of the 2016 IEEE international conference on robotics and automation (ICRA), Stockholm, Sweden, pp.4135–4142. New York: IEEE.
11.
ZhuJLyuLXuY, et al. Intelligent soft surgical robots for next-generation minimally invasive surgery. Adv Intell Syst2021; 3(5): 2100011.
12.
WangYCaoQZhuX, et al. A cable-driven distal end-effector mechanism for single-port robotic surgery. Int J Comput Assist Radiol Surg2021; 16(2): 301–309.
13.
PedramSAShinCFergusonPW, et al. Autonomous suturing framework and quantification using a cable-driven surgical robot. IEEE Trans Robot2021; 37(2): 404–417.
14.
HwangMThananjeyanBParadisS, et al. Efficiently calibrating cable-driven surgical robots with RGBD fiducial sensing and recurrent neural networks. IEEE Robot Autom Lett2020; 5(4): 5937–5944.
15.
KangJMChoiSHParkJW, et al. Position error prediction using hybrid recurrent neural network algorithm for improvement of pose accuracy of cable driven parallel robots. Microsyst Technol2020; 26(1): 209–218.
16.
HuangYYanLYangT, et al. Sensing design, trajectory planning, and motion control of a cable-driven redundant manipulator composed of quaternion joints. J Mech Robot2023; 15(5): 055001.
17.
GaoJZhouBZiB, et al. Kinematic uncertainty analysis of a cable-driven parallel robot based on an error transfer model. J Mech Robot2022; 14(5): 051008.
18.
BakloutiSCourteilleECaroS, et al. Dynamic and oscillatory motions of cable-driven parallel robots based on a nonlinear cable tension model. J Mech Robot2017; 9(6): 061014.
19.
WangZBaoSZiB, et al. Development of a novel 4-DOF flexible endoscopic robot using cable-driven multisegment continuum mechanisms. J Mech Robot2024; 16(3): 031011.
20.
WangYWangKZhangZ, et al. Control strategy and experimental research of a cable-driven lower limb rehabilitation robot. Proc IMechE, Part C: J Mechanical Engineering Science2021; 235(13): 2468–2481.
21.
XueRRenBYanZ, et al. A cable-pulley system modeling based position compensation control for a laparoscope surgical robot. Mech Mach Theory2017; 118: 283–299.
22.
YuXLiuGWangZ, et al. Full-closed loop tracking control based on multi-factor coupling compensations using artificial neural network for a cable-pulley-driven surgical robotic manipulator. In: Proceedings of the 2022 USCToMM symposium on mechanical systems and robotics, Rapid City, SD, pp.43–53. Cham: Springer.
23.
MiyasakaMHaghighipanahMLiY, et al. Modeling cable-driven robot with hysteresis and cable–pulley network friction. IEEE/ASME Trans Mechatron2020; 25(2): 1095–1104.
24.
WuXHongDChanussotJ.Convolutional neural networks for multimodal remote sensing data classification. IEEE Trans Geosci Remote Sens2022; 60: 1–10.
25.
HongDYaoJLiC, et al. Decoupled-and-coupled networks: self-supervised hyperspectral image super-resolution with subpixel fusion. IEEE Trans Geosci Remote Sens2023; 61: 1–12.
26.
WuXHongDChanussotJ.UIU-Net: U-net in U-net for infrared small object detection. IEEE Trans Image Process2022; PP: 364–376.
27.
XuWChenJLauHYK, et al. Data-driven methods towards learning the highly nonlinear inverse kinematics of tendon-driven surgical manipulators. Int J Med Robot Comput Assist Surg2017; 13(3): e1774.
28.
ShenWYangGZhengT, et al. An accuracy enhancement method for a cable-driven continuum robot with a flexible backbone. IEEE Access2020; 8: 37474–37481.
29.
LiSShaoZGuanY.A dynamic neural network approach for efficient control of manipulators. IEEE Trans Syst Man Cybern Syst2019; 49(5): 932–941.
30.
LiSZhangYJinL.Kinematic control of redundant manipulators using neural networks. IEEE Trans Neural Netw Learn Syst2017; 28(10): 2243–2254.
31.
ZhangYLiSZhouX.Recurrent-neural-network-based velocity-level redundancy resolution for manipulators subject to a joint acceleration limit. IEEE Trans Ind Electron2019; 66(5): 3573–3582.
32.
HuZHuYLiuJ, et al. A CRNN module for hand pose estimation. Neurocomputing2019; 333: 157–168.
33.
O’SheaKNashR.An introduction to convolutional neural networks. arXiv preprint arXiv:151108458 2015.
34.
HeYSainathTPrabhavalkarR, et al. Streaming end-to-end speech recognition for mobile devices. In: Proceedings of the 2019 IEEE international conference on acoustics, speech and signal processing (ICASSP), Brighton, UK, pp.6381–6385. New York: IEEE.
35.
JelodarHWangYOrjiR, et al. Deep sentiment classification and topic discovery on novel coronavirus or COVID-19 online discussions: NLP using LSTM recurrent neural network approach. IEEE J Biomed Health Inform2020; 24(10): 2733–2742.
36.
MaoSSejdićE.A review of recurrent neural network-based methods in computational physiology. IEEE Trans Neural Netw Learn Syst2023; 34(10): 6983–7003.
37.
YinWKannKYuM, et al. Comparative study of CNN and RNN for natural language processing. arXiv preprint arXiv:170201923 2017.
38.
SakHSeniorABeaufaysF.Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition. arXiv preprint arXiv:14021128 2014
39.
KumarR.Memory recurrent Elman neural network-based identification of time-delayed nonlinear dynamical system. IEEE Trans Syst Man Cybern Syst2023; 53(2): 753–762.
40.
KumarR.A Lyapunov-stability-based context-layered recurrent pi-sigma neural network for the identification of nonlinear systems. Appl Soft Comput2022; 122: 108836.
41.
KumarRSrivastavaS.Externally recurrent neural network based identification of dynamic systems using Lyapunov stability analysis. ISA Trans2020; 98: 292–308.
42.
KumarRSrivastavaSGuptaJRP, et al. Temporally local recurrent radial basis function network for modeling and adaptive control of nonlinear systems. ISA Trans2019; 87: 88–115.
43.
ChungJGulcehreCChoK, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling. In: Proceedings of the annual conference on neural information processing systems (NIPS) 2014 workshop on deep learning.
44.
ZhaoRWangDYanR, et al. Machine health monitoring using local feature-based gated recurrent unit networks. IEEE Trans Ind Electron2018; 65(2): 1539–1548.
45.
ZhuJJiangQShenY, et al. Application of recurrent neural network to mechanical fault diagnosis: a review. J Mech Sci Technol2022; 36(2): 527–542.
46.
JiangPXuXTaoH, et al. Convolutional-recurrent neural networks with multiple attention mechanisms for speech emotion recognition. IEEE Trans Cogn Dev Syst2022; 14(4): 1564–1573.
47.
BrauwersGFrasincarF.A general survey on attention mechanisms in deep learning. IEEE Trans Knowl Data Eng2023; 35(4): 3279–3298.
48.
VaswaniAShazeerNParmarN, et al. Attention is all you need. In: Proceedings of the conference on neural information processing systems (NIPS)2017.
49.
WangYYangGLiS, et al. Arrhythmia classification algorithm based on multi-head self-attention mechanism. Biomed Signal Process Control2023; 79: 104206.
50.
DingYJiaMMiaoQ, et al. A novel time–frequency transformer based on self–attention mechanism and its application in fault diagnosis of rolling bearings. Mech Syst Signal Process2022; 168: 108616.
51.
LeiXXiaYWangA, et al. Mutual information based anomaly detection of monitoring data with attention mechanism and residual learning. Mech Syst Signal Process2023; 182: 109607.
52.
WenQZhouTZhangC, et al. Transformers in time series: a survey. arXiv preprint arXiv:2202071252022.
53.
ShiXQiHShenY, et al. A spatial–temporal attention approach for traffic prediction. IEEE Trans Intell Transp Syst2021; 22(8): 4909–4918.
54.
LiuYLiuJZhuX, et al. Learning task-specific representation for video anomaly detection with spatial-temporal attention. In: Proceedings of the 2022 IEEE international conference on acoustics, speech and signal processing (ICASSP), Singapore, pp.2190–2194. New York: IEEE.
55.
ZhangDYaoLChenK, et al. A convolutional recurrent attention model for subject-independent EEG signal analysis. IEEE Signal Process Lett2019; 26(5): 715–719.
56.
XiaXPanJWangY. Audio sound determination using feature space attention based convolution recurrent neural network. In: Proceedings of the 2020 IEEE international conference on acoustics, speech and signal processing (ICASSP), Virtual Event, pp.3382–3386. New York: IEEE.
57.
WangZLiuGQianS, et al. Tracking control with external force self-sensing ability based on position/force estimators and non-linear hysteresis compensation for a backdrivable cable-pulley-driven surgical robotic manipulator. Mech Mach Theory2023; 183: 105259.
58.
KurasumiTChoSGDingM, et al. Simultaneous estimation of upper limb pose and joint torque based on upper arm deformation. IEEE Trans Med Robot Bionics2020; 2(3): 374–381.
59.
SunXQiuJTaoY, et al. Data-driven combined central and distributed Volt/Var control in active distribution networks. IEEE Trans Smart Grid2023; 14(3): 1855–1867.
60.
SuHQiWLiZ, et al. Deep neural network approach in EMG-based force estimation for human–robot interaction. IEEE Trans Artif Intell2021; 2(5): 404–412.
61.
O’NeillJJStephensTKKowalewskiTM.Evaluation of torque measurement surrogates as applied to grip torque and jaw angle estimation of robotic surgical tools. IEEE Robot Autom Lett2018; 3(4): 3027–3034.
