In this article, we propose a cascade control framework to attenuate the residual vibration of the underactuated manipulator. The control framework is divided into two phases. In the first phase, a path generator trained by the reinforcement learning produces the leading signal for the tracking controller. In the second phase, the leading signal stabilizes the underactuated manipulator, and the adaptive proportional derivative controller is implemented to reduce the vibration. In the process, a novel path planning method is proposed to improve exploration efficiency, and a negative reward is introduced to avoid unsafe strategies and simulation instability. The effectiveness of the proposed control scheme is verified in the simulations of the double pendulum crane and the two-link flexible manipulator.
WuJSheJWangY, et al. Position and posture control of planar four-link underactuated manipulator based on neural network model. IEEE Trans Ind Electron2020; 67(6): 4721–4728.
2.
HeWYanZSunY, et al. Neural-learning-based control for a constrained robotic manipulator with flexible joints. IEEE Trans Neural Networks Learn Syst2018; 29(12): 5993–6003.
3.
SunCGaoHHeW, et al. Fuzzy neural network control of a flexible robotic manipulator using assumed mode method. IEEE Trans Neural Networks Learn Syst2018; 29(11): 5214–5227.
4.
ColeMOWongratanaphisanT. A direct method of adaptive FIR input shaping for motion control with zero residual vibration. IEEE/ASME Trans Mechatronics2013; 18(1): 316–327.
5.
BarbazzaLZanottoDRosatiG, et al. Design and optimal control of an underactuated cable-driven micro-macro robot. IEEE Robot Autom Lett2017; 2(2): 896–903.
6.
IdàEBruckmannTCarricatoM, et al. Rest-to-rest trajectory planning for underactuated cable-driven parallel robots. IEEE Trans Robot2019; 35(6): 1338–1351.
7.
SingerNCSeeringWP. Preshaping command inputs to reduce system vibration. J Dyn Syst Meas Control1990; 112(1): 76–82.
8.
RamliLMohamedZJaafarHI. A neural network-based input shaping for swing suppression of an overhead crane under payload hoisting and mass variations. Mech Syst Signal Process2018; 107: 484–501.
9.
XuJFangHZhouT, et al. Optimal robust position control with input shaping for flexible solar array drive system: a fuzzy-set theoretic approach. IEEE Trans Fuzzy Syst2019; 27(9): 1807–1817.
10.
GhorbaniHAlipourKTarvirdizadehB, et al. Comparison of various input shaping methods in rest-to-rest motion of the end-effecter of a rigid-flexible robotic system with large deformations capability. Mech Syst Signal Process2019; 118: 584–602.
11.
HullDG. Conversion of optimal control problems into parameter optimization problems. J Guid Control Dyn1997; 20(1): 57–60.
12.
FanCLiaoY. Robust optimal motion planning for vibration reduction of free-floating flexible manipulator system. In: 2016 IEEE international conference on real-time computing and robotics (RCAR), Angkor Wat, Cambodia, 6–10 June 2016, vol. 3, pp.489–494. New York: IEEE.
13.
LismondeASonnevilleVBrülsO. A geometric optimization method for the trajectory planning of flexible manipulators. Multibody Syst Dyn2019; 47(4): 347–362.
14.
AbeA. Trajectory planning for flexible cartesian robot manipulator by using artificial neural network: numerical simulation and experimental verification. Robotica2011; 29(5): 797–804.
15.
CaoBSunKLiT, et al. Trajectory modified in joint space for vibration suppression of manipulator. IEEE Access2018; 6(c): 57969–57980.
16.
GamkrelidzeRV. Discovery of the maximum principle. J Dyn Control Syst1999; 5(4): 437–451.
17.
The VuMChoiHSKangJ, et al. A study on hovering motion of the underwater vehicle with umbilical cable. Ocean Eng2017; 135: 137–157.
18.
ZengXLiuX. Searching for time optimal periodic orbits near irregularly shaped asteroids by using an indirect method. IEEE Trans Aerosp Electron Syst2017; 53(3): 1221–1229.
19.
BoscariolPRichiedeiD. Robust point-to-point trajectory planning for nonlinear underactuated systems: theory and experimental assessment. Robot Comput Integr Manuf2018; 50: 256–265.
20.
Al KhudirKDe LucaA. Faster motion on cartesian paths exploiting robot redundancy at the acceleration level. IEEE Robot Autom Lett2018; 3(4): 3553–3560.
21.
WangZZhangSFengX, et al. Autonomous underwater vehicle path planning based on actor-multi-critic reinforcement learning. Proc IMechE Part I: J Syst Contr Eng2021; 235(10): 1787–1796.
22.
SongQYeDSunZ, et al. Autonomous reconfiguration of homogeneous pivoting cube modular satellite by deep reinforcement learning. Proc IMechE Part I: J Syst Contr Eng2021; 235(10): 1777–1786.
23.
JiGYanJDuJ, et al. Towards safe control of continuum manipulator using shielded multiagent reinforcement learning. IEEE Robot and Autom Lett2021; 6(4): 7461–7468.
24.
JiangHWangZJinY, et al. Hierarchical control of soft manipulators towards unstructured interactions. Int J Robot Res2021; 40(1): 411–434.
25.
LiSHeJLiY, et al. Distributed recurrent neural networks for cooperative control of manipulators: a game-theoretic perspective. IEEE Trans Neural Networks Learn Syst2017; 28(2): 415–426.
26.
ZhangJJinLChengL. RNN for perturbed manipulability optimization of manipulators based on a distributed scheme: a game-theoretic perspective. IEEE Trans Neural Networks Learn Syst2020; 31(12): 5116–5126.
27.
SuttonRSBartoAG. Reinforcement learning: an introduction. Cambridge, MA: A Bradford Book, 2018.
28.
ArulkumaranKDeisenrothMPBrundageM, et al. Deep reinforcement learning: a brief survey. IEEE Signal Process Mag2017; 34(6): 26–38.
SuttonRSMcAllesterDSinghS, et al. Policy gradient methods for reinforcement learning with function approximation. Adv Neural Inf Process Syst2000; 2000: 1057–1063.
31.
SilverDLeverGHeessN, et al. Deterministic policy gradient algorithms. In: International conference on machine learning, 2014, vol. 32, pp.605–619, http://proceedings.mlr.press/v32/silver14.pdf
32.
MnihVKavukcuogluKSilverD, et al. Human-level control through deep reinforcement learning. Nature2015; 518(7540): 529–533.
33.
LillicrapTPHuntJJPritzelA, et al. Continuous control with deep reinforcement learning. Arxiv, 2015, pp.1–14, https://arxiv.org/abs/1509.02971.
LiangDSongYQiY, et al. Efficient modeling and integrated control for tracking and vibration of a lightweight parallel manipulator including servo motor dynamics. Mech Syst Signal Proces2021; 153(4): 107502.
36.
LongTLiEHuY, et al. A vibration control method for hybrid-structured flexible manipulator based on sliding mode control and reinforcement learning. IEEE Trans Neural Networks Learn Syst2021; 32: 841–852.
37.
NorouziAGordonDAliramezaniM, et al. Machine learning-based diesel engine-out NOx reduction using a plug-in PD-type iterative learning control. In: IEEE CCTA, Montreal, QC, Canada, 24–26 August 2020. New York: IEEE.
38.
NorouziAKochCR. Robotic manipulator control using PD-type fuzzy iterative learning control. In: IEEE CCECE, Edmonton, AB, Canada, 5–8 May 2019. New York: IEEE.
39.
CongWZhengMWangZ, et al. Robust iterative learning control for vibration suppression of industrial robot manipulators. J Dyn Syst Meas Control2018; 140(1): 011003.
40.
NorouziAKochCR. Integration of PD-type iterative learning control with adaptive sliding mode control. IFAC Proc2020; 53: 6213–6218.
41.
HeWMengTHuangD, et al. Adaptive boundary iterative learning control for an Euler-Bernoulli beam system with input constraint. IEEE Trans Neural Networks Learn Syst2018; 29(5): 1539–1549.
42.
ZhiCChengLXianM. Experimental study on active vibration control for a kind of two-link flexible manipulator. Mech Syst Signal Process2019; 118: 623–644.
43.
PradhanSKSubudhiB. Position control of a flexible manipulator using a new nonlinear self-tuning PID controller. IEEE/CAA J Autom Sin2020; 7(1): 136–149.
44.
PradhanSKSubudhiB. Real-time adaptive control of a flexible manipulator using reinforcement learning. IEEE Trans Autom Sci Eng2012; 9(2): 237–249.
45.
SpongMWVidyasagarM. Robot dynamics and control. New York: John Wiley & Sons, 2008.
46.
MacfarlaneSCroftEA. Jerk-bounded manipulator trajectory planning: design for real-time applications. IEEE Trans Robot Autom2003; 19(1): 42–52.
47.
TheodoreRJGhosalA. Comparison of the assumed modes and finite element models for flexible multilink manipulators. Int J Rob Res1995; 14(2): 91–111.
48.
HeKZhangXRenS, et al. Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. In: International conference on computer vision, 2015, pp.1026–1034. DOI: 10.1109/ICCV.2015.123.
49.
GlorotXBengioY. Understanding the difficulty of training deep feedforward neural networks. In: International conference on artificial intelligence and statistics, 2010, vol. 9, pp.249–256, https://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf
50.
SchulmanJMoritzPLevineS, et al. High-dimensional continuous control using generalized advantage estimation. Arxiv, 2015, pp.1-14, https://arxiv.org/abs/1506.02438.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
