Markov game based controllers are robust but lack guarantee on the stability of the designed controller. In this work, we attempt to address this shortcoming by proposing a lyapunov fuzzy Markov game controller for safe and stable tracking control of two link robotic manipulators. Lyapunov theory has been used to generate fuzzy linguistic rules for implementing a reinforcement learning (RL) based Markov game controller. We employ fuzzy inference system as a generic function approximator to deal with the “curse of dimensionality” issue. Proposed RL based Markov game controller is self-learning, adaptive and optimal. We implement the proposed control paradigm on: a) Two link robot manipulator and b) SCARA manipulator for the cases: i) controller handles disturbances and parameter variations, and ii) disturbances and no parameter variations. We give comparative evaluation of our approach against: a) fuzzy Q learning controller, and b) fuzzy Markov game controller. Simulation results illustrate stable and superior tracking performance and advantage in terms of lower control torque requirements.
DixonW.E., BehalA., DawsonD.M. and NagarkattiS.P., Nonlinear control of engineering systems: A Lyapunov-based approach: Springer Science & Business Media; 2013.
2.
FradkovA.L.V., MiroshnikI.I.A.V.E. and NikiforovV.O., Nonlinear and adaptive control of complex systems: Springer Science & Business Media; 2013.
3.
KumarV. and RanaK., Nonlinear adaptive fractional order fuzzy PID control of a 2-link planar rigid manipulator with payload, Journal of the Franklin Institute354(2) (2017), 993–1022.
4.
KoberJ., BagnellJ.A. and PetersJ., Reinforcement learning in robotics: A survey, The International Journal of Robotics Research32(11) (2013), 1238–1274.
5.
HeW., DavidA.O., YinZ. and SunC., Neural network control of a robotic manipulator with input deadzone and output constraint, IEEE Transactions on Systems, Man, and Cybernetics: Systems46(6) (2016), 759–770.
6.
HeW., OuyangY. and HongJ., Vibration control of a flexible robotic manipulator in the presence of input deadzone, IEEE Transactions on Industrial Informatics13(1) (2017), 48–59.
7.
MahilS.M. and Al-DurraA., Modeling analysis and simulation of 2-DOF robotic manipulator, IEEE 59th International Midwest Symposium Circuits and Systems (MWSCAS), pp. 1–4, 2016.
8.
WangM. and YangA., Dynamic learning from adaptive neural control of robot manipulators with prescribed performance, IEEE Transactions on Systems, Man, and Cybernetics: SystemsPP(99) (2017), 1–12.
9.
XiaoB., YinS. and KaynakO., Tracking control of robotic manipulators with uncertain kinematics and dynamics, IEEE Transactions on Industrial Electronics63(10) (2016), 6439–6449.
10.
LewisF.L. and VrabieD., Reinforcement learning and adaptive dynamic programming for feedback control, IEEE Circuits and Systems Magazine9(3) (2009), 32–50.
11.
KhanS.G., HerrmannG., LewisF.L., PipeT. and MelhuishC., Reinforcement learning and optimal adaptive control: An overview and implementation examples, Annual Reviews in Control36(1) (2012), 42–59.
12.
OuyangY., HeW. and LiX., Reinforcement learning control of a single-link flexible robotic manipulator, IET Control Theory & Applications11(9) (2017), 1426–1433.
13.
TangL.,
LiuY.-J. and TongS., Adaptive neural control using reinforcement learning for a class of robot manipulator, Neural Computing and Applications25(1) (2014), 135–141.
14.
YousefianR. and KamalasadanS., Design and real-time implementation of optimal power system wide-area system-centric controller based on temporal difference learning, IEEE Transactions on Industry Applications52(1) (2016), 395–406.
15.
ZhaoB., WangD., ShiG., LiuD. and LiY., Decentralized Control for Large-Scale Nonlinear Systems With Unknown Mismatched Interconnections via Policy Iteration, IEEE Transactions on Systems, Man, and Cybernetics: SystemsPP(99) (2017), 1–11.
16.
YangE. and GuD., Multiagent reinforcement learning for multi-robot systems: A survey, tech. Rep; 2004.
17.
ShahH. and GopalM., Kernel recursive least squares function approximation in game theory based control, Procedia Technology23 (2016), 264–271.
18.
BusoniuL., BabuskaR. and De SchutterB., Multi-agent reinforcement learning: A survey, IEEE 9th International Conference on Control, Automation, Robotics and Vision (ICARCV) (2006), 1–6.
19.
HeW., DongY. and SunC., Adaptive neural impedance control of a robotic manipulator with input saturation, IEEE Transactions on Systems, Man, and Cybernetics: Systems46(3) (2016), 334–344.
20.
SharmaR. and GopalM., A markov game-adaptive fuzzy controller for robot manipulators, IEEE Transactions on Fuzzy Systems16(1) (2008), 171–186.
21.
KumarA. and SharmaR., Lyapunov theory based intelligent fuzzy controller for Inverted Pendulum, IEEE International Conference on Power Electronics, Intelligent Control and Energy Systems (ICPEICES) (2016), 1–5.
22.
KumarA. and SharmaR., Fuzzy lyapunov reinforcement learning for non linear systems, ISA Transactions67 (2017), 151–159.
23.
MargaliotM. and LangholzG., Fuzzy control of a benchmark problem: A computing with words approach, IEEE Transactions on Fuzzy Systems12(2) (2004), 230–235.
24.
SaghafiniaA., PingH.W., UddinM.N. and GaeidK.S., Adaptive fuzzy sliding-mode control into chattering-free IM drive, IEEE Transactions on Industry Applications51(1) (2015), 692–701.
25.
OhS. and KongK., Two-degree-of-freedom control of a two-link manipulator in the rotating coordinate system, IEEE Transactions on Industrial Electronics62(9) (2015), 5598–5607.
26.
MartiranoL., PariseG., PariseL. and ManganelliM., A fuzzy-based building automation control system: Optimizing the level of energy performance and comfort in an office space by taking advantage of building automation systems and solar energy, IEEE Industry Applications Magazine22(2) (2016), 10–17.
