This paper considers solving the zero-sum games (ZSGs) of nonlinear stochastic systems under the architecture of dynamic event-triggered mechanism (DETM). The adaptive dynamic programming (ADP) approach with great approximate computing power is proposed to seek the saddle-point equilibrium solution of ZSGs. First, the Hamilton–Jacobi–Isaacs equation of stochastic version is constructed which is the key to solving the two-player ZSGs. Then, the ADP method is employed to search the approximated solution under the dynamic triggering mechanism. On the foundation of static event-triggered mechanism (SETM), the DETM is derived through introducing an internal dynamic invariable to further lower the update frequency for the strategies. Additionally, the stability of the closed system is demonstrated in detail via the Lyapunov theory. Finally, two representative examples are provided to present the availability and reliability of this approach from different perspectives.
AsgharniaASchwartzHAtiaM (2022) Learning multi-objective deception in a two-player differential game using reinforcement learning and multi-objective genetic algorithm. International Journal of Innovative Computing, Information and Control18(6): 1667–1688.
2.
BasarTOlsderGJ (1999) Dynamic Noncooperative Game Theory. Philadelphia, PA: Society for Industrial and Applied Mathematics.
3.
BellmanRE (1957) Dynamic Programming. Princeton, NJ: Princeton University Press.
4.
DengCZhangDFengG (2022) Resilient practical cooperative output regulation for MASs with unknown switching exosystem dynamics under DoS attacks. Automatica139: 110172.
5.
DharNKVermaNKBeheraL (2020) An online event-triggered near-optimal controller for Nash solution in interconnected system. IEEE Transactions on Neural Networks and Learning Systems31(12): 5534–5548.
6.
FairbankMAlonsoEProkhorovD (2012) Simple and fast calculation of the second-order gradients for globalized dual heuristic dynamic programming in neural networks. IEEE Transactions on Neural Networks and Learning Systems23(10): 1671–1676.
7.
FairbankMAlonsoEProkhorovD (2013) An equivalence between adaptive dynamic programming with a critic and backpropagation through time. IEEE Transactions on Neural Networks and Learning Systems24(12): 2088–2100.
8.
FengYAndersonBDO (2010) An iterative algorithm to solve state-perturbed stochastic algebraic Riccati equations in LQ zero-sum games. Systems & Control Letters59(1): 50–56.
9.
GaoCZhangWXuD, et al. (2022) Event-triggered based model-free adaptive sliding mode constrained control for nonlinear discrete-time systems. International Journal of Innovative Computing, Information and Control18(2): 525–536.
10.
GirardA (2015) Dynamic triggering mechanisms for event-triggered control. IEEE Transactions on Automatic Control60(7): 1992–1997.
11.
GuoHLiuJAhnCK, et al. (2022) Dynamic event-triggered impulsive control for stochastic nonlinear systems with extension in complex networks. IEEE Transactions on Circuits and Systems I: Regular Papers69(5): 2167–2178.
12.
HarmonMEBairdLCKlopfAH (1995) Reinforcement learning applied to a differential game. Adaptive Behavior4(1): 3–28.
13.
KhalilHK (2002) Nonlinear Systems. Upper Saddle River, NJ: Prentice Hall.
14.
KimYHLewisFLAbdallahCT (1997) A dynamic recurrent neural-network-based adaptive observer for a class of nonlinear systems. Automatica33(8): 1539–1543.
15.
LewisFLSyrmosVL (1995) Optimal Control. New York: John Wiley & Sons.
16.
LewisFLVrabieD (2009) Reinforcement learning and adaptive dynamic programming for feedback control. IEEE Circuits and Systems Magazine9(3): 32–50.
17.
LiYYangTTongS (2020) Adaptive neural networks finite-time optimal control for a class of nonlinear systems. IEEE Transactions on Neural Networks and Learning Systems31(11): 4451–4460.
18.
LiangMWangYLiuD (2022) An efficient impulsive adaptive dynamic programming algorithm for stochastic systems. IEEE Transactions on Cybernetics. Epub ahead of print 5April. DOI: 10.1109/TCYB.2022.3158898.
19.
LiuDRLiHL (2012) Optimal control for discrete-time affine non-linear systems using general value iteration. IET Control Theory & Applications6(18): 2725–2736.
20.
LiuPZhangHSuH, et al. (2021) Online event-based adaptive critic design with experience replay to solve partially unknown multi-player nonzero-sum games. Neurocomputing458: 219–231.
21.
LiuPZhangHSunJ, et al. (2022) Event-triggered adaptive integral reinforcement learning method for zero-sum differential games of nonlinear systems with incomplete known dynamics. Neural Computing & Applications34: 10775–10786.
22.
LiuYHCaoGZTangSX, et al. (2018) Energy-based stabilisation and robust stabilisation of stochastic non-linear systems. IET Control Theory & Applications12(2): 318–325.
23.
LuoSDengF (2020) On event-triggered control of nonlinear stochastic systems. IEEE Transactions on Automatic Control65(1): 369–375.
24.
MingZZhangHLiW, et al. (2022) Neurodynamic programming and tracking control for nonlinear stochastic systems by PI algorithm. IEEE Transactions on Circuits and Systems II: Express Briefs69(6): 2892–2896.
25.
ModaresHLewisFL (2014) Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning. Automatica50(7): 1780–1792.
26.
MuCWangDSunC, et al. (2017) Robust adaptive critic control design with network-based event-triggered formulation. Nonlinear Dynamics90(3): 2023–2035.
27.
MuCWangKQiuT (2022) Dynamic event-triggering neural learning control for partially unknown nonlinear systems. IEEE Transactions on Cybernetics52(4): 2200–2213.
28.
NiJShiPZhaoY, et al. (2022) Fixed-time event-triggered output consensus tracking of high-order multi-agent systems under directed interaction graphs. IEEE Transactions on Cybernetics52(7): 6391–6405.
29.
NiZHeHWenJ, et al. (2013) Goal representation heuristic dynamic programming on maze navigation. IEEE Transactions on Neural Networks and Learning Systems24(12): 2038–2050.
30.
NieRHeWDuW, et al. (2022) Dynamic event-triggered SMC of multi-agent systems for consensus tracking. IEEE Transactions on Circuits and Systems II: Express Briefs69(3): 1188–1192.
ShiTShiPWuZG (2021) Dynamic event-triggered asynchronous MPC of Markovian jump systems with disturbances. IEEE Transactions on Cybernetics52: 11639–11648.
33.
SongRLiJLewisFL (2020) Robust optimal control for disturbed nonlinear zero-sum differential games based on single NN and least squares. IEEE Transactions on Systems, Man, and Cybernetics: Systems50(11): 4009–4019.
34.
TaoJXiaoZWeiC, et al. (2021) Event-triggered control for Markov jump systems subject to mismatched modes and strict dissipativity. IEEE Transactions on Cybernetics. Epub ahead of print 1September. DOI: 10.1109/TCYB.2021.3105179.
35.
VrabieDLewisFL (2011) Adaptive dynamic programming for online solution of a zero-sum differential game. Journal of Control Theory and Application9(3): 353–360.
36.
WangXFeiZShiP, et al. (2022) Zonotopic fault detection for 2-D systems under event-triggered mechanism. IEEE Transactions on Cybernetics52(5): 3510–3518.
37.
WeiQLiuDLinQ, et al. (2018) Adaptive dynamic programming for discrete-time zero-sum games. IEEE Transactions on Neural Networks and Learning Systems29(4): 957–969.
38.
WeiQZhuLSongR, et al. (2022) Model-free adaptive optimal control for unknown nonlinear multiplayer nonzero-sum game. IEEE Transactions on Neural Networks and Learning Systems33(2): 879–892.
39.
WeiWLvJ (2022) Multi-agent event-triggered containment control with joint connected switching topology. International Journal of Innovative Computing, Information and Control18(3): 957–971.
40.
YangDLiTXieX, et al. (2020) Event-triggered integral sliding-mode control for nonlinear constrained-input systems with disturbances via adaptive dynamic programming. IEEE Transactions on Systems, Man, and Cybernetics: Systems50(11): 4086–4096.
41.
YangXHeH (2021) Decentralized event-triggered control for a class of nonlinear-interconnected systems using reinforcement learning. IEEE Transactions on Cybernetics51(2): 635–648.
42.
YangXHeHLiuD (2019) Event-triggered optimal neuro-controller design with reinforcement learning for unknown nonlinear systems. IEEE Transactions on Systems, Man, and Cybernetics: Systems49(9): 1866–1878.
43.
ZhangGZhuQ (2021) Event-triggered optimal control for nonlinear stochastic systems via adaptive dynamic programming. Nonlinear Dynamics105: 387–401.
44.
ZhangHSuHZhangK, et al. (2019) Event-triggered adaptive dynamic programming for non-zero-sum games of unknown nonlinear systems via generalized fuzzy hyperbolic models. IEEE Transactions on Fuzzy Systems27(11): 2202–2214.
45.
ZhaoBLiuD (2020) Event-triggered decentralized tracking control of modular reconfigurable robots through adaptive dynamic programming. IEEE Transactions on Industrial Electronics67(4): 3054–3064.
46.
ZhaoGHuaC (2021) A hybrid dynamic event-triggered approach to consensus of multiagent systems with external disturbances. IEEE Transactions on Automatic Control66(7): 3213–3220.
47.
ZhongXHeH (2017) An event-triggered ADP control approach for continuous-time system with unknown internal states. IEEE Transactions on Cybernetics47(3): 683–694.
48.
ZhouKDoyleJCGloverK, et al. (1996) Robust and Optimal Control. Upper Saddle River, NJ: Prentice Hall.
49.
ZhuQ (2019) Stabilization of stochastic nonlinear delay systems with exogenous disturbances and the event-triggered feedback control. IEEE Transactions on Automatic Control64(9): 3764–3771.
50.
ZhuYZhaoDLiX (2017) Iterative adaptive dynamic programming for solving unknown nonlinear zero-sum game based on online data. IEEE Transactions on Neural Networks and Learning Systems28(3): 714–725.
