This paper develops a novel approximate optimal control method for a class of constrained continuous-time nonlinear systems in the presence of disturbances via adaptive dynamic programming (ADP) technique. First, an auxiliary dynamic compensator is introduced to deal with the input constraints. Through augmenting the original system with the designed auxiliary compensator, the design of constrained optimal controller is circumvented by stabilizing the equivalent augmented system. Then, the cost function is appropriately redefined by introducing an additional function connected with disturbances for the augmented nominal system in order to compensate the effect of unmatched disturbances. Next, the solution of associated Hamilton-Jacobi-Bellman (HJB) equation is solved online with weight adaptation law using neural networks (NNs). Furthermore, an additional robustifying term is utilized to compensate the effect of the approximation error of NNs, and thus the asymptotic stability of the closed-loop system is guaranteed. Finally, all signals of the closed-loop system are proved to be asymptotic convergence by using Lyapunov method. Simulation examples demonstrate the effectiveness of the proposed scheme.
Abu-KhalafMLewisFL (2005) Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica41(5): 779–791.
2.
ChenMTaoGJiangB (2015) Dynamic surface control using neural networks for a class of uncertain nonlinear systems with input saturation. IEEE Transactions on Neural Networks and Learning Systems26(9): 2086–2097.
3.
FanQYYangGH (2016) Adaptive actor-critic design-based integral sliding-mode control for partially unknown nonlinear systems with input disturbances. IEEE Transactions on Neural Networks and Learning Systems27(1): 165–177.
4.
HaddadWMChellaboinaVSFauszJL (1998) Robust nonlinear feedback control for uncertain linear systems with nonquadratic performance criteria. Systems and Control Letters33(5): 327–338.
5.
JiangYJiangZP (2015) Global adaptive dynamic programming for continuous-time nonlinear systems. IEEE Transactions on Automatic Control60(11): 2917–2929.
6.
LeeJYParkJBChoiYH (2012) Approximate dynamic programming for continuous-time linear quadratic regulator problems: Relaxation of known input-coupling matrix assumption. IET Control Theory and Applications6(13): 2063–2075.
7.
LiuDYangXWangD (2015) Reinforcement-learning-based robust controller design for continuous-time uncertain nonlinear systems subject to input constraints. IEEE Transactions on Cybernetics45(7): 1372–1385.
8.
LvYNaJYangQWuX, et al. (2016) Online adaptive optimal control for continuous-time nonlinear systems with completely unknown dynamics. International Journal of Control89(1): 99–112.
9.
ModaresHLewisFL (2014) Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning. Automatica50(7): 1780–1792.
10.
ModaresHLewisFLNaghibi-SistaniMB (2014) Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems. Automatica50(1): 193–202.
11.
SongRLewisFLWeiQ, et al. (2016) Off-policy actor-critic structure for optimal control of unknown systems with disturbances. IEEE Transactions on Cybernetics46(5): 1041–1050.
12.
SunJLiuCLiuN (2018) Data-driven adaptive critic approach for nonlinear optimal control via least squares support vector machine. Asian Journal of Control20(1): 104–114.
13.
SunJLiuCYeQ (2017) Robust differential game guidance laws design for uncertain interceptor-target engagement via adaptive dynamic programming. International Journal of Contro90(5): 990–1004.
14.
VamvoudakisKGVrabieDLewisFL (2011) Online adaptive learning of optimal control solutions using integral reinforcement learning. In: 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), IEEE, April, pp. 250–257.
15.
WangDMuC (2018) adaptive-critic-based robust trajectory tracking of uncertain dynamics and its application to a spring–mass–damper system. IEEE Transactions on Industrial Electronics65(1): 654–663.
16.
WangDLiuDLiH (2014a) policy iteration algorithm for online design of robust control for a class of continuous-time nonlinear systems. IEEE Transactions on Automation Science and Engineering11(2): 627–632.
17.
WangDLiuDLiH, et al. (2014b) Neural-network-based robust optimal control design for a class of uncertain nonlinear systems via adaptive dynamic programming. Information Sciences282: 167–179.
18.
WangDLiuDLiH, et al. (2015) Adaptive dynamic programming for infinite horizon optimal robust guaranteed cost control of a class of uncertain nonlinear systems. In: American Control Conference (ACC), Chicago, IL, pp. 2900–2905.
19.
WangDLiuDMuC, et al. (2017) neural network learning and robust stabilization of nonlinear systems with dynamic uncertainties. IEEE Transactions on Neural Networks and Learning Systems29(4): 1342–1351.
20.
WangDLiuDZhangQ, et al. (2016) Data-based adaptive critic designs for nonlinear robust optimal control with uncertain dynamics. IEEE Transactions on Systems, Man, and Cybernetics46(11): 1544–1555.
21.
WenCZhouJLiuZ (2011) Robust adaptive control of uncertain nonlinear systems in the presence of input saturation and external disturbance. IEEE Transactions on Automatic Control56(7): 1672–1678.
22.
WenGChenPGeSSYangHLiuX (2019) Optimized adaptive nonlinear tracking control using actor-critic reinforcement learning strategy. IEEE Transactions on Industrial Informatics15(9): 4969–4977.
23.
WerbosP (1974) Beyond regression: New tools for prediction and analysis in the behavioral sciences. PhD Dissertation, Harvard University.
24.
YangXHeH (2019) Adaptive critic designs for event-triggered robust control of nonlinear systems with unknown dynamics. IEEE Transactions on Cybernetics49(6): 2255–2267.
25.
YangXLiuDWangD (2014) Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraints. International Journal of Control, 87(3): 553–566.
26.
YuJZhong-PingJ (2014) robust adaptive dynamic programming and feedback stabilization of nonlinear systems. IEEE Transactions on Neural Networks and Learning Systems25(5): 882–893.
27.
ZhangHLuoYLiuD (2009) Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints. IEEE Transactions on Neural Networks20(9): 1490–1503.
28.
ZhangJZhangHLiuZWangY (2015) Model-free optimal controller design for continuous-time nonlinear systems by adaptive dynamic programming based on a precompensator. ISA Transactions57: 63–70.
29.
ZhaoXYangHXiaWWangX (2017) Adaptive fuzzy hierarchical sliding-mode control for a class of mimo nonlinear time-delay systems with input saturation. IEEE Transactions on Fuzzy Systems25(5): 1062–1077.
30.
ZhouQLiHWuCWangL.AhnCK (2017) Adaptive fuzzy control of nonlinear systems with unmodeled dynamics and input saturation using small-gain approach. IEEE Transactions on Systems, Man, and Cybernetics47(8): 1979–1989.
