Aiming at the attitude control problem of hypersonic flight vehicle, a compound control strategy based on the disturbance compensation technique and Q-learning optimization is proposed. Firstly, a three-channel independent control scheme based on the super-twisting extended state observer (STESO) is proposed such that coupling among channels, uncertainties, and external disturbances can be estimated and compensated in real time. Secondly, the Q-learning mechanism is introduced to optimize the control parameters while keeping the control structure unchanged. Lastly, the convergence of the STESO is analyzed by the Lyapunov theory, and some numerical simulation results are carried out to verify the effectiveness of the proposed strategy. Compared with the traditional linear active disturbance rejection control, the proposed method can avoid a lot of manual parameter tuning process and also has better performance.
SziroczakDSmithH. A review of design issues specific to hypersonic flight vehicles. Prog Aerospace Sci2016; 84: 1–28.
2.
McClintonC. X-43-scramjet power breaks the hypersonic barrier: dryden lectureship in research for 2006. In: 44th AIAA Aerospace Sciences Meeting and Exhibit, Reno, Nevada, 09–12 January 2006, p. 1.
3.
XuBShiZK. An overview on flight dynamics and control approaches for hypersonic vehicles. Sci China Inf Sci2015; 58(7): 1–19.
4.
FiorentiniLSerraniABolenderMA, et al. Nonlinear robust adaptive control of flexible air-breathing hypersonic vehicles. Journal of guidance, control, and dynamics2009; 32(2): 402–417.
5.
RehmanOUFidanBPetersenIR. Uncertainty modeling and robust minimax LQR control of multivariable nonlinear systems with application to hypersonic flight. Asian J Control2012; 14(5): 1180–1193.
6.
CaoLTangSZhangD. Flight control for air-breathing hypersonic vehicles using linear quadratic regulator design based on stochastic robustness analysis. Front Inf Technol Electron Eng2017; 18(7): 882–897.
7.
XuBHuangXYWangDW, et al.Dynamic surface control of constrained hypersonic flight models with parameter estimation and actuator compensation. Asian J Control2014; 16(1): 162–174.
8.
BuXWWuXYZhangR, et al.A neural approximation-based novel back-stepping control scheme for air-breathing hypersonic vehicles with uncertain parameters. Proc Inst Mech Eng J Syst Control Eng2016; 230(3): 231–243.
9.
SunJLPuZQYiJQ. Conditional disturbance negation based active disturbance rejection control for hypersonic vehicles. Control Eng Pract2019; 84: 159–171.
10.
WuQXSunMWWangZH, et al.Practical solution to efficient flight path control for hypersonic vehicles. Trans Jpn Soc Aeronaut Space Sci2016; 59(4): 195–204.
11.
AnHWuQQWangCH. Differentiator based full-envelope adaptive control of air-breathing hypersonic vehicles. Aerospace Sci Technol2018; 82: 312–322.
12.
GuoJGGuXYGuoZY. Asymptotic adaptive tracking control for hypersonic vehicles with guaranteeing multi-performance requirements. Aerospac Sci Technol2020; 105: 106025.
13.
GuoJGWangGQGuoZY, et al.New adaptive sliding mode control for a generic hypersonic vehicle. Proc Inst Mech Eng G J Aerospace Eng2018; 232(7): 1295–1303.
14.
DouLQDuMMMaoQ, et al.Finite-time nonsingular terminal sliding mode control-based fuzzy smooth-switching coordinate strategy for AHV-VGI. Aerospace Sci Technol2020; 106: 106080.
MengYZJiangBQiRY. Adaptive non-singular fault-tolerant control for hypersonic vehicle with unexpected centroid shift. IET Control Theor Appl2019; 13(12): 1773–1785.
17.
PiaoMNYangZHSunMW, et al.A practical attitude control scheme for hypersonic vehicle based on disturbance observer. Proc Inst Mech Eng Part G J Aerospac Eng2019; 233(12): 4523–4540.
18.
ShaoXLWangHL. Active disturbance rejection based trajectory linearization control for hypersonic reentry vehicle with bounded uncertainties. ISA Trans2015; 54: 27–38.
19.
GongKJLiaoYMeiYF. Extended state observer–based output feedback control for spacecraft pose tracking with control input saturation. Proc Inst Mech Eng Part G J Aerospac Eng2022; 236(4): 645–660.
20.
LiZBDongQLZhangXY, et al.Impact angle-constrained integrated guidance and control for supersonic skid-to-turn missiles using backstepping with global fast terminal sliding mode control. Aerospac Sci Technol2022; 122: 107386.
21.
ZhouJChangJGuoZY. A fault-tolerant control scheme within adaptive disturbance observer for hypersonic vehicle. Proc Inst Mech Eng Part G J Aerospac Eng2019; 233(3): 1071–1088.
22.
YuXLiPZhangYM. The design of fixed-time observer and finite-time fault-tolerant control for hypersonic gliding vehicles. IEEE Trans Ind Electron2017; 65(5): 4135–4144.
23.
GuoZYChangJGuoJG, et al.Adaptive twisting sliding mode algorithm for hypersonic reentry vehicle attitude control based on finite-time observer. ISA Trans2018; 77: 20–29.
24.
MadońskiRHermanP. Survey on methods of increasing the efficiency of extended state disturbance observers. ISA Trans2015; 56: 18–27.
25.
XuBWangDWSunFC, et al.Direct neural discrete control of hypersonic flight vehicle. Nonlinear Dyn2012; 70(1): 269–278.
26.
BuXWWuXYMaZ, et al.Nonsingular direct neural control of air-breathing hypersonic vehicle via back-stepping. Neurocomputing2015; 153: 164–173.
27.
ChaiRQTsourdosASavvarisA, et al.Six-DOF spacecraft optimal trajectory planning and real-time attitude control: a deep neural network-based approach. IEEE Trans Neural Networks Learn Syst2019; 31(11): 5005–5013.
28.
BoubertakhHTadjineMGlorennecPY, et al.Tuning fuzzy PD and PI controllers using reinforcement learning. ISA Trans2010; 49(4): 543–551.
29.
CarluchoIDe PaulaMVillarSA, et al.Incremental Q-learning strategy for adaptive PID control of mobile robots. Expert Syst Appl2017; 80: 183–199.
30.
KofinasPDounisAI. Online tuning of a PID controller with a fuzzy reinforcement learning MAS for flow rate control of a desalination unit. Electronics2019; 8(2): 231.
31.
PongfaiJSuXJZhangHY, et al.PID controller autotuning design by a deterministic Q-SLP algorithm. IEEE Access2020; 8: 50010–50021.
32.
MnihVKavukcuogluKSilverD, et al.Human-level control through deep reinforcement learning. Nature2015; 518(7540): 529–533.
33.
SilverDHuangAMaddisonCJ, et al.Mastering the game of Go with deep neural networks and tree search. Nature2016; 529(7587): 484–489.
34.
SilverDSchrittwieserJSimonyanK, et al.Mastering the game of go without human knowledge. Nature2017; 550(7676): 354–359.
35.
LiuCDongCYZhouZJ, et al.Barrier Lyapunov function based reinforcement learning control for air-breathing hypersonic vehicle with variable geometry inlet. Aerospace Sci Technol2020; 96: 105537.
36.
HuLLiRFXueT, et al.Neuro-adaptive tracking control of a hypersonic flight vehicle with uncertainties using reinforcement synthesis. Neurocomputing2018; 285: 141–153.
KeshmiriSColgrenRMirmiraniM. Six-DOF modeling and simulation of a generic hypersonic vehicle, for control and navigation purposes. In: AIAA guidance, navigation, and control conference and exhibit, Colorado, USAV, 21–24 August 2006: AIAA Press. paper no. AIAA-2006-6694.
39.
KeshmiriSColgrenRMirmiraniM. Six DoF nonlinear equations of motion for a generic hypersonic vehicle. In: AIAA atmospheric flight mechanics conference and exhibit, South Carolina: Virginia, 20–23 August 2007: AIAA Press, paper no. AIAA-2007-6626.
40.
MorenoJA. A linear framework for the robust stability analysis of a generalized super-twisting algorithm. In: Proceedings of the 6th International Conference on Electrical Engineering, Computing Science and Automatic Control, Toluca, Mexicon, 10–13 January 2009, pp. 1–6.
41.
KoberJBagnellJAPetersJ. Reinforcement learning in robotics: a survey. Int J Robotics Res2013; 32(11): 1238–1274.
42.
ChenZQQinBBSunMW, et al.Q-learning-based parameters adaptive algorithm for active disturbance rejection control and its application to ship course control. Neurocomputing2020; 408: 51–63.
