With the increasing level of automation in autonomous vehicles, consideration of comfort and stability will further enhance the public acceptance of autonomous driving technology. This paper presents a cooperative lateral and vertical control strategy for autonomous vehicles based on multi-agent deep reinforcement learning, which integrates path tracking and suspension control for different planar learning tasks. By developing the lateral and vertical dynamic models, the multi-objective coordinated exploration of path tracking and active suspension systems is imposed by using the deep deterministic policy gradient (DDPG) algorithm. In the multi-agent deep reinforcement learning, a feedforward steering of steering subsystem and a PID compensation control of suspension subsystem are added to the DDPG control process for efficiently searching the strategic action of the coupling system. Furthermore, the learning reward function of autonomous vehicle is designed by comprehensively considering the accuracy, safety and comfort performance. Through the trained learning process and simulation results under different driving conditions, the proposed method can achieve the simultaneous optimization of path tracking and suspension comfort performance, and effectively improve the ride comfort and stability in the high-performance path tracking process. This study provides an efficient control scheme for improving the ride comfort of autonomous vehicles.
AndreottiEBoyrazP.Potential impact of autonomous vehicles in mixed traffic from simulation using real traffic flow. J Intell Connect Veh2023; 6(1): 1–15.
2.
NarayananSChaniotakisEAntoniouC.Shared autonomous vehicle services: a comprehensive review. Transp Res Part C: Emerg Technol2020; 111: 255–293.
3.
WangJHuangHLiK, et al. Towards the unified principles for level 5 autonomous vehicles. Engineering2021; 7(9): 1313–1325.
4.
TermousHShraimHTaljR, et al. Coordinated control strategies for active steering, differential braking and active suspension for vehicle stability, handling and safety improvement. Veh Syst Dyn2019; 57(10): 1494–1529.
5.
YaoXGuXJiangP.Coordination control of active front steering and direct yaw moment control based on stability judgment for AVs stability enhancement. Proc IMechE, Part D: J Automobile Engineering2022; 236(1): 59–74.
6.
CaoXXuTTianY, et al. Gain-scheduling LPV synthesis H∞ robust lateral motion control for path following of autonomous vehicle via coordination of steering and braking. Veh Syst Dyn2023; 61(4): 968–991.
7.
Viadero-MonasterioFNguyenATLauberJ, et al. Event-triggered robust path tracking control considering roll stability under network-induced delays for autonomous vehicles. IEEE Trans Intell Transp Syst2023; 24(12): 14743–14756.
8.
HuCWangRYanF, et al. Output constraint control on path following of four-wheel independently actuated autonomous ground vehicles. IEEE Trans Vehicular Technol2016; 65(6): 4033–4043.
9.
DongJLiJGaoQ, et al. Optimal coordinated control of active steering and direct yaw moment for distributed-driven electric vehicles. Control Eng Pract2023; 134: 105486.
10.
BobierCGGerdesJC.Staying within the nullcline boundary for vehicle envelope control using a sliding surface. Veh Syst Dyn2013; 51(2): 199–217.
11.
ErlienSMFujitaSGerdesJC.Shared steering control using safe envelopes for obstacle avoidance and vehicle stability. IEEE Trans Intell Transp Syst2016; 17(2): 441–451.
12.
BrownMFunkeJErlienS, et al. Safe driving envelopes for path tracking in autonomous vehicles. Control Eng Pract2017; 61: 307–316.
13.
CadenaCCarloneLCarrilloH, et al. Past, present, and future of simultaneous localization and mapping: toward the robust-perception age. IEEE Trans Robot2016; 32(6): 1309–1332.
14.
FengSYanXSunH, et al. Intelligent driving intelligence test for autonomous vehicles with naturalistic and adversarial environment. Nat Commun2021; 12(1): 748.
15.
LiGLiSLiS, et al. Deep reinforcement learning enabled decision-making for autonomous driving at intersections. Automot Innov2020; 3(4): 374–385.
16.
FrancisAFaustAChiangHTL, et al. Long-range indoor navigation with PRM-RL. IEEE Trans Robot2020; 36(4): 1115–1134.
17.
HouZFeiJDengY, et al. Data-efficient hierarchical reinforcement learning for robotic assembly control applications. IRE Trans Ind Electron2021; 68(11): 11565–11575.
18.
KiranBRSobhITalpaertV, et al. Deep reinforcement learning for autonomous driving: a survey. IEEE Trans Intell Transp Syst2022; 23(6): 4909–4926.
19.
DingHLiWXuN, et al. An enhanced eco-driving strategy based on reinforcement learning for connected electric vehicles: cooperative velocity and lane-changing control. J Intell Connect Veh2022; 5(3): 316–332.
20.
ChenIMChanCY.Deep reinforcement learning based path tracking controller for autonomous vehicle. Proc IMechE, Part D: J Automobile Engineering2021; 235(2-3): 541–551.
21.
ZhangLZhangZWangZ, et al. Chassis coordinated control for full x-by-wire vehicles-a review. Chin J Mech Eng2021; 34(1): 42.
22.
WangCCuiXZhaoS, et al. A deep reinforcement learning-based active suspension control algorithm considering deterministic experience tracing for autonomous vehicle. Appl Soft Comput2024; 153: 111259.
23.
TanZWenGPanZ, et al. Control of a nonlinear active suspension system based on deep reinforcement learning and expert demonstrations. Proc IMechE, Part D: J Automobile Engineering2024; 238(13): 4093–4113.
24.
LeeDJinSLeeC.Deep reinforcement learning of semi-active suspension controller for vehicle ride comfort. IEEE Trans Vehicular Technol2023; 72(1): 327–339.
25.
DuYChenJZhaoC, et al. Comfortable and energy-efficient speed control of autonomous vehicles on rough pavements using deep reinforcement learning. Transp Res Part C: Emerg Technol2022; 134: 103489.
26.
DuYChenJZhaoC, et al. A hierarchical framework for improving ride comfort of autonomous vehicles via deep reinforcement learning with external knowledge. Comput Aided Civ Infrastruct Eng2023; 38(8): 1059–1078.
27.
YongHSeoJKimJ, et al. Suspension control strategies using switched soft actor-critic models for real roads. IRE Trans Ind Electron2023; 70(1): 824–832.
28.
LiuQSongSHuH, et al. Extended model predictive control scheme for smooth path following of autonomous vehicles. Front Mech Eng2022; 17(1): 4.
29.
HuGLiuQDingR, et al. Vibration control of semi-active suspension system with magnetorheological damper based on hyperbolic tangent model. Adv Mech Eng2017; 9(5): 1–15.
30.
HanZY.Research on magnetorehological semi-active vehicle suspension system design and integrated control. PhD Dissertation, Jilin University, Changchun, 2019 (in Chinese).
31.
FuYLiCYuFR, et al. A decision-making strategy for vehicle autonomous braking in emergency via deep reinforcement learning. IEEE Trans Vehicular Technol2020; 69(6): 5876–5888.
32.
SpielbergNABrownMKapaniaNR, et al. Neural network vehicle models for high-performance automated driving. Sci Robot2019; 4(28): eaaw1975.
33.
CuiQDingRWeiC, et al. Path-tracking and lateral stabilisation for autonomous vehicles by using the steering angle envelope. Veh Syst Dyn2021; 59(11): 1672–1696.
34.
YaoJLTaheriSTianSM, et al. A novel semi-active suspension design based on decoupling skyhook control. J Vibroengineering2014; 16(3): 1318–1325.
