Complex traffic scenes greatly challenge the road safety of automated vehicles (AVs). This paper integrates the decision-making, path-planning, and motion-control modules to ensure the autonomous driving performance in the high-speed cruising scenario. First, to guarantee deep exploration of the reinforcement learning method, a Bootstrapped deep-Q-Network (BDQN) is proposed to address the adaptive decision-making of AVs. The artificial potential field (APF) is introduced in the reward function to improve the autonomous obstacle-avoidance ability. Then, quantifying the multi-performance requirements of AVs under high-speed cruising can be complex. We employ an inverse reinforcement learning (IRL) approach to learn path-planning ability from skilled drivers, generating a reference path for executing lane changes. An embedded linear-quadratic regulator (LQR) tracking controller is further developed to output the trajectory points. These points effectively satisfy the vehicle safety constraints and ensure reference feasibility. By introducing the IRL and LQR into the training process of BDQN, the path-planning, motion-control modules are coupled with decision-making. Finally, simulation results demonstrate the proposed framework can guarantee the high-speed cruising performance.
LiangJLiYYinG, et al. A MAS-based hierarchical architecture for the cooperation control of connected and automated vehicles. IEEE Trans Veh Technol2023; 72(2): 1559–1573.
2.
LiangJTianQFengJ, et al. A polytopic model-based robust predictive control scheme for path tracking of autonomous vehicles. IEEE Trans Intell Veh2024; 9(2): 3928–3939.
3.
Van BrummelenJO’BrienMGruyerD, et al. Autonomous vehicle perception: the technology of today and tomorrow. Transp Res Part C Emerg Technol2018; 89: 384–406.
4.
WeiCPaschalidisEMeratN, et al. Human-like decision making and motion control for smooth and natural car following. IEEE Trans Intell Veh2023; 8(1): 263–274.
5.
LiGQiuYYangY, et al. Lane change strategies for autonomous vehicles: a deep reinforcement learning approach based on transformer. IEEE Trans Intell Veh2023; 8(3): 2197–2211.
6.
LiLGanJJiX, et al. Dynamic driving risk potential field model under the connected and automated vehicles environment and its application in car-following modeling. IEEE Trans Intell Transp Syst2022; 23(1): 122–141.
7.
ChenBChenXWuQ, et al. Adversarial evaluation of autonomous vehicles in lane-change scenarios. IEEE Trans Intell Transp Syst2022; 23(8): 10333–10342.
8.
AhmedHUHuangYLuP. A review of car-following models and modeling tools for human and autonomous-ready driving behaviors in micro-simulation. Smart Cities2021; 4(1): 314–335.
9.
RakhaHWangW. Procedure for calibrating Gipps car-following model. Transp Res Rec2009; 2124(1): 113–124.
10.
ZhouMQuXJinS. On the impact of cooperative autonomous vehicles in improving freeway merging: a modified intelligent driver model-based approach. IEEE Trans Intell Transp Syst2017; 18(6): 1422–1428.
11.
LiTGuoFKrishnanR, et al. Right-of-way reallocation for mixed flow of autonomous vehicles and human driven vehicles. Transp Res Part C Emerg Technol2020; 115: 102630.
12.
YeLYamamotoT. Modeling connected and autonomous vehicles in heterogeneous traffic flow. Physica A2018; 490: 269–277.
13.
XuTLavalJ. Statistical inference for two-regime stochastic car-following models. Transp Res B Methodol2020; 134: 210–228.
14.
LinYWangPZhouY, et al. Platoon trajectories generation: a unidirectional interconnected LSTM-based car-following model. IEEE Trans Intell Transp Syst2022; 23(3): 2071–2081.
15.
ZhengYHeSYiR, et al. Categorizing car-following behaviors: wavelet-based time series clustering approach. J Transp Eng A Syst2020; 146(8): 04020072.
16.
ZhuMWangXWangY. Human-like autonomous car-following model with deep reinforcement learning. Transp Res Part C Emerg Technol2018; 97: 348–368.
17.
BalalECheuRLSarkodie-GyanT. A binary decision model for discretionary lane changing move based on fuzzy inference system. Transp Res Part C Emerg Technol2016; 67: 47–61.
18.
DouYYanFFengD. Lane changing prediction at highway lane drops using support vector machine and artificial neural network classifiers. In: 2016 IEEE international conference on advanced intelligent mechatronics (AIM), Banff, AB, Canada, 12–15 July 2016, pp.901–906. New York: IEEE.
19.
XiongGKangZLiH, et al. Decision-making of lane change behavior based on RCS for automated vehicles in the real environment. In: 2018 IEEE intelligent vehicles symposium (IV), Changshu, China, 26–30 June 2018, pp.1400–1405. New York: IEEE.
20.
LiBAcarmanTZhangY, et al. Optimization-based trajectory planning for autonomous parking with irregularly placed obstacles: a lightweight iterative framework. IEEE Trans Intell Transp Syst2022; 23(8): 11970–11981.
21.
HanZWangDLiuF, et al. Multi-AGV path planning with double-path constraints by using an improved genetic algorithm. PLoS One2017; 12(7): e0181747.
22.
ClaussmannLRevilloudMGruyerD, et al. A review of motion planning for highway autonomous driving. IEEE Trans Intell Transp Syst2020; 21(5): 1826–1848.
23.
ScheffePHennekenTMKloockM, et al. Sequential convex programming methods for real-time optimal trajectory planning in autonomous vehicle racing. IEEE Trans Intell Veh2023; 8(1): 661–672.
24.
FayaziSAVahidiA. Mixed-integer linear programming for optimal scheduling of autonomous vehicle intersection crossing. IEEE Trans Intell Veh2018; 3(3): 287–299.
25.
BojarskiMDel TestaDDworakowskiD, et al. End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316, 2016.
26.
HoelC-JWolffKLaineL.Automated speed and lane change decision making using deep reinforcement learning. In: 2018 21st international conference on intelligent transportation systems (ITSC), Maui, HI, USA, 4–7 November 2018, pp.2148–2155. New York: IEEE.
27.
AnHJungJ-I. Decision-making system for lane change using deep reinforcement learning in connected and automated driving. Electronics2019; 8(5): 543.
28.
YeFChengXWangP, et al. Automated lane change strategy using proximal policy optimization-based deep reinforcement learning. In: 2020 IEEE intelligent vehicles symposium (IV), Las Vegas, NV, USA, 19 October–13 November 2020, pp.1746–1752. New York: IEEE.
29.
YeYZhangXSunJ. Automated vehicle’s behavior decision making using deep reinforcement learning and high-fidelity simulation environment. Transp Res C Emerg Technol2019; 107: 155–170.
30.
ZhaoJQuTXuF. A deep reinforcement learning approach for autonomous highway driving. IFAC Pap OnLine2020; 53(5): 542–546.
31.
PengJZhangSZhouY, et al. An integrated model for autonomous speed and lane change decision-making based on deep reinforcement learning. IEEE Trans Intell Transp Syst2022; 23(11): 21848–21860.
32.
NaveedKBQiaoZDolanJM. Trajectory planning for autonomous vehicles using hierarchical reinforcement learning. In: 2021 IEEE international intelligent transportation systems conference (ITSC), Indianapolis, IN, USA, 19–22 September 2021, pp.601–606. New York: IEEE.
33.
LiSWeiCWangY. Combining decision making and trajectory planning for lane changing using deep reinforcement learning. IEEE Trans Intell Transp Syst2022; 23(9): 16110–16136.
34.
FangZLiangJTanC, et al. Enhancing robust driver assistance control in distributed drive electric vehicles through integrated AFS and DYC technology. IEEE Trans Intell Veh. Epub ahead of print 21 February 2024. DOI: 10.1109/TIV.2024.3368050.
35.
IrshayyidAChenJXiongG. A review on reinforcement learning-based highway autonomous vehicle control. Green Energy Intell Transp2024; 3(4): 100156.
36.
SongLLiJWeiZ, et al. Longitudinal and lateral control methods from single vehicle to autonomous platoon. Green Energy Intell Transp2023; 2(2): 100066.
37.
TanTChuTWangJ. Multi-agent bootstrapped deep Q-network for large-scale traffic signal control. In: 2020 IEEE conference on control technology and applications (CCTA), Montreal, QC, Canada, 24–26 August 2020, pp.358–365. New York: IEEE.
38.
BaheriANageshraoSTsengHE, et al. Deep reinforcement learning with enhanced safety for autonomous highway driving. In: 2020 IEEE intelligent vehicles symposium (IV), Las Vegas, NV, USA, 19 October–13 November 2020, pp.1550–155. New York: IEEE.
39.
LiangJLuYYinG, et al. A distributed integrated control architecture of AFS and DYC based on MAS for distributed drive electric vehicles. IEEE Trans Veh Technol2021; 70(6): 5565–5577.
40.
LiangJWangFFengJ, et al. A hierarchical control of independently driven electric vehicles considering handling stability and energy conservation. IEEE Trans Intell Veh2024; 9(1): 738–751.
41.
JohnsonRW. An introduction to the bootstrap. Teach Stat2001; 23(2): 49–54.
42.
ZiebartBDMaasALBagnellJA, et al. Maximum entropy inverse reinforcement learning. In: Proceedings of the twenty-third AAAI conference on artificial intelligence, Chicago, IL, USA, 2008, vol. 8, pp.1433–1438. Menlo Park, CA: AAAI Press.
43.
LiangJLuYWangF, et al. A robust dynamic game-based control framework for integrated torque vectoring and active front-wheel steering system. IEEE Trans Intell Transp Syst2023; 24(7): 7328–7341.
44.
LiangJLuYFengJ, et al. Robust shared control system for aggressive driving based on cooperative modes identification. IEEE Trans Syst Man Cybern Syst2023; 53(11): 6672–6684.
45.
DoganODeveciMCanitezF, et al. A corridor selection for locating autonomous vehicles using an interval-valued intuitionistic fuzzy AHP and Topsis method. Soft Comput2020; 24(12): 8937–8953.
