Aiming at the safety issues in lane-changing decision-making for autonomous vehicles in urban environments, this study proposed a new lane-changing decision-making method for autonomous vehicles that considers epistemic uncertainty (EUTAS-DDQN). In the training phase, a Transformer layer was introduced into the Double Deep Q-Network algorithm (DDQN) to extract data features, which guides the fully connected layer in making decisions. Then, a trainable temperature coefficient was incorporated to develop an adaptive Softmax strategy for generating the final action. In the testing phase, the standard deviation of the Q-values was used as an indicator of epistemic uncertainty. An epistemic uncertainty threshold was defined such that when the standard deviation of the agent’s Q-values exceeds the threshold, the agent’s output action was executed; otherwise, the system switched to a rule-based module to execute a corrected action, thereby enhancing safety. Experimental results demonstrate that the proposed EUTAS-DDQN model achieves a collision rate of only 0.75%, representing a 70.47% reduction compared to the DDQN model.
WangHYuanSGuoM, et al. Tactical driving decisions of unmanned ground vehicles in complex highway environments: a deep reinforcement learning approach. Proc Inst Mech Eng D J Automobile Eng2021; 235: 1113–1127.
2.
WuQChengSLiL, et al. A fuzzy-inference-based reinforcement learning method of overtaking decision making for automated vehicles. Proc Inst Mech Eng D J Automobile Eng2022; 236: 75–83.
3.
HanLMaXWangY, et al. Safe and efficient DRL driving policies using fuzzy logic for urban lane changing scenarios. J Intell Connect Veh2025; 8: 9210054–9210061.
ZhangKQuDSongH, et al. Analysis of lane-changing decision-making behavior and molecular interaction potential modeling for connected and automated vehicles. Sustainability2022; 14: 11049.
6.
LiDLiuGXiaoB.Human-like driving decision at unsignalized intersections based on game theory. Proc Inst Mech Eng D J Automobile Eng2023; 237: 159–173.
7.
AmmourahRTalebpourA.Deep reinforcement learning approach for automated vehicle mandatory lane changing. Transp Res Rec2023; 2677: 712–724.
8.
LvKPeiXChenC, et al. A safe and efficient lane change decision-making strategy of autonomous driving based on deep reinforcement learning. Mathematics2022; 10: 1551.
9.
TianYJiYChangH, et al. Deep reinforcement learning of widening decision model for intelligent driving vehicles. J Jilin Univ Eng Ed2023; 53: 682–692.
10.
ZhuBJiaSZhaoJ, et al. Sequential game-like human decision-making for autonomous vehicles considering subjective cognition. Automot Eng2025; 1: 13–22.
11.
YiPPanYWangW, et al. A review of multi-vehicle intelligent driving interaction decision-making based on game theory. Control Decis2023; 38: 1159–1175.
12.
CuiJZhaoBQuM.An integrated lateral and longitudinal decision-making model for autonomous driving base deep reinforcement learning. J Adv Transport2023; 2023: 1–13.
13.
WangJWangHFeiM, et al. Vehicle lane changing game model based on improved SVM algorithm. World Electr Veh J2024; 15: 505.
14.
LiSWeiCWangY.Combining decision making and trajectory planning for lane changing using deep reinforcement learning. IEEE Trans Intell Transp Syst2022; 23: 16110–16136.
15.
ZhuBJiaSZhaoJ, et al. A review on decision-making and planning of autonomous vehicles. Chin J Highw2024; 37: 215–240.
16.
HeXYangHHuZ, et al. Robust lane change decision making for autonomous vehicles: an observation adversarial reinforcement learning approach. IEEE Trans Intell Vehicles2023; 8: 184–193.
17.
LiGQiuYYangY, et al. Lane change strategies for autonomous vehicles: a deep reinforcement learning approach based on transformer. IEEE Trans Intell Vehicles2023; 8: 2197–2211.
18.
PengJZhangSZhouY, et al. An integrated model for autonomous speed and lane change decision-making based on deep reinforcement learning. IEEE Trans Intell Transp Syst2022; 23: 21848–21860.
19.
ZhangYXuQWangJ, et al. A learning-based discretionary lane-change decision-making model with driving style awareness. IEEE Trans Intell Transp Syst2023; 24: 68–78.
20.
LvPHanJNieJ, et al. Cooperative decision-making of connected and autonomous vehicles in an emergency. IEEE Trans Vehicular Technol2023; 72: 1464–1477.
21.
GuanJChenGHuangJ, et al. A discrete soft actor-critic decision-making strategy with sample filter for freeway autonomous driving. IEEE Trans Vehicular Technol2023; 72: 2593–2598.
22.
XuCZhaoWLiuJ, et al. An integrated decision-making framework for highway autonomous driving using combined learning and rule-based algorithm. IEEE Trans Vehicular Technol2022; 71: 3621–3632.
23.
YaoFSunCLanY, et al. A hybrid expert model-based lane-changing decision-making method for intelligent networked vehicles. Automot Eng2024; 46: 882–892.
24.
XuQGuoJ.Research on intelligent vehicle decision system based on reinforcement learning. Meas Sci Technol2025; 36: 035114.
25.
LiWMaCShiH, et al. Decision control algorithm for autonomous driving based on hierarchical reinforcement learning. J Jilin Univ Eng Ed2023; 5: 1–8.
26.
JiangWWuZXuH, et al. Reinforcement learning-based lane-changing decision making for self-driving cars. Comput Eng Appl2025; 61: 359–371.
27.
JinLHanGXieX, et al. A review of reinforcement learning-based decision-making for autonomous driving. Automot Eng2023; 45: 527–540.
28.
ZhangXWangZ.Automatic lane-changing decision model based on double-duel deep Q-network. J Northeast Univ Nat Sci Ed2023; 44: 1369–1376.
29.
WangHChenWWangQ, et al. Lane-changing decision based on neighboring lanes’ safety posture classification. J Mech Eng2023; 59: 233–244.
30.
ZhangZLiuQLiY, et al. Safe reinforcement learning in autonomous driving with epistemic uncertainty estimation. IEEE Trans Intell Transp Syst2024; 25: 13653–13666.
