Future risk evolution is critical for the decision-making of autonomous vehicles (AVs), since it enables AVs to be aware of risks in advance and take action to ensure safety. This study proposes a reinforcement learning-based decision-making method by fusing future speculated risk for mixed traffic. Based on the current state of the environment and the predicted trajectories of pedestrians and vehicles, a driving risk field fusing current and future speculated risk is constructed to describe the driving risk. A Twin Delayed Deep Deterministic Policy Gradients (TD3) reinforcement learning algorithm is introduced as the main framework for decision-making training. The reward function accounting for both risk and efficiency is included in the TD3 training framework to optimize safety and efficiency performance. The proposed method is evaluated with the CARLA simulator in two scenarios (forward and unsignalized intersection scenarios). The experimental results show that the proposed method achieves better driving performance compared to methods that only consider instantaneous risk and rules.
QuinteroAR. Analysis of automated vehicle accidents across the U.S. California State University, 2022.
2.
SinhaAChandSVuV, et al. Crash and disengagement data of autonomous vehicles on public roads in California. Sci Data2021; 8: 298.
3.
MokhtariKWagnerAR. Don’t get into trouble! Risk-aware decision-making for autonomous vehicles. In: IEEE conference on robot and human interactive communication, Napoli, Italy, 29 August–2 September 2022, pp.1570–1577. New York: IEEE.
4.
NguyenHDChoiMHanK.Risk-informed decision-making and control strategies for autonomous vehicles in emergency situations. Accid Anal Prev2023; 193: 107305.
5.
AksjonovAKyrkiV. Rule-based decision-making system for autonomous vehicles at intersections with mixed traffic environment. In: IEEE intelligent transportation systems conference, Indianapolis, IN, USA, 19–22 September 2021, pp.660–666. New York: IEEE.
6.
ChengZZhangYDuC, et al. A rule-based decision-making framework for dilemma zone protection at signalized intersections. In: IEEE conference on intelligent transportation engineering, Beijing, China, 11–13 November 2022, pp. 493–496. New York: IEEE.
7.
WangYJiangJLiS, et al. Decision-making driven by driver intelligence and environment reasoning for high-level autonomous vehicles: a survey. IEEE Trans Intell Transp Syst2023; 24: 10362–10381.
8.
HuJWangYChengS, et al. A survey of decision-making and planning methods for self-driving vehicles. Front Neurorobot2025; 19: 1451923.
9.
ShuKMehriziRVLiS, et al. Human inspired autonomous intersection handling using game theory. IEEE Trans Intell Transp Syst2023; 24: 11360–11371.
10.
JiaSZhangYLiX, et al. Interactive decision-making with switchable game modes for automated vehicles at intersections. IEEE Trans Intell Transp Syst2023; 24: 11785–11799.
11.
QinLHuangZZhangC, et al. Deep imitation learning for autonomous navigation in dynamic pedestrian environments. In: IEEE conference on robotics and automation, Xi’an, China, 30 May–5 June 2021, pp. 4108–4115. New York: IEEE.
12.
CaiPWangSSunY, et al. Probabilistic end-to-end vehicle navigation in complex dynamic environments with multimodal sensor fusion. IEEE Robot Autom Lett2020; 5: 4218–4244.
13.
ElallidBBBenamarNHafidAS, et al. A comprehensive survey on the application of deep and reinforcement learning approaches in autonomous driving. J King Saud Univ Comput Inf Sci2022; 34: 7366–7390.
14.
IrshayyidAChenJXiongG.A review on reinforcement learning-based highway autonomous vehicle control. Green Energy Intell Transp2024; 3: 100156.
15.
IrshayyidAChenJ.Highway merging control using multi-agent reinforcement learning. In: IEEE conference on computing and machine intelligence, Mt Pleasant, MI, USA, 13–14 April 2024, pp. 1–2. New York: IEEE.
16.
GotoTKizumiYIwasakiS.Design of reward function on reinforcement learning for automated driving. IFAC Pap OnLine2023; 56: 7948–7953.
17.
LiTRuanJZhangK.The investigation of reinforcement learning-based end-to-end decision-making algorithms for autonomous driving on the road with consecutive sharp turns. Green Energy Intell Transp2025; 4: 100288.
18.
ChenHCaoXGuvencL, et al. Deep-reinforcement-learning-based collision avoidance of autonomous driving system for vulnerable road user safety. Electronics2024; 13: 1952.
19.
EverettMChenYFHowJP.Collision avoidance in pedestrian-rich environments with deep reinforcement learning. IEEE Access2021; 9: 10357–10377.
20.
MokhtariKWagnerAR.Pedestrian collision avoidance for autonomous vehicles at unsignalized intersection using deep Q-Network. arXiv preprint arXiv:2105.00153, 2021.
21.
LiJWuXXuM, et al. Deep reinforcement learning and reward shaping based eco-driving control for automated HEVs among signalized intersections. Energy2022; 251: 123924.
22.
TongHChuLChenZ, et al. Multi-objective autonomous eco-driving strategy: a pathway to future green mobility. Green Energy Intell Transp2025; 4: 100279.
23.
MaXChenJXiaL, et al. DSAC: distributional soft actor-critic for risk-sensitive reinforcement learning. J Artif Intell Res2025; 83: 4.
24.
LiGYangYLiS, et al. Decision making of autonomous vehicles in lane change scenarios: deep reinforcement learning approaches with risk awareness. Transp Res Part C Emerg Technol2022; 134: 103452.
25.
LiaoYYuGChenP, et al. Lateral motion control for obstacle avoidance in autonomous driving based on deep reinforcement learning. In: IEEE conference on intelligent transportation systems, Bilbao, Spain, 24–28 September 2023, pp. 5229–5234. New York: IEEE.
26.
CaiYZhanLSunX, et al. Research on obstacle avoidance strategy of automated heavy vehicle platoon in high-speed scenarios. Proc Inst Mech Eng Part D: J Automob Eng2025; 239: 4706–4722.
27.
SchmidtLMRietschSPlingeA, et al. How to learn from risk: explicit risk-utility reinforcement learning for efficient and safe driving strategies. In: IEEE conference on intelligent transportation systems, Macau, China, 8–12 October 2022, pp. 1913–1920. New York: IEEE.
28.
WangHYuanSGuoM, et al. Tactical driving decisions of unmanned ground vehicles in complex highway environments: a deep reinforcement learning approach. Proc Inst Mech Eng Part D: J Automob Eng2021; 235: 1113–1127.
29.
LiuWWangWYangC, et al. Human-like decision making for automated vehicles using driving style-guided inverse reinforcement learning method. Proc Inst Mech Eng Part D: J Automob Eng. Epub ahead of print 5 August 2025. DOI: 10.1177/09544070251350874.
30.
GaoAZhangWFuZ, et al. Trajectory planning algorithm considering obstacle risk in dynamic traffic scenarios. IEEE Trans Veh Technol2026; 75: 274–289.
31.
ZhanHWangGShanX, et al. Risk-aware lane-change trajectory planning with rollover prevention for autonomous light trucks on curved roads. Mech Syst Signal Process2024; 211: 111126.
32.
WangXHuJWeiC, et al. A novel lane-change decision-making with long-time trajectory prediction for autonomous vehicle. IEEE Access2023; 11: 137437–137449.
33.
LuoYSunAHongJ.Autonomous driving decision-making method based on spatial-temporal fusion trajectory prediction. Appl Sci2024; 14: 11913.
34.
ZhangSSunPPangY, et al. An autonomous driving decision-making framework for joint prediction and planning in unsignalized intersection scenarios. Proc Inst Mech Eng Part D: J Automob Eng. Epub ahead of print 19 February 2025. DOI: 10.1177/09544070251317687.
35.
XiongLLiZZhongD, et al. Rule-guidance reinforcement learning for lane change decision-making: a risk assessment approach. Chin J Mech Eng2025; 38: 30.
36.
CandelaEDoustalyOParadaL, et al. Risk-aware controller for autonomous vehicles using model-based collision prediction and reinforcement learning. Artif Intell2023; 320: 103923.
37.
LaiXYangZXieJ, et al. Reinforcement learning in transportation research: frontiers and future directions. Multimodal Transp2024; 3: 100164.
38.
HanJZhaoJZhuB, et al. Spatial-temporal risk field for intelligent connected vehicle in dynamic traffic and application in trajectory planning. IEEE Trans Intell Transp Syst2023; 24: 2963–2975.
39.
NianRLiuJHuangB.A review on reinforcement learning: introduction and applications in industrial process control. Comput Chem Eng2020; 139: 106886.
40.
VaswaniSKazemiABabanezhad HarikandehR, et al. Decision-aware actor-critic with function approximation and theoretical guarantees. Adv Neural Inf Process Syst2023; 36: 66451–66498.
41.
GuptaAJohnsonJFei-FeiL, et al. Social GAN: socially acceptable trajectories with generative adversarial networks. In: IEEE conference on computer vision and pattern recognition, Salt Lake City, UT, USA, 18–23 June 2018, pp. 2255–2264. New York: IEEE.
42.
AltcheFDe La FortelleA. An LSTM network for highway trajectory prediction. In: IEEE conference on intelligent transportation systems, Yokohama, Japan, 16–19 October 2017, pp. 353–359. New York: IEEE.
43.
LiLZhaoWWangC.POMDP motion planning algorithm based on multi-modal driving intention. IEEE Trans Intell Veh2023; 8: 1777–1786.
44.
WangXLiuTLiuZ.Study on collision avoidance behavior in the social force-based pedestrian–vehicle interaction simulation model at unsignalized intersections. Appl Sci2025; 15: 4885.
45.
ZhangZLiHChenT, et al. Decision-making of autonomous vehicles in interactions with jaywalkers: a risk-aware deep reinforcement learning approach. Accid Anal Prev2025; 210: 107843.
