Traditional adaptive cruise control (ACC) would choose to follow when encountering low-speed cars in front, but this may lead to low driving efficiency. This paper proposes a controller of enhanced adaptive cruise control with lane-change assistance (LCACC) for an articulated vehicle. A two-layer hierarchical control structure is adopted in this study. The upper determines high-level commands, while the bottom consisting of two modified deep deterministic policy gradient (DDPG) networks, controls the steering wheel and throttle/brake, respectively, according to the commands made by the upper layer. The vehicle’s lateral and longitudinal control are decoupled and controlled by two modified DDPG networks. Through proper design of the state and reward function, the actions of the articulated vehicle such as steering and acceleration/braking are closer to those of humans, thus ensuring ride comfort. Compared with traditional ACC, the LCACC proposed in this study increases the driving efficiency of the articulated vehicle by an average of 16% when there is a low-speed lead car.
GaoZSunTXiaoH. Decision-making method for vehicle longitudinal automatic driving based on reinforcement Q-learning. Int J Adv Robot Syst2019; 16:172988141985318.
2.
DolkVSPloegJHeemelsWPMH. Event-triggered control for string-stable vehicle platooning. IEEE Trans Intell Transp Syst2017; 18:3486–3500.
3.
SchwartingWAlonso-MoraJRusD. Planning and decision-making for autonomous vehicles. Annu Rev Contr Robot Auton Syst2018; 1:187–210.
4.
PerssonMBotlingFHesslowE, et al. Stop and go controller for adaptive cruise control. In: Proceedings of the 1999 IEEE international conference on control applications (Cat No. 99CH36328), 1999, pp.1692–1697. IEEE.
5.
VenhovensPNaabKAdiprasitoB. Stop and go cruise control. Int J Automot Technol2000; 1:61–69.
6.
LiSLiKRajamaniR, et al. Model predictive multi-objective vehicular adaptive cruise control. IEEE Trans Control Syst Technol2011; 19:556–566.
7.
GuoLGeP-SQiaoY, et al. Multi-objective adaptive cruise control strategy based on variable time headway. In: 2018 IEEE intelligent vehicles symposium (IV), Changshu, China, 2018, pp.203–208.
8.
ZhangJIoannouP. Longitudinal control of heavy trucks in mixed traffic: Environmental and fuel economy considerations. IEEE Trans Intell Transp Syst2006; 7:92–104.
9.
ChenTLuoYLiK. Multi-objective adaptive cruise control based on nonlinear model predictive algorithm. In: Proceedings of 2011 IEEE international conference on vehicular electronics and safety, Beijing, China, 2011, pp.274–279. IEEE.
10.
DangRWangJLiSE, et al. Coordinated adaptive cruise control system with lane-change assistance. IEEE Trans Intell Transp Syst2015; 16:2373–2383.
11.
MukaiMKawabeT. Model predictive control for lane change decision assist system using hybrid system representation. In: 2006 SICE-ICASE international joint conference, Busan, South Korea, 2006, pp.5120–5125. IEEE.
12.
YoshidaHShinoharaSNagaiM. Lane change steering manoeuvre using model predictive control theory. Veh Syst Dyn2008; 46:669–681.
13.
KimDMoonSParkJ, et al. Design of an adaptive cruise control/collision avoidance with lane change support for vehicle autonomous driving. In: 2009 ICCAS-SICE, Fukuoka, Japan, 18–21August2009, pp.2938–2943.
14.
VinyalsOBabuschkinICzarneckiWM, et al. Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature2019; 575:350–354.
15.
BernerCBrockmanGChanB, et al. Dota 2 with large scale deep reinforcement learning. ArXiv 1912.06680, 2019.
16.
MirchevskaBPekCWerlingM, et al. High-level decision making for safe and reasonable autonomous lane changing using reinforcement learning. In: 2018 21st international conference on intelligent transportation systems (ITSC), Maui, HI, USA, 2018, pp.2156–2162. IEEE.
17.
MnihVKavukcuogluKSilverD, et al. Human-level control through deep reinforcement learning. Nature2015; 518:529–533.
18.
SilverDHuangAMaddisonCJ, et al. Mastering the game of go with deep neural networks and tree search. Nature2016; 529:484–489.
19.
SallabAEAbdouMPerotE, et al. Deep reinforcement learning framework for autonomous driving. Electron Imaging2017; 29:70–76.
20.
LiDZhaoDZhangQ, et al. Reinforcement learning and deep learning based lateral control for autonomous driving [application notes]. IEEE Comput Intell Mag2019; 14:83–98.
21.
WolfPHubschneiderCWeberM, et al. Learning how to drive in a real world simulation with deep Q-networks. In: 2017 IEEE intelligent vehicles symposium (IV), Los Angeles, CA, USA, 2017, pp.244–250. IEEE.
22.
HoelC-JWolffKLaineL. Automated speed and lane change decision making using deep reinforcement learning. In: 2018 21st international conference on intelligent transportation systems (ITSC), Maui, HI, 2018, pp.2148-2155. IEEE.
23.
ShiTWangPChengX, et al. Driving decision and control for automated lane change behavior based on deep reinforcement learning. In: 2019 IEEE intelligent transportation systems conference (ITSC), Auckland, New Zealand, 2019, pp.2895–2900. IEEE.
24.
LiDZhaoDZhangQ. Reinforcement learning based lane change decision-making with imaginary sampling. In: 2019 IEEE symposium series on computational intelligence (SSCI), Xiamen, China, 2019, pp.16–21. IEEE.
25.
HuHLuZWangQ, et al. End-to-End Automated Lane-change maneuvering considering driving style using a deep deterministic policy gradient algorithm. Sensors2020; 20:5443.
26.
DuanJEben LiSGuanY, et al. Hierarchical reinforcement learning for self-driving decision-making without reliance on labelled driving data. IET Intell Transp Syst2020; 14:297–305.
27.
ChenZHuangX. End-to-end learning for lane keeping of self-driving cars. In: 2017 IEEE intelligent vehicles symposium (IV), Los Angeles, CA, USA, 2017, pp.1856–1860. IEEE.
28.
YangZZhangYYuJ, et al. End-to-end multi-modal multi-task vehicle control for self-driving cars with visual perceptions. In: 2018 24th international conference on pattern recognition (ICPR), Beijing, China, 2018, pp.2289–2294. IEEE.
29.
OnishiTMotoyoshiTSugaY, et al. End-to-end learning method for self-driving cars with trajectory recovery using a path-following function. In: 2019 international joint conference on neural networks (IJCNN), Budapest, Hungary, 2019, pp.1–8. IEEE.
30.
YenY-TChouJ-JShihC-S, et al. Proactive car-following using deep-reinforcement learning. In: 2020 IEEE 23rd international conference on intelligent transportation systems (ITSC), Rhodes, Greece, 2020, pp.1–6. IEEE.
31.
ZhuMWangYPuZ, et al. Safe, efficient, and comfortable velocity control based on reinforcement learning for autonomous driving. Transp Res Part C Emerg Technol2020; 117:102662.
32.
SunMZhaoWSongG, et al. DDPG-Based decision-making strategy of adaptive cruising for heavy vehicles considering stability. IEEE Access2020; 8:59225–59246.
33.
JonesWD. Keeping cars from crashing. IEEE Spectr2001; 38:40–45.
34.
JingHWeiWDebiaoZ. Research on multi-mode switching strategy of adaptive cruise control system based on personalized spacing strategy. Automot Eng2020; 42:1302–1311.
35.
SuttonRSBartoAG. Reinforcement learning: an introduction. Cambridge, MA: MIT press, 2018.
36.
FujimotoSHoofHMegerD. Addressing function approximation error in actor-critic methods. In: International conference on machine learning, Stockholm, Sweden, 2018, pp.1587–1596. PMLR.
37.
LillicrapTPHuntJJPritzelA, et al. Continuous control with deep reinforcement learning. arXiv preprint arXiv:150902971, 2015.
38.
NilssonJSilvlinJBrannstromM, et al. If, when, and how to perform lane change maneuvers on highways. IEEE Intell Transp Syst Mag2016; 8:68–78.
39.
ISO 15622:2018. Intelligent transport systems – Adaptive cruise control systems – Performance requirements and test procedures.
40.
LiuZYuanQNieG, et al. A multi-objective model predictive control for vehicle adaptive cruise control system based on a new safe distance model. Int J Autom Technol2021; 22:475–487.
