To ensure safe overtaking of intelligent vehicles in dynamic interactive environments, this paper proposes an end-to-end learning method for autonomous overtaking based on Graph Convolutional Network (GCN) and Conditional Imitation Learning (CIL). This method completes the autonomous overtaking behavior by directly mapping the environmental perception data to the underlying vehicle control actions (e.g. throttle and steer angle). This method fully considers the influence of other vehicles’ driving behavior on the overtaking behavior of the ego vehicle. Firstly, the dynamic interactive environments information around the ego vehicle is aggregated in the form of graph-structured data, and the aggregated global features are used as the input of the GCN to output the optimal action instructions that the ego vehicle should take. Secondly, combined with CIL, the action instructions output by the GCN are used as high-level commands to guide CIL. Finally, combined with other perception data, the underlying control actions of the vehicle will be output by CIL to complete safe overtaking in dynamic interactive environments. The method proposed in this paper can effectively extract the global information of the driving scene and complete the collision-free autonomous overtaking behavior, which greatly improves the intelligence of the driving system. The feasibility of the method has been verified by experiments on the CARLA simulation platform. The experimental results prove that the performance of this method is better than that of the conventional end-to-end learning framework, and it has better success rate and generalization performance.
MilanésVLlorcaDFVillagráJ, et al. Intelligent automatic overtaking system using vision for vehicle detection. Expert Syst Appl2012; 39(3): 3362–3373.
2.
FletcherLTellerSOlsonE, et al. The MIT-Cornell collision and why it happened. J Field Robot2009; 25(10): 775–807.
3.
WeiCHeYHTianHQ, et al. Game theoretic merging behavior control for autonomous vehicle at highway on-ramp. IEEE Trans Intell Transp Syst2022; 23(11): 21127–21136.
4.
LiLHGanJJiXK, et al. Dynamic driving risk potential field model under the connected and automated vehicles environment and its application in car-following modeling. IEEE Trans Intell Transp Syst2022; 23(1): 122–141.
5.
KipfTNWellingM.Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.
6.
CodevillaFMiillerMLópezA, et al. End-to-end driving via conditional imitation learning. In: 2018 IEEE international conference on robotics and automation (ICRA), Brisbane, QLD, Australia, 21–25 May 2018, pp.1–9. New York: IEEE.
7.
DosovitskiyARosGCodevillaF, et al. CARLA: an open urban driving simulator. In: 1st annual conference on robot learning (CoRL), 2017, pp.1–16.
8.
YurtseverELambertJCarballoA, et al. A survey of autonomous driving: common practices and emerging technologies. IEEE Access2019; 8: 58443–58469.
9.
NaranjoJEGonzalezCGarciaR, et al. Lane-change fuzzy control in autonomous vehicles for the overtaking maneuver. IEEE Trans Intell Transp Syst2008; 9(3): 438–450.
10.
PerezJMilanesVOnievaE, et al. Longitudinal fuzzy control for autonomous overtaking. In: 2011 IEEE international conference on mechatronics, Istanbul, Turkey, 13–15 April 2011, pp.188–193. New York: IEEE.
11.
WangFYangMYangR.Conflict-probability-estimation-based overtaking for intelligent vehicles. IEEE Trans Intell Transp Syst2009; 10(2): 366–370.
12.
XieSHuJBhowmickP, et al. Distributed motion planning for safe autonomous vehicle overtaking via artificial potential field. IEEE Trans Intell Transp Syst2022; 23: 21531–21547.
13.
LiuTTangXZhangJ, et al. Reinforcement learning-enabled decision-making strategies for a vehicle cyber physical system in connected environment. arXiv preprint arXiv:2007.09101, 2020.
14.
BojarskiMTestaDDDworakowskiD, et al. End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316, 2016.
15.
PomerleauDA.ALVINN: an autonomous land vehicle in a neural network. In: TouretzkyD (ed.) Advances in neural information processing systems 1. San Mateo, CA: Morgan Kaufmann, 1989, pp.305–313.
16.
ZhuLYuFRHeY, et al. Communication based train control system performance optimization using deep reinforcement learning. IEEE Trans Veh Technol2017; 66(12): 10705–10717.
17.
MnihVKavukcuogluKSilverD, et al. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013.
18.
KendallAHawkeJJanzD, et al. Learning to drive in a day. In: 2019 international conference on robotics and automation (ICRA), Montreal, QC, Canada, 20–24 May 2019, pp.8248–8254. New York: IEEE.
19.
LiXXuXZuoL.Reinforcement learning based overtaking decision-making for highway autonomous driving. In: 2015 sixth international conference on intelligent control and information processing (ICICIP), Wuhan, China, 26–28 November 2015, pp.336–342. New York: IEEE.
20.
NgaiDCKYungNHC. A multiple-goal reinforcement learning method for complex vehicle overtaking maneuvers. IEEE Trans Intell Transp Syst2011; 12: 509–522.
21.
ArgallBChernovaSVelosoMM, et al. A survey of robot learning from demonstration. Rob Auton Syst2009; 57(5): 469–483.
22.
RatliffNDBagnellJASrinivasaSS.Imitation learning for locomotion and manipulation. In: 2007 7th IEEE-RAS international conference on humanoid robots, Pittsburgh, PA, USA, 29 November–1 December 2007, pp.392–397. New York: IEEE.
23.
EnglertPParaschosAPetersJ, et al. Model-based imitation learning by probabilistic trajectory matching. In: 2013 IEEE international conference on robotics and automation, Karlsruhe, Germany, 6–10 May 2013, pp.1922–1927. New York: IEEE.
24.
AbbeelPCoatesANgAY.Autonomous helicopter aerobatics through apprenticeship learning. Int J Rob Res2010; 29(13): 1608–1639.
25.
RossSMelik-BarkhudarovNShankarKS, et al. Learning monocular reactive UAV control in cluttered natural environments. In: 2013 IEEE international conference on robotics and automation, Karlsruhe, Germany, 6–10 May 2013, pp.1765–1772. New York: IEEE.
26.
GiustiAGuzziJCiresanD, et al. A machine learning approach to visual perception of forest trails for mobile robots. IEEE Robot Autom Lett2016; 1(2): 661–667.
27.
ZiebartBDMaasALBagnellJA, et al. Maximum entropy inverse reinforcement learning. Proc Conf AAAI Artif Intell2008; 8: 1433–1438.
28.
ZiebartBDMaasALDeyAK, et al. Navigate like a cabbie: probabilistic reasoning from observed context-aware behavior. In: UbiComp, September2008, Seoul, Korea, pp.322–331. New York: ACM.
29.
ChenCSeffAKornhauserAL, et al. DeepDriving: learning affordance for direct perception in autonomous driving. In: 2015 IEEE international conference on computer vision (ICCV), Santiago, Chile, 7–13 December 2015, pp.2722–2730. New York: IEEE.
LiQChenLLiM, et al. A sensor-fusion drivable-region and lane-detection system for autonomous vehicle navigation in challenging road scenarios. Mach Vis Appl2014; 63(2): 540–550.
32.
RenSHeKGirshickR, et al. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Multi-Scale Comput Syst2017; 39(6): 1137–1149.
33.
ChenLFanLHuangK, et al. Moving-object detection from consecutive stereo pairs using slanted plane smoothing. Mach Vis Appl2017; 18(11): 3093–3102.
34.
ManenSGygliMDaiD, et al. Pathtrack: fast trajectory annotation with path supervision. In: 2017 IEEE international conference on computer vision (ICCV), Venice, Italy, 22–29 October 2017, pp.290–299. New York: IEEE.
35.
ScarselliFGoriMTsoiAC, et al. The graph neural network model. IEEE Trans Neural Netw2009; 20(1): 61–80.
36.
VeličkovićPCucurullGCasanovaA, et al. Graph attention networks. arXiv preprint arXiv:1710.10903, 2017.
37.
HuegleMKalweitGWerlingM, et al. Dynamic interaction-aware scene understanding for reinforcement learning in autonomous driving. In: 2020 IEEE international conference on robotics and automation (ICRA), Paris, France, 31 May–31 August 2020, pp.4329–4335. New York: IEEE.
38.
HartPKnollA.Graph neural networks and reinforcement learning for behavior generation in semantic environments. arXiv preprint arXiv:2006.12576, 2020.
39.
DosovitskiyAKoltunV.Learning to act by predicting the future. arXiv preprint arXiv:1611.01779, 2016.
40.
DixitSMontanaroUDianatiM, et al. Trajectory planning for autonomous high-speed overtaking in structured environments using robust MPC. IEEE Trans Intell Transp Syst2020; 21(6): 2310–2323.
41.
KingmaDPBaJ.Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
