Deep learning-based path tracking control using lane detection and traffic sign detection for autonomous driving

Abstract

Automated vehicles are a significant advancement in transportation technique, which provides safe, sustainable, and reliable transport. Lane detection, maneuver forecasting, and traffic sign recognition are the fundamentals of automated vehicles. Hence, this research focuses on developing a dynamic real-time decision-making system to obtain an effective driving experience in autonomous vehicles with the advancement of deep learning techniques. The deep learning classifier such as deep convolutional neural network (Deep CNN), SegNet and are utilized in this research for traffic signal detection, road segmentation, and lane detection. The main highlight of the research relies on the proposed Finch Hunt optimization, which involves the hyperparameter tuning of a deep learning classifier. The proposed real-time decision-making system achieves 97.44% accuracy, 97.56% of sensitivity, and 97.83% of specificity. Further, the proposed segmentation model achieves the highest clustering accuracy with 90.37% and the proposed lane detection model attains the lowest mean absolute error, mean square error, and root mean error of 17.76%, 11.32%, and 5.66% respectively. The proposed road segmentation model exceeds all the competent models in terms of clustering accuracy. Finally, the proposed model provides a better output for lane detection with minimum error, when compared with the existing model.

Keywords

Automated vehicles smart transport system lane detection maneuver prediction traffic sign detection deep learning

Get full access to this article

View all access options for this article.

References

Azad,

Azad and

I.T.

Kazerooni, Optimized method for Iranian road signs detection and recognition system, 2014, arXiv preprint arXiv:1407.5324.

Benenson,

Fraichard and

Parent, Achievable safety of driverless ground vehicles, in: Proceedings of 10th International Conference on Control, Automation, Robotics and Vision, 2008, pp. 515–521.

Breuer,

Kirschner,

Homoceanu and

Fingscheidt, Towards tactical maneuver detection for autonomous driving based on vision only, in: Proceedings of IEEE Intelligent Vehicles Symposium (IV), 2019, pp. 941–948.

Hu,

Xiong,

Zha and

Fu, Lane detection and trajectory tracking control of autonomous vehicle based on model predictive control, International journal of automotive technology 21(2) (2020), 285–295. doi:10.1007/s12239-020-0027-6.

Izquierdo,

Á.

Quintanar,

Lorenzo,

García-Daza,

Parra,

Fernández-Llorca and

M.Á.

Sotelo, Vehicle lane change prediction on highways using efficient environment representation and deep learning, IEEE Access 9 (2021), 119454–119465. doi:10.1109/ACCESS.2021.3106692.

Junaid,

Ghafoor,

Hassan,

Khalid,

S.A.

Tariq,

Ahmed and

Zia, Multi-feature view-based shallow convolutional neural network for road segmentation, IEEE Access 8 (2020), 36612–36623. doi:10.1109/ACCESS.2020.2968965.

Khairdoost,

Shirpour,

M.A.

Bauer and

S.S.

Beauchemin, Real-time driver maneuver prediction using LSTM, IEEE Transactions on Intelligent Vehicles 5(4) (2020), 714–724. doi:10.1109/TIV.2020.3003889.

D.H.

Lee and

J.L.

Liu, End-to-end deep learning of lane detection and path prediction for real-time autonomous driving, in: Signal, Image, and Video Processing, 2022, pp. 1–7.

Li,

Abdel-Aty and

Islam, Driving maneuvers detection using semi-supervised long short-term memory and smartphone sensors, Transportation research record 2675(9) (2021), 1386–1397. doi:10.1177/03611981211007483.

10.

Liu and

Khattak, Informed decision-making by integrating historical on-road driving performance data in high-resolution maps for connected and automated vehicles, Journal of Intelligent Transportation Systems 24(1) (2020), 11–23. doi:10.1080/15472450.2019.1699076.

11.

Liu,

Wang,

Li,

Cheng and

Chen, A novel lane change decision-making model of autonomous vehicle based on support vector machine, IEEE access 7 (2019), 26543–26550. doi:10.1109/ACCESS.2019.2900416.

12.

Loy and

Barnes, Fast shape-based road sign detection for a driver assistance system, in: Proceedings of 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No. 04CH37566), Vol. 1, 2004, pp. 70–75. doi:10.1109/IROS.2004.1389331.

13.

Mirjalili,

S.M.

Mirjalili and

Lewis, Grey wolf optimizer, Advances in engineering software 69 (2014), 46–61. doi:10.1016/j.advengsoft.2013.12.007.

14.

Ou and

Karray, Deep learning-based driving maneuver prediction system, IEEE transactions on vehicular technology 69(2) (2019), 1328–1340. doi:10.1109/TVT.2019.2958622.

15.

C.F.

Paulo and

P.L.

Correia, Automatic detection and classification of traffic signs, in: Proceedings of Eighth International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS’07), 2007, pp. 11.

16.

Robust-Lane-Detection, https://github.com/qinnzou/Robust-Lane-Detection, Accessed on February 2023.

17.

H.G.

Seif and

Hu, Autonomous driving in the iCity – HD maps as a key challenge of the automotive industry, Engineering 2(2) (2016), 159–162. doi:10.1016/J.ENG.2016.02.010.

18.

Traffic Sign Dataset – Classification, https://www.kaggle.com/datasets/ahemateja19bec1025/traffic-sign-dataset-classification, Accessed on February 2023.

19.

Wang,

Huang,

Khajepour,

Cao and

Lv, Ethical decision-making platform in autonomous vehicles with lexicographic optimization based model predictive controller, IEEE transactions on vehicular technology 69(8) (2020), 8164–8175. doi:10.1109/TVT.2020.2996954.

20.

Wang,

Qie,

Yang,

Liu,

Xiang and

Huang, An intelligent lane-changing behavior prediction and decision-making strategy for an autonomous vehicle, IEEE Transactions on Industrial Electronics 69(3) (2021), 2927–2937. doi:10.1109/TIE.2021.3066943.

21.

Wang,

Ren and

Qiu, Lanenet: Real-time lane detection networks for autonomous driving, 2018, arXiv preprint arXiv:1807.01726.

22.

Xue and

Shen, A novel swarm intelligence optimization approach: Sparrow search algorithm, Systems Science & Control Engineering 8(1) (2020), 22–34. doi:10.1080/21642583.2019.1708830.

23.

Zhang and

Zhu, Autonomous path tracking control of intelligent electric vehicles based on lane detection and optimal preview method, Expert Systems with Applications 121 (2019), 38–48. doi:10.1016/j.eswa.2018.12.005.