To address the issues of misdetection and missed detection in nighttime lane marking detection due to factors such as occlusion, insufficient lighting, glare, and interference from objects similar to lane markings, this paper proposes a nighttime structured road lane marking detection method based on LiDAR and camera fusion. LiDAR is used in the first stage to detect road boundaries and define the lane detection region based on the extracted boundary points. In the second stage, a camera performs lane detection within the LiDAR-defined region using an improved UFLD method. The first-stage method achieves an accuracy of 93.2% on a randomly selected 300 frames from the KITTI dataset. The second-stage method achieves F1 scores of 73.8% on the CULane dataset and accuracies of 95.26% and 77.61% on the TuSimple and self-built datasets, respectively. The real-world deployment of the fusion method shows a 1.05% improvement in accuracy compared to the non-fused algorithm. The experimental results demonstrate that the proposed method has good performance in lane marking detection on structured roads at night.
LiYChenLHuangH, et al. Nighttime lane markings recognition based on Canny detection and Hough transform. In: 2016 IEEE international conference on real-time computing and robotics (RCAR), Angkor Wat, Cambodia, 06–10 June 2016, vol. 1, pp.411–415. IEEE.
2.
MaCLuoDHuangH. Lane line detection based on improved semantic segmentation in complex road environment. Sens Mater2021; 33: 4545.
3.
ZhengFLuoSSongK, et al. Improved lane line detection algorithm based on Hough transform. Pattern Recognit Image Anal2018; 28: 254–260.
4.
CaoXLiuDRenX. Detection method for auto guide vehicle’s walking deviation based on image thinning and Hough transform. Meas Control2019; 52(3–4): 252–261.
5.
LiXYinPZhiY, et al. Vertical lane line detection technology based on Hough transform. IOP Conf Ser: Earth Environ Sci2020; 440(3): 1–12.
6.
RuyiJReinhardKTobiV, et al. Lane detection and tracking using a new lane model and distance transform. Mach Vis and Appl2010; 22: 721–737.
7.
AjaykumarRGuptaAMerchantSN. Automated lane detection by K-means clustering: a machine learning approach. Electron Imaging2016; 28: 1–6.
8.
LeeSKimJShin YoonJ, et al. Vpgnet: vanishing point guided network for lane and road marking detection and recognition. In: Proceedings of the IEEE international conference on computer vision (ICCV), Venice, Italy, 22–29 October 2017, pp.1947–1955. IEEE.
9.
HouYMaZLiuC, et al. Learning lightweight lane detection cnns by self attention distillation. In: Proceedings of the IEEE/CVF international conference on computer vision (ICCV), Seoul, Korea (South), 27 October–02 November 2019, pp.1013–1021. IEEE.
10.
TabeliniLBerrielRPaixaoTM, et al. Keep your eyes on the lane: real-time attentionguided lane detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), Nashville, TN, USA, 20–25 June 2021, pp.294–302. IEEE.
11.
SongSChenWLiuQ, et al. A novel deep learning network for accurate lane detection in low-light environments. Proc Inst Mech Eng D: J Automob Eng2022; 236(2–3): 424–438.
12.
GuanHLiJYuY, et al. Using mobile laser scanning data for automated extraction of road markings. ISPRS J Photogramm Remote Sens2014; 87: 93–107.
13.
ZhaoRHengYWangH, et al. Advancements in 3d lane detection using lidar point clouds: from data collection to model development. In: 2024 IEEE international conference on robotics and automation (ICRA), Yokohama, Japan, 13–17 May 2024, vol. 1, pp.5382–5388. IEEE.
14.
HuJChenNXuWC, et al. Dual-lidar 3D lane detection based on adaptive gating. Chin J. Lasers2023; 50(22): 2210001.
15.
WangYTsaiY. A lane detection method based on 3D-LiDAR. In: Proceedings of the 16th ITS Asia-Pacific Forum, Fukuoka, Japan, 8–10 May 2018, pp. 1–10.
16.
ShenZHeYDuX, et al. YCANet: target detection for complex traffic scenes based on camera-LiDAR fusion. IEEE Sens J2024; 24(6): 8379–8389.
17.
BaiMMattyusGHomayounfarN, et al. Deep multi-sensor lane detection. In: 2018 IEEE/RSJ International conference on intelligent robots and systems (IROS), Madrid, Spain, 01-05 October 2018, vol. 1, pp.3102–3109. IEEE.
18.
ZouZZhangXLiuH, et al. A novel multimodal fusion network based on a joint-coding model for lane line segmentation. Inf Fusion2022; 80: 167–178.
19.
HanYWangBGuanT, et al. Research on road environmental sense method of intelligent vehicle based on tracking check. IEEE Trans Intell Transp Syst2022; 24(1): 1261–1275.
20.
SchrammJVödischNPetekK, et al. BEVCar: camera-radar fusion for BEV map and object segmentation. In: 2024 IEEE/RSJ international conference on intelligent robots and systems (IROS), Abu Dhabi, United Arab Emirates, 14–18 October 2024, pp.1435–1442. IEEE.
21.
LiuSZouRXuP. A fast and stable road edge detection method based on LiDAR and camera. Meas Sci Technol2025; 36(7): 075205.
22.
LyuWKeWShengH, et al. Dynamic downsampling algorithm for 3D point cloud map based on voxel filtering. Appl Sci2024; 14(8): 3160.
23.
LeeSAnSKimJ, et al. Grid-Based DBSCAN clustering accelerator for LiDAR’s point cloud. Electronics2024; 13(17): 3395.
24.
DąbrowskiPSZienkiewiczMH. 3D point-cloud spatial expansion by total least-squares line fitting. Photogramm Rec2020; 35(172): 509–527.
25.
LiJWenYHeL. Scconv: spatial and channel reconstruction convolution for feature redundancy. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), Vancouver, BC, Canada, 17–24 June 2023, pp.6153–6162. IEEE.
26.
NiuZZhongGYuH. A review on the attention mechanism of deep learning. Neurocomputing2021; vol. 452: pp.48–62.
27.
GeigerALenzPStillerC, et al. Vision meets robotics: the kitti dataset. Int J Robotics Res2013; 32(11): 1231–1237.
28.
National Technical Committee of Automotive Standardization (SAC/TC 114). Performance requirements and test methods for lane keeping assist systems of commercial vehicles: GB/T 41796-2022. China Standards Press, 2022.
HeKZhangXRenS, et al. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016, pp.770–778. IEEE.
32.
QinZWangHLiX. Ultra fast structure-aware deep lane detection. In: Proceedings of the European conference on computer vision (ECCV), Glasgow, UK, 23–28 August 2020, vol. 1, pp.276–291. Springer International Publishing.
33.
PanXShiJLuoP, et al. Spatial as deep: spatial CNN for traffic scene understanding. In: Proceedings of the AAAI conference on artificial intelligence, New Orleans, Louisiana, USA, 02 February 2018, pp.7276–7283. AAAI Press.
34.
WangZRenWQiuQ. Lanenet: real-time lane detection networks for autonomous driving. arXiv2018. DOI: 10.48550/arXiv.1807.01726.
35.
PhilionJ. FastDraw: addressing the long tail of lane detection by adapting a sequential prediction network. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Long Beach, CA, USA, 15–20 June 2019, pp.11582–11591.
36.
ChenLCPapandreouGKokkinosI, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Machine Intell2017; 40(4): 834–848.
