Vision-based multi-task learning (V-MTL) has significantly promoted the autonomous driving. However, the state-of-the-art (SOTA) V-MTL method still has limited receptive fields and unsatisfied feature extraction. It is insufficient to handle fast scale variation, frequent illumination variation, and partial occlusion, especially in tunnels, skyways, and highways. To address these issues, this work proposes a novel ghosting fusion network, that is, GF-Net, for autonomous driving. Specifically, a receptive field-aware parallel atrous ghosting is presented to enlarge the receptive field and capture context information with low computation cost. A spatial-aware dynamic feature fusion is developed to integrate features generated with different receptive fields and suppress the conflicting spatial information. Comprehensive experiments on the well-known autonomous driving benchmark demonstrate the superiority of GF-Net in comparison with the SOTA approaches. Besides, real-world tests with various challenges have proven the robustness and accuracy of GF-Net.
RenSHeKGirshickR, et al. Faster R-CNN: towards realtime object detection with region proposal networks. In: Proceedings conference neural information processing systems (NeurIPS), 2015, pp.91–99.
2.
HeKZhangXRenS, et al. Deep residual learning for image recognition. In: Proceedings of IIEEE conference on computer vision and pattern recognition (CVPR), June 2016, pp.770–778.
3.
LiuWAnguelovDErhanD, et al. SSD: single shot multibox detector. In: Proceedings of European conference on computer vision (ECCV), October 2016, pp.21–37.
4.
BochkovskiyAWangCYLiaoHYM. YOLOv4: optimal speed and accuracy of object detection, arXiv preprint arXiv:2004.10934, pp.1–17, 2020.
5.
WangCYBochkovskiyALiaoHYM. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), June 2023, pp.7464–7475.
6.
BadrinarayananVKendallACipollaR.SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell2021; 39(12): 2481–2495.
7.
HuangHLinLTongR, et al. UNet 3+: a full-scale connected UNet for medical image segmentation. In: Proceedings of international conference on acoustics, speech and signal processing (ICASSP), 2020, pp.1055–1059.
8.
ZhaoHShiJQiX, et al. Pyramid scene parsing network. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), 2017, pp.2881–2890.
9.
RomeraEAlvarezJMBergasaLM, et al. ERFNet: efficient residual factorized ConvNet for real-time semantic segmentation. IEEE Trans Intell Transp Syst2018; 19(1): 263–272.
10.
PanXShiJLuoP, et al. Spatial as deep: spatial CNN for traffic scene understanding. Proc AAAI Conf Artif Intell (AAAI)2018; 32(1): 1–8.
11.
HouYMaZLiuC.Learning lightweight lane detection CNNs by self attention distillation. In: Proceedings of IEEE/CVF international conference on computer vision (ICCV), 2019, pp.1013–1021.
12.
WangZRenWQiuQ.Lanenet: real-time lane detection networks for autonomous driving. arXiv preprint arXiv:1807.01726, pp.1–9, 2018.
13.
TabeliniLBerrielRPaixaoTM, et al. Keep your eyes on the lane: real-time attention-guided lane detection. In: Proceedings of IEEE/CVF international conference on computer vision (ICCV), 2021, pp.294–302.
14.
TeichmannMWeberMZoellnerM, et al. Multinet: real-time joint semantic reasoning for autonomous driving. In: Proceedings of IEEE intelligent vehicles symposium (IV), 2018, pp.1013–1020.
15.
QianYDolanJMYangM.DLT-Net: joint detection of drivable areas, lane lines, and traffic objects. IEEE Trans Intell Transp Syst2020; 21(11): 4670–4679.
WuDLiaoMWZhangWT, et al. YOLOP: you only look once for panoptic driving perception. Mach Intell Res2022; 19: 550–562.
18.
LinTYDollárPGirshickR, et al. Feature pyramid networks for object detection. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), 2017, pp.2117–2125.
19.
GhiasiGLinTYLeQV. NAS-FPN: learning scalable feature pyramid architecture for object detection. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), 2019, pp.7036–7045.
20.
YuCXiaoBGaoC, et al. Lite-HRnet: a lightweight high-resolution network. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), 2021, pp.10440–10450.
21.
YuCWangJPengC, et al. BiSeNet: bilateral segmentation network for real-time semantic segmentation. In: Proceedings of European conference on computer vision (ECCV), 2018, pp.325–341.
22.
XianWChenYLiuF, et al. Bdd100k: a diverse driving dataset for heterogeneous multitask learning. in Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), 2020, pp.2636–2645.
23.
HeKGkioxariGDollarP, et al. Mask R-CNN. In: Proceedings of IEEE/CVF international conference on computer vision (ICCV), 2017, pp.2961–2969.
24.
GirshickR. Fast R-CNN. In: Proceedings of IEEE/CVF international conference on computer vision (ICCV), 2015, pp.1440–1448.
25.
RonnebergerOFischerPBroxT. U-Net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer assisted intervention (MICCAI), 2015, pp.234–241.
26.
TabeliniLBerrielRPaixaoTM, et al. Keep your eyes on the lane: real-time attentionguided lane detection. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), 2021, pp.294–302
27.
WuBWanAIandolaF, et al. SqueezeDet: unified, small, low power fully convolutional neural networks for real-time object detection for autonomous driving. In: Proceedings of IEEE conference on computer vision and pattern recognition workshops (CVPRW), 2017, pp.446–454.
28.
GurghianAKoduriTBailurSV, et al. DeepLanes: end-to-end lane position estimation using deep neural networks. In: Proceedings of IEEE conference on computer vision and pattern recognition workshops (CVPRW), 2016, pp.38–45.
29.
Ravi KumarVYogamaniSRashedH, et al. OmniDet: surround view cameras based MultiTask visual perception network for autonomous driving. IEEE Robot Autom Lett2021; 6: 2830–2837.
30.
HanCZhaoQZhangS, et al. Yolopv2: Better, faster, stronger for panoptic driving perception. arXiv preprint arXiv:2208.11434, pp.1–8, 2022.
31.
HuJJinLGaoS. FPN++: a simple baseline for pedestrian detection. In: IEEE International conference on multimedia and expo (ICME), 2019, pp.1138–1143
32.
HeKZhangXRenS, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell2015; 37(9): 1904–1916.
33.
ChenYWangYZhangY, et al. PANet: a context based predicate association network for scene graph generation. In: IEEE International conference on multimedia and expo (ICME), 2019, pp.508–513.
34.
WangCYBochkovskiyALiaoHYM. Scaled-yolov4: scaling cross stage partial network. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), 2021, pp.13029–13038.
35.
HanKWangYTianQ, et al. Ghostnet: more features from cheap operations. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), 2020, pp.1580–1589.
LiuSQiLQinH, et al. Path aggregation network for instance segmentation. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), 2018, pp.8759–8768.
WangJJonathan WuQMZhangN.You only look at once for real-time and generic multi-task. IEEE Trans Vehicular Technol2024; 73(9): 12625–12637.
42.
CheQHNguyenDPPhamMQ, et al. TwinLiteNet: an efficient and lightweight model for driveable area and lane segmentation in self-driving cars. In: Proceedings of international conference multimedia analysis and pattern recognition (MAPR), 2023, pp.1–6.
43.
PaszkeAChaurasiaAKimS, et al. Enet: a deep neural network architecture for real-time semantic segmentation. arXiv preprint arXiv:1606.02147, pp.1–10, 2016.
44.
XiaoLLiXYangS, et al. ADNet: lane shape prediction via anchor decomposition. In: In: Proceedings of IEEE/CVF international conference on computer vision (ICCV), 2023, pp.6381–6390.
45.
ZhengTHuangYLiuY, et al. CLRNet: cross layer refinement network for lane detection. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), 2022, pp.888–897.
