DL-YOLOX: Real-time object detection via adjustable dilated enhancement for autonomous driving scene

Abstract

In the domain of autonomous driving, object detection presents several complex challenges, particularly concerning the accurate identification of small and salient objects. This paper introduces DL-YOLOX (Dilated Enhancement YOLOX), which flexibly uses dilated convolution to enhance features to achieve the purpose of improving small objects and silent objects. As we all know, a large receptive field covers a larger area and has greater contextual information, which is more advantageous for detecting large targets. A small receptive field helps capture local details and has better detection capabilities for detecting small targets. To bolster the representation of objects across various scales, we propose the integration of Dilated Adaptive Feature Fusion (DAFF) which has the ability to adaptively fuse features with different receptive fields. This innovative fusion mechanism allows for a more comprehensive understanding of objects, enabling improved detection accuracy even for objects of varying sizes. In addition, we tackle the issue of small object loss during feature propagation by introducing Stack Dilated Module (SDM), a powerful module that mitigates this phenomenon and contributes to better detection performance. Moreover, we endeavor to enhance small object detection further by replacing the conventional Intersection over Union (IoU) metric with Normalized Gaussian Wasserstein Distance (NWD), a novel distance metric that proves to be more effective in accurately gauging small object detection, thus elevating the precision of our algorithm. To thoroughly evaluate the robustness and generalization capabilities of our proposed method, we conduct extensive experiments on two benchmark datasets, namely MS COCO 2017 and BDD100K. The results from our evaluation not only affirm the significant improvements achieved in multi-scale object detection but also highlight the real-time capability of our approach. The impressive performance across these datasets demonstrates the promising potential of DL-YOLOX in revolutionizing object detection techniques in the context of autonomous driving.

Keywords

Object detection autonomous driving feature fusion dilated convolution Gaussian Wasserstein Distance

Get full access to this article

View all access options for this article.

References

Cai

Vasconcelos

(2018) Cascade R-CNN: Delving into high quality object detection. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, Salt Lake City, UT, 18–23 June.

Carion

Massa

Synnaeve

, et al. (2020) End-to-end object detection with transformers. Computer Vision—ECCV 2020: 213–229.

Chen

Papandreou

Kokkinos

, et al. (2018) DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence 40: 834–848.

Chen

Papandreou

Schroff

, et al. (2017) Rethinking atrous convolution for semantic image segmentation. Available at: https://arxiv.org/abs/1706.05587

Dosovitskiy

Beyer

Kolesnikov

, et al. (2020) An image is worth 16x16 words: Transformers for image recognition at scale. Available at: https://arxiv.org/abs/2010.11929

Duan

Bai

Xie

, et al. (2019) CenterNet: Keypoint triplets for object detection. In: 2019 IEEE/CVF international conference on computer vision (ICCV), Seoul, Korea, 27 October–2 November, pp. 6569–6578. New York: IEEE.

Liu

Wang

, et al. (2021) YOLOX: Exceeding YOLO series in 2021. Available at: https://arxiv.org/abs/2107.08430

Girshick

Donahue

Darrell

, et al. (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: 2014 IEEE conference on computer vision and pattern recognition, Columbus, OH, 23–28 June, pp. 580–587. New York: IEEE.

Hamaguchi

Fujita

Nemoto

, et al. (2018) Effective use of dilated convolutions for segmenting small object instances in remote sensing imagery. In: 2018 IEEE winter conference on applications of computer vision (WACV), Lake Tahoe, NV, 12–15 March, pp. 1442–1450. New York: IEEE.

10.

Shen

Sun

(2018) Squeeze-and-excitation networks. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, Salt Lake City, UT, 18–23 June, pp. 7132–7141. New York: IEEE.

11.

Huang

Liu

Van Der Maaten

, et al. (2017) Densely connected convolutional networks. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), Honolulu, HI, 21–26 July, pp. 2261–2269. New York: IEEE.

12.

Kalchbrenner

Oord

Simonyan

, et al. (2017) Video pixel networks. Available at: https://arxiv.org/abs/1610.00527

13.

Law

Deng

(2019) 08. Cornernet: Detecting objects as paired keypoints. International Journal of Computer Vision 128: 642–656.

14.

Wang

, et al. (2019) Selective kernel networks. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR), Long Beach, CA, 15–20 June, pp. 510–519. New York: IEEE.

15.

Lin

Dollàr

Girshick

, et al. (2017) Feature pyramid networks for object detection. In: 2017 IEEE conference on computer vision and pattern recognition, Honolulu, HI, 21–26 July, pp. 2117–2125. New York: IEEE.

16.

Lin

Goyal

Girshick

, et al. (2020) Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 42: 318–327.

17.

Lin

Maire

Belongie

, et al. (2014) Microsoft COCO: Common objects in context. Computer Vision—ECCV 20148693: 740–755. Available at: https://doi.org/10.1007/978-3-319-10602-1_48

18.

Liu

Qin

, et al. (2018) Path aggregation network for instance segmentation. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, Salt Lake City, UT, 18–23 June, pp. 8759–8768. New York: IEEE.

19.

Liu

Anguelov

Erhan

, et al. (2016) SSD: Single shot multibox detector. Computer Vision—ECCV 20169905: 21–37. Available at: https://doi.org/10.1007/978-3-319-46448-0_2

20.

Liu

Lin

Cao

, et al. (2021) Swin transformer: Hierarchical vision transformer using shifted windows. In: 2021 IEEE/CVF international conference on computer vision (ICCV), Montreal, QC, Canada, 10–17 October, pp. 10012–10022. New York: IEEE.

21.

Long

Shelhamer

Darrell

(2015) Fully convolutional networks for semantic segmentation. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), Boston, MA, 7–12 June, pp. 3431–3440. New York: IEEE.

22.

Oord

Dieleman

Zen

, et al. (2016) WaveNet: A generative model for raw audio. Available at: https://arxiv.org/abs/1609.03499

23.

Purkait

Zhao

Zach

(2017) SPP-Net: Deep absolute pose regression with synthetic views. Available at: https://arxiv.org/abs/1712.03452

24.

Redmon

Divvala

Girshick

, et al. (2016) You only look once: Unified, real-time object detection. In: 2016 IEEE conference on computer vision and pattern recognition, Las Vegas, NV, 27–30 June, pp. 779–788. New York: IEEE.

25.

Ren

Girshick

, et al. (2015) Faster R-CNN: Towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems 28: 1440–1448.

26.

Szegedy

Liu

Jia

, et al. (2015) Going deeper with convolutions. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), Boston, MA, 7–12 June, pp. 1–9. New York: IEEE.

27.

Tan

(2019) EfficientNet: Rethinking model scaling for convolutional neural networks. International Conference on Machine Learning 97: 6105–6114.

28.

Tan

Pang

(2020) EfficientDet: Scalable and efficient object detection. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), Seattle, WA, 13–19 June, pp. 10781–10790. New York: IEEE.

29.

Wang

Yang

, et al. (2021) A normalized Gaussian Wasserstein Distance for tiny object detection. Available at: https://arxiv.org/abs/2110.13389

30.

Wang

Chen

Yuan

, et al. (2018) Understanding convolution for semantic segmentation. In: 2018 IEEE winter conference on applications of computer vision (WACV), Lake Tahoe, NV, 12–15 March, pp. 1451–1460. New York: IEEE.

31.

Yang

Zhang

, et al. (2018) DenseASPP for semantic segmentation in street scenes. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, Salt Lake City, UT, 18–23 June, pp. 3684–3692. New York: IEEE.

32.

Yao

, et al. (2021) Efficient DETR: Improving end-to-end object detector with dense prior. Available at: https://arxiv.org/abs/2104.01318

33.

Koltun

(2016) Multi-scale context aggregation by dilated convolutions. Available at: https://arxiv.org/abs/1511.07122

34.

Chen

Wang

, et al. (2020) BDD100K: A diverse driving dataset for heterogeneous multitask learning. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), Seattle, WA, 13–19 June, pp. 2636–2645. New York: IEEE.

35.

Koltun

Funkhouser

(2017) Dilated residual networks. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), Honolulu, HI, 21–26 July, pp. 472–480. New York: IEEE.

36.

Zheng

Dong

, et al. (2023) Less is more: Focus attention for efficient DETR. In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 6674–6683. Available at: https://openaccess.thecvf.com/content/ICCV2023/papers/Zheng_Less_is_More_Focus_Attention_for_Efficient_DETR_ICCV_2023_paper.pdf

37.

Zhu

, et al. (2021) Deformable detr: Deformable transformers for end-to-end object detection. Available at: https://arxiv.org/abs/2010.04159

38.

Zou

Fang

, et al. (2021) Group-based atrous convolution stereo matching network. Wireless Communications and Mobile Computing 2121: 7386280.