DSRCF: Radar-camera fusion with dynamic filtering and coupled matching for object detection across driving scenarios

Abstract

Multi-modal fusion plays a crucial role in enhancing the accuracy and robustness of autonomous driving perception systems. Due to the robust performance of millimeter-wave radar in adverse weather, the fusion of economical radar and camera sensors for object detection is an active research topic. However, the sparsity, multi-noise, and multipath interference of radar data pose significant challenges in effectively extracting and utilizing radar information, limiting the performance of the camera and radar fusion system. To address these issues, a novel fusion framework of radar and camera for efficient object detection across scenes is proposed, which achieves high detection accuracy while maintaining a fast inference speed. Specifically, a multi-dimensional dynamic filtering radar preprocessing method, which performs dynamic multi-target matching across consecutive frames, is developed to address discontinuity and low accuracy in radar feature extraction. During this period, the information expansion of the extracted radar features improves the contribution value of radar information within the fusion architecture. Besides, a coupled matching fusion strategy is implemented, enhancing the robustness of the proposed framework in cross-scenario detection tasks. Comprehensive experiments and ablation studies conducted on the nuScenes dataset validate the effectiveness of the proposed method, yielding a 4.8% improvement in cross-scenario mAP and up to a 7.5% improvement in single-scenario mAP.