Research on multi-task visual environment perception method for intelligent driving

Abstract

Environment perception is a key technology to ensure the safe and stable driving of intelligent vehicles. Aiming at the problem that the existing single-task model cannot satisfy diversified perception, a multi-task environment perception algorithm MTP-Net (Multi Task Perception-Network) is proposed, which can simultaneously complete the vehicle detection task and the lane line segmentation task. MTP-Net uses CSP-Darknet53 as an efficient shared backbone network, introduces BiFPN (Bidirectional Feature Pyramid Network) multi-scale weighted feature fusion and a small target detection layer to achieve vehicle detection with high accuracy, adopts DySample dynamic up-sampling and encoder-decoder structure to achieve accurate lane line segmentation, and combines the design of weighted loss function to effectively improve the overall performance of multi-task learning. In addition, to address the problem of insufficient performance of existing multi-target tracking methods, based on the prediction results of the MTP-Net vehicle detection branch, an improved ByteTrack_UKF algorithm is proposed by introducing the acceleration state variable and using UKF (Unscented Kalman Filter), which achieves the real-time stable tracking of multi-target vehicles. The experimental results show that the mAP50 of MTP-Net vehicle detection is 82.1%, the IoU of lane line segmentation is 28.2% and the IDsw of multi-target vehicle tracking is reduced by 42.86% compared to the original algorithm. The MTP-Net integrates the ByteTrack_UKF algorithm and deploys it on the edge computing device NVIDIA Jetson Xavier NX. The inference frame rate is 47.91 FPS, which can efficiently complete the vehicle detection, tracking and lane line segmentation tasks, and realise the real-time and diversified sensing of the environment by intelligent vehicles.