An image stitching method based on multi-stage feature matching and GAN for autonomous driving

Abstract

Image stitching plays a significant role in the environmental perception of autonomous vehicles by generating panoramic images from the images captured by surrounding cameras. However, existing deep learning methods still face the problem of poor homography estimation when handing images with complex backgrounds, resulting in ghosting artifacts in the stitching results. In this paper, we propose an image stitching method based on multi-stage feature matching and GAN for autonomous vehicles. It consists of a stitching network, an optimization network, and a GAN framework. The stitching network employs deformable convolutions to match the nonlinear contour features of the same object among images from different viewpoints, and extract the homography relationship to warp the images and perform dynamic fusion. The optimization network adopts a skip connection structure to fuses feature maps of different scales. To further improve visual quality of image stitching, a GAN framework is employed to provide feedback to the model during the training process and guide it to produce seamless image stitching results. Both quantitative and qualitative evaluations demonstrate that the proposed image stitching method outperforms the existing state-of-the-art methods.