CNN-transformer hybrid network for automated segmentation of multiple defects in concrete bridges

Abstract

Regular inspection of damaged areas of concrete bridges is critical to the safe service of bridges. However, the complexity of bridge environments and multiple defect types pose challenges to deep learning-based inspection methods. To address these issues, this study proposes a novel segmentation model for multiple defects in concrete bridges, named ConcreteBridgeSeg-Net (CBS-Net). Firstly, it employs a convolutional neural network (CNN) to extract local features, followed by a novel Transformer integrated with a Multi-head Transposed Attention (MDTA) mechanism for global feature extraction. Subsequently, an auxiliary feature extraction branch parallel to Transformer was designed to enhance local feature extraction, and a channel feature fusion module (CFFM) was employed to fuse the feature information from both. Lastly, Efficient Channel Attention (ECA) is embedded within skip connections to minimize interference from redundant information. The experimental results show that the mean intersection over union (mIou), mean pixel accuracy (mPA), and mean precision of CBS-Net in the test set are 78.51%, 88.79%, and 86.76%, respectively, and the accuracy metrics are higher than those of other state-of-the-art segmentation models. Notably, CBS-Net achieves an inference speed of 38.72 frames per second (FPS) with an input image size of 512 × 512, demonstrating a fast segmentation rate while maintaining segmentation accuracy. Furthermore, it exhibits excellent generalization performance on public datasets, thereby confirming its applicability to real-world bridge inspection scenarios.