Aiming to automatically, precisely, and rapidly detect tunnel lining cracks from images and extract geometric information for structural condition assessment, this study proposes a novel tunnel lining crack segmentation network (TCSegNet) and establishes a framework for calculating key geometric parameters of cracks. A tunnel lining crack segmentation dataset is first built by conducting on-site inspections of metro tunnels and collecting open-sourced tunnel images. Afterward, the TCSegNet, conforming to the encoder–decoder architectural paradigm, is designed to separate cracks from lining images pixel-to-pixel. An improved ConvNeXt and developed efficient atrous spatial pyramid pooling module constitute the encoder. The skip connections, upsampling modules, and tailored segmentation head form the decoder. Upon the segmentation results of TCSegNet, a computing framework integrating multiple digital image processing techniques is proposed to obtain the length, average width, and maximum width of cracks. The experimental results show that the TCSegNet achieves leading results among several dominant models, with 70.78% mean intersection over union (mIoU) and 57.43% F1 score. Furthermore, the TCSegNet has 32.01 million parameters, requires 55.13 billion floating point operations, and gets 107.28 frames per second, proving that it has low time and space complexities and implements real-time segmentation. Also, the rationality and effectiveness of TCSegNet in alleviating the crack disjoint problem and preserving crack edge details are verified through comparative experiments. In addition, the TCSegNet achieves 71.99%, 70.45%, and 70.23% mIoU in high-resolution image segmentation, robustness, and generalization tests, respectively, demonstrating that it is competent for detecting high-resolution lining images, has a solid resistance to illumination variations, and can be well generalized to other tunnel lining image datasets. Finally, the applicability of the crack quantification framework is validated by practical application examples. The developed approaches in this study provide pixel-level segmentation results and detailed measurements of concrete lining cracks to assess tunnel structural safety status.
JiangYWangLZhangB, et al. Tunnel lining detection and retrofitting. Autom Constr2023; 152: 104881.
2.
FengSJFengYZhangXL, et al. Deep learning with visual explanations for leakage defect segmentation of metro shield tunnel. Tunnelling Underground Space Technol2023; 136: 105107.
3.
FengYFengSJZhangXL, et al. A two-step deep learning-based framework for metro tunnel lining defect recognition. Tunnelling Underground Space Technol2024; 150: 105832.
4.
LeiMLiuLShiC, et al. A novel tunnel-lining crack recognition system based on digital image technology. Tunnelling Underground Space Technol2021; 108: 103724.
5.
ZhouZYanLZhangJ, et al. Automatic segmentation of tunnel lining defects based on multiscale attention and context information enhancement. Constr Build Mater2023; 387: 131621.
6.
FengYZhangXFengS, et al. Improved SOLOv2 detection method for shield tunnel lining water leakages. J Intell Constr2023; 1: 9180004.
7.
ZhangXLinXZhangW, et al. Intelligent recognition of voids behind tunnel linings using deep learning and percussion sound. J Intell Constr2023; 1: 9180029.
8.
LiQZouQLiaoJ, et al. Deep learning with spatial constraint for tunnel crack detection. In: Computing in civil engineering 2019. Atlanta, Georgia, pp. 393–400. American Society of Civil Engineers.
9.
OuyangADi MurroVCullM, et al. Automated pixel-level crack monitoring system for large-scale underground infrastructure – a case study at CERN. Tunnelling Underground Space Technol2023; 140: 105310.
10.
WangHLiYDangLM, et al. Pixel-level tunnel crack segmentation using a weakly supervised annotation approach. Comput Ind2021; 133: 103545.
11.
LiDXieQGongX, et al. Automatic defect detection of metro tunnel surfaces using a vision-based inspection system. Adv Eng Inform2021; 47: 101206.
12.
YinZLeiZZhengA, et al. Automatic detection and association analysis of multiple surface defects on shield subway tunnels. Sensors2023; 23: 7106.
13.
HuangZFuHChenW, et al. Damage detection and quantitative analysis of shield tunnel structure. Autom Constr2018; 94: 303–316.
14.
KimJShimSKangS-J, et al. Learning structure for concrete crack detection using robust super-resolution with generative adversarial network. Struct Control Health Monitor2023; 2023: 1–16.
15.
XueYLiY.A fast detection method via region-based fully convolutional neural networks for shield tunnel lining defects. Comput Aided Civil Infrastruct Eng2018; 33: 638–654.
16.
XueYCaiXShadabfarM, et al. Deep learning-based automatic recognition of water leakage area in shield tunnel lining. Tunnelling Underground Space Technol2020; 104: 103524.
17.
ZhaoSShadabfarMZhangD, et al. Deep learning-based classification and instance segmentation of leakage-area and scaling images of shield tunnel linings. Struct Control Health Monit. Epub ahead of print June2021. DOI: 10.1002/stc.2732.
18.
ManKLiuRLiuX, et al. Water leakage and crack identification in tunnels based on transfer-learning and convolutional neural networks. Water2022; 14: 1462.
19.
ZhouZZhangJGongC, et al. Automatic tunnel lining crack detection via deep learning with generative adversarial network-based data augmentation. Underground Space2023; 9: 140–154.
20.
ZhouZZhangJGongC.Automatic detection method of tunnel lining multi-defects via an enhanced You Only Look Once network. Comput Aided Civil Eng2022; 37: 762–780.
21.
LiuJZhaoZLvC, et al. An image enhancement algorithm to improve road tunnel crack transfer detection. Constr Build Mater2022; 348: 128583.
22.
ChaYChoiWSuhG, et al. Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Comput Aided Civil Eng2018; 33: 731–747.
23.
AliRKangDSuhG, et al. Real-time multiple damage mapping using autonomous UAV and deep faster region-based neural networks for GPS-denied structures. Autom Constr2021; 130: 103831.
24.
HuangHLiQZhangD.Deep learning based image recognition for crack and leakage defects of metro shield tunnel. Tunnelling Underground Space Technol2018; 77: 166–176.
25.
LiGMaBHeS, et al. Automatic tunnel crack detection based on U-Net and a convolutional neural network with alternately updated clique. Sensors2020; 20: 717.
26.
MiaoXWangJWangZ, et al. Automatic recognition of highway tunnel defects based on an improved U-net model. IEEE Sens J2019; 19: 11413–11423.
27.
DangLMWangHLiY, et al. Automatic tunnel lining crack evaluation and measurement using deep learning. Tunnelling Underground Space Technol2022; 124: 104472.
28.
SongQWuYXinX, et al. Real-time tunnel crack analysis system via deep learning. IEEE Access2019; 7: 64186–64197.
29.
XuLWangYDongA, et al. Image-based intelligent detection of typical defects of complex subway tunnel surface. Tunnelling Underground Space Technol2023; 140: 105266.
30.
ZhaoSZhangDXueY, et al. A deep learning-based approach for refined crack evaluation from shield tunnel lining images. Autom Constr2021; 132: 103934.
31.
XuYLiDXieQ, et al. Automatic defect detection and segmentation of tunnel surface using modified Mask R-CNN. Measurement2021; 178: 109316.
32.
KangDBenipalSSGopalDL, et al. Hybrid pixel-level concrete crack segmentation and quantification across complex backgrounds using deep learning. Autom Constr2020; 118: 103291.
LinDLiYPrasadS, et al. CAM-UNET: class activation MAP guided UNET with feedback refinement for defect segmentation. In: 2020 IEEE international conference on image processing (ICIP). Abu Dhabi, United Arab Emirates, pp. 2131–2135. IEEE.
35.
RenYHuangJHongZ, et al. Image-based concrete crack detection in tunnels using deep fully convolutional networks. Constr Build Mater2020; 234: 117367.
36.
QinSQiTLeiB, et al. Batched-image detection model and deployment method for tunnel lining defects using line-scan cameras based on experiments study. Tunnelling Underground Space Technol2023; 142: 105428.
37.
WangZChaY-J.Unsupervised deep learning approach using a deep auto-encoder with a one-class support vector machine to detect damage. Struct Health Monitor2021; 20: 406–425.
38.
WadaK. Labelme: image polygonal annotation with Python, GitHub repository. https://github.com/wkentaro/labelme (2020, accessed 15 March 2024).
39.
LiuZMaoHWuC-Y, et al. A ConvNet for the 2020s. In: 2022 IEEE/CVF conference on computer vision and pattern recognition (CVPR), New Orleans, LA, USA.
40.
DosovitskiyABeyerLKolesnikovA, et al. An image is worth 16x16 words: transformers for image recognition at scale, http://arxiv.org/abs/2010.11929.
41.
LiuZLinYCaoY, et al. Swin transformer: hierarchical vision transformer using shifted windows, http://arxiv.org/abs/2103.14030.
42.
WangZJiS. Smoothed dilated convolutions for improved dense prediction. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. London, United Kingdom, pp. 2486–2495. ACM.
43.
ZhouLZhangCWuM. D-LinkNet: LinkNet with pretrained encoder and dilated convolution for high resolution satellite imagery road extraction. In: 2018 IEEE/CVF conference on computer vision and pattern recognition workshops (CVPRW). Salt Lake City, UT, USA, pp. 192–1924. IEEE.
44.
ImDHanDChoiS, et al. DT-CNN: dilated and transposed convolution neural network accelerator for real-time image segmentation on mobile devices. In: 2019 IEEE international symposium on circuits and systems (ISCAS). Sapporo, Japan, pp. 1–5. IEEE.
45.
LiuQKampffmeyerMJenssenR, et al. Dense dilated convolutions’ merging network for land cover classification. IEEE Trans Geosci Remote Sensing2020; 58: 6309–6320.
46.
LiuFWangL.UNet-based model for crack detection integrating visual explanations. Constr Build Mater2022; 322: 126265.
47.
GengPJiaMRenX.Tunnel lining water leakage image Segmentation based on improved BlendMask. Structural Health Monitoring2023; 22: 865–878.
48.
ChenL-CZhuYPapandreouG, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation, http://arxiv.org/abs/1802.02611.
49.
YangXLiHYuY, et al. Automatic pixel-level crack detection and measurement using fully convolutional network: pixel-level crack detection and measurement using FCN. Comput Aided Civil Infrastruct Eng2018; 33: 1090–1109.
50.
MarieRLabbani-IgbidaOMouaddibEM.The delta medial axis: a fast and robust algorithm for filtered skeleton extraction. Pattern Recognition2016; 56: 26–39.
51.
ZhangTSuenC.A fast parallel algorithm for thinning digital patterns. Commun ACM1984; 27: 236–239.
HilditchCJ.Comparison of thinning algorithms on a parallel processor. Image Vision Comput1983; 1(3): 115–132.
54.
HuLNooshabadiSAhmadiM. Massively parallel KD-tree construction and nearest neighbor search algorithms. In: 2015 IEEE international symposium on circuits and systems (ISCAS). Lisbon, Portugal, pp. 2752–2755. IEEE.
55.
AndrewsHPattersonC.Singular value decomposition (SVD) image coding. IEEE Trans Commun1976; 24: 425–432.
ChoiWChaY-J.SDDNet: real-time crack segmentation. IEEE Trans Ind Electron2020; 67: 8016–8025.
58.
KangDHChaY-J.Efficient attention-based deep encoder and decoder for automatic crack segmentation. Struct Health Monitor2022; 21: 2190–2205.
59.
HeKZhangXRenS, et al. Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR). Las Vegas, NV, USA, pp. 770–778. IEEE.
60.
SandlerMHowardAZhuM, et al. MobileNetV2: inverted residuals and linear bottlenecks. In: 2018 IEEE/CVF conference on computer vision and pattern recognition. Salt Lake City, UT, pp. 4510–4520. IEEE.
ChenL-CPapandreouGSchroffF, et al. Rethinking atrous convolution for semantic image segmentation. Epub ahead of print 17June2017. DOI: 10.48550/arXiv.1706.05587.
63.
RonnebergerOFischerPBroxT. U-Net: convolutional networks for biomedical image segmentation. In: NavabNHorneggerJWellsWM, et al. (eds) Medical image computing and computer-assisted intervention – MICCAI 2015. Cham: Springer International Publishing, pp. 234–241.
64.
ZhouZSiddiqueeMMRTajbakhshN, et al. UNet++: a nested U-Net architecture for medical image segmentation, http://arxiv.org/abs/1807.10165.
65.
ZhaoHShiJQiX, et al. Pyramid scene parsing network. arXiv:1612.01105, 2017. DOI: 10.48550/arXiv.1612.01105.
66.
ChaurasiaACulurcielloE. LinkNet: Exploiting encoder representations for efficient semantic segmentation. In: 2017 IEEE Visual Communications and Image Processing (VCIP), St. Petersburg, FL, USA, 2017, pp. 1–4
67.
ZhaoSZhangGZhangD, et al. A hybrid attention deep learning network for refined segmentation of cracks from shield tunnel lining images. J Rock Mech Geotechl Eng2023; 15: 3105–3117.
68.
BochkovskiyAWangC-YLiaoH-YM. YOLOv4: optimal speed and accuracy of object detection, arXiv:2004.10934, http://arxiv.org/abs/2004.10934.
69.
YuanJRenQJiaC, et al. Automated pixel-level crack detection and quantification using deep convolutional neural networks for structural condition assessment. Structures2024; 59: 105780.
