Manual detection methods of tunnel surface defects with mobile laser scanning (MLS) data are increasingly inadequate to meet the high standards for modern tunnel maintenance. Meanwhile, investigations on MLS data-based automatic multitype surface defect detection are rarely reported. This study proposes an optimized method, named AMD-MLS (automatic multitype damage detection method based on MLS data), for automatically identifying leakage and spalling of tunnel linings through MLS data mining. The AMD-MLS method offers enhanced accuracy owing to the incorporation of mix transformer and multiple region of interest (ROI) aligns in mask region-based convolutional neural network. Ablation experiments with metro tunnel MLS data showcased that the proposed scheme attained scores of 92.0% (b-AP) and 86.8% (s-AP) for leakage detection and scores of 78.2% (b-AP) and 72.9% (s-AP) for spalling detection. The proposed method exhibited higher multitype detection accuracy compared to the baseline models. The defect identification is followed by automated calculation of areas and locations, supporting more informed decision-making for tunnel maintenance.
DawoodTZhuZZayedT.Machine vision-based model for spalling detection and quantification in subway networks. Autom Constr2017; 81: 149–160.
2.
GengPJiaMRenX.Tunnel lining water leakage image Segmentation based on improved BlendMask. Struct Health Monit2022; 22: 865–878.
3.
MonteroRVictoresJGMartínezS, et al. Past, present and future of robotic tunnel inspection. Autom Constr2015; 59: 99–112.
4.
AttardLDebonoCJValentinoG, et al. Tunnel inspection using photogrammetric techniques and image processing: a review. ISPRS J Photogrammetry Remote Sens2018; 144: 180–188.
5.
SjölanderABelloniVAnsellA, et al. Towards automated inspections of tunnels: a review of optical inspections and autonomous assessment of concrete tunnel linings. Sensors2023; 23: 3189.
6.
HuangHSunYXueY, et al. Inspection equipment study for subway tunnel defects by grey-scale image processing. Adv Eng Inform2017; 32: 188–201.
7.
UkaiM.Advanced inspection system of tunnel wall deformation using image processing. Quart Rep RTRI2007; 48: 94–98.
8.
HuCPZhuHHLiXJ.Detection of tunnel water leakage based on image processing. Inform Technol Geo-Eng2010, 254–262.
9.
HanLChenJFLiHB, et al. Multispectral water leakage detection based on a one-stage anchor-free modality fusion network for metro tunnels. Autom Constr2022; 140: 104345.
10.
PuenteIGonzälez-JorgeHMartínez-SánchezJ, et al. Review of mobile mapping and surveying technologies. Measurement2013; 46: 2127–2145.
11.
CuiHRenXCMaoQZ, et al. Shield subway tunnel deformation detection based on mobile laser scanning. Autom Constr2019; 106: 102889.
12.
ZhouYHWangSHMeiX, et al. Railway tunnel clearance inspection method based on 3D point cloud from mobile laser scanning. Sensors2017; 17: 2055.
13.
HuangHWChengWZhouML, et al. Towards automated 3D inspection of water leakages in shield tunnel linings using mobile laser scanning data. Sensors2020; 20: 6669.
14.
ZhouMLChengWHuangHW, et al. A novel approach to automated 3d spalling defects inspection in railway tunnel linings using laser intensity and depth information. Sensors2021; 21: 5725.
15.
WuHBAoXRChenZ, et al. Concrete spalling detection for metro tunnel from point cloud based on roughness descriptor. J Sens2019; 2019: 8574750.
16.
AiQYuanYBiXL.Acquiring sectional profile of metro tunnels using charge-coupled device cameras. Struct Infrastruct Eng2016; 12: 1065–1075.
17.
FitzgibbonAPiluMFisherRB.Direct least squares fitting of ellipses. IEEE Trans Pattern Anal Machine Intell1999; 21: 476–480.
18.
ConsensusRSFischlerMABollesRC.A paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM1981; 6: 381–395.
19.
HuangTYangGTangG.A fast two-dimensional median filtering algorithm. IEEE Trans Acoust Speech Signal Process1979; 27: 13–18.
20.
TsaiCM.Adaptive local power-law transformation for color image enhancement. Appl Math Inform Sci2013; 7: 2019–2026.
21.
CuiXGoelVKingsburyB. Data augmentation for deep convolutional neural network acoustic modeling. In: 2015 IEEE International conference on acoustics, speech and signal processing (ICASSP), South Brisbane, Queensland, Australia. 2015.
22.
TorralbaARussellBCYuenJ.LabelMe: online image annotation and applications. Proc IEEE2010; 98: 1467–1484.
23.
LinTYMaireMBelongieS, et al. Microsoft COCO: common objects in context. In: Computer Vision–ECCV 2014, 13th European conference, Zurich, Switzerland, lecture notes in computer science, 2014, pp. 740–755.
24.
HeKGkioxariGDollarP, et al. Mask R-CNN. IEEE Trans Pattern Anal Mach Intell 2020; 42: 386–397.
25.
HeKZhangXRenS, et al. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016.
26.
LinTYDollarPGirshickR, et al. Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 2017.
27.
RenSHeKGirshickR.Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Machine Intell2017; 28: 1137–1149.
28.
VaswaniAShazeerNParmarN, et al. Attention is all you need. In: 31st Conference on neural information processing systems (NIPS 2017), Long Beach, CA, USA, 2017, pp. 1–11.
29.
ChunPYamaneTMaemuraY.A deep learning-based image captioning method to automatically generate comprehensive explanations of bridge damage. Comput-Aid Civil Infrastruct Eng2022; 37: 1387–1401.
30.
DosovitskiyABeyerLKolesnikovA.An image is worth 16x16 words: transformers for image recognition at scale. Computer Vision and Pattern Recognition at Scale, arXiv:2020.11929, 2022.
31.
XieEWangWYuZ, et al. SegFormer: simple and efficient design for semantic segmentation with transformers. Adv Neural Inform Process Syst2021; 34: 12077–12090.
32.
LongJShelhamerEDarrellT. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, Boston, MA, pp. 3431–3440.
33.
WanHPNiYQ.Bayesian multi-task learning methodology for reconstruction of structural health monitoring data. Struct Health Monit2018; 18: 1282–1309.
34.
FuCYLiuWRangaA, et al. DSSD: deconvolutional single shot detector. Cornell University – arXiv1701.06659, 2017.
35.
ZhangFLGuDKLiX, et al. Structural damage detection based on fundamental Bayesian two-stage model considering the modal parameters uncertainty. Struct Health Monit2022; 22: 2305–2324.
36.
GhiasiGFowlkesCC. Laplacian pyramid reconstruction and refinement for semantic segmentation. In: Computer Vision–ECCV 2016: 14th European conference, Amsterdam, The Netherlands, 11–14 October 2016, Proceedings, Part III 14, 2016, pp. 519–534.
37.
LinTYDollarPGirshickR, et al. Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 2017, Honolulu, Hawaii, pp. 2117–2125.
38.
LiXLinXZhuH, Condition assessment of shield tunnel using a new indicator: the tunnel serviceability index. Tunnell Underground Space Technol2017; 67: 98–106.
39.
WuHNShenSLChenRP.Three-dimensional numerical modeling on localised leakage in segmental lining of shield tunnels. Comput Geotechnics2020; 122: 103549.
40.
JulierSJUhlmannJK.New extension of the Kalman filter to nonlinear systems. Signal Process Sensor Fusion Target Recognition VI1997; 3068: 182–193.
41.
HuangZHuangLGongY.Mask scoring R-CNN. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), 2019, Long Beach, CA, pp. 6409–6418.
42.
DuXLinTYJinP. SpineNet: learning scale-permuted backbone for recognition and localization. In: Proceedings of the 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), 2020, Seattle, WA, pp. 11592–11601.
43.
ZhangHWuCZhangZ.ResNeSt: split-attention networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, New Orleans, LA, pp. 2736–2743.
