High-precision crack segmentation is crucial for analyzing and maintaining the apparent state of structures. The introduction of large vision models, such as the segment anything model (SAM), has brought significant advancements in object segmentation due to their remarkable generalization capabilities. However, SAM cannot be directly used for the purpose of automatic crack segmentation. This study introduces a novel approach that fine-tunes SAM specifically for crack segmentation by incorporating low-rank adaptation (LoRA). This method involves adding a dedicated crack segmentation head to SAM, enabling automatic crack segmentation. Additionally, the application of LoRA technology facilitates the readjustment of SAM’s features without incurring the substantial costs typically associated with fine-tuning entire networks. A comparative analysis with current leading crack segmentation models demonstrated a significant increase in accuracy across eight different crack datasets. This study offers guidelines for the application of large vision models for crack identification.
AdhikariRSMoselhiOBagchiA.Image-based retrieval of concrete crack properties for bridge inspection. Autom Constr2014; 39: 180–194.
2.
YeumCMDykeSJ.Vision-based automated crack detection for bridge inspection. Comput Aid Civ Infrastruct Eng2015; 30: 759–770.
3.
SpencerBFHoskereVNarazakiY.Advances in computer vision-based civil infrastructure inspection and monitoring. Engineering2019; 5: 199–222.
4.
ChaY-JChoiWBüyüköztürkO.Deep learning-based crack damage detection using convolutional neural networks. Comput Aid Civ Infrastruct Eng2017; 32: 361–378.
5.
ChaY-JChoiWSuhG, et al. Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Comput Aided Civ Infrastruct Eng2018; 33: 731–747.
6.
DorafshanSThomasRJMaguireM.Comparison of deep convolutional neural networks and edge detectors for image-based crack detection in concrete. Constr Build Mater2018; 186: 1031–1045.
7.
KimBChoS.Automated vision-based detection of cracks on concrete surfaces using a deep learning technique. Sensors2018; 18: 3452.
8.
XuYBaoYChenJ, et al. Surface fatigue crack identification in steel box girder of bridges by a deep fusion convolutional neural network based on consumer-grade camera images. Struct Health Monit2018; 18: 653–674.
9.
XuHSuXWangY, et al. Automatic bridge crack detection using a convolutional neural network. Appl Sci2019; 9: 2867.
DengJLuYLeeVC-S. Imaging-based crack detection on concrete surfaces using You Only Look Once network. Struct Health Monit2020; 20: 484–499.
12.
ZhangJQianSTanC.Automated bridge surface crack detection and segmentation using computer vision-based deep learning model. Eng Appl Artif Intell2022; 115: 105225.
13.
KangDBenipalSSGopalDL, et al. Hybrid pixel-level concrete crack segmentation and quantification across complex backgrounds using deep learning. Autom Constr2020; 118: 103291.
14.
YeXJinTYunC.A review on deep learning-based structural health monitoring of civil infrastructures. Smart Struct Syst2019; 24: 567–585.
15.
DongC-ZCatbasFN.A review of computer vision–based structural health monitoring at local and global levels. Struct Health Monit2021; 20: 692–743.
16.
KochCGeorgievaKKasireddyV, et al. A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure. Adv Eng Inform2015; 29: 196–210.
17.
AzimiMEslamlouADPekcanG.Data-driven structural health monitoring and damage detection through deep learning: state-of-the-art review. Sensors2020; 20.
18.
HouYLiQZhangC, et al. The state-of-the-art review on applications of intrusive sensing, image processing techniques, and machine learning methods in pavement monitoring and analysis. Engineering2021; 7: 845–856.
19.
LiSZhaoXZhouG.Automatic pixel-level multiple damage detection of concrete structure using fully convolutional network. Comput Aided Civ Infrastruct Eng2019; 34: 616–634.
20.
HoskereVNarazakiYHoangTA, et al. MaDnet: multi-task semantic segmentation of multiple types of structural materials and damage in images of civil infrastructure. J Civ Struct Health Monit2020; 10: 757–773.
21.
NarazakiYHoskereVHoangTA, et al. Vision-based automated bridge component recognition with high-level scene consistency. Comput Aided Civ Infrastruct Eng2020; 35: 465–482.
22.
ChenF-CJahanshahiMR.ARF-Crack: rotation invariant deep fully convolutional network for pixel-level crack detection. Mach Vision Appl2020; 31: 47.
23.
DengJLuYLeeVCS. A hybrid lightweight encoder-decoder network for automatic bridge crack assessment with real-world interference. Measurement2023; 216: 112892.
24.
HuyanJLiWTigheS, et al. CrackU-net: a novel deep convolutional neural network for pixelwise pavement crack detection. Struct Control Health Monit2020; 27: e2551.
25.
DongCLiLYanJ, et al. Pixel-level fatigue crack segmentation in large-scale images of steel structures using an encoder–decoder network. Sensors2021; 21: 4135.
26.
ZhangLLiaoYWangG, et al. A multi-scale contextual information enhancement network for crack segmentation. Appl Sci2022; 12: 11135.
27.
DengLZhangAGuoJ, et al. An integrated method for road crack segmentation and surface feature quantification under complex backgrounds. Remote Sensing2023; 15: 1530.
28.
TangYHuangZChenZ, et al. Novel visual crack width measurement based on backbone double-scale features for improved detection automation. Eng Struct2023; 274: 115158.
29.
XuYFanYLiH.Lightweight semantic segmentation of complex structural damage recognition for actual bridges. Struct Health Monit2023; 22: 3250–3269.
30.
ZhangKZhangYChengHD.CrackGAN: pavement crack detection using partially accurate ground truths based on generative adversarial learning. IEEE Trans Intell Transportation Syst2021; 22: 1306–1319.
31.
PanZLauSLHYangX, et al. Automatic pavement crack segmentation using a generative adversarial network (GAN)-based convolutional neural network. Results Eng2023; 19: 101267.
32.
KimCHwangSSohnH.Weld crack detection and quantification using laser thermography, mask R-CNN, and CycleGAN. Autom Constr2022; 143: 104568.
33.
MaDFangHWangN, et al. Automatic detection and counting system for pavement cracks based on PCGAN and YOLO-MF. IEEE Trans Intell Transp Syst2022; 23: 22166–22178.
34.
SongJLiPFangQ, et al. Data augmentation by an additional self-supervised CycleGAN-based for shadowed pavement detection. Sustainability2022; 14: 14304.
35.
WengXHuangYLiY, et al. Unsupervised domain adaptation for crack detection. Autom Constr2023; 153: 104939.
36.
ZhangTWangDMullinsA, et al. Integrated APC-GAN and AttuNet framework for automated pavement crack pixel-level segmentation: a new solution to small training datasets. IEEE Trans Intell Transp Syst2023; 24: 4474–4481.
37.
XiangXWangZQiaoY.An improved YOLOv5 crack detection method combined with transformer. IEEE Sens J2022; 22: 14328–14335.
38.
LuoHLiJCaiL, et al. STrans-YOLOX: fusing swin transformer and YOLOX for automatic pavement crack detection. Appl Sci2023; 13: 1999.
39.
WangSChenXDongQ.Detection of asphalt pavement cracks based on vision transformer improved YOLO V5. J Transp Eng Part B Pavements2023; 149: 04023004.
40.
YangLBaiSLiuY, et al. Multi-scale triple-attention network for pixelwise crack segmentation. Autom Constr2023; 150: 104853.
41.
WangWSuC.Automatic concrete crack segmentation model based on transformer. Autom Constr2022; 139: 104275.
42.
DingWYangHYuK, et al. Crack detection and quantification for concrete structures using UAV and transformer. Autom Constr2023; 152: 104929.
43.
GuoFQianYLiuJ, et al. Pavement crack detection based on transformer network. Autom Constr2023; 145: 104646.
44.
TongZMaTZhangW, et al. Evidential transformer for pavement distress segmentation. Comput Aided Civ Infrastruct Eng2023; 38: 2317–2338.
45.
Asadi ShamsabadiEXuCRaoAS, et al. Vision transformer-based autonomous crack detection on asphalt and concrete surfaces. Autom Constr2022; 140: 104316.
46.
ZhongQYanxinLQiangZ.CrackT-net: a method of convolutional neural network and transformer for crack segmentation. J Electron Imaging2022; 31: 023040.
47.
ZhouZZhangJGongC.Hybrid semantic segmentation for tunnel lining cracks based on swin transformer and convolutional neural network. Comput Aided Civ Infrastruct Eng2023; 38: 2491–2510.
48.
KirillovAMintunERaviN, et al. Segment anything. arXiv e-prints. arXiv:2304.02643, 2023.
49.
WangLYeXZhuL, et al. When SAM meets sonar images. arXiv e-prints. arXiv:2306.14109, 2023.
50.
AhmadiMGholizadeh LonbarASharifiA, et al. Application of segment anything model for civil infrastructure defect assessment. arXiv e-prints. arXiv: 2304.12600, 2023.
51.
HuEJShenYWallisP, et al. LoRA: low-rank adaptation of large language models. arXiv e-prints. arXiv:2106.09685, 2021.
52.
ZhangJChenXQiuZ, et al. Hard exudate segmentation supplemented by super-resolution with multi-scale attention fusion module. In: 2022 IEEE international conference on bioinformatics and biomedicine (BIBM), Las Vegas, NV, 2022, pp. 1375–1380.
ShiYCuiLQiZ, et al. Automatic road crack detection using random structured forests. IEEE Trans Intell Transp Syst2016; 17: 3434–3445.
55.
YangFZhangLYuS, et al. Feature pyramid and hierarchical boosting network for pavement crack detection. IEEE Trans Intell Transp Syst2020; 21: 1525–1535.
56.
ZouQCaoYLiQ, et al. CrackTree: automatic crack detection from pavement images. Pattern Recognit Lett2012; 33: 227–238.
57.
LiuYYaoJLuX, et al. DeepCrack: a deep hierarchical feature learning architecture for crack segmentation. Neurocomputing2019; 338: 139–153.
58.
EisenbachMStrickerRSeichterD, et al. How to get pavement distress detection ready for deep learning? a systematic approach. In: 2017 International joint conference on neural networks (IJCNN), Anchorage, Alaska, 2017, pp. 2039–2047.
59.
ChenL-CZhuYPapandreouG, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation. arXiv e-prints. arXiv:1802.02611, 2018.
60.
XieEWangWYuZ, et al. SegFormer: simple and efficient design for semantic segmentation with transformers. arXiv e-prints. arXiv:2105.15203, 2021.
61.
CordtsMOmranMRamosS, et al. The cityscapes dataset for semantic urban scene understanding. arXiv e-prints. arXiv:1604.01685, 2016.
