Accurately measuring visible cracks in bridges is crucial for their structural health diagnosis, damage detection, performance evaluation, and maintenance planning. The primary means of visual crack detection still relies heavily on manual visual inspection, an inefficient process that can pose significant safety risks. This article develops a unmanned aerial vehicle (UAV) vision-based surface crack measurement methodology and visualization scheme for the bridges that can detect and measure cracks automatically with improved efficiency. The surface crack measurement methodology is achieved by designing a three-stage crack sensing system including the You Only Look Once-based crack recognition, U-shaped network-based crack segmentation, and deep-vision-based crack width calculation. This workflow is integrated into a comprehensive UAV inspection system, which is intended for operation at the field. The surface crack visualization scheme is accomplished by taking advantage of time-series image fusion, GPS information migration, and three-dimensional (3D) point cloud technique to reconstruct the 3D geometrical model of the tested bridge, which is convenient for unveiling the crack information in the bridge. The proposed methodology was successfully validated by a case study on an arch bridge. The achievement of this article promotes the UAV vision-based bridge’s surface crack inspection technology to a new status that no preparation for pasting calibration marker is needed, and crack identification, segmentation, and width calculation are realized promptly during the UAV flying on-site, as well as damage evaluation for bridges is visually fulfilled based on the reconstructed digital-graphical 3D model. The working environments and influencing factors to the developed system are sufficiently discussed. Certain limitations in the current application are pointed out for future improvements.
TongXHGuoJLingY, et al. A new image-based method for concrete bridge bottom crack detection. In: 2011 International conference on image analysis and signal processing, Hubei, China, 2011, pp. 568–571. New York: IEEE.
6.
JangKJungHAnYK. Automated bridge crack evaluation through deep super resolution network-based hybrid image matching. Autom Constr2022; 137: 104229.
7.
NishikawaTYoshidaJSugiyamaT, et al. Concrete crack detection by multiple sequential image filtering. Comput Aided Civ Inf2012; 27: 29–47.
8.
LeeBYKimYYYiST, et al. Automated image processing technique for detecting and analysing concrete surface cracks. Struct Infrastruct Eng2013; 9(6): 567–577.
9.
WangHFZhaiLHuangH, et al. Measurement for cracks at the bottom of bridges based on tethered creeping unmanned aerial vehicle. Automat Constr2020; 119: 103330.
10.
ZhangJQianSRTanC. Automated bridge surface crack detection and segmentation using computer vision-based deep learning model. Eng Appl Artif Intel2022; 115: 105225.
11.
YangXLiSZhangD, et al. Identification framework for cracks on a steel structure surface by a restricted Boltzmann machines algorithm based on consumer-grade camera images. Struct Control Health Monit2018; 25(2): e2075.
12.
RenSQHeKMGirshickR, et al. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Machine Intel2017; 39(6): 1137–1149.
13.
WangCMuhammadJWangY, et al. Towards complete and accurate iris segmentation using deep multi-task attention network for non-cooperative Iris recognition. IEEE Trans Inf Foren Sec2020; 15: 2944–2959.
14.
ShelhamerELongJDarrellT. Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Machine Intel2017; 39(4): 640–651.
15.
ZhangYXHuangJCaiFH. On bridge surface crack detection based on an improved YOLO V3 algorithm. IFAC-PapersOnLine2020; 53(2): 8205–8210.
16.
QiuQWLauD. Real-time detection of cracks in tiled sidewalks using YOLO-based method applied to unmanned aerial vehicle (UAV) images. Automat Constr2023; 147: 104745.
17.
GaoQFWangYLiJ, et al. An enhanced percolation method for automatic detection of cracks in concrete bridges. Adv Civ Eng2020; 2020; 8896176.
18.
NieMXWangC. Pavement crack detection based on Yolo V3. In: 2nd International conference on safety produce informatization (IICSPI), Chongqing, China, 2019, pp. 327–330. New York: IEEE.
19.
LiRXYuJYLiF, et al. Automatic bridge crack detection using unmanned aerial vehicle and Faster R-CNN. Constr Build Mater2023; 362: 129659.
20.
NiYHMaoJXWangH, et al. Toward high-precision crack detection in concrete bridges using deep learning. J Perform Constr Fac2023; 37(3): 04023017.
21.
WangMYangWWangL, et al. FE-YOLOv5: feature enhancement network based on YOLOv5 for small object detection. J Vis Commun Image Represent2023; 90: 103752.
22.
RonnebergerOFischerPBroxT. U-Net: convolutional networks for biomedical image segmentation. In: Proceedings of the 18th international conference on medical image computing and computer-assisted intervention. Munich, Germany, 2015, pp. 234–241.
23.
GouCPengBLiT, et al. Pavement crack detection based on the improved faster-RCNN. In: IEEE 14th international conference on intelligent systems and knowledge engineering (ISKE), Dalian, China, 2019, pp. 962–967. New York: IEEE.
24.
LiuZCaoYWangY, et al. Computer vision-based concrete crack detection using U-Net fully convolutional networks. Autom Constr2019; 104: 129–139.
25.
ZhangYYuenK-V. Crack detection using fusion features-based broad learning system and image processing. Comput Aided Civ Inf2021; 36: 1568–1584.
26.
YangXBaoYQChenJH, et al. Surface fatigue crack identification in steel box girder of bridges by a deep fusion convolutional neural network based on consumer-grade camera images. Struct Health Monit2019; 18(3): 653–674.
27.
ZhangLYangFDaniel ZhangY, et al. Road crack detection using deep convolutional neural network. In: IEEE international conference on image processing (ICIP), Phoenix, AZ, USA, 2016, pp. 3708–3712.
28.
ChaYJChoiWBüyüköztürkO. Deep learning-based crack damage detection using convolutional neural networks. Comput Aided Civ Inf2017; 32(5): 361–378.
29.
SpencerBFJrHoskereVNarazakiY. Advances in computer vision-based civil infrastructure inspection and monitoring. Engineering2019; 5(2): 199–222.
30.
XuYWeiSYBaoYQ, et al. Automatic seismic damage identification of reinforced concrete columns from images by a region-based deep convolutional neural network. Struct Control Health Monit2019; 26(3): e2313.
31.
LiYHMaoYXWangJH, et al. Cracking enabled unclonability in colloidal crystal patterns authenticated with computer vision. Nanoscale2022; 14(24): 8833–8841.
32.
SongJHGaoHWLiuYJ, et al. Road damage identification and degree assessment based on UGV. Int J Smart Sens Intel Syst2016; 9(4): 2069–2087.
33.
LauSLHChongEKPYangX, et al. Automated pavement crack segmentation using U-Net-based convolutional neural network. IEEE Access2020; 8(1): 114892–114899.
34.
XuHYSuXXuHY, et al. Autonomous bridge crack detection using deep convolutional neural networks. In: Proceedings of the 3rd international conference on computer engineering, information science & application technology (ICCIA), 2019.
35.
NiFTZhangJChenZQ. Pixel-level crack delineation in images with convolutional feature fusion. Struct Control Health Monit2018; 26(1): e2286.
36.
JiangSZhangJ. Real-time crack assessment using deep neural networks with wall-climbing unmanned aerial system. Comput Aided Civ Inf2020; 35(6): 549–564.
37.
WangDWangXLRenBY, et al. Vision-based productivity analysis of cable crane transportation using augmented reality–based synthetic image. J Comput Civil Eng2022; 36(1): 04021030.
38.
EllenbergAKontsosAMoonF, et al. Bridge related damage quantification using unmanned aerial vehicle imagery. Struct Control Health Monit2016; 23(9): 1168–1179.
39.
TianYDZhangCJiangS, et al. Noncontact cable force estimation with unmanned aerial vehicle and computer vision. Comput Aided Civ Inf2021; 36: 73–88.
40.
IvićSCrnkovićBGrbčićL, et al. Multi-UAV trajectory planning for 3D visual inspection of complex structures. Autom Constr2023; 147: 104709.
41.
KongQGuJXiongB, et al. Vision-aided three-dimensional damage quantification and finite element model geometric updating for reinforced concrete structures. Comput Aided Civ Inf. Epub ahead of print 6January2023. DOI: 10.1111/mice.12967.
42.
JahanshahiMRMasriSF. Adaptive vision-based crack detection using 3D scene reconstruction for condition assessment of structures. Automat Constr2012; 22: 567–576.
43.
LiuYFChoSSpencerBFJr.Concrete crack assessment using digital image processing and 3D scene reconstruction. J Comput Civil Eng. Epub ahead of print 2016. DOI: 10.1061/(ASCE)CP.1943-5487.0000446.
44.
TorokMMGolparvar-FardMKochersbergerKB. Image-based automated 3D crack detection for post-disaster building assessment. J Comput Civil Eng. Epub ahead of print 2013. DOI: 10.1061/(ASCE)CP.1943-5487.0000334.
45.
ZhangAWangKCPLiB, et al. Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning networkComput Aided Civ Infrastruct Eng2017; 32(10): 805–819.
46.
KhalooALattanziDCunninghamK, et al. Unmanned aerial vehicle inspection of the placer river trail bridge through image-based 3D modelling. Struct Infrastruct Eng2018; 14(1): 1–13.
47.
Golparvar-FardMBohnJTeizerJ, et al. Evaluation of image-based modeling and laser scanning accuracy for emerging automated performance monitoring techniques. Automat Constr2011; 20(8): 1143–1155.
48.
LiuYFNieXFanJS, et al. Image-based crack assessment of bridge piers using unmanned aerial vehicles and three-dimensional scene reconstruction. Comput Aided Civ Infrastruct Eng2020; 35(5): 511–529.
49.
YoonSGwonGHLeeJH, et al. Three-dimensional image coordinate-based missing region of interest area detection and damage localization for bridge visual inspection using unmanned aerial vehicles. Struct Health Monit2021; 20(4): 1462–1475.
50.
DorafshanSThomasRJMaguireM. Fatigue crack detection using unmanned aerial systems in fracture critical inspection of steel Bridges. J Bridge Eng2018; 23(10): 04018078.
51.
KimHLeeJAhnE, et al. Concrete crack identification using a UAV incorporating hybrid image processing. Sensors2017; 17(9): 2052.
52.
DungCVSekiyaHHiranoS, et al. A vision-based method for crack detection in gusset plate welded joints of steel bridges using deep convolutional neural networks. Automat Constr2019; 102: 217–229.
XiongPZhongXGZhaoC, et al. A UAV-based machine vision method for bridge crack recognition and width quantification through hybrid feature learning. Constr Build Mater2021; 299: 123896.
55.
LiRYuJLiF, et al. Automatic bridge crack detection using unmanned aerial vehicle and faster R-CNN. Constr Build Mater2023; 362: 129659.
56.
LengJWuY. CMOS real-time image acquisition and display system design based on FPGA. Appl Mech Mater2014; 644–650: 4403–4406.
57.
DongXYanS. A lightweight vehicles detection network model based on YOLOv5. Eng Appl Artif Intel2022; 113: 104914.
58.
HamzenejadiM. Fine-tuned YOLOv5 for real-time vehicle detection in UAV imagery: architectural improvements and performance boost. Expert Syst Appl2023; 231: 120845.
59.
TranTNguyenS. Advanced crack detection and segmentation on bridge decks using deep learning. Constr Build Mater2023; 400: 132839.
60.
ZhangF. Real 3D modeling with context capture base on DJI UAV. Guangdong Water Resources and Hydropower, 2017; 8: 88–103. [in Chinese].
61.
The Ministry of Railways of the People’s Republic of China ([2004] 120). Code for rating existing railway bridges. Beijing: China Railway Publishing House, 2004.
62.
MOT (Ministry of Transport) JTG/T J21-2011. Specification for inspection and evaluation of load-bearing capacity of highway bridges. Beijing: MOT, 2011.
