The structural condition of bridges is generally assessed using manual visual inspection. However, this approach consumes labor, time, and capital, and produces subjective results. Therefore, industries today are using automated visual inspection approaches, which quantify and localize damages such as cracks using robots and computer vision. This paper proposes an instant damage identification and localization approach that uses an image capturing and geo-tagging system and deep convolutional neural network for crack detection. The image capturing and geo-tagging allows the geo-tagging of three-dimensional coordinates and camera pose data with bridge inspection images; the deep convolutional neural network is trained for automated crack identification. The damages extracted by the convolutional neural network are instantly transformed into a global bridge damage map, with georeferencing data acquired using the image capturing and geo-tagging. This method is experimentally validated through a lab-scale test on a wall and a field test on a bridge to demonstrate the performance of the instant damage map.
DongYSongRLiuH. Bridges structural health monitoring and deterioration detection synthesis of knowledge and technology. Alaska University Transportation Center, Alaska Department of Transportation and Public Facilities, Report, 2010, https://scholarworks.alaska.edu:443/handle/11122/7476 (accessed 13 December 2019).
2.
DorafshanSMaguireM. Bridge inspection: human performance, unmanned aerial systems and automation. J Civil Struct Health Monit2018; 8(3): 443–476.
3.
ChuaCACawleyP. Crack growth monitoring using fundamental shear horizontal guided waves. Struct Health Monit. Epub ahead of print 23October2019. DOI: 10.1177/1475921719882330.
4.
KimHAhnEShinM, et al. Crack and noncrack classification from concrete surface images using machine learning. Struct Health Monit2019; 18(3): 725–738.
5.
AliRChaY-J. Subsurface damage detection of a steel bridge using deep learning and uncooled micro-bolometer. Construct Build Mater2019; 226: 376–387.
6.
KimI-HJeonHBaekS-C, et al. Application of crack identification techniques for an aging concrete bridge inspection using an unmanned aerial vehicle. Sensors2019; 18(6): 1881.
7.
PrasannaPDanaKJGucunskiN, et al. Automated crack detection on concrete bridges. IEEE Trans Automat Sci Eng2016; 13(2): 591–599.
8.
OrtizABonnin-PascualFGarcia-FidalgoE, et al. Vision-based corrosion detection assisted by a micro-aerial vehicle in a vessel inspection application. Sensors2016; 16(12): 2118.
9.
ZhuZGermanSBrilakisI. Visual retrieval of concrete crack properties for automated post-earthquake structural safety evaluation. Automat Construct2011; 20(7): 874–883.
10.
KimJ-WKimS-BParkJ-C, et al. Development of crack detection system with unmanned aerial vehicles and digital image processing. In: Proceedings of the 2015 World Congress on Advances in Structural Engineering and Mechanics (ASEM’15), Incheon, Korea, 25–29 August 2015, p. 11. Daejeon, Korea: ASEM.
11.
NecatiCFBrown DavidLEminAA. Parameter estimation for multiple-input multiple-output modal analysis of large structures. J Eng Mech2004; 130(8): 921–930.
12.
Sanchez-CuevasPJRamon-SoriaPArrueB, et al. Robotic system for inspection by contact of bridge beams using UAVs. Sensors2019; 19(2): 305.
13.
LeeBJShinDHSeoJW, et al. Intelligent bridge inspection using remote controlled robot and image processing technique. In: 28th International Symposium on Automation and Robotics in Construction (ISARC), Seoul, Korea, 29 June–2 July 2011, pp. 1426–1431.
14.
LimRSLaHMShengW. A robotic crack inspection and mapping system for bridge deck maintenance. IEEE Trans Automat Sci Eng2014; 11(2): 367–378.
15.
LiBUshirodaKYangL, et al. Wall-climbing robot for non-destructive evaluation using impact-echo and metric learning SVM. Int J Intell Robot Appl2017; 1(3): 255–270.
16.
BaoYTangZLiH, et al. Computer vision and deep learning–based data anomaly detection method for structural health monitoring. Struct Health Monit. Epub ahead of print 19February2018. DOI: 10.1177/1475921718757405.
17.
HuangJLiDLiH, et al. Damage identification of a large cable-stayed bridge with novel cointegrated Kalman filter method under changing environments. Struct Contr Health Monit2018; 25(5): e2152.
18.
Mohtasham KhaniMVahidniaSGhasemzadehL, et al. Deep-learning-based crack detection with applications for the structural health monitoring of gas turbines. Struct Health Monit. Epub ahead of print 11November2019. DOI: 10.1177/1475921719883202.
19.
DavoudiRMillerGRKutzJN. Computer vision based inspection approach to predict damage state and load level for RC members. In: 11th International Workshop on Structural Health Monitoring, Stanford, CA, 12–14 September 2017.
20.
KhalooALattanziDJachimowiczA, et al. Utilizing UAV and 3D computer vision for visual inspection of a large gravity dam. Front Built Environ. Epub ahead of print 2July2018. DOI: 10.3389/fbuil.2018.00031.
21.
LeeK-BShinH-S. An application of a deep learning algorithm for automatic detection of unexpected accidents under bad CCTV monitoring conditions in tunnels. In: 2019 International Conference on Deep Learning and Machine Learning in Emerging Applications (Deep-ML), Istanbul, Turkey, 26–28 August 2019.
22.
ParlangeRMartinez-CarranzaJSucarLE, et al. Vision-based autonomous navigation for wind turbine inspection using an unmanned aerial vehicle. In: 10th International Micro Air Vehicle Conference (IMAV), Melbourne, Australia, November 2018.
23.
ShihavuddinASMChenXFedorovV, et al. Wind turbine surface damage detection by deep learning aided drone inspection analysis. Energies12(4): 676.
24.
SpencerBFHoskereVNarazakiY. Advances in computer vision-based civil infrastructure inspection and monitoring. Engineering5(2): 199–222.
25.
KangDChaY-J. Autonomous UAVs for structural health monitoring using deep learning and an ultrasonic beacon system with geo-tagging. Comput Aid Civil Infra Eng2018; 33(10): 885–902.
26.
JangKKimNAnY-K. Deep learning–based autonomous concrete crack evaluation through hybrid image scanning. Struct Health Monit2019; 18(5–6): 1722–1737.
27.
AkbarMAQidwaiUJahanshahiMR. An evaluation of image-based structural health monitoring using integrated unmanned aerial vehicle platform. Struct Control Health Monit2019; 26(1): e2276.
28.
LeiBRenYWangN, et al. Design of a new low-cost unmanned aerial vehicle and vision-based concrete crack inspection method. Struct Health Monit. Epub ahead of print 3February2020. DOI: 10.1177/1475921719898862.
29.
SongSJungSKimH, et al. A method for mapping and localization of quadrotors for inspection under bridges using camera and 3 D-LiDAR. In: Proceedings of the 7th Asia-Pacific Workshop on Structural Health Monitoring, Hong Kong SAR, P.R. China, November 2019.
30.
BrooksCRichardJDDavidMB, et al. Evaluating the use of unmanned aerial vehicles for transportation purposes, https://trid.trb.org/view/1352503 (2015, accessed 2 March 2020).
31.
LaHMLimRSBasilyBB, et al. Mechatronic systems design for an autonomous robotic system for high-efficiency bridge deck inspection and evaluation. IEEE/ASME Trans Mechatron2013; 18(6): 1655–1664.
32.
GibbSLaHMLeT, et al. Nondestructive evaluation sensor fusion with autonomous robotic system for civil infrastructure inspection. J Field Robot2018; 35(6): 988–1004.
33.
FirooziYSGolrooAJahanshahiMR. Automated rutting measurement using an inexpensive RGB-D sensor fusion approach. J Trans Eng, Part B: Pavements2019; 145(1): 04018061.
34.
JiangSZhangJ. Real-time crack assessment using deep neural networks with wall-climbing unmanned aerial system. Comput Aid Civil Infra Eng. Epub ahead of print 8December2019. DOI: 10.1111/mice.12519.
35.
WangRKawamuraY. An automated sensing system for steel bridge inspection using GMR sensor array and magnetic wheels of climbing robot. J Sensors2016; 2016: 8121678 (15 pp.).
36.
LaHM. Automated robotic monitoring and inspection of steel structures and bridges. arXiv:1705.04888, http://arxiv.org/abs/1705.04888 (2017, accessed 2 March 2020).
37.
MoussaAEl-SheimyN. A fast approach for stitching of aerial images. Int Arch Photogramm Remote Sens Spatial Inf Sci2016; XLI-B3: 769–774.
38.
LiuQLiuWZouL, et al. A new approach to fast mosaic UAV images. Int Arch Photogramm Remote Sens Spatial Inf Sci2012; XXXVIII-1-C22: 271–276.
RenSHeKGirshickR, et al. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell2017; 39(6): 1137–1149.
41.
RedmonJDivvalaSGirshickR, et al. You only look once: unified, real-time object detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, 27–30 June 2016, pp. 779–788. New York: IEEE.
DorafshanSThomasRJMaguireM. SDNET2018: an annotated image dataset for non-contact concrete crack detection using deep convolutional neural networks. Data Brief2018; 21: 1664–1668.
ShiYCuiLQiZ, et al. Automatic road crack detection using random structured forests. IEEE Transactions on Intelligent Transportation Systems2016; 17: 3434–3445.
HeKZhangXRenS, et al. Deep residual learning for image recognition. arXiv:1512.03385, http://arxiv.org/abs/1512.03385 (2015, accessed 9 December 2019).
49.
KimBChoS. Image-based concrete crack assessment using mask and region-based convolutional neural network. Struct Contr Health Monit2019; 26(8): e2381.
ZhangZ. A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell2000; 22(11): 1330–1334.
52.
VanegasFGastonKJRobertsJ, et al. A framework for UAV navigation and exploration in GPS-denied environments. In: 2019 IEEE Aerospace Conference, Big Sky, MT, 2–9 March 2019, pp. 1–6. New York: IEEE.
