The visual test for cracks is a tough work for human eyes. With the help of cameras, methods for object detection based on deep learning can detect the cracks automatically from digital images. However, an anomaly is observed that the models for crack detection are obviously underestimated by the widely used mean Average Precision (mAP) standard. In this study, by theoretical analyses about the topology of cracks, we attribute the failure of mAP standard to the random fractal feature. More deeply, this issue is due to the strict box matching during the calculation. To solve this problem, we construct a practical fractal-available evaluation method adopting the idea of covering box matching. Several metrics including Extended Recall (XR), Extended Precision (XP), and Extended F-score (Fext) are defined for scoring the crack detectors. Then, we conduct experiments on several mainstream models for object detection. Results show two advantages of the proposed evaluation method: (1) the proposed metrics evaluate cases better; (2) the proposed evaluation is non-maximum suppression-unrelated. Moreover, we present a case study to optimize a faster region-based convolutional neural networks model for crack detection adopting the proposed metrics. Recall (represented by the XR) of our best model achieves an industrial-level at 95.8%. These results denote that the proposed evaluation method removes the obstacle in evaluating the methods for object detection for cracks, which show great potential for automatic industrial inspections.
DengJLuYLeeVCS. Imaging-based crack detection on concrete surfaces using you only look once network. Struct Health Monit2020; 20(2), 484–499.
2.
AldeaEHégaratSL.Robust crack detection strategies for aerial inspection. In: International conference on quality control by artificial vision, Le Creusot, France, June 2015. HAL Open Science.
3.
VarneyPItzhakG.Crack detection in a rotor dynamic system by vibration monitoring—part II: extended analysis and experimental results. J Eng Gas Turbines Power2012; 134(11): 112501.
4.
DiamantiKSoutisC.Structural health monitoring techniques for aircraft composite structures. Prog Aerosp Sci2010; 46: 342–352.
5.
TimCRomuloLSidneyG.Fpga-based design for real-time crack detection based on particle filter. IEEE Trans Industr Inform2020; 16(9): 5703–5711.
6.
OliveiraHCorreiaPL.Automatic road crack detection and characterization. IEEE Trans Intell Transp Syst2013; 14(1): 155–168.
7.
DongHSongKHeY, et al. PGA-Net: pyramid feature fusion and global context attention network for automated surface defect detection. IEEE Trans Industr Inform2020; 16(12): 7448–7458.
8.
RaoASNguyenTPalaniswamiM, et al. Vision-based automated crack detection using convolutional neural networks for condition assessment of infrastructure. Struct Health Monit2020; 20(4): 2124–2142.
MohtashamKMVahidniaSGhasemzadehL, et al. Deep-learning-based crack detection with applications for the structural health monitoring of gas turbines. Struct Health Monit2020; 19(5): 1440–1452.
11.
ChenFCJahanshahiMR.NB-CNN: deep learning-based crack detection using convolutional neural network and naïve Bayes data fusion. IEEE Trans Industr Electron2018; 65(5): 4392–4400.
12.
JangKKimNAnYK.Deep learning–based autonomous concrete crack evaluation through hybrid image scanning. Struct Health Monit2019; 18(5–6): 1722–1737.
13.
ZhouSSongW.Deep learning–based roadway crack classification with heterogeneous image data fusion. Struct Health Monit2020; 20(3): 1274–1293.
14.
ZhangLShenJZhuB.A research on an improved Unet-based concrete crack detection algorithm. Struct Health Monit2020; 20(4): 1864–1879.
15.
ChoiWChaYJ.SDDNet: real-time crack segmentation. IEEE Trans Ind Electron2020; 67: 8016–8025.
16.
WuXSahooDHoiSC.Recent advances in deep learning for object detection. Neurocomputing2020; 396: 39–64.
17.
RenSHeKGirshickR, et al. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell2017; 39(6): 1137–1149.
18.
CaiZVasconcelosN.Cascade R-CNN: delving into high quality object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Salt Lake City, UT, USA, 18–23 June 2018, pp. 6154–6162. New York, NY: IEEE.
19.
ChaYJChoiWSuhG, et al. Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Comput Aided Civ Infrastruct Eng2018; 33: 731–747.
20.
SuhGChaYJ.Deep faster R-CNN-based automated detection and localization of multiple types of damage. In: Proceedings of sensors and smart structures technologies for civil, mechanical, and aerospace systems (ed HSohn), Denver, CO, USA, 3–7 March 2019, p. 1059812.
21.
DengJLuYLeeVC.Concrete crack detection with handwriting script interferences using faster region-based convolutional neural network. Comput Aided Civ Infrastruct Eng2020; 35: 373–388.
22.
RedmonJFarhadiA.YOLO9000: better, faster, stronger. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, HI, USA, 21–26 July 2017. New York, NY: IEEE.
23.
YuZShenYShenC.A real-time detection approach for bridge cracks based on yolov4-fpm. Automat Constr2021; 122: 103514.
24.
KangDBenipalSSGopalDL, et al. Hybrid pixel-level concrete crack segmentation and quantification across complex backgrounds using deep learning. Automat Constr2020; 118: 103291.
FawcettT.An introduction to roc analysis. Pattern Recognit Lett2006; 27(8): 861–874.
27.
HandDJ.Measuring classifier performance: a coherent alternative to the area under the ROC curve. Mach Learn2009; 77(1): 103–123.
28.
EveringhamMWinnJ.The pascal visual object classes challenge 2007 (voc2007) development kit. Int J Comput Vis2006; 111(1): 98–136.
29.
LinTYMaireMBelongieS, et al. Microsoft COCO: common objects in context. In: FleetDPajdlaTSchieleBTuytelaarsT (eds.) European conference on computer vision. Lecture Notes in Computer Science, vol. 8693. Cham: Springer, 2014; pp. 740–755.
30.
ChavaliNAgrawalHMahendruA, et al. Object-proposal evaluation protocol is ‘gameable’. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA, 2016, pp. 835–844. New York, NY: IEEE.
31.
HosangJBenensonRDollárP, et al. What makes for effective detection proposals?IEEE Trans Pattern Anal Mach Intell2016; 38(4): 814.
32.
SkinnerJHallDZhangH, et al. The probabilistic object detection challenge. arXiv:1903.07840, 2019.
33.
GirshickRDonahueJDarrellT, et al. Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Columbus, OH, USA, 23–28 June 2014, pp. 580–587. New York, NY: IEEE.
34.
GirshickR.Fast R-CNN. In: Proceedings of the IEEE international conference on computer vision, Santiago, Chile, 07–13 December 2015, pp. 1440–1448. New York, NY: IEEE.
35.
DaiJLiYHeK, et al. R-FCN: object detection via region-based fully convolutional networks. arXiv: 1605.06409, 2016.
36.
LiuWAnguelovDErhanD, et al. SSD: single shot multibox detector. In: LeibeBMatasJSebeNWellingM (eds.) European conference on computer vision. Lecture Notes in Computer Science, vol. 9905. Cham: Springer, 2016, pp. 21–37.
37.
RedmonJDivvalaSGirshickR, et al. You only look once: unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA, 27–30 June 2016. New York, NY: IEEE.
38.
RedmonJFarhadiA.YOLOv3: an incremental improvement. arXiv:180402767, 2018.
39.
BochkovskiyAWangCYLiaoHYM. YOLOv4: optimal speed and accuracy of object detection. arXiv: 2004.10934, 2020.
40.
SimonyanKZissermanA.Very deep convolutional networks for large-scale image recognition. arXiv: 1409.1556, 2015.
41.
HeKZhangXRenS, et al. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA, 27–30 June 2016. pp. 770–778. New York, NY: IEEE.
42.
HowardAGZhuMChenB, et al. MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv:170404861, 2017.
43.
MandelbrotB.How long is the coast of Britain? Statistical self-similarity and fractional dimension. Science1967; 156(3775): 636–638.
44.
SaupeD.Algorithms for random fractals. New York, NY: Springer.
45.
BannonTJ.Fractals and transformations. Math Teach1991; 84(3): 178–185.
46.
KangDChaYJ.Efficient attention-based deep encoder and decoder for automatic crack segmentation. Struct Health Monit2022; 21: 2190–2205.
47.
GoutteCGaussierE.A probabilistic interpretation of precision, recall and f-score, with implication for evaluation. Int J Radiat Biol Relat Stud Phys Chem Med2005; 51(5): 345–359.
48.
AbadiMBarhamPChenJ, et al. Tensorflow: a system for large-scale machine learning. arXiv:1605.08695, 2016.
49.
EggertCZechaDBrehmS, et al. Improving small object proposals for company logo detection. In: Proceedings of the ACM on international conference on multimedia retrieval, Bucharest, Romania, 6 June 2017, pp. 167–174. ACM.
50.
YangTZhangXLiZ, et al. MetaAnchor: learning to detect objects with customized anchors. In: Proceedings of the 32nd international conference on neural information processing systems, Palais des Congrès de Montréal, Montréal, Canada, 3 December 2018, pp. 318–328. ACM.
