Ultrasonic wave propagation imaging enables the detection of anomalies in various structures; hence, it has been applied as one of the promising techniques for damage identification in structural health monitoring (SHM). The interpretation of imaging data is vital to SHM; however, it relies significantly on expert subjective judgment, rendering the results vulnerable to human errors. Recent advances in the field of computer vision arising from the adoption of deep neural networks have resulted in new perspectives for substituting human roles in laborious data interpretation tasks. This paper presents an effective learning architecture that can characterize the ultrasonic wave propagation videos for automatic non-destructive inspection. The main contribution is threefold: 1. To the best of our knowledge, this is the first study to leverage video content analysis techniques to exploit ultrasonic wave propagation image series. Previous approaches that focused on the still wavefield images are likely to lose critical temporal information, thereby resulting in an inferior performance. 2. We devise a model that progressively aggregates both temporal and spatial information encoded in multiple adjacent snapshots of ultrasonic wave propagation motions for efficient data analysis. We presented the details regarding the system implementation and critical parameter settings. 3. The proposed approach is validated through extensive experimental comparisons with other state-of-the-art computer vision techniques on a real dataset which is publicly available. We hope that this study will encourage further investigations into video-based non-destructive data interpretation, not limited to ultrasonic signals.
SchmerrLW. Fundamentals of Ultrasonic Nondestructive Evaluation. New York, NY: Springer, 2016.
2.
YashiroSTakatsuboJMiyauchiH, et al.A novel technique for visualizing ultrasonic waves in general solid media by pulsed laser scan. NDT E Int2008; 41(2): 137–144.
3.
MichaelsJE. Ultrasonic wavefield imaging: research tool or emerging nde method? In: 43rd Annual Review of Progress in Quantitative Nondestructive Evaluation AIP Conference Proceedings, Atlanta, Georgia, 17–22 July, 1806, p. 020001.
4.
YeagerMGregoryBKeyC, et al.On using robust mahalanobis distance estimations for feature discrimination in a damage detection scenario. Struct Health Monit2019; 18(1): 245–253.
5.
WangRChenchoAnS, et al.Deep residual network framework for structural health monitoring. Struct Health Monit2021; 20(4): 1443–1461.
6.
RobiniMCMagninIEBenoit-CattinH, et al.Two-dimensional ultrasonic flaw detection based on the wavelet packet transform. IEEE Trans Ultrason Ferroelectrics Frequency Control1997; 44(6): 1382–1394.
7.
KudelaPRadzieńskiMOstachowiczW. Identification of cracks in thin-walled structures by means of wavenumber filtering. Mech Syst Signal Process2015; 50-51: 456–466.
8.
RadzienskiMKrawczukM´ ZakA, et al.Application of rms for damage detection by guided elastic waves. In: Journal of Physics: Conference Series,Volume 305, Oxford, United Kingdom, IOP Publishing, 11–13July 2011.
9.
MesnilOYanHRuzzeneM, et al.Fast wavenumber measurement for accurate and automatic location and quantification of defect in composite. Struct Health Monit2016; 15(2): 223–234.
10.
DafyddISharif KhodaeiZ. Analysis of barely visible impact damage severity with ultrasonic guided lamb waves. Struct Health Monit2020; 19(4): 1104–1122, p. 012085.
11.
BaoYLiH. Machine learning paradigm for structural health monitoring. Struct Health Monit2021; 20(4): 1353–1372.
12.
VirkkunenIKoskinenT. Flaw detection in ultrasonic data using deep learning.In: Baltica XI 2019: International Conference on Life Management and Maintenance for Power Plants, Helsinki, 11–13 June 2019. VTT Technical Research Centre of Finland.
13.
YeJItoSToyamaN. Computerized ultrasonic imaging inspection: From shallow to deep learning. Sensors2018; 18(11): 3820.
LeshnoMLinVYPinkusA, et al.Multilayer feedforward networks with a nonpolynomial activation function can approximate any function. Neural Networks1993; 6(6): 861–867.
16.
LuZPuHWangF, et al.The expressive power of neural networks: a view from the width. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NY, USA, 4 December 2017. pp. 6232–6240.
17.
GoodfellowIBengioYCourvilleA, et al.Deep Learning, Cambridge: MIT press Cambridge, 2016.
18.
LawsonSWParkerGA. Automatic detection of defects in industrial ultrasound images using a neural network. In: Vision Systems: Applications, volume 2786, pp. 37–47, International Society for Optics and Photonics. SPIE, 1996.
19.
GuarneriGAJuniorFNde ArrudaLVR. Comparative evaluation of artificial neural networks models to classify weld flaws using pulse-echo ultrasonic signals. In: 22nd International Congress of Mechanical Engineering, Brazil, 3–7 November 2013, pp. 2126–2134.
20.
VirupakshappaKOrukluE. Ultrasonic flaw detection using support vector machine classification. In: 2015 IEEE International Ultrasonics Symposium (IUS), Taipei, Taiwan, 21–24 October 2015, pp. 1–4.
21.
ChunP-jUjikeIMishimaK, et al.Random forest-based evaluation technique for internal damage in reinforced concrete featuring multiple nondestructive testing results. Construction Building Mater2020; 253: 119238.
22.
LeCunYBengioYHintonG. Deep learning. Nature2015; 521(7553): 436–444.
23.
MunirNParkJKimH-J, et al.Performance enhancement of convolutional neural network for ultrasonic flaw classification by adopting autoencoder. NDT E Int2020; 111: 102218.
24.
MengMChuaYJWoutersonE, et al.Ultrasonic signal classification and imaging system for composite materials via deep convolutional neural networks. Neurocomputing2017; 257: 128–135.
25.
PyleRJBevanRLHughesRR, et al.Deep learning for ultrasonic crack characterization in nde. IEEE Trans Ultrason Ferroelectrics, Frequency Control2021; 68: 1854–1865.
26.
VirkkunenIKoskinenTJessen-JuhlerO, et al.Augmented ultrasonic data for machine learning. J Nondestructive Eval2021; 40(1): 1–11.
27.
ZhuYLiXLiuC, et al.A comprehensive study of deep video action recognition. arXiv preprint arXiv:201206567 2020.
28.
WangHKläserASchmidC, et al.Dense Trajectories and Motion Boundary Descriptors for Action Recognition. Int Journal Computer Vision2013; 103(1): 60–79.
29.
HeKZhangXRenS, et al.Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA, 27–30 June 2016, pp. 770–778.
30.
HuangGLiuZVan Der MaatenL, et al.Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, HI, USA, 21–26 July 2017, pp. 4700–4708.
31.
DraiRKhelilMBenchaalaA. Time frequency and wavelet transform applied to selected problems in ultrasonics nde. NDT E Int2002; 35(8): 567–572.
32.
CruzFCSimas FilhoEFAlbuquerqueMCS, et al.Efficient feature selection for neural network based detection of flaws in steel welded joints using ultrasound testing. Ultrasonics2017; 73: 1–8.
33.
AlguriKSChiaCCHarleyJB. Model-driven, wavefield baseline subtraction for damage visualization using dictionary learning. In: Structural Health Monitoring, Stanford, CA, 12–14 September 2017.
34.
ZahranOAl-NuaimyW. Utilising phase relationships for automatic weld flaw categorisation in time-of-flight diffraction images. In: 11th International Conference on Fracture (ICF), Turin, Italy, pp. 20–25.
35.
ShitoleCSNZahranOAl-NuaimyW. Combining fuzzy logic and neural networks in classification of weld defects using ultrasonic time-of-flight diffraction. Insight - Non-destructive Test Condition Monit2007; 49(2): 79–82.
36.
SambathSNagarajPSelvakumarN. Automatic defect classification in ultrasonic ndt using artificial intelligence. J Nondestructive Evaluation2011; 30(1): 20–28.
37.
JinHYangKWuS, et al.Sparse deconvolution method for ultrasound images based on automatic estimation of reference signals. Ultrasonics2016; 67: 1–8.
38.
TianYMaitraRMeekerWQ, et al.A statistical framework for improved automatic flaw detection in nondestructive evaluation images. Technometrics2017; 59(2): 247–261.
39.
YeJToyamaN. Benchmarking deep learning models for automatic ultrasonic imaging inspection. IEEE Access2021; 9: 36986–36994. DOI: 10.1109/ACCESS.2021.3062860.
40.
IjjehAAUllahSKudelaP. Full wavefield processing by using fcn for delamination detection. Mech Syst Signal Process2021; 153: 107537.
41.
TranDBourdevLFergusR, et al.Learning spatiotemporal features with 3D convolutional networks. In Proceedings of the IEEE international conference on computer vision, Santiago, Chile, 7–13 December 2015, pp. 4489–4497.
42.
SunLJiaKYeungDY, et al.Human action recognition using factorized spatio-temporal convolutional networks. In Proceedings of the IEEE international conference on computer vision, Santiago, Chile, 7–13 December 2015, pp. 4597–4605.
43.
TranDWangHTorresaniL, et al.A closer look at spatiotemporal convolutions for action recognition. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018, pp. 6450–6459.
44.
ToyamaNYamamotoTUrabeK, et al.Ultrasonic inspection of adhesively bonded CFRP/aluminum joints using pulsed laser scanning. Adv Compos Mater2019; 28(1): 27–35.
45.
ZhongZZhengLKangG, et al.Random erasing data augmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, 7–12 February 2020, volume 34. pp. 13001–13008.
46.
DalalNTriggsB. Histograms of oriented gradients for human detection. In: IEEE computer society conference on computer vision and pattern recognition (CVPR’05), San Diego, CA, USA, 20–25 June 2005, pp. 886–893.
47.
OshiroTMPerezPSBaranauskasJA. How many trees in a random forest? In: International workshop on machine learning and data mining in pattern recognition, Berlin, Germany, 13 July 2012, Springer, pp. 154–168.
48.
McInnesLHealyJSaulN, et al.Umap: uniform manifold approximation and projection. J Open Source Softw2018; 3(29): 861.
49.
KrizhevskyASutskeverIHintonGE. Imagenet classification with deep convolutional neural networks. Adv Neural Information Processing Systems2012; 25: 1097–1105.
50.
MurphyKP. Machine Learning: A Probabilistic Perspective. MA: MIT press, 2012.
