As image acquisition devices have outstanding potential for gathering vibration information, computer vision has received a lot of interest in structural health monitoring (SHM). In this work, a fully automated peak picking methodology based on computer vision in tandem with deep learning is proposed to realize vibration measurements and identify natural frequencies from the plot of the power spectral density transmissibility (PSDT). A deep-learning-enhanced image processing technology was used to extract the vibration signals with automatic active pixel selection, while a convolutional neural network was used to further process the vibration measurements so that the frequencies could be identified from PSDT-based functions. The proposed method was verified by three case studies, including the dynamic testing of two laboratory models and the field testing of the stay cable. The findings showed that the proposed deep-learning-enhanced approach has a high potential for use in SHM by automatically performing vibration measurement and frequency extraction.
AgarwalAGuptaSSinghDK (2016). Review of optical flow technique for moving object detection. In: 2016 2nd International Conference on Contemporary Computing and Informatics (IC3I), Greater Noida, India, 14–17 December 2016, pp. 409–413.
2.
ArbeláezPMaireMFowlkesC, et al. (2011) Contour detection and hierarchical image segmentation. IEEE transactions on pattern analysis and machine intelligence33: 898–916.
3.
AslaniSMahdavinasabH (2013) Optical flow based moving object detection and tracking for traffic surveillance. International Journal of Electrical, Computer, Energetic, Electronic and Communication Engineering7(9): 1252–1256.
4.
BaqersadJPoozeshPNiezreckiC, et al. (2016) Photogrammetry and optical methods in structural dynamics – A review. Mechanical Systems and Signal Processing86: 17–34.
5.
ChaY-JChoiWBüyüköztürkO (2017) Deep learning-based crack damage detection using convolutional neural networks. Computer-Aided Civil and Infrastructure Engineering32(5): 361–378.
6.
ChenZ-WLiuK-MYanW-J, et al. (2021) Two-stage automated operational modal analysis based on power spectrum density transmissibility and support-vector machines. International Journal of Structural Stability and Dynamics21: 2150068. DOI: 10.1142/S0219455421500681.
7.
CherriAKKarimMA (1989) Optical symbolic substitution: edge detection using Prewitt, Sobel, and Roberts operators. Applied optics28(21): 4644–4648.
8.
ChingJBeckJL (2004) Bayesian analysis of the phase II IASC-ASCE structural health monitoring experimental Benchmark data. Journal of Engineering Mechanics130: 1233–1244.
9.
ChoSLynchJPLeeJ-J, et al. (2010) Development of an automated wireless tension force estimation system for cable-stayed bridges. Journal of Intelligent Material Systems and Structures21: 361–376.
10.
CuevasEEchavarríaAZaldívarD, et al. (2013) A novel evolutionary algorithm inspired by the states of matter for template matching. Expert Systems with Applications40: 6359–6373.
11.
DevriendtCGuillaumeP (2007) The use of transmissibility measurements in output-only modal analysis. Mechanical Systems and Signal Processing21: 2689–2696.
12.
DiamondDHHeynsPSOberholsterAJ (2017) Accuracy evaluation of sub-pixel structural vibration measurements through optical flow analysis of a video sequence. Measurement95: 166–172.
13.
DingLGoshtasbyA (2000) On the canny edge detector. Pattern Recognition34: 721–725.
14.
DungCVSekiyaHHiranoS, et al. (2019) A vision-based method for crack detection in gusset plate welded joints of steel bridges using deep convolutional neural networks. Automation in Construction102: 217–229.
15.
FengDFengMOzerE, et al. (2015) A vision-based sensor for noncontact structural displacement measurement. Sensors15(7): 16557–16575.
16.
FuLFengYMajeedY, et al. (2018) Kiwifruit detection in field images using Faster R-CNN with ZFNet. IFAC-PapersOnLine51(17): 45–50.
17.
GunesB (2013) Health Monitoring of a Benchmark Structure Using Vibration Data. Dordrecht: Springer, pp. 1053–1058.
18.
GuoJWuXLiuJ, et al. (2021) Non-contact vibration sensor using deep learning and image processing. Measurement183: 109823.
19.
HavaranAMahmoudiM (2020) Markers tracking and extracting structural vibration utilizing randomized hough transform. Automation in Construction116: 103235.
20.
HelfrickMNNiezreckiCAvitabileP, et al. (2011) 3D digital image correlation methods for full-field vibration measurement. Mechanical Systems and Signal Processing25: 917–927.
21.
JeongSKimHLeeJ, et al. (2021) Automated wireless monitoring system for cable tension forces using deep learning. Structural Health Monitoring20(4): 1805–1821.
22.
JiYFChangCC (2008) Nontarget image-based technique for small cable vibration measurement. Journal of Bridge Engineering13(1): 34–42.
23.
JinS-SJeongSSimS-H, et al. (2021) Fully automated peak-picking method for an autonomous stay-cable monitoring system in cable-stayed bridges. Automation in Construction126: 103628.
24.
KimDKimMBaeSG, et al. (2021) Field measurements for identification of modal parameters for high-rise buildings under construction or in use. Automation in Construction121: 103446.
25.
KimHSimSH (2019) Automated peak picking using region-based convolutional neural network for operational modal analysis. Structural Control and Health Monitoring26(11): e2436.
26.
KimJ-TNguyenK-DHuynhT-C (2013) Wireless health monitoring of stay cable using piezoelectric strain response and smart skin technique. Smart Struct. Syst12(3–4): 381–397.
27.
KrizhevskyASutskeverIHintonGE (2017) ImageNet classification with deep convolutional neural networks. Communications of the Acm60(6): 84–90.
28.
KuangZYuJLiZ, et al. (2018) Integrating multi-level deep learning and concept ontology for large-scale visual recognition. Pattern Recognition78: 198–214.
29.
KuddusMALiJHaoH, et al. (2019) Target-free vision-based technique for vibration measurements of structures subjected to out-of-plane movements. Engineering Structures190: 210–222.
30.
LeiYShenWASongY, et al. (2010) Intelligent wireless sensors with application to the identification of structural modal parameters and steel cable forces: from the lab to the field. Advances in Civil Engineering2010: 1–9.
31.
LiJHaoHLoJV (2015) Structural damage identification with power spectral density transmissibility: numerical and experimental studies. Smart Structures and Systems15(1): 15–40.
32.
LiuXChiMZhangY, et al. (2018). Classifying high resolution remote sensing images by fine-tuned VGG deep networks. In: IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium, Valencia, Spain, 22–27 July 2018, pp. 7137–7140.
33.
LiuXWangZWangL, et al. (2021) A hybrid Rao-NM algorithm for image template matching. Entropy23: 678.
34.
LiuYLewM (2016). Learning relaxed deep supervision for better edge detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016, pp. 231–240.
35.
LuZ-RLinGWangL (2021) Output-only modal parameter identification of structures by vision modal analysis. Journal of Sound and Vibration497: 115949.
36.
NguyenPMLChoJHChoSB (2014). An architecture for real-time hardware co-simulation of edge detection in image processing using Prewitt edge operator. In:2014 International Conference on Electronics, Information and Communications (ICEIC), Kota Kinabalu, Malaysia, 15–18 January 2014, pp. 1–2.
37.
ParkHSOhBK (2018) Real-time structural health monitoring of a supertall building under construction based on visual modal identification strategy. Automation in Construction85: 273–289.
38.
PathirageCSNLiJLiL, et al. (2018) Structural damage identification based on autoencoder neural networks and deep learning. Engineering Structures172: 13–28.
39.
PeetersBDe RoeckG (1999) Reference-based stochastic subspace identification for output-only modal analysis. Mechanical Systems and Signal Processing13: 855–878.
40.
PeetersBVan der AuweraerHGuillaumeP, et al. (2004) The PolyMAX frequency-domain method: a new standard for modal parameter estimation?Shock and Vibration11: 395–409.
41.
PeetersBVenturaCE (2003) Comparative study of modal analysis techniques for bridge dynamic characteristics. Mechanical Systems and Signal Processing17: 965–988.
42.
PieracciniMLuziGMecattiD, et al. (2004) Remote sensing of building structural displacements using a microwave interferometer with imaging capability. NDT & E International37: 545–550.
43.
ReuPRoheDJacobsL (2016) Comparison of DIC and LDV for practical vibration and modal measurements. Mechanical Systems and Signal Processing86: 2–16.
44.
ShaoYLiLLiJ, et al. (2021) Computer vision based target-free 3D vibration displacement measurement of structures. Engineering Structures246: 113040.
45.
ShenWWangXWangY, et al. (2015). DeepContour: a deep convolutional feature learned by positive-sharing loss for contour detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 07–12 June 2015, pp. 3982–3991.
46.
ShiWCaballeroJHuszárF, et al. (2016). Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognitionm, Las Vegas, NV, USA, 27–30 June 2016, pp. 1874–1883.
47.
ShrivakshanGTChandrasekarC (2012) A comparison of various edge detection techniques used in image processing. International Journal of Computer Science Issues9: 269–276.
48.
SonK-SJeonH-SParkJ-H, et al. (2015) Vibration displacement measurement technology for cylindrical structures using camera images. Nuclear Engineering and Technology47: 488–499.
49.
SzegedyCLiuWJiaY, et al. (2014). Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Boston, MA, 07–12 June 2015, pp. 1–9.
50.
TianLPanB (2016) Remote bridge deflection measurement using an advanced video deflectometer and actively illuminated LED targets. Sensors16: 1344.
51.
VincentOFolorunsoO (2009) A descriptive algorithm for sobel image edge detection. In: Proceedings of informing science & IT education conference (InSITE), Macon, GA, USA, 12–15 June 2009, pp. 97–107.
52.
WangYZhaoXHuangK (2017). Deep crisp boundaries. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017
53.
WuZShenCvan den HengelA (2019) Wider or deeper: revisiting the ResNet model for visual recognition. Pattern Recognition90: 119–133.
54.
XieSTuZ (2015). Holistically-nested edge detection. In: Proceedings of the IEEE international conference on computer vision, Santiago, Chile, 07–13 December 2015, pp. 1395–1403.
55.
XinYHaoHLiJ (2019) Time-varying system identification by enhanced empirical wavelet transform based on synchroextracting transform. Engineering Structures196: 109313.
56.
YanW-JRenW-X (2012) Operational modal parameter identification from power spectrum density transmissibility. Computer-Aided Civil and Infrastructure Engineering27: 202–217.
57.
YanW-JRenW-X (2015) An enhanced power spectral density transmissibility (EPSDT) approach for operational modal analysis: theoretical and experimental investigation. Engineering Structures102: 108–119.
58.
YangX-MYiT-HQuC-X, et al. (2019) Automated eigensystem realization algorithm for operational modal identification of bridge structures. Journal of Aerospace Engineering32(2): 04018148.
59.
YeXWDongCZLiuT (2016a) Image-based structural dynamic displacement measurement using different multi-object tracking algorithms. Smart Structures and Systems17(6): 935–956.
60.
YeXWDongCZLiuT (2016b) A review of machine vision-based structural health monitoring: methodologies and applications. Journal of Sensors2016: 7103039.
61.
YeXWNiYQWaiTT, et al. (2013) A vision-based system for dynamic displacement measurement of long-span bridges: algorithm and verification. Smart Structures and Systems12(3): 363–379.
62.
YuLPanB (2017) Single-camera high-speed stereo-digital image correlation for full-field vibration measurement. Mechanical Systems and Signal Processing94: 374–383.
