In this study, a vision-based multi-point structural dynamic monitoring framework is proposed. This framework aims to solve issues in current vision-based structural health monitoring. Limitations are due to manual markers, single-point monitoring, and synchronization between a multiple-camera setup and a sensor network. The proposed method addresses the first issue using virtual markers—features extracted from an image—instead of physical manual markers. The virtual markers can be selected according to each scenario, which makes them versatile. The framework also overcomes the issue of single-point monitoring by utilizing an advanced visual tracking algorithm based on optical flow, allowing multi-point displacement measurements. Besides, a synchronization mechanism between a multiple-camera setup and a sensor network is built. The proposed method is first tested on a grandstand simulator located in the laboratory. The experiment is to verify the performance of displacement measurement of the proposed method and conduct structural identification of the grandstand through multi-point displacement records. The results from the proposed method are then compared to the data gathered by traditional displacement sensors and accelerometers. A second experiment is conducted at a stadium during a football game to validate the feasibility of field application and the operational modal identification of the stadium under human crowd jumping through the measured displacement records. From these experiments, it is concluded that the proposed method can be employed to identify modal parameters for structural health monitoring.
GulMCatbasFN.Ambient vibration data analysis for structural identification and global condition assessment. J Eng Mech2008; 134: 650–662.
2.
CatbasFNAktanAE.Condition and damage assessment: issues and some promising indices. J Struct Eng2002; 128: 1026–1036.
3.
GulMCatbasFN.Damage assessment with ambient vibration data using a novel time series analysis methodology. Construction2011; 137: 1518–1526.
4.
GulMCatbasFN.Structural health monitoring and damage assessment using a novel time series analysis methodology with sensor clustering. J Sound Vib2011; 330: 1196–1210.
5.
AktanAEGrimmelsmanKABarrishRA, et al. Structural identification of a long-span truss bridge. Transp Res Rec2000; 1696: 210–218.
6.
GulMCatbasFN.Statistical pattern recognition for structural health monitoring using time series modeling: theory and experimental verifications. Mech Syst Signal Pr2009; 23: 2192–2204.
7.
PerformanceLGhosnMFrangopolDM, et al. Reliability-based structural performance indicators for structural members. J Struct Eng2016; 142: 1–13.
8.
GulMDumlupinarTHattoriH, et al. Structural monitoring of movable bridge mechanical components for maintenance decision-making2014; 1: 249–271.
9.
ZaurinRKhucTCatbasFN, et al. Hybrid sensor-camera monitoring for damage detection: case study of a real bridge. J Bridg Eng2015; 21: 1–27.
10.
CatbasNZaurinRGulM, et al. Sensor networks, computer imaging, and unit influence lines for structural health monitoring: case study for bridge load rating. J Bridg Eng2012; 17: 662–670.
11.
ZaurinRCatbasFN.Structural health monitoring using video stream, influence lines, and statistical analysis. Struct Health Monit2010; 10: 309–332.
12.
ZaurinRCatbasFN.Integration of computer imaging and sensor data for structural health monitoring of bridges. Smart Mater Struct2010; 19: 1–15.
YeXWDongCZLiuT.A review of machine vision-based structural health monitoring: methodologies and applications. J Sensors2016; 2016: 7103039.
15.
FengDFengMQOzerE, et al. A vision-based sensor for noncontact structural displacement measurement. Sensors2015; 15: 16557–16575.
16.
YeXWYiT-HDongCZ, et al. Vision-based structural displacement measurement: system performance evaluation and influence factor analysis. Measurement2016; 88: 372–384.
17.
YoneyamaSKitagawaAIwataS, et al. Bridge deflection measurement using digital image correlation. Exp Tech2007; 31: 34–40.
18.
KohutPHolakKUhlT, et al. Monitoring of a civil structure’s state based on noncontact measurements. Struct Health Monit2013; 12: 411–429.
19.
YeXWDongCZLiuT.Image-based structural dynamic displacement measurement using different multi-object tracking algorithms. Smart Struct Syst2016; 17: 935–956.
20.
YeXWDongCZLiuT.Force monitoring of steel cables using vision-based sensing technology: methodology and experimental verification. Smart Struct Syst2016; 18: 585–599.
21.
HenkeKPawlowskiRSchregleP, et al. Use of digital image processing in the monitoring of deformations in building structures. J Civ Struct Health Monit2015; 5: 141–152.
22.
ParkJ-WLeeJ-JJungH-J, et al. Vision-based displacement measurement method for high-rise building structures using partitioning approach. NDT&E Int2010; 43: 642–647.
23.
JáureguiDVWhiteKRWoodwardCB, et al. Noncontact photogrammetric measurement of vertical bridge deflection. J Bridg Eng2003; 8: 212–222.
24.
LeeJJChoSShinozukaM.Evaluation of bridge load carrying capacity based on dynamic displacement measurement using real-time image processing techniques. Steel Struct2006; 6: 377–385.
25.
OlaszekP.Investigation of the dynamic characteristic of bridge structures using a computer vision method. Measurement1999; 25: 227–236.
26.
LeeJJShinozukaM.Real-time displacement measurement of a flexible bridge using digital image processing techniques. Exp Mech2006; 46: 105–114.
27.
FengMQFukudaYFengD, et al. Nontarget vision sensor for remote measurement of bridge dynamic response. J Bridg Eng2015; 20: 4015023.
28.
KhucTCatbasFN.Computer vision-based displacement and vibration monitoring without using physical target on structures. Struct Infrastruct E2016; 13: 1–12.
29.
KhucTCatbasFN.Completely contactless structural health monitoring of real-life structures using cameras and computer vision. Struct Control Hlth2017; 24: e1852.
30.
FukudaYFengMQShinozukaM.Cost-effective vision-based system for monitoring dynamic response of civil engineering structures. Struct Control Hlth2010; 17: 918–936.
31.
YeXWYiTHDongCZ, et al. Multi-point displacement monitoring of bridges using a vision-based approach. Wind Struct Int J2015; 20: 315–326.
32.
HoHNLeeJHParkYS, et al. A synchronized multipoint vision-based system for displacement measurement of civil infrastructures. Sci World J2012; 2012: 1–9.
33.
YoonHElanwarHChoiH, et al. Target-free approach for vision-based structural system identification using consumer-grade cameras. Struct Control Hlth2016; 23: 1405–1416.
34.
CelikODongCZCatbasFN.A computer vision approach for the load time history estimation of lively individuals and crowds. Comput Struct2018; 200: 3252.
35.
ChenJGWadhwaNChaYJ, et al. Modal identification of simple structures with high-speed video using motion magnification. J Sound Vib2015; 345: 58–71.
36.
YangYDornCManciniT, et al. Blind identification of full-field vibration modes from video measurements with phase-based video motion magnification. Mech Syst Signal Pr2017; 85: 567–590.
37.
YangYDornCManciniT, et al. Blind identification of full-field vibration modes of output-only structures from uniformly-sampled, possibly temporally-aliased (sub-Nyquist), video measurements. J Sound Vib2017; 390: 232–256.
38.
YangYDornCManciniT, et al. Reference-free detection of minute, non-visible, damage using full-field, high-resolution mode shapes output-only identified from digital videos of structures. Struct Health Monit2017; 514–531.
39.
ZhangZ.A flexible new technique for camera calibration. IEEE T Pattern Anal2002; 22: 1330–1334.
40.
BrownjohnJMWXuYHesterD.Vision-based bridge deformation monitoring. Front Built Environ2017; 3: 1–16.
41.
SzeliskiR.Computer vision: algorithms and applications. London: Springer.
42.
HarrisCStephensM. A combined corner and edge detector. In: Proceedings of the Fourth Alvey Vision Conference, Manchester, 31 August–2 September 1988, pp. 10–5244.
43.
ShiJTomasiC.Good features to track. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Seattle, WA, 21–23 June 1994, pp. 593–600. New York: IEEE.
44.
LoweDG.Distinctive image features from. Int J Comput Vis2004; 60: 91–110.
45.
BayHEssA.Speeded-up robust features (SURF). Comput Vis Image Und2008; 110: 346–359.
46.
RostenEDrummondT.Machine learning for high speed corner detection. Comput Vis2006; 1: 430–443.
RubleeEBradskiG.ORB: an efficient alternative to SIFT or SURF. In: Proceedings of the international conference on computer vision, Barcelona, 6–13 November 2011, pp. 2564–2571.
49.
AlahiAOrtizRVandergheynstP.FREAK: fast retina keypoint. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Providence, RI, 16–21 June 2012, pp. 510–517. New York: IEEE.
50.
LucasBDKanadeT.An iterative image registration technique with an application to stereo vision. In: Proceedings of the 7th international joint conference on artificial intelligence (IJCAI), Vancouver, BC, Canada, 24–28 August 1981, vol. 2, pp. 674–679. New York: ACM.
FarnebäckG.Two-frame motion estimation based on polynomial expansion. Imag Anal2003; 2749: 363–370.
53.
LiuBZaccarinA.New fast algorithms for the estimation of block motion vectors. IEEE T Circuit Syst1993; 3: 148–157.
54.
FleetDJJepsonAD.Computation of component image velocity from local phase information. Int J Comput Vis1990; 5: 77–104.
55.
XuYBrownjohnJKongD.A non-contact vision-based system for multipoint displacement monitoring in a cable-stayed footbridge. Struct Control Hlth2018; 25: 1–23.
WangTCelikOCatbasFN, et al. A frequency and spatial domain decomposition method for operational strain modal analysis and its application. Eng Struct2016; 114: 104–112.
58.
WangTCelikOCatbasF.Damage detection of a bridge model based on operational dynamic strain measurements. Adv Struct Eng2016; 19: 1379–1389.
59.
CatbasFNBrownDLAktanAE.Parameter estimation for multiple-input multiple-output modal analysis of large structures. J Eng Mech2004; 130: 921930.
