Abnormal data in structural health monitoring (SHM) system can have a significant impact on the assessment of the true condition of a structure. Most existing studies are costly and need the manual screening of training data, which have significantly hindered the development of automated and efficient cleaning of monitoring data. To address this, an unsupervised learning-based data cleaning framework for bridge structural health monitoring is proposed. The continuous wavelet transform is used to convert monitoring data into wavelet time–frequency (WTF) images. A pre-trained GoogLeNet model is used as a feature extractor to capture the features from the WTF images. The extracted high-dimensional features are clustered using K-means algorithm. The results of cluster assignments serve as pseudo-labels, providing initial supervised labels for subsequent model refinement. The main-girder acceleration data from the SHM system of a large-span cable-stayed bridge is taken as an example to verify the effectiveness of the proposed framework. In the large-scale test, the precision of normal data is 98.4%. In addition, the proposed framework can also accurately clean all data types with an accuracy of 97.9%. The results show that all data types can be accurately detected. Meanwhile, the proposed framework does not require any manual prior screening of the dataset and accomplishes automatic cleaning of monitoring data at low cost.
EntezamiAShariatmadarH. An unsupervised learning approach by novel damage indices in structural health monitoring for damage localization and quantification. Struct Health Monit2017; 17: 325–345.
2.
JuHZhaiWDengY, et al. Temperature time-lag effect elimination method of structural deformation monitoring data for cable-stayed bridges. Case Stud Therm Eng2023; 42: 102696.
3.
JuHShiHShenW, et al. An accurate and low-cost vehicle-induced deflection prediction framework for long-span bridges using deep learning and monitoring data. Eng Struct2024; 310: 118094.
4.
WickramarachchiCTMaguireECrossEJ, et al. Measuring data similarity in population-based structural health monitoring using distance metrics. Struct Health Monit2023; 23(4): 2609–2635.
5.
JuHDengYZhaiW, et al. Recovery of abnormal data for bridge structural health monitoring based on deep learning and temporal correlation. Sens Mater2022; 34(12): 4491.
6.
WangQ-ALiuQMaZ-G, et al. Data interpretation and forecasting of SHM heteroscedastic measurements under typhoon conditions enabled by an enhanced Hierarchical sparse Bayesian Learning model with high robustness. Measurement2024; 230: 114509.
7.
DengYZhaoYJuH, et al. Abnormal data detection for structural health monitoring: state-of-the-art review. Dev Built Environ2024; 17: 100337.
8.
DengYJuHZhongG, et al. A general data quality evaluation framework for dynamic response monitoring of long-span bridges. Mech Syst Signal Process2023; 200: 110514.
9.
HuangS-KLinT-X. Development of data anomaly classification for structural health monitoring based on iterative trimmed loss minimization and human-in-the-loop learning. Struct Health Monit2024; 24: 548–565.
10.
TabatabaeiMKimiaefarRHajianA, et al. Robust outlier detection in geo-spatial data based on LOLIMOT and KNN search. Earth Sci Inform2021; 14: 1065–1072.
11.
JianXZhongHXiaY, et al. Faulty data structural health monitoring via statistical and deep-learning approach. Struct Control Health Monit2021; 28: e2824.
12.
ZhangYTangZYangR. Data anomaly detection for structural health monitoring by multi-view representation based on local binary patterns. Measurement2022; 202: 111804.
13.
CaiSLiLLiS, et al. An efficient approach for outlier detection from uncertain data streams based on maximal frequent patterns. Expert Syst Appl2020; 160: 113646.
14.
KaoJ-BJiangJ-R. Anomaly detection for univariate time series with statistics and deep learning. In: 2019 IEEE Eurasia conference on IOT, communication and engineering (ECICE), 2019.
15.
YangJYangFZhangL, et al. Bridge health anomaly detection using deep support vector data description. Neurocomputing2021; 444: 170–178.
16.
WangHBahMJHammadM. Progress in outlier detection techniques: a survey. IEEE Access2019; 7: 107964–108000.
17.
ChoiKYiJParkC, et al. Deep learning for anomaly detection in time-series data: review, analysis, and guidelines. IEEE Access2021; 9: 120043–120065.
18.
QuBLiaoPHuangY. Outlier detection and forecasting for bridge health monitoring based on time series intervention analysis. Struct Durab Health Monit2022; 16: 323–341.
19.
ZhangJZhangJWuZ. Long-short term memory network-based monitoring data anomaly detection of a long-span suspension bridge. Sensors2022; 22: 6045.
20.
ChaY-JAliRLewisJ, et al. Deep learning-based structural health monitoring. Autom Constr2024; 161: 105328.
21.
WangZChaY-J. Unsupervised deep learning approach using a deep auto-encoder with a one-class support vector machine to detect structural damage. Struct Health Monit2020; 20(1): 406–425.
22.
BaoYTangZLiH, et al. Computer vision and deep learning–based data anomaly detection method for structural health monitoring. Struct Health Monit2018; 18: 401–421.
23.
TangZChenZBaoY, et al. Convolutional neural network-based data anomaly detection method using multiple information for structural health monitoring. Struct Control Health Monit2018; 26: e2296.
24.
LeiXXiaYWangA, et al. Mutual information-based anomaly detection of monitoring data with attention mechanism and residual learning. Mech Syst Signal Process2023; 182: 109607.
25.
DengYJuHZhongG, et al. Data quality evaluation for bridge structural health monitoring based on deep learning and frequency-domain information. Struct Health Monit2022; 22: 2925–2847.
26.
MaoJWangHSpencerBF. Toward data anomaly detection for automated structural health monitoring: exploiting generative adversarial nets and autoencoders. Struct Health Monit2021; 20(4): 1609–1626.
27.
PiomboBFasanaAMarchesielloS, et al. Modelling and identification of the dynamic response of a supported bridge. Mech Syst Signal Process2000; 14: 75–89.
28.
JianXXiaYSunL. An indirect method for bridge mode shapes identification based on wavelet analysis. Struct Control Health Monit2020; 27: e2630.
29.
PuriNValeroETurkanY, et al. Assessment of compliance of dimensional tolerances in concrete slabs using TLS data and the 2D continuous wavelet transform. Autom Constr2018; 94: 62–72.
30.
GhaffariAGolbayaniHGhasemiM. A new mathematical-based QRS detector using continuous wavelet transform. Comput Electr Eng2008; 34: 81–91.
31.
WangHXuZWuT, et al. Evolutionary power spectral density of recorded typhoons at Sutong Bridge using harmonic wavelets. J Wind Eng Ind Aerodyn2018; 177: 197–212.
32.
MarraVPaulaACunhaAL, et al. Vibration monitoring of a small concrete bridge using wavelet transforms on GPS data. J Civ Struct Health Monit2019; 9: 397–409.
33.
JuHZhaoYChenX, et al. Real-time identification framework for instantaneous frequency of bridge cables considering multiple types of abnormal monitoring data. Eng Struct2025; 340: 120734.
34.
QuHLiTWangR, et al. Application of adaptive wavelet transform-based multiple analytical mode decomposition for damage progression identification of cable-stayed bridge via shake table test. Mech Syst Signal Process2021; 149: 107055.
35.
JuHZhaoYDengY, et al. Deflection prediction and early warning for long-span bridges using a fusion ATT-BiGRU model and real-time updated thresholds. Structures2025; 76: 109061.
36.
SzegedyCLiuWJiaY, et al. Going deeper with convolutions. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), 2015.
37.
AkarsuEOzbekIYKaracaliT. Effects of different pre-training algorithms on video classification results. Trait Signal2024; 41: 2221–2231.
38.
EhtishamRQayyumWCampCV, et al. Classification of defects in wooden structures using pre-trained models of convolutional neural network. Case Stud Constr Mater2023; 19: e02530.
39.
QayyumWEhtishamRBahramiA, et al. Assessment of convolutional neural network pre-trained models for detection and orientation of cracks. Materials2023; 16: 826.
40.
NasrBElloumi OueslatiA. Human exons and introns classification using pre-trained Resnet-50 and GoogLeNet models and 13-layers CNN model. J Genet Eng Biotechnol2024; 22: 100359.
41.
ZhangJ. Advancements of outlier detection: a survey. Trans Scalable Inf Syst2013; 13: e2.
42.
YangXJeffersonWBulbulT. Evaluation of construction worker perceptions of wearable proximity sensors during the COVID-19 pandemic. J Constr Eng Manage2025; 151(3) : 04024217.
43.
SyakurMAKhotimahBKRochmanEMS, et al. Integration K-means clustering method and elbow method for identification of the best customer profile cluster. IOP Conf Ser: Mater Sci Eng2018; 336: 012017.
44.
SaiLNShreyaMSSubudhiAA, et al. Optimal K-means clustering method using silhouette coefficient. Int J Appl Res Inf Technol Comput2017; 8: 335.
45.
BagirovAMAliguliyevRMSultanovaN. Finding compact and well-separated clusters: clustering using silhouette coefficients. Pattern Recognit2023; 135: 109144.
46.
JohnsonJDouzeMJegouH. Billion-scale similarity search with GPUs. IEEE Trans Big Data2019; 7: 1.
47.
DoerschCGuptaAEfrosAAA. Unsupervised visual representation learning by context prediction. In: 2015 IEEE International Conference on Computer Vision (ICCV), 2015.
48.
DengYJuHLiY, et al. Abnormal data recovery of structural health monitoring for ancient city wall using deep learning neural network. Int J Archit Herit2024; 18(3): 389–407.
49.
VanDGeversTSnoekCGM. Evaluating color descriptors for object and scene recognition. IEEE Trans Pattern Anal Mach Intell2010; 32: 1582–1596.
50.
AounAMasadehMTaharS. Design space exploration for energy-efficient approximate Sobel filter. AEÜ Int J Electron Commun2023; 172: 154887.
51.
PaulinMDouzeMHarchaouiZ, et al. Local convolutional features with unsupervised training for image retrieval. In: 2015 IEEE international conference on computer vision (ICCV), 2015.
52.
JuHDengYZhaoY, et al. Automatic modal frequency identification of bridge cables under influence of abnormal monitoring data. J Perform Constr Fac2024; 38(6): 04024046.
53.
AmelioAPizzutiC. Correction for closeness: adjusting normalized mutual information measure for clustering comparison. Comput Intell2016; 33: 579–601.
