Machinery label data is necessary for training intelligent fault diagnosis models. However, unlabeled and abnormal data are commonly seen in these data, resulting in the reduction of data quality. As a result, these low-quality data may lead to inaccurate diagnosis models. To address this issue, a kernel cluster local outlier factor (CLOF) method is proposed for automated labeling and abnormal data detection. The suggested approach can establish the relationship among different samples of label data based on a parameter-free method, that is, the natural neighbor spectrum. Through this relationship, different clusters are searched. Then, CLOF is calculated to evaluate the abnormal degree of different clusters, and clusters whose CLOF is larger than the predetermined threshold value are detected as abnormal data. The natural neighbor spectrum is reconstructed after cleaning abnormalities. Finally, fault types of the data can be labeled automatically based on the relationship between unlabeled and labeled data through the reconstructed spectrum. The proposed method is validated through different experimental data collected from a gear test bench, a real wind turbine, and a centrifugal pump, respectively. The results indicate that the proposed approach is effective in detecting abnormal data with different condition types and labeling data accurately and automatically.
ShaoHXiaoYLengJ, et al. Collaborative human-computer fault diagnosis via calibrated confidence estimation. Adv Eng Inform2025; 65: 103349.
2.
WangJShaoHHeJ, et al. A novel interpretable fault diagnosis method using multi-image feature extraction and attention fusion. Pattern Recognit Lett2025; 189: 38–47.
3.
XuXYangXHeC, et al. Adversarial domain adaptation model based on LDTW for extreme partial transfer fault diagnosis of rotating machines. IEEE Trans Instrum Meas2024; 73: 3538811.
4.
XuXLiBQiaoZ, et al. Caputo-Fabrizio fractional order derivative stochastic resonance enhanced by ADOF and its application in fault diagnosis of wind turbine drivetrain. Renew Energy2023; 219: 119398.
5.
YanXLiuYJiaM. Health condition identification for rolling bearing using a multi-domain indicator-based optimized stacked denoising autoencoder. Struct Health Monit2020; 19(5): 1602–1626.
6.
XuXOuXGeL, et al. Simulated data-assisted fault diagnosis framework with dual-path feature fusion for rolling element bearings under incomplete data. IEEE Trans Instrum Meas2025; 74: 3542671.
7.
WangSTianJLiangP, et al. Single and simultaneous fault diagnosis of gearbox via wavelet transform and improved deep residual network under imbalanced data. Eng Appl Artif Intell2024; 133: 108146.
8.
LuoJShaoHLinJ, et al. Meta-learning with elastic prototypical network for fault transfer diagnosis of bearings under unstable speeds. Reliab Eng Syst Saf2024; 245: 110001.
9.
LiangPXuLShuaiH, et al. Semi-supervised subdomain adaptation graph convolutional network for fault transfer diagnosis of rotating machinery under time-varying speeds. IEEE/ASME Trans Mechatron2024; 29(1): 730–741.
10.
ChenMMaoSLiuY. Big data: a survey. Mobile Netw Appl2014; 19: 171–209.
11.
WangJShaoHYanS, et al. C-ECAFormer: a new lightweight fault diagnosis framework towards heavy noise and small samples. Eng Appl Artif Intell2023; 126: 107031.
12.
MaleklooAOzerEAlHamaydehM, et al. Machine learning and structural health monitoring overview with emerging technology and high-dimensional data source highlights. Struct Health Monit2022; 21(4): 1906–1955.
13.
KongYQinZWangT, et al. Data-driven dictionary design–based sparse classification method for intelligent fault diagnosis of planet bearings. Struct Health Monit2022; 21(4): 1313–1328.
14.
ZhaoDShaoDWangT, et al. Time-frequency self-similarity enhancement network and its application in wind turbines fault analysis. AdvEng Inform2025; 65: 103322.
15.
TalebIDssouliRSerhaniMA. Big Data pre-processing: a quality framework. In: 2015 IEEE international congress on Big Data (BigData Congress), New York, NY, USA, 27 June–2 July 2015, pp. 191–198. Washington, DC: IEEE.
16.
NajafabadiMMVillanustreFKhoshgoftaarTM, et al. Deep learning applications and challenges in big data analytics. J Big Data2015; 2(1): 2–21.
17.
AtlaATadaRShengV, et al. Sensitivity of different machine learning algorithms to noise. J Comput Sci Coll2011; 26(5): 96–103.
18.
LeiXSunLXiaY. Lost data reconstruction for structural health monitoring using deep convolutional generative adversarial networks. Struct Health Monit2021; 20(4): 2069–2087.
19.
García-GilDLuengoJGarcíaS, et al. Enabling smart data: noise filtering in big data classification. Inform Sci2019; 479: 135–152.
20.
ShangZZhaoZYanR, et al. Core loss: mining core samples efficiently for robust machine anomaly detection against data pollution. Mech Syst Signal Process2023; 189: 110046.
21.
WangSLeiYYangB, et al. A graph neural network-based data cleaning method to prevent intelligent fault diagnosis from data contamination. Eng Appl Artif Intell2023; 126: 107071.
22.
XieQWangJTaoG, et al. Research on one-dimensional data quality assurance for machinery health monitoring using compressive sensing and enhanced context encoder. Expert Syst Appl2025; 297(Part B): 129407.
23.
HodgeVAustinJ. A survey of outlier detection methodologies. Artif Intell2004; 22(2): 85–126.
24.
Vera-TudelaLKühnM. Analyzing wind turbine fatigue load prediction: the impact of wind farm flow conditions. Renew Energy2017; 107: 352–360.
25.
PapatheouEDervilisNMaguireA, et al. Performance monitoring of a wind turbine using extreme function theory. Renew Energy2017; 113: 1490–1502.
26.
GuoLLeiYLiN, et al. Machinery health indicator construction based on convolutional neural networks considering trend burr. Neurocomputing2018; 292: 142–150.
27.
ShenXFuXZhouC. A combined algorithm for cleaning abnormal data of wind turbine power curve based on change point grouping algorithm and quartile algorithm. IEEE Trans Sustain Energy2018; 10(1): 46–54.
28.
MckinnonCCarrollJMcdonaldA, et al. Comparison of new anomaly detection technique for wind turbine condition monitoring using gearbox SCADA data. Energies2020; 13: 5152.
29.
XiaoYWangHXuW, et al. Robust one-class SVM for fault detection. Chemometr Intell Lab Syst2016; 151: 15–25.
30.
ChenGZhangXWangZ, et al. Robust support vector data description for outlier detection with noise or uncertain data. Knowl Based Syst2015; 90: 129–137.
31.
BaoYTangZLiH, et al. Computer vision and deep learning–based data anomaly detection method for structural health monitoring. Struct Health Monit2019; 18(2): 401–421.
32.
WangBMaoZ. Outlier detection based on Gaussian process with application to industrial processes. Appl Soft Comput2019; 76: 505–516.
33.
DervilisNWordenKCrossE. On robust regression analysis as a means of exploring environmental and operational conditions for SHM data. J Sound Vib2015; 347: 279–296.
34.
FerdowsiHJagannathanSZawodniokM. An online outlier identification and removal scheme for improving fault detection performance. IEEE Trans Neural Netw Learn Syst2013; 25(5): 908–919.
35.
DaiJSongHShengG, et al. Cleaning method for status monitoring data of power equipment based on stacked denoising autoencoders. IEEE Access2017; 5: 22863–22870.
36.
LiHMedinaDVilà-VallsJ, et al. Robust variational-based kalman filter for outlier rejection with correlated measurements. IEEE Trans Signal Process2020; 69: 357–369.
37.
JiaXJinCBuzzaM, et al. Wind turbine performance degradation assessment based on a novel similarity metric for machine performance curves. Renew Energy2016; 99: 1191–1201.
38.
TangBHeH. A local density-based approach for outlier detection. Neurocomputing2017; 241: 171–180.
39.
HeYWuPLiY, et al. A generic energy prediction model of machine tools using deep learning algorithms. Appl Energy2020; 275: 115402.
40.
XuXLeiYLiZ. An incorrect data detection method for big data cleaning of machinery condition monitoring. IEEE Trans Ind Electron2019; 67(3): 2326–2336.
41.
XieQTaoGXieC, et al. Abnormal data detection based on adaptive sliding window and weighted multiscale local outlier factor for machinery health monitoring. IEEE Trans Ind Electron2022; 70(11): 11725–11734.
42.
LiCGuoLGaoH, et al. Similarity-measured isolation forest: anomaly detection method for machine monitoring data. IEEE Trans Instrum Meas2021; 70: 1–12.
43.
HuYXiYPanC, et al. Daily condition monitoring of grid-connected wind turbine via high fidelity power curve and its comprehensive rating. Renew Energy2020; 146: 2095–2111.
44.
ReyesOVenturaS. Evolutionary strategy to perform batch-mode active learning on multi-label data. ACM Trans Intell Syst Technol2018; 9(4): 1–26.
45.
SarrJMABrochierTBrehmerP, et al. Complex data labeling with deep learning methods: lessons from fisheries acoustics. ISA Trans2021; 109: 113–125.
46.
KramerO. K-nearest neighbors. In: Dimensionality reduction with unsupervised nearest neighbors. Berlin, Heidelberg: Springer Berlin Heidelberg, 2013, pp. 13–23.
47.
ZhangZ. Introduction to machine learning: k-nearest neighbors. Ann Transl Med2016; 4(11): 218.
48.
AgrawalR. K-nearest neighbor for uncertain data. Int J Comput Appl2014; 105(11): 13–16.
49.
ZhuQFengJHuangJ. Natural neighbor: a self-adaptive neighborhood method without parameter k. Pattern Recognit Lett2016; 80: 30–36.
50.
WangTLuGLiuJ, et al. Graph-based change detection for condition monitoring of rotating machines: techniques for graph similarity. IEEE Trans Reliab2018; 68(3): 1034–1049.
51.
BreunigMMKriegelHPNgRT, et al. LOF: identifying density-based local outliers. In: Proceedings of the 2000 ACM SIGMOD international conference on Management of data and symposium on principles of database systems, Texas, USA, 15–18 May 2000, pp. 93–104. New York: Association for Computing Machinery.
