As critical components of a machine, bearings are fundamental to maintaining system reliability, which emphasizes the importance of intelligent fault diagnosis. In industrial applications, bearing failures are episodic. As a result, a large amount of samples collected is normal, and the extreme imbalance of samples poses a challenge for bearing fault diagnosis (BFD) in industrial applications. In addition, industrial applications impose higher requirements for model lightweight due to the limitations in computational capability of the device. To address these challenges, a novel knowledge distillation framework with improved activation function and adaptive feature selection strategy was proposed for imbalanced BFD. Firstly, an enhanced activation function was introduced to improve the model’s nonlinear feature extraction capabilities, thereby boosting its performance when handling imbalanced datasets. Secondly, an adaptive feature selection strategy was proposed to ensure the extraction of sufficient information, thereby enhancing the model’s generalization capability across diverse devices. Finally, a novel knowledge distillation framework was proposed to downsize the model, which makes the model lightweight to meet the requirements of industrial applications. The effectiveness of the proposed methodology was experimentally validated on two imbalanced bearing datasets. The experiments demonstrate that the proposed methodology outperforms existing models in diagnostic performance.
ZhouKYangCLiuJ, et al. Deep graph feature learning-based diagnosis approach for rotating machinery using multi-sensor data. J Intell Manuf2023; 34(4): 1965–1974.
2.
ChenBSongDZhangW, et al. A novel spectral coherence-based envelope spectrum for railway axle-box bearing damage identification. Struct Health Monit2023; 22(2): 879–896.
3.
ChengYWangSChenB, et al. An improved envelope spectrum via candidate fault frequency optimization-gram for bearing fault diagnosis. J Sound Vib2022; 523: 116746.
4.
VerstraeteDBDroguettELMeruaneV, et al. Deep semi-supervised generative adversarial fault diagnostics of rolling element bearings. Struct Health Monit2020; 19(2): 390–411.
5.
ChengYZhouNWangZ, et al. Cffsbd: a candidate fault frequencies-based blind deconvolution for rolling element bearings fault feature enhancement. IEEE Trans Instrum Meas2023; 72: 1–12.
6.
ZhaoZSunSXuW, et al. Dynamics modeling and fault diagnosis of flexible thin-walled elliptical bearings in harmonic reducers. Measurement2024; 238: 115378.
7.
LeiYYangBJiangX, et al. Applications of machine learning to machine fault diagnosis: a review and roadmap. Mech Syst Signal Process2020; 138: 106587.
8.
YangCQiaoZZhuR, et al. An intelligent fault diagnosis method enhanced by noise injection for machinery. IEEE Trans Instrum Meas2023; 72: 1–11.
9.
ChengYYanJKZhangF, et al. Surrogate modeling of pantograph-catenary system interactions. Mech Syst Signal Process2025; 224: 112134.
10.
HeDZhaoJJinZ, et al. Dcaggcn: a novel method for remaining useful life prediction of bearings. Reliab Eng Syst Saf2025; 260: 110978.
11.
ZhaoJHeDJinZ, et al. A new method for bearing remaining useful life prediction based on dynamic wavelet and physical information constraints. Expert Syst Appl2025; 296: 129023.
12.
YanJChengYZhangF, et al. Multi-modal imitation learning for arc detection in complex railway environments. IEEE Trans Instrum Meas2025; 74: 3529413.
13.
YanJChengYWangQ, et al. Transformer and graph convolution-based unsupervised detection of machine anomalous sound under domain shifts. IEEE Trans Emerg Top Comput Intell2024; 8: 2827–2842.
14.
YangCQiaoZLiuL, et al. Positive-incentive noise in artificial intelligence-enabled machine fault diagnosis. Struct Health Monit. Epub ahead of print 2025.
15.
LiuLZhangWGuF, et al. An unsupervised transfer network with adaptive input and dynamic channel pruning for train axle bearing fault diagnosis. Struct Health Monit2024; 24(5): 2837–2856.
16.
ZhaoJWangWHuangJ, et al. A comprehensive review of deep learning-based fault diagnosis approaches for rolling bearings: advancements and challenges. AIP Adv2025; 15(2): 020702.
17.
WuCZhaoDHanT, et al. Bearing fault diagnosis using gradual conditional domain adversarial network. Appl Soft Comput2024; 158: 111580.
18.
ChenPZhaoRHeT, et al. A novel bearing fault diagnosis method based joint attention adversarial domain adaptation. Reliab Eng Syst Saf2023; 237: 109345.
19.
ChangYChenJWuW, et al. Intelligent fault quantitative identification for industrial internet of things (IIoT) via a novel deep dual reinforcement learning model accompanied with insufficient samples. IEEE Internet Things J2022; 9(20): 19811–19822.
20.
ZhangXLiuJZhangX, et al. Self-supervised graph feature enhancement and scale attention for mechanical signal node-level representation and diagnosis. Adv Eng Inform2025; 65: 103197.
21.
LiangPDengCYuanX, et al. A deep capsule neural network with data augmentation generative adversarial networks for single and simultaneous fault diagnosis of wind turbine gearbox. ISA Trans2023; 135: 462–475.
22.
JavanmardiAHüllermeierE.Conformal prediction intervals for remaining useful lifetime estimation. International Journal of Prognostics and Health Management2022; 14(2): 1–13.
23.
ZhaoMKangMTangB.Contrastive learning-based domain-invariant feature extraction for intelligent fault diagnosis. Mech Syst Signal Process2022; 162: 108018.
24.
LiXZhangWDingQ.Deep domain adaptation for fault diagnosis under different working conditions. IEEE Trans Ind Inform2020; 16(12): 7619–7629.
25.
WenLLiXGaoL.A new deep transfer learning based on sparse auto-encoder for fault diagnosis. IEEE Trans Ind Electron2019; 66(5): 3867–3875.
26.
BaiSKolterJZKoltunV.An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv:1803.01271, 2018.
27.
WuTHeSLiuJ, et al. A brief overview of chatgpt: Tthe history, status quo and potential future development. IEEE/CAA J Autom Sinica2023; 10(5): 1122–1136.
28.
HowardAGZhuMChenB, et al. Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861, 2017.
29.
ZhangXZhouXLinM, et al. Shufflenet: an extremely efficient convolutional neural network for mobile devices. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Utah, USA, 18–22 June, 2018, pp. 6848–6856. Los Alamitos: IEEE.
30.
HouYLiTWangJ, et al. A lightweight transformer based on feature fusion and global-local parallel stacked self-activation unit for bearing fault diagnosis. Measurement2024; 236: 115068.
31.
WangSZhangHXuK, et al. Beta-crown: Efficient bound propagation with per-neuron split constraints for neural network robustness verification. Adv Neural Inf Process Syst2021; 34: 29909–29921.
32.
QinHGongRLiuX, et al. Binary neural networks: a survey. Pattern Recognit2020; 105: 107281.
33.
YimJJooDBaeJ, et al. A gift from knowledge distillation: fast optimization, network minimization and transfer learning. In: Proceedings of the IEEE conference on computer vision and pattern recognitionHonolulu, HI, USA, 21–26 July 2017, pp. 4133–4141. New York: IEEE.
34.
JiMPengGLiS, et al. A neural network compression method based on knowledge-distillation and parameter quantization for the bearing fault diagnosis. Appl Soft Comput2022; 127: 109331.
35.
ChangYLiFChenJ, et al. Efficient temporal flow transformer accompanied with multi-head probsparse self-attention mechanism for remaining useful life prognostics. Reliab Eng Syst Saf2022; 226: 108701.
36.
ZhangXHuYYinA, et al. Inferable deep distilled attention network for diagnosing multiple motor bearing faults. IEEE Trans Transp Electrif2022; 9: 2207–2216.
37.
HuKHeQChengC, et al. Adaptive incremental diagnosis model for intelligent fault diagnosis with dynamic weight correction. Reliab Eng Syst Saf2024; 241: 109705.
38.
WangCWangHLiuM.A dual-attention feature fusion network for imbalanced fault diagnosis with two-stream hybrid generated data. J Intell Manuf2023; 35: 1707–1719.
39.
ChenYYangXDaiH.Cost-sensitive continuous ensemble kernel learning for imbalanced data streams with concept drift. Knowl Based Syst2023; 284: 111272.
40.
XuHPanHZhengJ, et al. Intelligent fault identification in sample imbalance scenarios using robust low-rank matrix classifier with fuzzy weighting factor. Appl Soft Comput2024; 152: 111229.
41.
HanHWangW-YMaoB-H.Borderline-smote: a new over-sampling method in imbalanced data sets learning. In: International conference on intelligent computing, Hefei, China, 23–26 August 2005, pp. 878–887. Berlin, Heidelberg: Springer.
42.
BunkhumpornpatCSinapiromsaranKLursinsapC. Safe-level-smote: safe-level-synthetic minority over-sampling technique for handling the class imbalanced problem. In: Advances in knowledge discovery and data mining: 13th Pacific-Asia conference, Bangkok, Thailand, April 27–30, 2009, pp. 475–482. Springer.
43.
MathewJPangCKLuoM, et al. Classification of imbalanced data by oversampling in kernel space of support vector machines. IEEE Trans Neural Netw Learn Syst2017; 29(9): 4065–4076.
44.
LiuSJiangHWuZ, et al. Rolling bearing fault diagnosis using variational autoencoding generative adversarial networks with deep regret analysis. Measurement2021; 168: 108371.
45.
ZhouFYangSFujitaH, et al. Deep learning fault diagnosis method based on global optimization gan for unbalanced data. Knowl Based Syst2020; 187: 104837.
46.
ShaoSWangPYanR.Generative adversarial networks for data augmentation in machine fault diagnosis. Comput Ind2019; 106: 85–93.
47.
ZhaoBYuanQ.Improved generative adversarial network for vibration-based fault diagnosis with imbalanced data. Measurement2021; 169: 108522.
48.
WangCXinCXuZ.A novel deep metric learning model for imbalanced fault diagnosis and toward open-set classification. Knowl Based Syst2021; 220: 106925.
HeKZhangXRenS, et al. Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, Santiago, Chile, 7–13 December 2015, pp. 1026–1034. New York: IEEE.
51.
ClevertD-AUnterthinerTHochreiterS.Fast and accurate deep network learning by exponential linear units (elus). arXiv:1511.07289, 2015.
52.
ShaoHJiangHZhaoH, et al. A novel deep autoencoder feature learning method for rotating machinery fault diagnosis. Mech Syst Signal Process2017; 95: 187–204.
53.
JiaFLeiYGLuN, et al. Deep normalized convolutional neural network for imbalanced fault classification of machinery and its understanding via visualization. Mech Syst Signal Process2018; 110: 349–367.
54.
HanHWangHLiuZ, et al. Intelligent vibration signal denoising method based on non-local fully convolutional neural network for rolling bearings. ISA Trans2022; 122: 13–23.
55.
WenLLiXYGaoL, et al. A new convolutional neural network-based data-driven fault diagnosis method. IEEE Trans Ind Electron2018; 65(7): 5990–5998.
56.
LiuJZhangCJiangX.Imbalanced fault diagnosis of rolling bearing using improved MSR-GAN and feature enhancement-driven capsnet. Mech Syst Signal Process2022; 168: 108664.
57.
PanTChenJXieJ, et al. Deep feature generating network: a new method for intelligent fault detection of mechanical systems under class imbalance. IEEE Trans Ind Inform2020; 17(9): 6282–6293.
58.
LiuSJiangHWuZ, et al. Data synthesis using deep feature enhanced generative adversarial networks for rolling bearing imbalanced fault diagnosis. Mech Syst Signal Process2022; 163: 108139.
59.
YuYGuoLGaoH, et al. Pareto-optimal adaptive loss residual shrinkage network for imbalanced fault diagnostics of machines. IEEE Trans Ind Inform2021; 18(4): 2233–2243.
60.
GuoLLeiYXingS, et al. Deep convolutional transfer learning network: a new method for intelligent fault diagnosis of machines with unlabeled data. IEEE Trans Ind Electron2018; 66(9): 7316–7325.
61.
StantonSIzmailovPKirichenkoP, et al. Does knowledge distillation really work?Adv Neural Inform Process Syst2021; 34: 6906–6919.
