The intelligent fault diagnosis (IFD) method based on incremental learning (IL) can expand new fault categories without retraining the model, making it a research hotspot in the field of fault diagnosis. Currently, the combination of knowledge distillation (KD) and replay techniques has been widely used to alleviate catastrophic forgetting in IL. However, this method still has some limitations: first, the difference in data distribution of different incremental tasks may cause concept drift, hindering the model’s adaptation to new tasks; second, replay techniques may lead to an imbalance in the number of samples between new and old classes due to the limited storage of exemplar library, resulting in classifier learning bias. To address these limitations, this article proposes an incremental IFD method (IIFD-DDRF) based on dual-teacher knowledge distillation (DTKD) and dynamic residual fusion (DRF) (IIFD-DDRF). First, a DTKD strategy is proposed, which transmits new and old knowledge through two teacher models, helping the student model better adapt to new tasks while retaining old knowledge. Second, a DRF method is proposed to handle dynamic data imbalance. This method incorporates lightweight branch layers specific to the task, encoding old task knowledge and performing residual fusion to optimize classifier output. Additionally, a dynamic branch layer merging mechanism is adopted to effectively prevent excessive growth of the model. Finally, the effectiveness and advancement of this method are validated on three datasets: bearings and gearboxes.
GuoWLiXShenZ.A lightweight residual network based on improved knowledge transfer and quantized distillation for cross-domain fault diagnosis of rolling bearings. Expert Syst Appl2024; 245: 123083.
2.
LiuDCuiLWangG, et al. Interpretable domain adaptation transformer: a transfer learning method for fault diagnosis of rotating machinery. Struct Health Monit2024; 14759217241249656. doi:10.1177/14759217241249656
3.
LiuFShangZGaoM, et al. Bearing failure diagnosis at time-varying speed based on adaptive clustered fractional Gabor transform. Meas Sci Technol2023; 34: 095002.
4.
LiWShangZGaoM, et al. Intelligent fault diagnosis of partial deep transfer based on multi-representation structural intraclass compact and double-aligned domain adaptation. Mech Syst Signal Process2023; 197: 110412.
5.
HuangFZhangKLiZ, et al. A rolling bearing fault diagnosis method based on interactive generative feature space oversampling-based autoencoder under imbalanced data. Struct Health Monit2024; 14759217241248209.
6.
ShangZZhangJLiW, et al. A novel small samples fault diagnosis method based on the self-attention Wasserstein generative adversarial network. Neural Process Lett2023; 55: 6377–6407.
7.
ShaoHLiWCaiB, et al. Dual-threshold attention-guided GAN and limited infrared thermal images for rotating machinery fault diagnosis under speed fluctuation. IEEE Trans Ind Inf2023; 19: 9933–9942.
8.
ZhuJHuTJiangB, et al. Intelligent bearing fault diagnosis using PCA–DBN framework. Neural Comput Appl2020; 32: 10773–10781.
9.
WangJGuoSHuangR, et al. Dual-channel capsule generation adversarial network for hyperspectral image classification. IEEE Trans Geosci Remote Sens2022; 60: 1–16.
10.
LiWLanHChenJ, et al. WavCapsNet: an interpretable intelligent compound fault diagnosis method by backward tracking. IEEE Trans Instrum Meas2023; 72: 1–11.
11.
ChenJWenKXiaJ, et al. Knowledge embedded autoencoder network for harmonic drive fault diagnosis under few-shot industrial scenarios. IEEE Internet Things J2024; 11: 22915–22925.
12.
LiWHuangRLiJ, et al. A perspective survey on deep transfer learning for fault diagnosis in industrial scenarios: theories, applications and challenges. Mech Syst Signal Process2022; 167: 108487.
13.
ChenJHuangRChenZ, et al. Transfer learning algorithms for bearing remaining useful life prediction: a comprehensive review from an industrial application perspective. Mech Syst Signal Process2023; 193: 110239.
14.
ParisiGIKemkerRPartJL, et al. Continual lifelong learning with neural networks: a review. Neural Netw2019; 113: 54–71.
15.
FuYCaoHChenX, et al. Broad auto-encoder for machinery intelligent fault diagnosis with incremental fault samples and fault modes. Mech Syst Signal Process2022; 178: 109353.
16.
ZhangSWangRWangL, et al. Fault diagnosis for power converters based on incremental learning. IEEE Trans Instrum Meas2023; 72: 1–13.
17.
FuYCaoHChenX, et al. Improved broad learning system for machinery intelligent fault diagnosis with increasing fault samples, fault modes, and running conditions. ISA Trans2023; 136: 400–416.
18.
KirkpatrickJPascanuRRabinowitzN, et al. Overcoming catastrophic forgetting in neural networks. Proc Natl Acad Sci USA2017; 114: 3521–3526.
19.
LiaoJ-XWeiS-LXieC-L, et al. BearingPGA-Net: a lightweight and deployable bearing fault diagnosis network via decoupled knowledge distillation and FPGA acceleration. IEEE Trans Instrum Meas2024; 73: 1–14.
20.
ZhengJXiongHZhangY, et al. Bearing fault diagnosis via incremental learning based on the repeated replay using memory indexing (R-REMIND) method. Machines2022; 10: 338.
21.
LiuYChenBWangD, et al. A lifelong learning method based on generative feature replay for bearing diagnosis with incremental fault types. IEEE Trans Instrum Meas2023; 72: 1–11.
22.
ZhuJWangYHuangC-G, et al. A new incremental learning for bearing fault diagnosis under noisy conditions using classification and feature-level information. IEEE Trans Instrum Meas2024; 73: 1–14.
23.
HuKHeQChengC, et al. Adaptive incremental diagnosis model for intelligent fault diagnosis with dynamic weight correction. Reliab Eng Syst Saf2024; 241: 109705.
24.
MaDZhangKCaoQ, et al. Coordinate attention guided dual-teacher adaptive knowledge distillation for image classification. Expert Syst Appl2024; 250: 123892.
25.
LiZLiXYangL, et al. Dual teachers for self-knowledge distillation. Pattern Recognit2024; 151: 110422.
26.
YangYLongYLinY, et al. Two-stage edge-side fault diagnosis method based on double knowledge distillation. Comput Mater Con2023; 76: 3623–3651.
27.
ZhongHYuSTrinhH, et al. A novel small-sample dense teacher assistant knowledge distillation method for bearing fault diagnosis. IEEE Sensors J2023; 23: 24279–24291.
28.
DingAYiXQinY, et al. Self-driven continual learning for class-added motor fault diagnosis based on unseen fault detector and propensity distillation. Eng Appl Artif Intell2024; 127: 107382.
29.
LiZHoiemD.Learning without forgetting. IEEE Trans Pattern Anal Mach Intell2018; 40: 2935–2947.
30.
CuiJLiuSTianZ, et al. ResLT: residual learning for long-tailed recognition. IEEE Trans Pattern Anal Mach Intell2022; 45: 3695–3706.
31.
PanCShangZLiW, et al. Bearing fault diagnosis based on high-confidence pseudo-labels and dual-view multi-adversarial sparse joint attention network under variable working conditions. Eng Appl Artif Intell2024; 133: 108625.
32.
WangF-YZhouD-WYeH-J, et al. FOSTER: feature boosting and compression for class-incremental learning. In: AvidanSBrostowGCisséM, et al. (eds) Computer vision – ECCV. Cham: Springer Nature Switzerland, 2022, pp. 398–414.
33.
YanSXieJHeX. DER: dynamically expandable representation for class incremental learning. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 3013–3022. Nashville, TN: IEEE.
34.
WuYChenYWangL, et al. Large Scale Incremental Learning. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 374–382. Long Beach, CA: IEEE.
35.
ZhouD-WWangQ-WYeH-J, et al. A model or 603 exemplars: towards memory-efficient class-incremental learning, 2022. arXiv:abs/2205.13218. DOI: 10.48550/arXiv.2205.13218.
36.
ZhouD-WCaiZ-WYeH-J, et al. Revisiting class-incremental learning with pre-trained models: generalizability and adaptivity are all you need. Int J Comput Vis. Epub ahead of print 31August2024. DOI: 10.1007/s11263-024-02218-0.
37.
DengKChengGYangR, et al. Alleviating ASR long-tailed problem by decoupling the learning of representation and classification. IEEE/ACM Trans Audio Speech Lang Process2022; 30: 340–354.
38.
ZhaoBXiaoXGanG, et al. Maintaining discrimination and fairness in class incremental learning. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 13205–13214. Seattle, WA: IEEE.
