To address weak fault signatures in rolling bearing vibration signals under strong noise conditions, coupled fault mode interference, and limitations of existing methods in modal representation and feature fusion, a deep learning-based time-, frequency-, and time–frequency representation (TFT) multi-modal fusion framework for fault diagnosis with an attention mechanism is proposed. Specifically, three complementary modalities are constructed, including the time-domain, frequency-domain via fast Fourier transform (FFT), and time–frequency-domain via Multi-scale wavelet convolution (MWC), to comprehensively characterize fault-related information. A parallel convolutional neural network–long short-term memory (CNN–LSTM) architecture is employed to extract spatial and temporal features from each modality. Subsequently, a multi-head attention mechanism with learnable modal weights is introduced to adaptively fuse multi-modal features while enhancing discriminative information and suppressing redundant representations. Experimental results on the HUST bearing dataset demonstrate that the proposed framework achieves 100% classification accuracy for both single and compound faults, outperforming all baseline methods. Under varying signal-to-noise ratio (SNR) conditions, the model maintains consistently stable performance, exhibiting strong noise robustness and excellent generalization capability. Furthermore, t-distributed stochastic neighbor embedding (T-SNE) visualization and cross-dataset validation further confirm the effectiveness and separability of the learned feature representations.
PengBBiYXueB, et al. A survey on fault diagnosis of rolling bearings. Algorithms2022; 15: 347.
2.
ChenZZhangQZhangJ, et al. A knowledge-based fault diagnosis method for rolling bearings without fault sample training. Proc Inst Mech Eng C J Mech Eng Sci2024; 238(20): 10253–10265.
3.
DengWLiZLiX, et al. Compound fault diagnosis using optimized MCKD and sparse representation for rolling bearings. IEEE Trans Instrum Meas2022; 71: 1–9.
4.
FanHChenBCaoX, et al. An intelligent multi-element fault diagnosis method of rolling bearings considering damage degrees and sensor abnormity under small samples. Proc Inst Mech Eng C J Mech Eng Sci2025; 239(7): 2334–2356.
5.
WangXLiJJingZ, et al. Fault diagnosis method of rolling bearing based on SSA-VMD and RCMDE. Sci Rep2024; 14(1): 30637.
6.
WangYLiangJGuX, et al. Multi-scale attention mechanism residual neural network for fault diagnosis of rolling bearings. Proc Inst Mech Eng C J Mech Eng Sci2022; 236(20): 10615–10629.
7.
GangLGongYZhangH, et al. Infrared spectral noise reduction algorithm based on wavelet transform optimized EEMD combined with SG. Infrared Technol2024; 46(12): 1453–1458.
8.
HanTZhangLYinZ, et al. Rolling bearing fault diagnosis with combined convolutional neural networks and support vector machine. Measurement2021; 177: 109022.
9.
ZhengJPanHChengJ. Rolling bearing fault detection and diagnosis based on composite multiscale fuzzy entropy and ensemble support vector machines. Mech Syst Signal Process2017; 85: 746–759.
10.
CuiKLiuMMengY. A new fault diagnosis of rolling bearing on FFT image coding and L-CNN. Meas Sci Technol2024; 35(7): 076108.
11.
PangBTangGTianT. Enhanced singular spectrum decomposition and its application to rolling bearing fault diagnosis. IEEE Access2019; 7: 87769–87782.
12.
YuanXZhangHLiuH. A novel fault diagnosis approach for rolling bearing based on CWT and adaptive sparse representation. Shock Vib2022; 2022: 1–10.
13.
YingWZhengJPanH, et al. Use of Holo-Hilbert spectral analysis to reveal the amplitude modulation features of faulty rolling bearing signals. J Vib Control2023; 29(21-22): 5067–5077.
14.
RaiAUpadhyaySH. A review on signal processing techniques utilized in the fault diagnosis of rolling element bearings. Tribol Int2016; 96: 289–306.
15.
LiXMaZYuanZ, et al. A review on convolutional neural network in rolling bearing fault diagnosis. Meas Sci Technol2024; 35(7): 072002.
16.
TaoHQiuJChenY, et al. Unsupervised cross-domain rolling bearing fault diagnosis based on time-frequency information fusion. J Franklin Inst2023; 360(2): 1454–1477.
17.
SinitsinVIbryaevaOSakovskayaV, et al. Intelligent bearing fault diagnosis method combining mixed input and hybrid CNN-MLP model. Mech Syst Signal Process2022; 180: 109454.
18.
ZhouZWangHLiZ, et al. Fault diagnosis of rolling bearing based on deep convolutional neural network and gated recurrent unit. J Adv Mech Des Syst Manuf2023; 17(2): JAMDSM0017.
19.
QiaoMYanSTangX, et al. Deep convolutional and LSTM recurrent neural networks for rolling bearing fault diagnosis under strong noises and variable loads. IEEE Access2020; 8: 66257–66269.
20.
DiaoNWangZMaH, et al. Fault diagnosis of rolling bearing under variable working conditions based on CWT and T-ResNet. J Vib Eng Technol2023; 11(8): 3747–3757.
21.
ZhouYLongXSunM, et al. Bearing fault diagnosis based on Gramian angular field and DenseNet. Math Biosci Eng2022; 19(12): 14086–14101.
22.
MengDWangHYangS, et al. Fault analysis of wind power rolling bearing based on EMD feature extraction. Comput Model Eng Sci2022; 130(1): 543–558.
23.
ZhouJXiaoMNiuY, et al. Rolling bearing fault diagnosis based on WGWOA-VMD-SVM. Sensors2022; 22(16): 6281.
24.
LiuBYanCHeC, et al. An interpretable physics-informed subdomain moment-enhanced adaptation network for unsupervised transfer fault diagnosis of rolling bearing. Adv Eng Inform2025; 67: 103491.
25.
LiuBYanCLiuY, et al. ISEANet: an interpretable subdomain enhanced adaptive network for unsupervised cross-domain fault diagnosis of rolling bearing. Adv Eng Inform2024; 62: 102610.
26.
YouKWangPGuY. Toward efficient and interpretative rolling bearing fault diagnosis via quadratic neural network with Bi-LSTM. IEEE Internet Things J2024; 11(13): 23002–23019.
27.
YouKGuYLinY, et al. A novel physical constraint-guided quadratic neural networks for interpretable bearing fault diagnosis under zero-fault sample. Nondestructive Test Eval. Epub ahead of print 22 July 2025. DOI: 10.1080/10589759.2025.2534429
28.
HuDHouXWeiL, et al. MM-DFN: multimodal dynamic fusion network for emotion recognition in conversations. In: ICASSP 2022, 2022, pp.7037–7041.
29.
ChittaKPrakashAJaegerB, et al. Transfuser: imitation with transformer-based sensor fusion for autonomous driving. IEEE Trans Pattern Anal Mach Intell2023; 45(11): 12878–12895.
LipkovaJChenRJChenB, et al. Artificial intelligence for multimodal data integration in oncology. Cancer Cell2022; 40(10): 1095–1110.
32.
CheCWangHNiX, et al. Hybrid multimodal fusion with deep learning for rolling bearing fault diagnosis. Measurement2021; 173: 108655.
33.
PengCShengYGuiW, et al. A rolling bearing fault diagnosis method based on multimodal knowledge graph. IEEE Trans Ind Inform2024; 20(11): 13047–13057.
34.
ChangHZhangXLongY, et al. WCNN-RSN: a novel fault diagnosis method for rolling bearing using multimodal feature fusion. Meas Sci Technol2024; 35(12): 126145.
35.
FangXZhengJJiangB. A rolling bearing fault diagnosis method based on vibro-acoustic data fusion and fast Fourier transform (FFT). Int J Data Sci Anal2025; 20(3): 2377–2386.
36.
YouKGuYShaoH, et al. A liquid-impulse neural network model based on heterogeneous fusion of multimodal information for interpretable rotating machinery fault diagnosis. Mech Syst Signal Process2026; 246: 113923.
37.
YouKLianZGuY. A performance-interpretable intelligent fusion of sound and vibration signals for bearing fault diagnosis via dynamic CAME. Nonlinear Dyn2024; 112(23): 20903–20940.
38.
ZhaoCZioEShenW. Domain generalization for cross-domain fault diagnosis: an application-oriented perspective and a benchmark study. Reliab Eng Syst Saf2024; 245: 109964.
39.
WangHLiuZPengD, et al. Interpretable convolutional neural network with multilayer wavelet for noise-robust machinery fault diagnosis. Mech Syst Signal Process2023; 195: 110314.
40.
YanSShaoHWangJ, et al. LiConvFormer: a lightweight fault diagnosis framework using separable multiscale convolution and broadcast self-attention. Expert Syst Appl2024; 237: 121338.
