This paper proposes a novel multimodal fault diagnosis method for rotating machinery based on a dual-branch 2D-ResNet-CBAM and 1D-GRU architecture. It simultaneously processes shaft trajectory images and vibration signals to leverage complementary spatial and temporal features effectively. The ResNetCBAM-GRU model was rigorously validated. Experimental results demonstrate superior performance, achieving 98.09% accuracy, 97.08% precision, 96.97% recall, and 97.86% F1-score, all surpassing existing methods. Ablation studies further confirmed the contributions: the dual-branch design achieved a 0.48% accuracy gain compared to the single 2D-ResNet-CBAM branch. This method provides robust and accurate support for rotating machinery fault diagnosis.
AasiATabatabaeiRAasiE, et al. (2022) Experimental investigation on time-domain features in the diagnosis of rolling element bearings by acoustic emission. Journal of Vibration and Control28(19–20): 2585–2595.
2.
ChengHKongXWangQ, et al. (2023) Deep transfer learning based on dynamic domain adaptation for remaining useful life prediction under different working conditions. Journal of Intelligent Manufacturing34(2): 587–613.
3.
DuXJiaWYuP, et al. (2022) A remaining useful life prediction method based on Time-frequency images of the mechanical vibration signals. Measurement202: 111782.
4.
GuptaMWadhvaniRRasoolA (2023) A Real-time adaptive model for bearing fault classification and remaining useful life estimation using deep neural network. Knowledge-Based Systems259: 110070.
5.
LiSYanQLiuP (2020) An efficient fire detection method based on multiscale feature extraction, implicit deep supervision and channel attention mechanism. IEEE Transactions on Image Processing29: 8467–8475.
6.
LiuSFanL (2022) An adaptive prediction approach for rolling bearing remaining useful life based on multistage model with three-source variability. Reliability Engineering & System Safety218: 108182.
7.
LuWMaoHLinF, et al. (2022) Recognition of rolling bearing running state based on genetic algorithm and convolutional neural network. Advances in Mechanical Engineering14(4): 1–11.
8.
Rohani BastamiAAasiAArghandHA (2019) Estimation of remaining useful life of rolling element bearings using wavelet packet decomposition and artificial neural network. Iranian Journal of Science and Technology, Transactions of Electrical Engineering43(1): 233–245.
9.
ShenYTangBLiB, et al. (2022) Remaining useful life prediction of rolling bearing based on multi-head attention embedded Bi-LSTM network. Measurement202: 111803.
10.
TaoJLiuYYangD, et al. (2015) Fault diagnosis of rolling bearing using deep belief networks. In: 2015 International Symposium on Material, Energy and Environment Engineering, 29 November. Amsterdam: Atlantis Press, pp. 566–569.
11.
WangYLiSLiuC, et al. (2024) Multiscale feature fusion and Semi-supervised Temporal-spatial learning for performance monitoring in the flotation industrial process. IEEE Transactions on Cybernetics54(2): 974–987.
12.
WenLLiXGaoL, et al. (2017) A new convolutional neural network-based data-driven fault diagnosis method. IEEE Transactions on Industrial Electronics65(7): 5990–5998.
13.
XuJXieHLiuC, et al. (2021) Hip landmark detection with dependency mining in ultrasound image. IEEE Transactions on Medical Imaging40(12): 3762–3774.
14.
XuQFanZJiaW, et al. (2019) Quantile regression neural network-based fault detection scheme for wind turbines with application to monitoring a bearing. Wind Energy22(10): 1390–1401.
15.
YinCWangYMaG, et al. (2022) Weak fault feature extraction of rolling bearings based on improved ensemble noise-reconstructed EMD and adaptive threshold denoising. Mechanical Systems and Signal Processing171: 108834.
16.
ZengFLiYJiangY, et al. (2021) An online transfer Learning-based remaining useful life prediction method of ball bearings. Measurement176(1): 109201.
17.
ZhuJChenNPengW (2018) Estimation of bearing remaining useful life based on multiscale convolutional neural network. IEEE Transactions on Industrial Electronics66(4): 3208–3216.
18.
ZossoDDragomiretskiyKBertozziAL, et al. (2017) Two-dimensional compact variational mode decomposition: Spatially compact and spectrally sparse image decomposition and segmentation. Journal of Mathematical Imaging and Vision58(2): 294–320.
