Intelligent fault diagnosis based on multisensor data fusion techniques and two-dimensional convolutional neural network (CNN) has been widely developed and achieved numerous excellent results. Existing studies usually develop multi-input models to facilitate data fusion, lacking schemes for realizing the fusion in the process of data-to-image. Traditional methods that convert signals to grayscale maps and concatenate them into RGB images lose temporal correlation and are susceptible to interference. Besides, few studies integrated the favorable features at different stages in the CNN for diagnosis, which limits the diagnostic performance. To this end, this article proposes a multisource signal-to-image method called multidimensional distance matrix (MDM) and a multi-scale adaptive feature fusion convolutional neural network (MAFFCNN). First, MDM images emphasize the interrelationships between points of different moments in time series data and preserve temporal correlation. Then, the conv block of MAFFCNN can extract features at different scales in the image, and its attention branch can better aggregate the location information. Also, the MAFFCNN introduces efficient attention and cross-spatial learning to generate learnable weights based on the importance of features at different stages to achieve adaptive feature fusion. Finally, the above method is validated using the established gear fault dataset and the public bearing dataset. The experimental results demonstrate the effectiveness of the proposed methods and their excellent robustness in complex environments.
ZhangYJiJRenZ, et al. Digital twin-driven partial domain adaptation network for intelligent fault diagnosis of rolling bearing. Reliab Eng Syst Saf2023; 234: 109186.
2.
FengKJiJCNiQ, et al. A review of vibration-based gear wear monitoring and prediction techniques. Mech Syst Signal Proc2023; 182: 109605.
3.
ZhangYYuKRenZ, et al. Joint domain alignment and class alignment method for cross-domain fault diagnosis of rotating machinery. IEEE Trans Instrum Meas2021; 70: 1–12.
4.
GuH-HWangR-ZTangM-J, et al. Data-physics-model based fatigue reliability assessment methodology for high-temperature components and its application in steam turbine rotor. Reliab Eng Syst Saf2023; 241: 109633.
5.
XuXBaoSShaoH, et al. A multi-sensor fused incremental broad learning with D-S theory for online fault diagnosis of rotating machinery. Adv Eng Inform2024; 60: 102419.
6.
ZhangXZhangXLiuJ, et al. Graph features dynamic fusion learning driven by multi-head attention for large rotating machinery fault diagnosis with multi-sensor data. Eng Appl Artif Intell2023; 125: 106601.
7.
ZhangXShengCOuyangW, et al. Fault diagnosis of marine electric thruster bearing based on fusing multi-sensor deep learning models. Measurement2023; 214: 112727.
8.
XuYFengKYanX, et al. CFCNN: a novel convolutional fusion framework for collaborative fault identification of rotating machinery. Inform Fusion2023; 95: 1–16.
9.
RajabiSAzariMSSantiniS, et al. Fault diagnosis in industrial rotating equipment based on permutation entropy, signal processing and multi-output neuro-fuzzy classifier. Expert Syst Appl2022; 206: 117754.
10.
LinYXiaoMLiuH, et al. Gear fault diagnosis based on CS-improved variational mode decomposition and probabilistic neural network. Measuremet2022; 192: 110913.
11.
SuZTangBLiuZR, et al. Multi-fault diagnosis for rotating machinery based on orthogonal supervised linear local tangent space alignment and least square support vector machine. Neurocomputing2015; 157: 208–222.
12.
WangRJiangHZhuK, et al. A deep feature enhanced reinforcement learning method for rolling bearing fault diagnosis. Adv Eng Inform2022; 54: 101750.
13.
XuYYanXSunB, et al. Hierarchical multiscale dense networks for intelligent fault diagnosis of electromechanical systems. IEEE Trans Instrum Meas2022; 71: 1–12.
14.
LeiYJiaFLinJ, et al. An intelligent fault diagnosis method using unsupervised feature learning towards mechanical big data. IEEE Trans Indus Electr2016; 63: 3137–3147.
15.
ZhongJYangYMaoH, et al. Contrastive regularization guided label refurbishment for fault diagnosis under label noise. Adv Eng Inform2024; 61: 102478.
16.
WangHLiuZPengD, et al. Interpretable convolutional neural network with multilayer wavelet for noise-robust machinery fault diagnosis. Mech Syst Signal Proc2023; 195: 110314.
17.
ZhangYRenZZhouS, et al. Supervised contrastive learning-based domain adaptation network for intelligent unsupervised fault diagnosis of rolling bearing. IEEE/ASME Trans Mechatr2022; 27(6): 5371–5380.
18.
WangDLiYJiaL, et al. Novel three-stage feature fusion method of multimodal data for bearing fault diagnosis. IEEE Trans Instrum Meas2021; 70: 1–10.
19.
XuYYanXSunB, et al. Multireceptive field denoising residual convolutional networks for fault diagnosis. IEEE Trans Indus Electr2022; 69: 11686–11696.
20.
WangHLiuZPengD, et al. Understanding and learning discriminant features based on multiattention 1dcnn for wheelset bearing fault diagnosis. IEEE Trans Indus Inform2020; 16: 5735–5745.
21.
YuJZhouX.One-dimensional residual convolutional autoencoder based feature learning for gearbox fault diagnosis. IEEE Trans Indus Inform2020; 16(10): 6347–6358.
22.
ZhaoJYangSLiQ, et al. A new bearing fault diagnosis method based on signal-to-image mapping and convolutional neural network. Measurement2021; 176: 109088.
23.
YangJLiuJXieJ, et al. Conditional GAN and 2-D CNN for bearing fault diagnosis with small samples. IEEE Trans Instrum Meas2021; 70: 1–12.
24.
ZhangKWangJShiH, et al. A fault diagnosis method based on improved convolutional neural network for bearings under variable working conditions. Measurement2021; 182: 109749.
25.
YuXWangSXuH, et al. Intelligent fault diagnosis of rotating machinery under variable working conditions based on deep transfer learning with fusion of local and global time-frequency features. Struct Health Monit2023; 23(4): 14759217231199427.
26.
GuYChenRHuangP, et al. A lightweight bearing compound fault diagnosis method with gramangle field and ghost-resnet model. IEEE Trans Reliab2023; 19: 1–14.
27.
XueLLeiCJiaoM, et al. Rolling bearing fault diagnosis method based on self-calibrated coordinate attention mechanism and multi-scale convolutional neural network under small samples. IEEE Sens J2023; 23(9): 10206.
28.
LiXWangYYaoJ, et al. Multi-sensor fusion fault diagnosis method of wind turbine bearing based on adaptive convergent viewable neural networks. Reliab Eng Syst Saf2024; 245: 109980.
29.
ZhangYJiJRenZ, et al. Multi-sensor open-set cross-domain intelligent diagnostics for rotating machinery under variable operating conditions. Mech Syst Signal Proc2023; 191: 110172.
30.
ZhangYRenZFengK, et al. Transformer-enabled cross-domain diagnostics for complex rotating machinery with multiple sensors. IEEE/ASME Trans Mechatr2023; 28(4): 2293–2304.
31.
AzamfarMSinghJBravo-ImazI, et al. Multisensor data fusion for gearbox fault diagnosis using 2-D convolutional neural network and motor current signature analysis. Mech Syst Signal Proc2020; 144: 106861.
32.
ZhangYDingJLiY, et al. Multi-modal data cross-domain fusion network for gearbox fault diagnosis under variable operating conditions. Eng Appl Artif Intell2024; 133: 108236.
33.
XieTHuangXChoiS-K.Intelligent mechanical fault diagnosis using multisensor fusion and convolution neural network. IEEE Trans Indus Inform2021; 18: 3213–3223.
34.
HeDLaoZJinZ, et al. Train bearing fault diagnosis based on multi-sensor data fusion and dual-scale residual network. Nonlinear Dyn2023; 111(16): 14901–14924.
35.
DaiYZhangLXuD, et al. Development of deep learning-based cooperative fault diagnosis method for multi-pmsm drive system in 4WID-EVS. IEEE Trans Instrum Meas2024; 73: 1–13.
36.
LiuZNingHWuM, et al. A comprehensive diagnosis method of valve leakage faults based on bi-sensor information fusion. Struct Health Monit2024; 23(1): 512–526.
37.
ShaoSYanRLuY, et al. DCNN-based multi-signal induction motor fault diagnosis. IEEE Trans Instrum Meas2019; 69(6): 2658–2669.
38.
YuTRenZZhangY, et al. Laficnn: a novel convolutional adaptive fusion framework for fault diagnosis of rotating machinery. IEEE Trans Instrum Meas2024; 73: 1–9.
39.
LiuZMaoHWuC, et al. A convnet for the 2020s. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 19–20 June 2022, pp. 11966–11976.
40.
HouQZhouDFengJ.Coordinate attention for efficient mobile network design. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Nashville, TN, USA, 19–25 June 2021, pp. 13713–13722.
41.
ShaoSMcAleerSMYanR, et al. Highly accurate machine fault diagnosis using deep transfer learning. IEEE Trans Indus Inform2019; 15: 2446–2455.
42.
HeKZhangXRenS, et al. Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), Las Vegas, NV, USA, 26 June 2016 to 1 July 2016, pp. 770–778.
43.
HowardAGSandlerMChuG, et al. Searching for mobilenetv3. In: 2019 IEEE/CVF international conference on computer vision (ICCV), Seoul, Korea (South), 27 October 2019, pp. 1314–1324.
