The operating environment inside submarines for valves is harsh, prone to leaks, and leak signal detection is often conducted in complex, noisy environments. To address the issue that a single sensor cannot fully characterize leak features, which are often indistinct and lead to insufficient detection accuracy, this study proposes a multi-sensor enhancement scheme and a diagnosis method based on multi-sensor data fusion and deep learning (DL). First, distributed optical fiber temperature sensors, distributed optical fiber acoustic sensors, and acoustic emission sensors are used to comprehensively collect leak signals. Adaptive enhancement strategies are proposed for different signal characteristics: temperature signals are converted into high-feature-intensity images using entropy values combined with Euclidean distance maps (SEED); acoustic vibration signals are transformed into Markov images based on fast Fourier transform (FFT), followed by multi-point image fusion to highlight shared features; acoustic emission signals are converted into high-energy-concentrated time-frequency images using multi-scale synchroextracting transform (MSST). Subsequently, the three enhanced images are input into a multi-scale dilated convolutional network for feature extraction. Feature-level fusion is combined with an attention mechanism (three-branch multi-scale fusion attention network) to select key features. Finally, the jellyfish optimizer is introduced to optimize hyperparameters, and gradient-weighted class activation mapping is used to visualize the model’s attention regions. Experiments show that the proposed signal enhancement method effectively highlights signal features and improves the reliability of multimodal processing schemes. Compared to traditional convolutional neural network and image combination methods, this multi-scale feature extraction network demonstrates significant advantages and higher accuracy. In this research context, the scheme combining feature fusion with an attention mechanism outperforms decision fusion and hybrid fusion by better utilizing key information from each modality, promoting deep interactions between modalities and global feature selection, reducing information loss, and thereby significantly enhancing classification performance, achieving an overall accuracy exceeding 99.5%.
LiuZNingHWuM, et al. A comprehensive diagnosis method of valve leakage faults based on bi-sensor information fusion. Struct Health Monitor2024; 23(1): 512–526.
2.
AgbakwuruJ. Oil/Gas pipeline leak inspection and repair in underwater poor visibility conditions: challenges and perspectives. J Environ Protect2012; 3: 49.
3.
AdegboyeMAFungWKKarnikA. Recent advances in pipeline monitoring and oil leakage detection technologies: principles and approaches. Sensors2019; 19(11): 2548.
4.
HanYFengXToddMD. A novel methodology for quantitative identification of pipeline leakage and negative pressure wave velocity. Struct Health Monitor2023; 22(4): 2267–2279.
5.
LiuBJiangZNieW, et al. Research on leak location method of water supply pipeline based on negative pressure wave technology and VMD algorithm. Measurement2021; 186: 110235.
6.
SharmaRR. Gas leakage detection in pipeline by svm classifier with automatic eddy current based defect recognition method. UCCT2021; 3(3): 196–212.
7.
ShenYZhangLT. Eddy current detecting of leak hole in pipeline by wavelet packet signal analysis method. Appl Mech Mater2013; 291: 2486–2491.
8.
RenLJiangTJiaZG, et al. Pipeline corrosion and leakage monitoring based on the distributed optical fiber sensing technology. Measurement2018; 122: 57–65.
9.
JacobszSWJahnkeSI. Leak detection on water pipelines in unsaturated ground by discrete fibre optic sensing. Struct Health Monitor2020; 19(4): 1219–1236.
10.
HuZTariqSZayedT. A comprehensive review of acoustic based leak localization method in pressurized pipelines. Mech Syst Sig Process2021; 161: 107994.
11.
FanHTariqSZayedT. Acoustic leak detection approaches for water pipelines. Automat Constr2022; 138: 104226.
12.
JiaZZhanghuaLZhaomingZ, et al. Leakage detection in a buried gas pipeline based on distributed optical fiber time-domain acoustic wave signal. Eng Failure Anal2022; 141: 106594.
13.
StajancaPChruscickiSHomannT, et al. Detection of leak-induced pipeline vibrations using fiber—optic distributed acoustic sensing. Sensors2018; 18(9): 2841.
14.
LiuZYangXWuM, et al. Leveraging deep learning techniques for ship pipeline valve leak monitoring. Ocean Eng2023; 288: P2.
15.
ZhengjieLWeileiMHaoN, et al. Pressure vessel leakage detection method based on online acoustic emission signals. Insight Non Destruct Test Cond Monitor2023; 65(1): 36–42.
16.
YinHHChenHPengZB. Real-time model method research in ship pipeline system leakage detecting. Appl Mech Mater2012; 105: 685–688.
17.
YanJLiaoBJGaoYJ, et al. Fusion of audio and vibration signals for bearing fault diagnosis based on a quadratic convolution neural network. Sensors2023; 23(22): 9155.
18.
XuBLiHDingR, et al. Fault diagnosis in electric motors using multi-mode time series and ensemble transformers network. Sci Rep2025; 15(1): 7834.
19.
OmerKEyupC. A deep-learning-based multi-modal sensor fusion approach for detection of equipment faults. Machines2022; 10(11): 1105.
20.
ShannonEC. A mathematical theory of communication. ACM SIGMOBILE Mobile Comput Commun Rev2001; 5(1): 3–55.
21.
DasDRayAHensC, et al. Complexity measure of extreme events. Chaos (Woodbury, N.Y.)2024; 34(12): 0232645.
22.
AldrichC. A comparative analysis of image encoding of time series for anomaly detection. In: Time series analysis-recent advances, new perspectives and applications. IntechOpen2023; 7: 1002535.
23.
ZhangJZhangQQinX, et al. Robust fault diagnosis of quayside container crane gearbox based on 2D image representation in frequency domain and CNN. Struct Health Monitor2024; 23(1): 324–342.
24.
De MartinoA.On the Clifford short-time Fourier transform and its properties. Appl Math Comput2022; 418: 126812.
25.
ZhangQDengL. An intelligent fault diagnosis method of rolling bearings based on short-time Fourier transform and convolutional neural network. J Failure Anal Prevent2023; 23(2): 795–811.
26.
WanTHTsangCWHuiK, et al. Anomaly detection of train wheels utilizing short-time Fourier transform and unsupervised learning algorithms. Eng Appl Artif Intell2023; 122: 106037.
27.
LiuZHuangWZhangY, et al. Multi-source elastic meta-transfer learning for few-shot fault diagnosis of aero-engine under unstable speed. IEEE Intern Thing J2025; 24(12): 52632–52643.
28.
WangYYangMLiY, et al. A multi-input and multi-task convolutional neural network for fault diagnosis based on bearing vibration signal. IEEE Sens J2021; 21(9): 10946–10956.
29.
ZhuXYangHZhangZ, et al. Frequency-chirprate reassignment. Digi Sig Process2020; 104: 102783.
30.
PlanteFMeyerGAinsworthWA. Improvement of speech spectrogram accuracy by the method of reassignment. IEEE Trans Speech Audio Process1998; 6(3): 282–287.
31.
DaubechiesIMaesS. A nonlinear squeezing of the continuous wavelet transform based on auditory nerve models. In: DaubechiesIMaesS (eds) Wavelets in medicine and biology. London: Routledge, 2017, pp. 527–546.
32.
ZhuXZhangZLiZ, et al. Multiple squeezes from adaptive chirplet transform. Sig Process2019; 163: 26–40.
33.
YuGYuMXuC. Synchroextracting transform. IEEE Trans Industr Electron2017; 64(10): 8042–8054.
34.
ZhuXZhangZGaoJ, et al. Synchroextracting chirplet transform for accurate IF estimate and perfect signal reconstruction. Digi Sig Process2019; 93: 172–186.
35.
YuGWangZZhaoP. Multisynchrosqueezing transform. IEEE Trans Industr Electron2019; 66(7): 5441–5455.
36.
LuBLLiuZHWeiHL, et al. A deep adversarial learning prognostics model for remaining useful life prediction of rolling bearing. IEEE Trans Artif Intell2021; 2(4): 329–340.
37.
TanXLiuMLvR. Rolling bearing fault diagnosis technology based on deep learning. In: 2022 international conference on computation, big-data and engineering (ICCBE), Yunlin, Taiwan, 27–29 May 2022, pp. 214–216. New York: IEEE.
38.
GuJPengYLuH, et al. A novel fault diagnosis method of rotating machinery via VMD, CWT and improved CNN. Measure2022; 200: 111635.
39.
WangHLiuZPengD, et al. Feature-level attention-guided multitask CNN for fault diagnosis and working conditions identification of rolling bearing. IEEE Trans Neural Netw Learn Syst2021; 33(9): 4757–4769.
40.
LiLXuJLiJ. Estimating remaining useful life of rotating machinery using relevance vector machine and deep learning network. Eng Failure Anal2023; 146: 107125.
41.
ChuCGeYQianQ, et al. A novel multi-scale convolution model based on multi-dilation rates and multi-attention mechanism for mechanical fault diagnosis. Digi Sig Process2022; 122: 103355.
42.
ChenQZhangFWangY, et al. Bearing fault diagnosis based on efficient cross space multiscale CNN transformer parallelism. Sci Rep2025; 15(1): 12344.
XiaMLiTXuL, et al. Fault diagnosis for rotating machinery using multiple sensors and convolutional neural networks. IEEE/ASME Trans Mech2017; 23(1): 101–110.
45.
JingLWangTZhaoM, et al. An adaptive multi-sensor data fusion method based on deep convolutional neural networks for fault diagnosis of planetary gearbox. Sensors2017: 17(2): 414.
46.
FengWZhenLXiaoZ, et al. Oil and gas pipeline leakage recognition based on distributed vibration and temperature information fusion. Result Optic2021; 5: 100131.
47.
JanssensOSlavkovikjVVervischB, et al. Convolutional neural network based fault detection for rotating machinery. J Sound Vib2016; 377: 331–345.
48.
GuerrieroMSvenssonLWillettP. Bayesian data fusion for distributed target detection in sensor networks. IEEE Trans Sig Process2010; 58(6): 3417–3421.
49.
SongningLXifengHHaoxuanX, et al. Multimodal sentiment analysis: a survey. Displays2023; 80: 102563.
50.
JiangWYangHLuX, et al. MCANet: a coherent multimodal collaborative attention network for advanced modulation recognition in adverse noisy environments. arXiv preprint. arXiv:2510.18336, 2025.
51.
DuanRLiXZhangH, et al. A multi-modal modulation recognition method with SNR segmentation based on time domain signals and constellation diagrams. Electronics2023; 12(14): 3175.
52.
BahdanauDChoKBengioY. Neural machine translation by jointly learning to align and translate. In: CoRR2014: abs/1409.0473.
53.
JieHLiSSamuelA, et al. Squeeze-and-excitation networks. IEEE Trans Pattern Anal Mach Intell2019; 42(8): 7132–7141.
54.
WooSParkJLeeJY, et al. Cbam: convolutional block attention module. In: FerrariVHebertMSminchisescuC, et al. (eds) Proceedings of the European conference on computer vision (ECCV). Cham: Springer, 2018, pp. 3–19.
55.
VinayVMilindSPranavN, et al. Estimation of lithium-ion battery discharge capacity by integrating optimized explainable-AI and stacked LSTM model. Batteries2023; 9(2): 125.
56.
SalemMTaheriSYuanSJ. An experimental evaluation of fault diagnosis from imbalanced and incomplete data for smart semiconductor manufacturing. Big Data Cognitive Comput2018; 2(4): 30.
57.
RuiyanCJunMengLYanW, et al. Hyperspectral imaging coupled with Dual-channel convolutional neural network for early detection of apple valsa canker. Comput Electron Agri2022; 202: 107411.
58.
ZhouBKhoslaALapedrizaÀ, et al. Learning deep features for discriminative localization. In: IEEE conference on computer vision and pattern recognition (CVPR), Las Vegas, NV, 27–30 June 2016.
59.
WangZOatesT. Imaging time-series to improve classification and imputation. In: Proceedings of the 24th international conference on artificial intelligence (CoRR), Buenos Aires, Argentina, 25–31 July 2015.
60.
WangZOatesT. Encoding time series as images for visual inspection and classification using tiled convolutional neural networks. In: Workshops at the twenty-ninth AAAI conference on artificial intelligence, Austin: AAAI, 2015.
