Fault diagnosis of wind turbine blades under wide-weather multi-operating conditions based on multi-modal information fusion and deep learning

Abstract

Continuous health monitoring of wind turbine blades under all-weather and multi-operating conditions represents a significant challenge in the renewable energy sector. In this article, we present a fault diagnosis approach leveraging multi-modal information fusion and deep learning with continuous state division, thereby overcoming the limitations of traditional methods in complex and noisy environments. The operational conditions of wind turbine blades are categorized into two primary states: the wind operation state and the sudden shutdown state. Additionally, various climate types, including sunny, foggy, windy conditions, differing lighting levels, and others, are considered in the analysis. During wind operation, sound and vibration signals exhibit higher efficacy for fault detection; however, high noise levels may introduce interferences. To address the issue of indistinct fault characteristics after deep convolution due to multiple noise factors, which could result in reduced diagnostic accuracy, we propose a robust fast Fourier transform-ResTransNet model. In the shutdown state, vibration and sound data features become less prominent, making image processing techniques advantageous. Nevertheless, diverse climate types can lead to challenges such as low visibility, high noise, and other interferences. Consequently, we design a Swin-Transformer model that integrates infrared thermal imaging and visible light imagery. This model resolves the problem of non-homologous data representation and ensures accurate information interaction under multi-source data fusion. Simulation results confirm that the proposed fault diagnosis method achieves substantial improvements over existing approaches. To validate the practical applicability of our method, we construct a real-world wind turbine operational environment, simulate several common blade fault scenarios, and collect actual vibration, sound, and image data under varying weather conditions. Based on these simulations, we establish a multi-modal information fusion model tailored for different weather types. Furthermore, to facilitate the integration of our research into real-world wind farm operations, we develop a human–computer interface that enables seamless deployment. The corresponding source code is publicly available at https://github.com/midfigher/Humancomputer-interaction.