State space model enhanced stacked convolutional long short-term memory for blade damage identification

Abstract

Current vibration-based damage identification methods face difficulties in accurately identify damage features due to the low richness of data feature for wind turbine blade. This article introduces convolutional long short-term memory (ConvLSTM) that can better characterize the spatiotemporal characteristics in deep learning and explores the damage identification method combining stacked ConvLSTM network with structural state space model. A state space model enhanced stacked ConvLSTM for blade damage identification is proposed. First, the vibration signals of the blades at different damage states are converted to time–frequency images through the preprocessing of normalization and wavelet transform. The preprocessing operation improved the damage characteristics of the original vibration signals. Then, the designed stacked ConvLSTM is used to train and test time–frequency images at different damage states and output damage states and corresponding probability values through Softmax component. During the training, the different between the state equation of blades and cell state of stacked ConvLSTM is taken as loss function. Finally, specific parameter of the proposed state space model enhanced stacked ConvLSTM are set using the displacement data of blades in OpenFast software, and the recognition results are compared and validated with the mainstream networks convolutional neural network (CNN), LSTM, and ConvLSTM. The results show that, among these networks, 1D CNN, 2D CNN, LSTM, ConvLSTM, and BConvLSTM, the proposed state space model enhanced stacked ConvLSTM network achieves the best recognition results. Compared with the standard ConvLSTM network, the accuracy and mean intersection over union of the proposed state space model enhanced stacked ConvLSTM network are improved by 1.69% and 4.4%, respectively. Moreover, the proposed state space model enhanced stacked ConvLSTM achieved recognition accuracy of over 97% at different wind turbine blades working conditions. This indicates that the proposed state space model enhanced stacked ConvLSTM for blade damage identification has high accuracy and robustness. The effectiveness of the proposed state space model enhanced stacked ConvLSTM in blade damage identification has been validated again through laboratory scale wind turbine blade damage test.

Keywords

Wind turbine blade damage identification state space model time–frequency image stacked ConvLSTM Softmax component

Get full access to this article

View all access options for this article.

References

Sun

, et al. Wind turbine blade breakage detection based on environment-adapted contrastive learning. Renewable Energy 2023; 219: 119487.

Zhao

Chen

NZ.

A single-sensor method for structural damage localization in wind turbine blades: laboratory assessment on a blade segment. Mech Syst Signal Process 2024; 214:111370.

Sarrafi

Mao

Niezrecki

, et al. Vibration-based damage detection in wind turbine blades using phase-based motion estimation and motion magnification. J Sound Vibr 2018; 421: 300–318.

Lifshitz

Rotem

Determination of reinforcement unbonding of composites by a vibration technique. J Compos Mater 1969; 3: 412–423.

Yuen

MMF

. A numerical study of the eigenparameters of a damaged cantilever. J Sound Vib 1985; 103(3): 301–310.

Rizos

Aspragathos

Dimarogonas

AD.

Identification of crack location and magnitude in a cantilever beam from the vibration modes. J Sound Vib 1990; 138(3): 381–388.

Wang

McFadden

PD.

Application of orthogonal wavelets to early gear damage detection. Mech Syst Signal Process 1995; 9(5): 497–507.

King

NE.

Detection of structural nonlinearity using Hilbert transform procedures. UK: The University of Manchester, 1994.

Lin

Deep diagnostics and prognostics: an integrated hierarchical learning framework in PHM applications. Appl Soft Comput 2018; 72: 555–564.

10.

Cha

Choi

Büyükoztürk

Deep learning-based crack damage detection using convolutional neural networks. Comput Aided Civ Inf Eng 2017; 32(5): 361–378.

11.

Ren

Girshick

, et al. Faster R-CNN: towards real-time object detection with region proposal networks. Adv Neural Inf Process Syst 2017; 39(6): 1137–1149.

12.

Cha

Choi

Suh

, et al. Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Comput Aided Civ Inf Eng 2018; 33(9): 731–747.

13.

Zhao

Chen

NZ.

An exponential smoothing multi-head graph attention network (ESMGAT) method for damage zone localization on wind turbine blades. Compos Struct 2024; 342: 118224.

14.

LeCun

Bengio

Hinton

Deep learning. Nature 2015; 521(7553): 436–444.

15.

Ren

Wang

Liang

, et al. Acoustic emission detection of filament wound CFRP composite structure damage based on Mel spectrogram and deep learning. Thin-Walled Struct 2024; 198: 111683.

16.

Lecun

Bottou

Bengio

, et al. Gradient-based learning applied to document recognition. Proc IEEE 1998; 86(11): 2278–2324.

17.

Hinton

Salakhutdinov

RR.

Reducing the dimensionality of data with neural networks. Science 2006; 313: 504–507.

18.

Hopfield

JJ.

Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci USA 1982; 79(8): 2554–2558.

19.

Elman

JL.

Finding structure in time. Cognitive Sci 1990; 14(2): 179–211.

20.

Miao

QL.

End-to-end remote sensing image classification framework based on convolutional block attention module. J Forestry Eng 2020; 5(04): 133–138.

21.

Adnan

Hamza

MZHM

Mortazavizadeh

, et al. Comparative evaluation of LSTM, CNN, and ConvLSTM for hourly short-term streamflow forecasting using deep learning approaches. Ecol Inform 2023; 75: 102119.

22.

Reddy

Indragandhi

Ravi

, et al. Detection of cracks and damage in wind turbine blades using artificial intelligence-based image analytics. Measurement 2019; 147: 106823.

23.

Bengio

Simard

Frasconi

Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Netw 1994; 5(2): 157–166.

24.

Hochreiter

Schmidhuber

JJ.

Long short-term memory. Neural Comput 1997; 9(8): 1735–1780.

25.

Choe

Kim

MH.

Sequence-based modeling of deep learning with LSTM and GRU networks for structural damage detection of floating offshore wind turbine blades. Renewable Energy 2021; 174: 218–235.

26.

Huo

Tang

Zhang

, et al. Semisupervised classification of remote sensing images with hierarchical spatial similarity. IEEE Geosci Remote Sens Lett 2015; 12(1): 150–154.

27.

Shi

Chen

Wang

, et al. Convolutional LSTM network: a machine learning approach for precipitation nowcasting. In: Proceedings of the 29th international conference on neural information processing systems, 2015, 802–810.

28.

Ghimire

Yaseen

Farooque

, et al. Streamflow prediction using an integrated methodology based on convolutional neural network and long short-term memory networks. Sci Rep 2021; 11(1): 7834616.

29.

Moishin

Deo

Prasad

, et al. Designing deep-based learning flood forecast model with ConvLSTM hybrid algorithm. IEEE Access 2021; 9: 50982–50993.

30.

Qiao

Truman

Sukkarieh

Cattle segmentation and contour extraction based on Mask R-CNN for precision livestock farming. Comput Electron Agric 2019; 165: 104958.

31.

Gao

Zhao

, et al. Fusion model of weight on bit in horizontal exploration hole based on wavelet transform and machine learning. Geoenergy Sci Eng 2024; 234: 212654.

32.

Haar

Zur Theorie der orthogonalen Funktionensysteme. Math Ann 1910; 69: 331–371.

33.

Paduart

Lauwers

Swevers

, et al. Identification of nonlinear systems using polynomial nonlinear state space models. Automatica 2010; 46(4): 647–656.

34.

Wei

Guan

Wang

, et al. Damage detection of jacket platforms through improved stacked autoencoder and softmax classifier. Ocean Eng 2024; 306: 118036.

35.

Krizhevsky

Sutskever

Hinton

GE.

Imagenet classification with deep convolutional neural networks. Commun ACM 2017; 60(6): 84–90.

36.

, et al. A review of deep-learning-based super-resolution: from methods to applications. Pattern Recognit 2025; 157: 110935.

37.

Cha

Mostafavi

Benipal

SS.

DNoiseNet: deep learning-based feedback active noise control in various noisy environments

Eng Appl Artif Intell 2023; 121:105971; Constr. 2023; 151: 104885.

38.

Mostafavi

Cha

. Deep learning-based active noise control on construction sites. Autom Constr 2023; 151: 104885.

39.

Liu

Wang

Zhang

, et al. Deep learning-based data augmentation for large-scale mineral image recognition and classification. Minerals Eng 2023; 204: 108411.

40.

Zhao

Zhang

Zhu

, et al. Predicting tongue motion in unlabeled ultrasound videos using convolutional LSTM neural networks. In: ICASSP 2019-2019 IEEE international conference on acoustics, speech and signal processing, Brighton, UK, 2019, pp. 5926–5930.

41.

Wang

Sprague

Jonkman

, et al. BeamDyn: a high-fidelity wind turbine blade solver in the FAST modular framework. Wind Energy 2017; 20(8): 5–9.

42.

Yang

JD.

Wind farm operation maintenance and overhaul. Beijing, China: WaterPower Press, 2014.

43.

Ciang

Lee

Bang

HJ.

Structural health monitoring for a wind turbine system: a review of damage detection methods. Meas Sci Technol 2008; 19(12): 310–314.

44.

Cha

Wang

Unsupervised novelty detection–based structural damage localization using a density peaks-based fast clustering algorithm. Struct Health Monit 2018; 17 (2): 313–324.

45.

Wang

Cha

YJ.

Unsupervised deep learning approach using a deep autoencoder with a one-class support vector machine to detect damage. Struct Health Monit 2012; 20 (1): 406–425.

46.

Wang

Cha

YJ.

Unsupervised machine and deep learning methods for structural damage detection: a comparative study. Eng Rep 2022; 7: e12551.

47.

SJ.

Extraction of forest burned area based on stacked ConvLSTM and remote sensing image time series. Beijing, China: University of Chinese Academy of Sciences, 2021.

48.

Azad

Asadi-Aghbolaghi

Fathy

, et al. Bi-directional ConvLSTM U-Net with Densley connected convolutions. In: 2019 IEEE/CVF International conference on computer vision workshop (ICCVW), Seoul, Korea (South), 2019, pp. 406–415.

49.

Zhang

Biswal

Wang

SHMnet: condition assessment of bolted connection with beyond human-level performance. Struct Health Monit 2020; 19(4): 1188–1201.

50.

Wang

, et al. A novel deep learning-based method for damage identification of smart building structures. Struct Health Monit 2019; 18(1): 143–163.

51.

Zhan

Zhang

, et al. High-precision parameter-free optical distortion measurement and correction via digital image correlation. Opt Laser Technol 2024; 177: 111129.