Ship hydrodynamic prediction by a novel transformer model with incremental learning

Abstract

With the rapid development of maritime commerce, the complexity of ship operation environments continues to grow, so that it is crutial for ship designers to make accurate forecasting of ship performance under various sea conditions. However, there exist the following challenges in ship performance evaluation and optimization. The first challenge is the computational overhead and slow response of numerical simulations based on Computational Fluid Dynamics (CFD), and the second is the inability of traditional machine learning to achieve the computational accuracy of numerical simulations. To address these issues, this paper proposes an online surrogate model, called Increformer, for ship performance prediction. The Increformer leverages continuous self-attention mechanisms to explore the temporal dependencies between feature variables and employs continuous normalization mechanisms to handle non-stationary data issues. In addition, in order to improve prediction accuracy by the model, we employ an incremental training strategy based on elastic weight consolidation to acquire new knowledge from data streams. Experiments are conducted by using historical performance data from various types of vessels including KCS, Wigley-III, and C60. The results demonstrate that the Increformer model effectively captures the temporal information and inter-dimensional correlations in the data, with the accuracy of ship performance prediction enhanced significantly. Furthermore, ablation experiments are also carried out to assess the effectiveness and necessity of the continuous normalization mechanism, continuous attention mechanism, and incremental training strategy for the Increformer model. The findings validate the accuracy and universality of the proposed model. It is also shown that the Increformer model adeptly captures trends and fluctuations in ship sequence data and thus provides a reliable solution for ship performance prediction.

Keywords

Time series prediction transformer incremental learning continual attention

Get full access to this article

View all access options for this article.

References

Peri

Campana

. Variable Fidelity and Surrogate Modeling in Simulation-based Design[C]. In: 27th Symposium on Naval Hydrodynamics. Seoul, Korea. 2008.

Pedersen

Larsen

. Prediction of full-scale propulsion power using artificial neural networks[C]. In: 8th International Conference of Computer and IT Applications in the Maritime Industries, Budapest. Budapest, Hungary, 2009, pp.10–12.

Hefazi

Mizine

Schmitz

, et al. Multidisciplinary synthesis optimization process in multihull ship design [J]. Nav Eng J 2010; 122: 29–47.

Petersen

Winther

. Mining of Ship Operation Data for Energy Conservation[D]. Technical University of Denmark, 2011.

Babadi

Ghassemi

. Effect of hull form coefficients on the vessel seakeeping performance[J]. J Mar Sci Technol 2013; 21: 594–604.

Wang

Liu

. Ship rolling motion prediction based on wavelet neural network[J]. Applied Mechanics and Materials 2012; 1923: 724–728.

Tang

Yao

, et al. Prediction about the vessel’s heave motion under different sea states based on hybrid PSO_ARMA model [J]. Ocean Eng 2022; 263: 112247. ISSN 0029-8018.

Ferrandis

Triantafyllou

Chryssostomidis

. Learning functionals via LSTM neural networks for predicting vessel dynamics in extreme sea states. Proceedings of The Royal Society A Mathematical Physical and Engineering Sciences 2021; 477: 20190897.

Ren

. Ship roll motion prediction using ConvLSTM with attention mechanism[C]. In: 2022 41st Chinese Control Conference (CCC). IEEE, 2022, pp.5616–5620.

10.

Kirkpatrick

Pascanu

Rabinowitz

, et al. Overcoming catastrophic forgetting in neural networks[J]. Proc Natl Acad Sci USA 2017; 114: 3521–3526.

11.

Aljundi

Kelchtermans

Tuytelaars

. Task-free continual learning[C]. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019, pp.11254–11263.

12.

Buzzega

Boschini

Porrello

, et al. Dark experience for general continual learning: a strong, simple baseline[J]. Adv Neural Inf Process Syst 2020; 33: 15920–15930.

13.

C-Z

Zou

Z-J

. Online prediction of ship roll motion in waves based on auto-moving gird search-least square support vector machine[J]. Math Probl Eng 2021; 2021: 366–372.

14.

Guan

Yang

Wang

, et al. Ship roll motion prediction based on ℓ1 regularized extreme learning machine[J]. PLOS ONE 2018; 13: e0206476.

15.

Yin

J-C

Perakis

Wang

. A real-time ship roll motion prediction using wavelet transform and variable RBF network[J]. Ocean Eng 2018; 160: 10–19.

16.

Hou

X-R

Zou

Z-J

Liu

. Nonparametric identification of nonlinear ship roll motion by using the motion response in irregular waves[J]. Appl Ocean Res 2018; 73: 88–89.

17.

Wei

Chen

Zhao

, et al. An ensemble multi-step forecasting model for ship roll motion under different external conditions: a case study on the South China Sea[J]. Measurement ( Mahwah N J) 2022; 201: 86–89.

18.

Mallya

Lazebnik

. Packnet: adding multiple tasks to a single network by iterative pruning[C]. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2018, pp.7765–7773.

19.

Wang

Han

. Online sequential extreme learning machine with kernels for nonstationary time series prediction[J]. Neurocomputing 2014; 145: 90–97.

20.

Dai

. Dwe IL: a new incremental learning algorithm for non-stationary time series prediction via dynamically weighting ensemble learning[J]. Applied Intelligence 2022; 52: 174–194.

21.

Wang

Yue

. IncLSTM: incremental ensemble LSTM model towards time series data[J]. Comput Electr Eng 2021; 92: 107156.

22.

Woo

Liu

Sahoo

, et al. CoST: Contrastive learning of disentangled seasonal-trend representations for time series forecasting[J]. arXiv preprint arXiv:2202.01575, 2022.

23.

Huang

Chen

Zhang

, et al. A transferable time series forecasting service using deep transformer model for online systems[C]. In: Proceedings of the 37th IEEE/ACM international conference on automated software engineering. 2022, pp.1–12.

24.

Chen

, et al. Long sequence time-series forecasting with deep learning: a survey[J]. Inf Fus 2023; 97: 101819.

25.

Wang

Dong

, et al. Deep time series models: A comprehensive survey and benchmark[J]. arXiv preprint arXiv:2407.13278, 2024.

26.

Zeng

Chen

Zhang

, et al. Are transformers effective for time series forecasting?[C]. Proceedings of the AAAI Conference on Artificial Intelligence 2023; 37: 11121–11128.

27.

Qiu

Lin

, et al. Duet: dual clustering enhanced multivariate time series forecasting[C]. In: Proceedings of the 31st ACM SIGKDD conference on knowledge discovery and data mining V. 1. 2025, pp.1185–1196.

28.

Lopez-Paz

Ranzato

. Gradient episodic memory for continual learning[J]. Adv Neural Inf Process Syst 2017: 30.

29.

Chaudhry

Ranzato

Rohrbach

, et al. Efficient lifelong learning with a-gem[J]. arXiv preprint arXiv:1812.00420, 2018.

30.

Zeng

Chen

Cui

, et al. Continual learning of context-dependent processing in neural networks[J]. Nature Machine Intelligence 2019; 1: 364–372.

31.

Kim

, et al. A comprehensive survey of deep learning for time series forecasting: architectural diversity and open challenges[J]. Artif Intell Rev 2025; 58: 216.

32.

Wang

, et al. Chattime: a unified multimodal time series foundation model bridging numerical and textual data[C]. Proceedings of the AAAI Conference on Artificial Intelligence 2025; 39: 12694–12702.

33.

Zhao

Wang

, et al. Predictive pretrained transformer (PPT) for real-time battery health diagnostics[J]. Appl Energy 2025; 377: 124746.

34.

Zhou

Zhang

Peng

, et al. Informer: beyond efficient transformer for long sequence time-series forecasting[C]. In: Proceedings of the 2021 AAAI conference on artificial intelligence. Menlo Park: AAAI, 2021, pp.11106–11115.

35.

. Group normalization[C]. In: Proceedings of the European conference on computer vision (ECCV). 2018, pp.3–19.

36.

Ioffe

Szegedy

. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]. In: International conference on machine learning. PMLR, 2015, pp.448–456.