A hybrid approach for long- and short-term time series analysis and prediction combining the Mamba and transformer models

Abstract

Accurate prediction of electricity load time series is crucial for optimizing power system scheduling and ensuring reliable operation. However, existing approaches face challenges in effectively capturing long-term global trends while addressing short-term local fluctuations. This study proposes a hybrid strategy that integrates the Mamba algorithm for long-term predictions with the Transformer model for short-term forecasting, leveraging the complementary strengths of both methods. The Mamba algorithm captures global trends in long-term sequences, while the Transformer focuses on local variations in short-term data. This combination enhances both long-term trend accuracy and the detection of short-term fluctuations. Additionally, a robust sequence decomposition module is introduced to refine the sliding window mechanism, which segments long sequences for Transformer input, thereby reducing computational overhead and memory demands. Experimental comparisons demonstrate that the hybrid approach outperforms both standalone Mamba and Transformer models in prediction precision, computational efficiency, and memory optimization. Specifically, compared to Mamba, the hybrid model reduces mean squared error (MSE) by 37%, and compared to Transformer, it achieves an 11% reduction in MSE. Additionally, the training time is shortened by 34%, and GPU memory usage is reduced by 47%. These findings confirm that the proposed strategy effectively integrates the advantages of Mamba and Transformer, achieving reliable long-term load forecasting alongside robust short-term prediction capabilities. The model’s efficacy has been validated using real-world power load datasets from Sichuan Province, China, covering the period from 2022 to July 2024.