Energy management strategy for hybrid electric vehicles based on deep reinforcement learning with driving condition recognition and expert demonstrations

Abstract

In order to improve the fuel economy of hybrid electric vehicles (HEV), this paper proposes an adaptive demonstration-guided dueling double deep Q-network (AD-D3QN) energy management strategy (EMS) based on parallel HEV architecture. The upper-layer network employs a sequential network with long short-term memory (LSTM) to capture dependencies in driving cycle speed sequences for driving condition identification. The lower-layer adversarial dueling double deep Q-network, combined with learning from demonstration (LfD), determines optimal weight parameters through multiple grid searches for different driving conditions, ultimately training to obtain optimal energy management strategies for various driving conditions. During operation, the lower layer selects the corresponding strategy in real-time based on the driving condition type identified by the upper layer, thereby improving generalization performance and reducing energy consumption in different driving conditions. The simulation results demonstrate that incorporating dynamic programming (DP) expert prior knowledge significantly enhances training speed. Moreover, the proposed strategy achieves an energy consumption only 1.91% higher than the global optimum obtained by DP, and it substantially outperforms other strategies. Additionally, hardware-in-the-loop (HIL) experiments show that the strategy’s performance in actual hardware closely matches simulation results, validating the algorithm’s reliability in engineering applications.