In recent years, Deep Reinforcement Learning (DRL) has demonstrated significant potential in Energy Management Strategies (EMS) for Fuel Cell Vehicles (FCVs). Among these, the Soft Actor-Critic (SAC) algorithm has garnered widespread attention for its superior performance, yet it exhibits shortcomings in convergence speed, stability, and reward value. Moreover, the effectiveness of SAC in mitigating the degradation of fuel cells and lithium batteries remains limited. Therefore, this paper proposes an improved SAC (I-SAC) algorithm, which incorporates Prioritized Experience Replay (PER) and Self-Adaptive Temperature Control (SATC) to enhance performance and effectively extend the lifespan of fuel cells and lithium batteries. Simulation results show that, compared with EMSs based on Double Deep Q-Network (DDQN) and Deep Deterministic Policy Gradient (DDPG), I-SAC significantly reduces hydrogen consumption under various operating conditions, while reducing the degradation of fuel cells and lithium batteries by up to 10.615% and 34.347%, respectively. This study presents a new efficient and robust EMS solution for FCVs.
AgyekumEBOdoi-YorkeFAbbeyAA, et al. A review of the trends, evolution, and future research prospects of hydrogen fuel cells – a focus on vehicles. Int J Hydrogen Energy2024; 72: 918–939.
2.
İnciM.Future vision of hydrogen fuel cells: a statistical review and research on applications, socio-economic impacts and forecasting prospects. Sustain Energy Technol Assess2022; 53: 102739.
3.
PramuanjaroenkijAKakaçS.The fuel cell electric vehicles: the highlight review. Int J Hydrogen Energy2023; 48: 9401–9425.
4.
ZhaoXWangLZhouY, et al. Energy management strategies for fuel cell hybrid electric vehicles: classification, comparison, and outlook. Energy Convers Manag2022; 270: 04022044.
5.
DuGZouYZhangX, et al. Heuristic energy management strategy of hybrid electric vehicle based on deep reinforcement learning with accelerated gradient optimization. IEEE Trans Transp Electrif2021; 7: 2194–2208.
6.
ZhangCLiTCuiW, et al. Proximal policy optimization based intelligent energy management for plug-in hybrid electric bus considering battery thermal characteristic. World Electr Veh J2023; 14: 47.
7.
ParkSAhnC.Power management controller for a hybrid electric vehicle with predicted future acceleration. IEEE Trans Veh Technol2019; 68: 10477–10488.
8.
HanJShuHTangX, et al. Predictive energy management for plug-in hybrid electric vehicles considering electric motor thermal dynamics. Energy Convers Manag2022; 251: 115022.
9.
HuXZouCTangX, et al. Cost-optimal energy management of hybrid electric vehicles using fuel cell/battery health-aware predictive control. IEEE Trans Power Electron2020; 35: 382–392.
10.
WuJWeiZLiuK, et al. Battery-involved energy management for hybrid electric bus based on expert-assistance deep deterministic policy gradient algorithm. IEEE Trans Veh Technol2020; 69: 12786–12796.
11.
SorleiI-SBizonNThounthongP, et al. Fuel cell electric vehicles—a brief review of current topologies and energy management strategies. Energies2021; 14: 252.
12.
RenXYeJXieL, et al. Battery longevity-conscious energy management predictive control strategy optimized by using deep reinforcement learning algorithm for a fuel cell hybrid electric vehicle. Energy2024; 286: 129344.
13.
HanXHeHWuJ, et al. Energy management based on reinforcement learning with double deep Q-learning for a hybrid electric tracked vehicle. Appl Energy2019; 254: 113708.
14.
TangXChenJLiuT, et al. Distributed deep reinforcement learning-based energy and emission management strategy for hybrid electric vehicles. IEEE Trans Veh Technol2021; 70: 9922–9934.
15.
LianRPengJWuY, et al. Rule-interposing deep reinforcement learning based energy management strategy for power-split hybrid electric vehicle. Energy2020; 197: 117297.
16.
DuanYChenXHouthooftR, et al. Benchmarking deep reinforcement learning for continuous control. In: International conference on machine learning (eds DuanYChenXHouthooftR, et al.), 2016, June 19–24, New York, NY, USA. pp.1329–1338. PMLR.
17.
HaarnojaTZhouAHartikainenK, et al. Soft actor-critic algorithms and applications. arXiv preprint arXiv:1812.05905, 2018.
18.
WuCRuanJCuiH, et al. The application of machine learning based energy management strategy in multi-mode plug-in hybrid electric vehicle, part I: twin delayed deep deterministic policy gradient algorithm design for hybrid mode. Energy2023; 262: 125084.
19.
WangHYeYZhangJ, et al. A comparative study of 13 deep reinforcement learning based energy management methods for a hybrid electric vehicle. Energy2023; 266: 126497.
20.
XuDCuiYYeJ, et al. A soft actor-critic-based energy management strategy for electric vehicles with hybrid energy storage systems. J Power Sources2022; 524: 231099.
21.
SunWZouYZhangX, et al. High robustness energy management strategy of hybrid electric vehicle based on improved soft actor-critic deep reinforcement learning. Energy2022; 258: 124806.
22.
LiuZMChangGFJiangSF, et al. Adaptive anti-surge control strategy for PEM fuel cell vehicle with online surge detection. IEEE Trans Transp Electrif2024; 10: 844–858.
23.
ShanmugamCNattuthuraiSMuthusamyS. Optimized control of hybrid energy storage systems for electric vehicles using BWPOA-MFPIDNN approach. Journal of Energy Storage2024; 104. DOI: 10.1016/j.est.2024.114317.
24.
LiuBWeiXSunC, et al. A controllable neural network-based method for optimal energy management of fuel cell hybrid electric vehicles. Int J Hydrogen Energy2024; 55: 1371–1382.
25.
SongKWangXLiF, et al. Pontryagin’s minimum principle-based real-time energy management strategy for fuel cell hybrid electric vehicle considering both fuel economy and power source durability. Energy2020; 205: 118064.
26.
ChenHPeiPSongM.Lifetime prediction and the economic lifetime of proton exchange membrane fuel cells. Appl Energy2015; 142: 154–163.
27.
ZhouJXueSXueY, et al. A novel energy management strategy of hybrid electric vehicle via an improved TD3 deep reinforcement learning. Energy2021; 224: 120118.
28.
TangLRizzoniGOnoriS.Energy management strategy for HEVs including battery life optimization. IEEE Trans Transp Electrif2015; 1: 211–222.
29.
OnoriSSpagnolPMaranoV, et al. A new life estimation method for lithium-ion batteries in plug-in hybrid electric vehicles applications. Int J Power Electron2012; 4: 302–319.
30.
ChenWPengJChenJ, et al. Health-considered energy management strategy for fuel cell hybrid electric vehicle based on improved soft actor critic algorithm adopted with Beta policy. Energy Convers Manag2023; 292: 117362.
31.
LiQMengXGaoF, et al. Approximate cost-optimal energy management of hydrogen electric multiple unit trains using double Q-learning algorithm. IEEE Trans Ind Electron2022; 69: 9099–9110.
32.
YangDWangLYuK, et al. A reinforcement learning-based energy management strategy for fuel cell hybrid vehicle considering real-time velocity prediction. Energy Convers Manag2022; 274: 116453.
33.
SchaulTQuanJAntonoglouI, et al. Prioritized experience replay. arXiv preprint arXiv:1511.05952, 2015.
