The integration of connected and autonomous vehicle (CAV) technologies with advanced deep reinforcement learning (DRL) techniques has opened new frontiers in vehicular energy efficiency optimization. This study presents a novel hierarchical energy management strategy (EMS) framework designed to address the dual challenges of driving performance enhancement and fuel economy optimization for fuel cell hybrid electric vehicles (FCHEV) operating in car-following scenarios. The proposed framework comprises two sequential components. Initially, a hybrid CNN-BiLSTM-Attention neural network is developed to generate precise velocity predictions for the preceding vehicle. These prediction results are then utilized to establish a preliminary predictive cruise control (PCC) strategy for maintaining appropriate following speeds. Subsequently, the twin delayed deep deterministic policy gradient (TD3) algorithm is implemented to optimize the power distribution among the FCHEV’s multiple energy sources, utilizing the planned speed trajectory as input. Extensive experimental evaluations confirm the system’s effectiveness in maintaining battery state-of-charge (SOC) within stable ranges while simultaneously achieving superior fuel efficiency and demonstrating robust performance across diverse driving cycles.
HuangYHuHTanJ, et al. Deep reinforcement learning based energy management strategy for range extend fuel cell hybrid electric vehicle. Energy Convers Manag2023; 277: 116678.
2.
LiuZMaoXKangZ, et al. Multi-objective optimization of ice distribution uniformity and startup failure time for cold start of PEMFC catalyst layer based on CTnet and PSO-GA. Fuel2024; 363: 130922.
3.
HuangYKangZMaoX, et al. Deep reinforcement learning based energy management strategy considering running costs and energy source aging for fuel cell hybrid electric vehicle. Energy2023; 283: 129177.
4.
WuYHuangZHofmannH, et al. Hierarchical predictive control for electric vehicles with hybrid energy storage system under vehicle-following scenarios. Energy2022; 251: 123774.
5.
de FuentesJMGonzález-ManzanoLGonzález-TablasAI, et al. Security models in vehicular ad-hoc networks: a survey. IETE Tech Rev2014; 31(1): 47–64.
6.
NdashimyeERaySKSarkarNI, et al. Vehicle-to-infrastructure communication over multi-tier heterogeneous networks: a survey. Comput Netw2017; 112: 144–166.
7.
MaoJYangLHuY, et al. Research on vehicle adaptive cruise control method based on fuzzy model predictive control. Machines2021; 9: 160.
8.
MakridisMMattasKAnesiadouA, et al. OpenACC. An open database of car-following experiments to study the properties of commercial ACC systems. Transp Res Part C Emerg Technol2021; 125: 103047.
9.
AsadiBVahidiA.Predictive cruise control: utilizing upcoming traffic signal information for improving fuel economy and reducing trip time. IEEE Trans Control Syst Technol2011; 19(3): 707–714.
10.
HeDHeWSongX.Efficient predictive cruise control of autonomous vehicles with improving ride comfort and safety. Meas Control2020; 53(1–2): 18–28.
11.
LiCXuXZhuH, et al. Research on car-following control and energy management strategy of hybrid electric vehicles in connected scene. Energy2024; 293: 130586.
12.
YangHHuYGongX, et al. Energy management strategy for fuel cell hybrid electric vehicles considering the inaccuracy of predicted vehicle speed. IEEE Trans Transp Electrif2024; 10(4): 8246–8262.
13.
AbduljabbarRLDiaHTsaiPW.Unidirectional and bidirectional LSTM models for short-term traffic prediction. J Adv Transp2021; 2021: 5589075.
14.
ChengZLuJZhouH, et al. Short-term traffic flow prediction: an integrated method of econometrics and hybrid deep learning. IEEE Trans Intell Transp Syst2021; 23(6): 5231–5244.
15.
WangYWuYTangY, et al. Cooperative energy management and eco driving of plug-in hybrid electric vehicle via multi-agent reinforcement learning. Appl Energy2023; 332: 120563.
16.
TangXChenJYangK, et al. Visual detection and deep reinforcement learning-based car following and energy management for hybrid electric vehicles. IEEE Trans Transp Electrif2022; 8(2): 2501–2515.
17.
WangSLinX.Eco-driving control of connected and automated hybrid vehicles in mixed driving scenarios. Appl Energy2020; 271: 115233.
18.
TangWWangYJiaoX, et al. Hierarchical energy management strategy based on adaptive dynamic programming for hybrid electric vehicles in car-following scenarios. Energy2023; 265: 126264.
19.
LiuYHuangBYangY, et al. Hierarchical speed planning and energy management for autonomous plug-in hybrid electric vehicle in vehicle-following environment. Energy2022; 260: 125212.
20.
ZhouYHuangYMaoX, et al. Research on energy management strategy of fuel cell hybrid power via an improved TD3 deep reinforcement learning. Energy2024; 293: 130564.
21.
HuangXZhangJOuK, et al. Deep reinforcement learning-based health-conscious energy management for fuel cell hybrid electric vehicles in model predictive control framework. Energy2024; 304: 131769.
22.
TangXZhouHWangF, et al. Longevity-conscious energy management strategy of fuel cell hybrid electric vehicle based on deep reinforcement learning. Energy2022; 238: 121593.
23.
LillicrapTP.Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971, 2015.
24.
WangJDuCYanF, et al. Hierarchical rewarding deep deterministic policy gradient strategy for energy management of hybrid electric vehicles. IEEE Trans Transp Electrif2023; 10(1): 1802–1815.
25.
WuCRuanJCuiH, et al. The application of machine learning-based energy management strategy in multi-mode plug-in hybrid electric vehicle, part I: twin delayed deep deterministic policy gradient algorithm design for hybrid mode. Energy2023; 262: 125084.
26.
QiYEspinoza-AndaluzMThernM, et al. Dynamic modelling and controlling strategy of polymer electrolyte fuel cells. Int J Hydrogen Energy2020; 45(54): 29718–29729.
27.
ZhengCOhCParkY, et al. Fuel economy evaluation of fuel cell hybrid vehicles based on equivalent fuel consumption. Energy Fuels2012; 37(2): 1790–1796.
28.
SunZWangYChenZ, et al. Min-max game based energy management strategy for fuel cell/supercapacitor hybrid electric vehicles. Appl Energy2020; 267: 115086.
29.
ChenHPeiPSongM.Lifetime prediction and the economic lifetime of proton exchange membrane fuel cells. Appl Energy2015; 142: 154–163.
30.
JohnsonVH.Battery performance models in ADVISOR. J Power Sources2002; 110(2): 321–329.
31.
WangYZhangYZhangC, et al. Genetic algorithm-based fuzzy optimization of energy management strategy for fuel cell vehicles considering driving cycles recognition. Energy2023; 263: 124585.
32.
LiuYLiJChenZ, et al. Research on a multi-objective hierarchical prediction energy management strategy for range extended fuel cell vehicles. J Power Sources2019; 429: 55–66.
33.
ChenSHuangYZhangJ, et al. Research on a novel multi-agent deep reinforcement learning eco-driving framework. Energy2025; 326: 136308.
34.
OldhamK.A Gouy–Chapman–Stern model of the double layer at a (metal)/(ionic liquid) interface. J Electroanal Chem2008; 613(2): 131–138.
35.
JiaCLiKHeH, et al. Health-aware energy management strategy for fuel cell hybrid bus considering air-conditioning control based on TD3 algorithm. Energy2023; 283: 128462.
36.
ZouWLiJYangQ, et al. A real-time energy management approach with fuel cell and battery competition-synergy control for the fuel cell vehicle. Appl Energy2023; 334: 120667.
