Multi-objective optimization of PV-powered EV charging stations: Minimizing energy costs and extending battery lifespan using DQN and DDQN reinforcement learning

Abstract

This paper addresses optimal power management for Photovoltaic (PV)-powered Electric Vehicle Charging Stations (EVCS) integrated within a Microgrid (MG) architecture, which includes a PV generator, a Battery Energy Storage System (BESS), five EV charging terminals, and the connection with the power grid. A multi-objective optimization problem is formulated to simultaneously minimize the total energy cost and extend BESS lifespan, considering constraints such as dynamic electricity pricing, PV intermittency, EV demand variability, State of Health (SOH) degradation, and physical system limitations. The optimization problem is solved using Deep Q-Network (DQN) and Double DQN (DDQN) Reinforcement Learning (DRL) algorithms. For battery health management, a Long Short-Term Memory (LSTM) network is employed to predict the SOH, serving as a basis for evaluating the effectiveness of DRL-based strategies in extending battery lifespan over a three-year horizon. Simulation results show that DDQN outperforms DQN in both slow (Scenario 1) and fast (Scenario 2) charging modes. In Scenario 1, DDQN achieves 82.71% PV utilization with a cost reduction to −384.7 c€/day, compared to 82.01% and −110.1 c€/day for DQN. In Scenario 2, DDQN reaches 53.12% PV usage and −77.06 c€/day of energy cost, while DQN attains 52.58% and −3.85 c€/day. Negative cost values indicate net economic benefits. Additionally, DDQN reduces battery degradation to 5.66%, whereas DQN shows negligible degradation compared to the case without optimization. Grid dependency is also lower with DDQN (11.62%) compared to DQN (17.98%). These findings demonstrate that the proposed DRL approach, incorporating SOH awareness, improves energy efficiency and extends battery lifespan in PV-based EVCS.