A dueling double deep Q-network with style-based recalibration module for traffic light optimization in deep reinforcement learning

Abstract

With the acceleration of urbanization and the increase in car ownership, urban traffic congestion has become an increasingly severe problem. Existing traffic light control methods based on fixed-time and inductive control are unable to adapt to the dynamic changes in large-scale urban traffic flows. Moreover, many deep reinforcement learning algorithms do not adequately consider the impact of vehicles near intersections on traffic flow. This paper introduces the SRM attention mechanism module into the D3QN algorithm, namely SRM-D3QN. By leveraging the SRM attention mechanism, which assigns higher weights to vehicles near intersections, the model can better focus on the traffic conditions in these critical areas. In the SRM-D3QN algorithm, the extracted features are fed into the SRM attention module. The SRM module assigns different weights to each feature based on their spatial relevance and importance. Specifically, the SRM assigns higher weights to the states of vehicles near intersections to ensure that the model pays more attention to these key regions. Simulation experiments were conducted in the traffic simulation software SUMO, using different traffic flow environments at a single intersection model. The performance of the SRM-D3QN algorithm outperforms the selected comparison algorithms and demonstrates faster convergence speed, proving the effectiveness of the algorithm’s performance and its faster learning speed.