Research on Intelligent Signal Timing Optimization of Signalized Intersection Based on Deep Reinforcement Learning Using Floating Car Data

Abstract

Adaptive signal control is a method that dynamically adjusts signal phases based on real-time traffic conditions, aiming to improve the efficiency of intersection light control. However, in practical applications, obtaining complete traffic spatiotemporal states at intersections is challenging given the limited data sources. To address this issue, this paper presents a novel deep reinforcement learning model, namely 3DQN-PSTER, which combines Dueling Deep Q Network, Double Deep Q Network, and Priority SumTree Experience Replay technology. This model effectively extracts traffic states from sampled floating car data (FCD) and achieves optimal timing schemes. To evaluate its performance, the proposed model is tested in a simulated intersection environment using actual intersection data. The model interacts with the environment by adjusting signal phases, updating traffic states, and obtaining optimal timing schemes. Simulation results demonstrate that the proposed model exhibits higher convergence efficiency and outperforms benchmark models. When applied, the reinforcement learning single control (RLSC) based on the 3DQN-PSTER model outperforms fixed-time signal control schemes and actuated signal control (ASC) schemes. Even in scenarios with low penetration ratios of FCD, RLSC still significantly outperforms ASC. Therefore, the signal timing optimization method proposed in this paper could greatly enhance intersection operation efficiency in both ideal overall data environments and partial states.