Deep Reinforcement Learning Technique for Traffic Metering in Connected Urban Street Networks

Abstract

Proper metering can improve traffic operations in congested urban street networks. The available approaches either (a) use macroscopic fundamental diagrams to model traffic dynamics or (b) use numerical time–space discretization of the hydrodynamic traffic flow model with high computational requirements. Therefore, they either (a) do not represent traffic dynamics accurately or (b) are not suitable for online applications. This study introduces a deep reinforcement learning (DRL) methodology to capture traffic dynamics on a micro-level scale with the capability of capturing detailed traffic dynamics with low computational time. The DRL methodology employs two neural networks that map the location of connected vehicles in a network to traffic metering signal indications and estimate the objective function of the traffic metering problem. The outputs of the neural networks are used to construct a loss function, whose optimization provides the optimal parameters for the neural networks. Because of the complexity of the loss function, the gradient descent optimization technique with Monte Carlo simulation is used to optimize the loss function. The proposed methodology was tested on a simulated case study network in Vissim software with 20 intersections. The numerical results showed that the methodology increased throughput by 41.2% and 21.3% and reduced the total travel time of vehicles by 3.4% and 15.5% compared to a no-metering strategy. Comparing the computational time of the proposed methodology with one of the existing traffic metering approaches also showed the potential of the methodology for online applications.