Q-Learning-Based Coordinated Variable Speed Limit and Hard Shoulder Running Control Strategy to Reduce Travel Time at Freeway Corridor

Abstract

To increase traffic mobility and safety, several types of active traffic management (ATM) strategies, such as variable speed limit (VSL), ramp metering, dynamic message signs, and hard shoulder running (HSR), are adopted in many countries. While all kinds of ATM strategies show promise in releasing traffic congestion, many studies indicate that stand-alone strategies have very limited capability. To remedy the defects of stand-alone strategies, cooperative ATM strategies have caught researchers’ attention and different combinations have been studied. In this paper, a coordinated VSL and HSR control strategy based on a reinforcement learning technique—Q-learning—is proposed. The proposed control strategy bridges up a direct connection between the traffic flow data and the ATM control strategies via intensive self-learning processes, thus reducing the need for human knowledge. A typical congested interstate highway, I-270 in Maryland, United States, was selected as the study area to evaluate the proposed strategy. A dynamic traffic assignment simulation model was introduced to calibrate the network with real-world data and was used to evaluate the regional impact of the proposed algorithm. Simulation results indicated that the proposed coordinated control could reduce corridor travel time by up to 27%. The performance of various control strategies were also compared. The results suggested that the proposed strategy outperformed the stand-alone control strategies and the traditional feedback-based VSL strategy in mitigating congestion and reducing travel time on the freeway corridor.