Automated Adaptive Traffic Corridor Control Using Reinforcement Learning

Abstract

Advancements in intelligent transportation systems and communication technology could considerably reduce delay and congestion through an array of networkwide traffic control and management strategies. The two most promising control tools for freeway corridors are traffic-responsive ramp metering and dynamic traffic diversion using variable message signs (VMSs). The use of these control methods independently could limit their usefulness. Therefore, integrated corridor control by using ramp metering and VMS diversion simultaneously could be beneficial. Administration of freeways and adjacent arterials often falls under different jurisdictional authorities. Lack of coordination among those authorities caused by lack of means for information exchange or “institutional gridlock” could hinder the full potential of technically possible integrated control. Fully automating corridor control could alleviate this problem. Research was conducted to develop a self-learning adaptive integrated freeway–arterial corridor control for both recurring and nonrecurring congestion. Reinforcement learning, an artificial intelligence method for machine learning, is used to provide a single, multiple, or integrated optimal control agent for a freeway or freeway–arterial corridor for both recurrent and nonrecurrent congestion. The microsimulation tool Paramics, which has been used to train and evaluate the agent in an offline mode within a simulated environment, is described. Results from various simulation case studies in the Toronto, Canada, area are encouraging and have demonstrated the effectiveness and superiority of the technique.

Get full access to this article

View all access options for this article.

References

Papageorgiou

An Integrated Control Approach for Traffic Corridors. Transportation Research Part C, Vol. 3, 1995, pp. 19–30.

Kotsialos

Papageorgiou

Mangeas

and Haj-Salem

Coordinated and Integrated Control of Motorway Networks via Nonlinear Optimal Control. Transportation Research Part C, Vol. 10, 2002, pp. 65–84.

Logi

and Ritchie

S. G.

Development and Evaluation of a Knowledge-Based System for Traffic Congestion Management and Control. Transportation Research Part C, Vol. 9, 2001, pp. 433–459.

Watkins

C. J. C. H.

Learning from Delayed Rewards. PhD thesis. King's College, University of Cambridge, Cambridge, U.K., 1989.

Modeller and Programmer V3.0 User Guide and Manual. Quadstone Ltd., Edinburgh, Scotland, 2000.

Mitchell

Machine Learning. McGraw-Hill, New York, 1997.

Sutton

R. S.

and Barto

A. G.

Reinforcement Learning—An Introduction. MIT Press, Cambridge, Mass., 1998.

Gasser

Computation in Cognitive Science. Indiana University. www.Indiana.edu/∼q320/Notes/rl-explore.html. Accessed Nov. 2004.

Miller

W. T.

and Glanz

F. H.

The University of New Hampshire Implementation of the Cerebellar Arithmetic Controller—CMAC. Robotic Laboratory, University of New Hampshire, Durham, 1996.

10.

Darken

Chang

and Moody

Learning Rate Schedules for Faster Stochastic Gradient Search, in Neural Networks for Signal Processing II (Fallside Sorenson Kung and Kamm, eds.). IEEE, Piscataway New Jersey, 1992.

11.

Papageorgiou

Diakaki

Dinopoulou

Kotsialos

and Wang

Review of Road Traffic Control Strategies. Proceedings of the IEEE, Vol. 91, No. 12, Dec. 2003.

12.

Papageorgiou

Hadj-Salem

and Blosseville

J.-M.

ALINEA: A Local Feedback Control Law for On-Ramp Metering. In Transportation Research Record 1320, TRB, National Research Council, Washington, D.C., 1991, pp. 58–64.