Cooperative adaptive cruise control system for electric vehicles through a predictive deep reinforcement learning approach

Abstract

In this paper, a predictive deep reinforcement learning-based control algorithm is introduced for the cooperative adaptive cruise control (CACC) system. The proposed control algorithm achieves an optimal response in the interaction of the host vehicle with the leading vehicles. For this purpose, at first, a new algorithm based on deep neural networks is proposed for modeling and predicting the driving behavior of the leading vehicle, according to the traffic behavior of its surrounding vehicles. Then, the outputs of the predictive model are used as additional states for the upper-level controller of the CACC system, which is designed based on a deep reinforcement learning approach. Therefore, the proposed controller uses conventional states such as relative distance and speed of the host vehicle along with the leading vehicles and employs the speed profile of the leading vehicle in the future time horizons to train the reinforcement learning agent. Thus, the upper-level controller learning process is improved. In addition, for regulating the lower-level controller of CACC system against parametric uncertainties in electric vehicle dynamics, a deep reinforcement learning-based controller is proposed, which can be retuned according to the real-time estimated parameters of vehicle mass and road grade. Simulation results represent the promising performance of the proposed control algorithm for maintaining a balance between travel safety, passenger comfort, and reducing vehicle energy consumption.