A multi-agent reinforcement learning-based approach with lateral-longitudinal coupling for highway on-ramp merging control of autonomous vehicle

Abstract

To improve the safety and efficiency of autonomous vehicles when lane changing occur in the parallel-type on-ramp merging condition on the highway, this paper proposes a multi-agent deep reinforcement learning-based control model in considering the lateral and longitudinal dynamic-coupling to deal with complex high-density traffic flow scenarios. The model takes dynamic traffic state as input, and accurately controls the longitudinal acceleration and lateral front wheel angle of the autonomous vehicle through the dynamics coupling and state interaction between lateral and longitudinal agents. The kernel of this model is a Multi-Head Adaptive Attention Proximal Policy Optimization (MAAPPO) algorithm, which improves the network learning efficiency and driving policy stability. To satisfy the vehicle safety, comfort, and efficient driving requirements, this paper proposes a Multi-Dimensional Balanced Optimization of Reward Scheme (MDBORS), which achieves multi-dimensional control objective cooperation. To train and test the model, we constructed a highway ramp merging simulation experiment by using different traffic flow inputs. Results show that the proposed approach can generate efficient and safe control instructions. We also compare our approach with a state-of-the-art approach, our model has significant advantages in key metrics such as lane change time, trajectory length, and Time-To-Collision (TTC), verifying its excellent performance in the field of ramp merging control strategy for autonomous vehicles.