Attention network-based reinforcement learning decision-making algorithm for unprotected left-turn maneuvers at intersections

Abstract

Autonomous vehicles (AVs) face challenges in unprotected left-turn maneuvers at intersections due to insufficient recognition of complex traffic scenarios and low traversal efficiency. This paper proposes a novel decision-making framework to address these issues. First, an improved multi-head cross-attention network, termed the Addego Multi-Head Cross-Attention (AEMA) Network, is introduced to tackle the difficulty of extracting critical information in complex traffic scenarios. By reconstructing the input matrix and incorporating ego-vehicle state information into the Key and Value matrices, this network achieves a rational allocation of attention weights, making it effectively applicable to intersection scenarios. Second, to circumvent the need for supervised data to train the attention network, an interactive simulation environment is constructed. The attention network is embedded within the actor network of the reinforcement learning (RL) framework to directly output continuous actions, while a Multi-Layer Perceptron (MLP) is employed to train the critic network. Finally, addressing the issue of difficult training convergence caused by the complex state space in unprotected left-turn scenarios at four-way intersections, conflict points are incorporated into the state space and discretized, which significantly accelerates the training convergence speed. Experimental results demonstrate that the proposed method significantly improves the accuracy of decision making in complex scenarios. Under the premise of maintaining a low collision rate, the average traffic speed is increased by at least 10%, providing an effective solution for the efficient navigation of autonomous vehicles in complex traffic environments.