Reinforcement learning-based controller for path tracking of an autonomous vehicle

Abstract

Path tracking controllers are crucially important in the performance and safety of autonomous vehicles. These controllers’ main duty is enabling the vehicle to accurately follow a predefined trajectory while maintaining stability and avoiding deviations, which becomes even more challenging in varying road and weather conditions, where maintaining accuracy and safety is critical. The controller must be able to handle sharp turns and adapt to varying road conditions, such as icy or rainy roads, which plays a significant role in assessing the efficacy of a path-tracking controller. The PID controller is widely used for its acceptable performance with fixed gains, while reinforcement learning-based controllers, such as the extensively used PPO algorithm, offer adaptive and accurate results through environmental interaction. However, when considering RL controllers, we face two major challenges: parameter tuning and fluctuating control signals. To address these issues, we propose a novel combination. One modifies the neural network structure, and the other incorporates two alterations in the algorithm’s loss function, resulting in parameter robustness and reduction in control signal fluctuations. Then, we evaluate and compare the performance of the PID controller and the RL-based controller. Our comparison focuses on path-tracking accuracy, highlighting the trade-offs between traditional and adaptive control methods under challenging conditions. Initial results show that the RL-based controller performs worse than the PID with fixed gains and reference speed, while the fixed PID lacks high path-tracking accuracy. Moreover, we propose a hybrid approach where RL optimizes the PID controller for adaptive gain and reference speed adjustment. Results show that while the fixed PID fails in low-friction conditions, the PID-RL combination outperforms each method alone, achieving superior performance in both normal and challenging road conditions. These results illustrate the potential of hybrid approaches in optimizing control systems for autonomous vehicles, particularly in enhancing path-tracking performance under varying road conditions.