This paper investigates the path-following control of autonomous vehicle by proposing a dual-controllers deep deterministic policy gradient. Dual-controllers match with the lateral and longitudinal control of vehicle, respectively. The main contributions are threefold: (1) We consider six lateral and three longitudinal state variates separately, aiming to reduce lateral and longitudinal coupling. (2) A dual-controllers is designed correspondingly to control lateral steering angle and longitudinal acceleration. It utilizes a shared experience replay pool to achieve efficient training. (3) A novel dual-stage reward mechanism is designed for the two controllers, which combines direct error feedback and indirect incremental guidance to balance lateral tracking accuracy and longitudinal safety coordination. This design yields better training performance compared to a simple reward function. Finally, MATLAB is employed for experimental verification, while CarSim simulations are utilized for physical validation. The simulation results show that it outperforms other methods.
RizkHChaibetAKribècheA.Model-based control and model-free control techniques for autonomous vehicles: a technical survey. Appl Sci2023; 13(11): 6700.
2.
NguyenATSentouhCPopieulJC.Fuzzy steering control for autonomous vehicles under actuator saturation: design and experiments. J Franklin Inst2018; 355(18): 9374–9395.
3.
JungHJungDChoiSB.Lqr control of an all-wheel drive vehicle considering variable input constraint. IEEE Trans Control Syst Technol2022; 30(1): 85–96.
4.
LiTBaiWLiuQ, et al. Distributed fault-tolerant containment control protocols for the discrete-time multiagent systems via reinforcement learning method. IEEE Trans Neural Netw Learn Syst2023; 34(8): 3979–3991.
5.
BaiWLiTLongY, et al. Event-triggered multigradient recursive reinforcement learning tracking control for multiagent systems. IEEE Trans Neural Netw Learn Syst2023; 34(1): 366–379.
6.
LiDZhaoDZhangQ, et al. Reinforcement learning and deep learning based lateral control for autonomous driving [application notes]. IEEE Comput Intell Mag2019; 14(2): 83–98.
7.
MaQYinXZhangX, et al. Game-theoretic receding-horizon reinforcement learning for lateral control of autonomous vehicles. IEEE Trans Vehicular Technol2024; 73(10): 14547–14562.
8.
YuWHuaXWangW.Investigating the longitudinal impact of cooperative adaptive cruise control vehicle degradation under communication interruption. IEEE Intell Transp Syst Mag2022; 14(4): 183–201.
9.
WangHZuoZXueW, et al. Switching longitudinal and lateral semi-decoupled active disturbance rejection control for unmanned ground vehicles. IEEE Trans Ind Electron2024; 71(3): 3034–3043.
10.
HuangZXuXHeH, et al. Parameterized batch reinforcement learning for longitudinal control of autonomous land vehicles. IEEE Trans Syst Man Cybern Syst2019; 49(4): 730–741.
11.
BaiLZhengFHouK, et al. Longitudinal control of automated vehicles: a novel approach by integrating deep reinforcement learning with intelligent driver model. IEEE Trans Vehicular Technol2024; 73(8): 11014–11028.
12.
WijnbergenPJeeningaMde HaanR, et al. Longitudinal and lateral control of vehicle platoons: a unifying framework to prevent corner cutting. Automatica2025; 177: 112340.
13.
JingSHuiFZhaoX, et al. Integrated longitudinal and lateral hierarchical control of cooperative merging of connected and automated vehicles at on-ramps. IEEE Trans Intell Transp Syst2022; 23(12): 24248–24262.
14.
LuYBiL.Combined lateral and longitudinal control of eeg signals-based brain-controlled vehicles. IEEE Trans Neural Syst Rehabil Eng2019; 27(9): 1732–1742.
15.
GaoHZhaoMZhengX, et al. An improved hierarchical deep reinforcement learning algorithm for multi-intelligent vehicle lane change. Neurocomputing2024; 609: 128482.
16.
MoSPeiXWuC.Safe reinforcement learning for autonomous vehicle using monte carlo tree search. IEEE Trans Intell Transp Syst2022; 23(7): 6766–6773.
17.
JiangLXieYEvansNG, et al. Reinforcement learning based cooperative longitudinal control for reducing traffic oscillations and improving platoon stability. Transp Res Part C: Emerg Technol2022; 141: 103744.
18.
FuYLiCYuFR, et al. A decision-making strategy for vehicle autonomous braking in emergency via deep reinforcement learning. IEEE Trans Vehicular Technol2020; 69(6): 5876–5888.
19.
WangYCaoJSunJ, et al. Path following control for unmanned surface vehicles: a reinforcement learning-based method with experimental validation. IEEE Trans Neural Netw Learn Syst2024; 35(12): 18237–18250.
20.
DjerbiRRouaneATalebZ, et al. Design and implementation of a self-driving car using deep reinforcement learning: a comprehensive study. Comput Ind Eng2025; 207: 111319.
21.
LiJGuoZJiangY, et al. An integrated model for coordinating adaptive platoons and parking decision-making based on deep reinforcement learning. Comput Ind Eng2025; 203: 110962.
22.
LuoLZhaoNZhuY, et al. A* guiding dqn algorithm for automated guided vehicle pathfinding problem of robotic mobile fulfillment systems. Comput Ind Eng2023; 178: 109112.
23.
HouXZhangJHeC, et al. Crash mitigation controller for unavoidable t-bone collisions using reinforcement learning. ISA Trans2022; 130: 629–654.
24.
GhafariRMansouriN.Reinforcement learning-based solution for resource management in fog computing: a comprehensive survey. Expert Syst Appl2025; 276: 127214.
25.
ZhangZQHuangTQianB, et al. Mamhsan: a multi-agent deep reinforcement learning framework based on multi-head self-attention network with heterogeneous graph embedding for flexible job shop scheduling. Comput Ind Eng2025; 210: 111466.
26.
LiCLinLQiuX, et al. Evolutionary experience-guided deep reinforcement learning for job shop scheduling with agv transportation constraints in flexible manufacturing system. Comput Ind Eng2025; 209: 111465.
27.
TianYCaoXHuangK, et al. Learning to drive like human beings: a method based on deep reinforcement learning. IEEE Trans Intell Transp Syst2022; 23(7): 6357–6367.
28.
CaponioCStanoPCarliR, et al. Modeling, positioning, and deep reinforcement learning path following control of scaled robotic vehicles: design and experimental validation. IEEE Trans Autom Sci Eng2025; 22: 9856–9871.
29.
NiHYuGChenP, et al. An integrated framework of lateral and longitudinal behavior decision-making for autonomous driving using reinforcement learning. IEEE Trans Vehicular Technol2024; 73(7): 9706–9720.
30.
RenJGuoSChenF.Orientation-preserving rewards’ balancing in reinforcement learning. IEEE Trans Neural Netw Learn Syst2022; 33(11): 6458–6472.
31.
CuiJZhaoBQuM.An integrated lateral and longitudinal decision-making model for autonomous driving based on deep reinforcement learning. J Adv Transport2023; 2023: 1–13.
32.
AcquaroneMMainoCMisulD, et al. Influence of the reward function on the selection of reinforcement learning agents for hybrid electric vehicles real-time control. Energies2023; 16(6): 2749.
33.
YuCNiALuoJ, et al. A novel dynamic lane-changing trajectory planning model for automated vehicles based on reinforcement learning. J Adv Transport2022; 2022(1): 1–16.
34.
RajamaniR.Vehicle dynamics and control. Springer, 2012.
35.
AradiS.Survey of deep reinforcement learning for motion planning of autonomous vehicles. IEEE Trans Intell Transp Syst2022; 23(2): 740–759.
36.
ZhangXWuLLiuH, et al. High-speed ramp merging behavior decision for autonomous vehicles based on multiagent reinforcement learning. IEEE Internet Things J2023; 10(24): 22664–22672.
37.
DangVTMacDLLeDT, et al. An online reinforcement learning-based conflict resolution approach for human-vehicle cooperative systems. ISA Trans2026; 169: 391–407. https://doi.org/10.1016/j.isatra.2025.12.017
38.
RoyABorkarVKarandikarA, et al. Online reinforcement learning of optimal threshold policies for markov decision processes. IEEE Trans Automat Contr2022; 67(7): 3722–3729.
39.
PeiHZhangJZhangY, et al. Self-organized routing for autonomous vehicles via deep reinforcement learning. IEEE Trans Vehicular Technol2024; 73(1): 426–437.
40.
WangYLinnettJARobertsJW.Motion feasibility of a wheeled vehicle with a steering angle limit. Robotica1994; 12(3): 217–226.
41.
ChenJZhuBLingX, et al. A multi-agent reinforcement learning-based approach with lateral-longitudinal coupling for highway on-ramp merging control of autonomous vehicle. Proc Inst Mech Eng Part D: J Automobile Eng2025. https://doi.org/10.1177/09544070251362075
42.
WangSWangZJiangR, et al. A multi-agent reinforcement learning-based longitudinal and lateral control of cavs to improve traffic efficiency in a mandatory lane change scenario. Transp Res Part C: Emerg Technol2024; 158: 104445.
43.
NishimuraY.Stabilization by artificial wiener processes. IEEE Trans Automat Contr2016; 61(11): 3574–3579.
44.
YangZMerrickKJinL, et al. Hierarchical deep reinforcement learning for continuous action control. IEEE Trans Neural Netw Learn Syst2018; 29(11): 5174–5184.
45.
ZouaidiaKRaisMSBouguerouaL. Upgraded decision making in continuous domains for autonomous vehicles in high complexity scenarios using escalated ddpg. Appl Intell2025; 55: 638.
