With the advancement of autonomous driving and vehicular networking technologies, platoon lane-changing (PLC) has become a research hotspot in intelligent transportation systems. This paper proposes a cooperative control model based on an end-to-end multi-agent deep meta-reinforcement learning (MADMRL) framework to address the technical challenges in PLC. The model considers the coupling effects among vehicles within the platoon, enabling precise control of longitudinal acceleration and lateral front-wheel steering angles. To improve training efficiency and learning outcomes, meta-learning is integrated with platoon dynamics models, proposing the Platoon-MMAPPO algorithm, which enhances model accuracy and accelerates policy network convergence. Additionally, a Platoon-Adaptive-Weight Reward Function (Platoon-Ada-Weight RF) is proposed to effectively guide the learning process, reduce unnecessary exploration, and accelerate the convergence to optimal policies. Highway simulation experiments and ablation studies validate the proposed model’s significant advantages in improving lane-changing efficiency, driving comfort, reducing road occupancy, and ensuring safety.
YuS-YFengY-YQuT, et al. A review of research on cooperative control of vehicle queues in smart grid environment. Control Decis Making. DOI: kzyjc/article/abstract/2023-1209
2.
ThormannSSchirrerAJakubekS.Safe and efficient cooperative platooning. IEEE Trans Intell Transp Syst2022; 23(2): 1368–1380.
3.
HsuHCHand Liu A. Platoon lane change maneuvers for automated highway systems. In: IEEE conference on robotics, automation and mechatronics, 2004, Singapore, 1–3 December 2004, pp.780–785. New York: IEEE.
4.
CheWWZhangLDengC, et al. Hierarchical lane-changing control for vehicle platoons in prescribed performance. Automatica2025; 171: 111972.
5.
JiangYTanLXiaoG, et al. Platoon-aware cooperative lane-changing strategy for connected automated vehicles in mixed traffic flow. Physica A2024; 640: 129689.
6.
NayakSHotaLNayakBP, et al. Dynamic lane change coordination and platoon management in vehicular ad hoc networks. In: 2024 3rd international conference for advancement in technology (ICONAT), Goa, India, 6–8 September 2024, pp.1–7. New York: IEEE.
7.
WangJTengFLiJ, et al. Intelligent vehicle lane change trajectory control algorithm based on weight coefficient adaptive adjustment. Adv Mech Eng2021; 13(3): 16878140211003393.
8.
ZhaoHGuoHSiS, et al. Automatic control method and performance analysis of platoon vehicles. J Northeastern Univ Nat Sci2018; 39(6): 781–786.
9.
ZhangZZhangLDengJ, et al. An enabling trajectory planning scheme for lane change collision avoidance on highways. IEEE Trans Intell Veh2023; 8(1): 147–158.
10.
GaoHZhaoMZhengX, et al. An improved hierarchical deep reinforcement learning algorithm for multi-intelligent vehicle lane change. Neurocomputing2025; 609: 128482.
11.
ChenYDongCPalanisamyP, et al. Attention-based hierarchical deep reinforcement learning for lane change behaviors in autonomous driving. In: 2019 IEEE/RSJ international conference on intelligent robots and systems (IROS), Macau, China, 3–8 November 2019, pp.3697–3703. New York: IEEE.
12.
CaiYZhanLSunX, et al. Research on obstacle avoidance strategy of automated heavy vehicle platoon in high-speed scenarios. Proc IMechE, Part D: J Automobile Engineering. Epub ahead of print 16September2024. DOI: 10.1177/09544070241276062.
13.
ChaeHJeongYLeeH, et al. Design and implementation of human driving data–based active lane change control for autonomous vehicles. Proc IMechE, Part D: J Automobile Engineering2021; 235(1): 55–77.
14.
LiuTLeiLZhengK, et al. Autonomous platoon control with integrated deep reinforcement learning and dynamic programming. IEEE Internet Things J2023; 10(6): 5476–5489.
15.
PengBSunQLiSE, et al. End-to-end autonomous driving through dueling double deep Q-network. Automot Innov2021; 4: 328–337.
16.
MalekiMTaghavipourAAzadiS.A real-time optimal cooperative lane change strategy via V2V communication. Proc IMechE, Part D: J Automobile Engineering2023; 237(13): 3094–3107.
17.
WanJLiuHXuM, et al. Lane-changing tracking control of automated vehicle platoon based on MA-DDPG and adaptive MPC. IEEE Access2024; 12: 58078–58096.
18.
ChenIMChanCY.Deep reinforcement learning based path tracking controller for autonomous vehicle. Proc IMechE, Part D: J Automobile Engineering2021; 235(2–3): 541–551.
19.
XavierPPanY-J. A practical PID-based scheme for the collaborative driving of automated vehicles. In: Proceedings of the 48th IEEE conference on decision and control (CDC) held jointly with 28th China control conference, Shanghai, China, 15–18 December 2009, pp.967–971. New York: IEEE.
20.
MohammadiMMalekiMTaghavipourA.Multi-parametric model predictive control for vehicle platoons. Proc IMechE, Part D: J Automobile Engineering. Epub ahead of print 12April2025. DOI: 10.1177/09544070251327233.
21.
LiuCLiuZXuZ, et al. A multistep cooperative lane change strategy for connected and autonomous vehicle platoons departing from dedicated lanes. Transp Res Part C Emerg Technol2024; 165: 104720.
22.
WeiZPLiuMMYaoMK, et al. Cooperative lane-changing control method for connected autonomous driving platoons. In: WangWLuGSiY (eds) International conference on green intelligent transportation system and safety. Singapore: Springer Nature Singapore, 2024, pp.483–498.
23.
XiaoLGaoFWangJ.On scalability of platoon of automated vehicles for leader-predecessor information framework. In: 2009 IEEE intelligent vehicles symposium, Xi’an, China, 3–5 June 2009, pp.1103–1108. New York: IEEE.
24.
ZhangHDuLShenJ.Hybrid MPC system for platoon based cooperative lane change control using machine learning aided distributed optimization. Transp Res Part B Methodol2022; 159: 104–142.
25.
WangHLaiJZhangX, et al. Make space to change lane: a cooperative adaptive cruise control lane change controller. Transp Res Part C Emerg Technol2022; 143: 103847.
26.
LiuDYangGH.Data-driven adaptive sliding mode control of nonlinear discrete-time systems with prescribed performance. IEEE Trans Syst Man Cybern Syst2019; 49(12): 2598–2604.
27.
XuJZhangJZhangR, et al. String stability guaranteed lane change maneuver for automated vehicles with vehicle-to-vehicle communication. IFAC Pap Online2021; 54(10): 330–335.
28.
LiuCMurpheyYL.Optimal power management based on Q-learning and neuro-dynamic programming for plug-in hybrid electric vehicles. IEEE Trans Neural Netw Learn Syst2020; 31(6): 1942–1954.
29.
ThorpeT. Multi-agent reinforcement learning: Independent vs. cooperative agents. Diss. Master's thesis, Department of Computer Science, Colorado State University, 1997, pp.330–337.
30.
XieGQZhaoJWLiY, et al. Cooperative lane-changing based on multi-cluster system. J Guangdong Univ Technol2021; 38(5): 1–9.
31.
FaragAAbdelAzizOM.HusseinA, et al. Reinforcement learning based approach for multi-vehicle platooning problem with nonlinear dynamic behavior. In Proceedings of the 34th International Conference on Neural Information Processing Systems, 2020, pp.1–10.
32.
SukhbaatarSFergusR.Learning multiagent communication with backpropagation. Adv Neural Inf Process Syst2016; 29: 2244–2252.
33.
SinghAJainTSukhbaatarS.Learning when to communicate at scale in multiagent cooperative and competitive tasks. arXiv preprint arXiv:1812.09755, 2018.
34.
LiMCaoZLiZ.A reinforcement learning-based vehicle platoon control strategy for reducing energy consumption in traffic oscillations. IEEE Trans Neural Netw Learn Syst2021; 32(12): 5309–5322.
35.
Hernandez-LealPKartalBTaylorME.A survey and critique of multiagent deep reinforcement learning. Auto Agents Multi Agent Syst2019; 33(6): 750–797.
36.
ChenJWuXLvZ, et al. Collaborative control of vehicle platoon based on deep reinforcement learning. IEEE Trans Veh Technol2024; 73(10): 14399–14414.
37.
LuRCaiYChenL, et al. Altruistic cooperative adaptive cruise control of mixed traffic platoon based on deep reinforcement learning. IET Intell Transp Syst2023; 17: 1951–1963.
38.
PrathibaSBRajaGDevK, et al. A hybrid deep reinforcement learning for autonomous vehicles smart-platooning. IEEE Trans Veh Technol2021; 70(12): 13340–13350.
39.
ZhaoJLiuHSunJ, et al. Deep reinforcement learning-based end-to-end control for UAV dynamic target tracking. Biomimetics2022; 7: 197.
