In modern transportation systems, dynamic platooning is a promising approach, leveraging adaptable vehicle formations to optimize traffic flow and fuel efficiency. However, relying on periodic broadcasts to preserve string stability during platoon maneuvers poses a challenge, demanding substantial resources within a constrained spectrum. This paper presents an event-triggered message scheduling that uses deep reinforcement learning specifically tailored for a dynamic platoon of autonomous vehicles on highways. In the proposed approach, the platoon leader intelligently triggers communication instances only when platoon speed variations threaten string stability. Furthermore, this study introduces a categorization system for triggered messages, distinguishing between safety and non-safety messages, ensuring their efficient storage and prioritized transmission. Simulation results demonstrate the efficacy of the trained leader in maintaining string stability, even amidst joining and leaving maneuvers, achieved through reduced communication frequency. Additionally, shorter waiting time for safety messages within the priority buffer guarantees reliable and expedited communication of critical information.
LeschV.BreitbachM.SegataM.BeckerC.KounevS.KrupitzerC.An Overview on Approaches for Coordination of Platoons. IEEE Transactions on Intelligent Transportation Systems, Vol. 23, No. 8, Aug.2022, pp. 10049–10065. https://doi.org/10.1109/TITS.2021.3115908.
2.
MaitiS.WinterS.KulikL.SarkarS.The Impact of Flexible Platoon Formation Operations. IEEE Transactions on Intelligent Vehicles, Vol. 5, No. 2, June2020, pp. 229–239. https://doi.org/10.1109/TIV.2019.2955898.
3.
JiaD.LuK.WangJ.ZhangX.ShenX.A Survey on Platoon-Based Vehicular Cyber-Physical Systems. IEEE Communications Surveys and Tutorials, Vol. 18, No. 1, 2016, pp. 263–284. https://doi.org/10.1109/COMST.2015.2410831.
4.
AgbajeP.AnjumA.MitraA.OseghaleE.BloomG.OlufowobiH.Survey of Interoperability Challenges in the Internet of Vehicles. IEEE Transactions on Intelligent Transportation Systems, Vol. 23, no. 12, Dec.2022, pp. 22838–22861. https://doi.org/10.1109/TITS.2022.3194413.
5.
MushtaqA.KhanZ.KoubaaA.BoulilaW.A Microscopic Platoon Stability Model Using Vehicle-to-Vehicle Communication. Electronics, Vol. 11, No. 13, 2022, p. 1994. https://doi.org/10.3390/electronics11131994.
6.
ChinW. H.FanZ.HainesR.Emerging Technologies and Research Challenges for 5G Wireless Networks. IEEE Wireless Communications, Vol. 21, No. 2, April2014, pp. 106–112. https://doi.org/10.1109/MWC.2014.6812298.
7.
ETSI TR 137 985 V16.0.0 (2020-07) LTE, 5G. Overall description of Radio Access Network (RAN), Aspects for Vehicle-to-Everything (V2X) Based on LTE and NR. (3GPP TR 37.985 version 16.0.0 Release 16).
8.
KamalM. A.RazaH. W.AlamM. M.Su’udM. M.BakarA. A.Resource Allocation Schemes for 5G Network: A Systematic Review. Sensors, Vol. 21, No. 19, 2021, p. 6588. https://doi.org/10.3390/s21196588.
9.
HegdeS.BlumeO.ShrivastavaR.BakkerH.Enhanced Resource Scheduling for Platooning in 5G V2X Systems. IEEE 2nd 5G World Forum (5GWF), Dresden, 2019, pp. 108–113. https://doi.org/10.1109/5GWF.2019.8911647.
10.
GunterG.JanssenC.BarbourW.SternR. E.WorkD. B.Model-Based String Stability of Adaptive Cruise Control Systems Using Field Data. IEEE Transactions on Intelligent Vehicles, Vol. 5, No. 1, March2020, pp. 90–99. https://doi.org/10.1109/TIV.2019.2955368.
11.
KhoundP.WillP.GronwaldF.Design Methodology to Derive Over-Damped String Stable Adaptive Cruise Control Systems. IEEE Transactions on Intelligent Vehicles, Vol. 7, No. 1, March2022, pp. 32–44. https://doi.org/10.1109/TIV.2021.3066056.
12.
FengS.SongZ.LiZ.ZhangY.LiL.Robust Platoon Control in Mixed Traffic Flow Based on Tube Model Predictive Control. IEEE Transactions on Intelligent Vehicles, Vol. 6, No. 4, Dec.2021, pp. 711–722. https://doi.org/10.1109/TIV.2021.3060626.
13.
LiangH.ZhouS.LiuX.ZhengF.HongX.ZhouX.ZhaoL.A Dynamic Resource Allocation Model Based on SMDP and DRL Algorithm for Truck Platoon in Vehicle Network. IEEE Internet of Things Journal, Vol. 9, No. 12, 2022, pp. 10295–10305. https://doi.org/10.1109/JIOT.2021.3123811.
14.
WenS.GuoG.ChenB.GaoX.Cooperative Adaptive Cruise Control of Vehicles Using a Resource-Efficient Communication Mechanism. IEEE Transactions on Intelligent Vehicles, Vol. 4, No. 1, March2019, pp. 127–140. https://doi.org/10.1109/TIV.2018.2886676.
15.
GeX.XiaoS.HanQ.L.ZhangX. M.DingD.Dynamic Event-Triggered Scheduling and Platooning Control Co-Design for Automated Vehicles Over Vehicular Ad-Hoc Networks. IEEE/CAA Journal of Automatica Sinica, Vol. 9, No. 1, January2022, pp. 31–46. https://doi.org/10.1109/JAS.2021.1004060.
16.
DongZ.ZhuX.JiangY.ZengH.WeiZ.ZhengF. C.LeungK. C.Dynamic Manager Selection Assisted Resource Allocation in URLLC With Finite Block Length for 5G-V2X Platoons. IEEE Transactions on Vehicular Technology, Vol. 71, No. 11, Nov.2022, pp. 11336–11350. https://doi.org/10.1109/TVT.2022.3190518.
17.
CaoL.YinH.Resource Allocation for Vehicle Platooning in 5G NR-V2X via Deep Reinforcement Learning. IEEE International Black Sea Conference on Communications and Networking (BlackSeaCom), Bucharest, Romania, 2021, pp. 1–7. https://doi.org/10.1109/BlackSeaCom52164.2021.9527765.
18.
FarzanullahM.Le-NgocT.Platoon Leader Selection, User Association and Resource Allocation on a C-V2X Based Highway: A Reinforcement Learning Approach. ICC 2023 – IEEE International Conference on Communications, Rome, Italy, 2023, pp. 5396–5401. https://doi.org/10.1109/ICC45041.2023.10279056.
19.
VuH. V.FarzanullahM.LiuZ.NguyenD. H. N.MorawskiR.Le-NgocT.Multi-Agent Reinforcement Learning for Channel Assignment and Power Allocation in Platoon-Based C-V2X Systems. IEEE 95th Vehicular Technology Conference (VTC2022-Spring), Helsinki, Finland, 2022, pp. 1–5. https://doi.org/10.1109/VTC2022-Spring54318.2022.9860518
20.
WangP.DiB.ZhangH.BianK.SongL.Platoon Cooperation in Cellular V2X Networks for 5G and Beyond. IEEE Transactions on Wireless Communications, Vol. 18, No. 8, Aug.2019, pp. 3919–3932. https://doi.org/10.1109/TWC.2019.2919602.
21.
YeH.Ye LiG.Fred JuangB.-H.Deep Reinforcement Learning Based Resource Allocation for V2V Communications. IEEE Transactions on Vehicular Technology, Vol. 68, No. 4, April2019, pp. 3163–3173. https://doi.org/10.1109/TVT.2019.2897134.
22.
SrokaP.StrömE.SvenssonT.KliksA.Autonomous Controller-Aware Scheduling of Intra-Platoon V2V Communications. Sensors, Vol. 23, 2022, p. 60. https://doi.org/10.3390/s23010060.
23.
WenS.GuoG.Control of Leader-Following Vehicle Platoons with Varied Communication Range. IEEE Transactions on Intelligent Vehicles, Vol. 5, No. 2, June2020, pp. 240–250. https://doi.org/10.1109/TIV.2019.2955899.
24.
HongC.ShanH.SongM.ZhuangW.XiangZ.WuY.A Joint Design of Platoon Communication and Control Based on LTE-V2V. IEEE Transactions on Vehicular Technology, Vol. 69, No. 12, Dec.2020, pp. 15893–15907. https://doi.org/10.1109/TVT.2020.3037239.
25.
XiaoS.GeX.HanQ. L.ZhangY.Resource-Efficient Platooning Control of Connected Automated Vehicles Over VANETs. IEEE Transactions on Intelligent Vehicles, Vol. 7, No. 3, Sept.2022, pp. 579–589. https://doi.org/10.1109/TIV.2022.3155640.
26.
SybisM.KryszkiewiczP.SrokaP.On the Context-Aware, Dynamic Spectrum Access for Robust Intraplatoon Communications. Mobile Information Systems, Vol. 2018, 2018, Article ID 3483298, 16 pages. https://doi.org/10.1155/2018/3483298.
27.
GuoG.WenS.Communication Scheduling and Control of a Platoon of Vehicles in VANETs. IEEE Transactions on Intelligent Transportation Systems, Vol. 17, No. 6, June2016, pp. 1551–1563. https://doi.org/10.1109/TITS.2015.2505407.
28.
CaoL.RoyS.YinH.Resource Allocation in 5G Platoon Communication: Modeling, Analysis and Optimization. IEEE Transactions on Vehicular Technology, Vol. 72, No. 4, April2023, pp. 5035–5048. https://doi.org/10.1109/TVT.2022.3223351.
29.
WangP.DiB.ZhangH.BianK.SongL.Joint Platoon Formation and Resource Allocation for Connected Vehicles by Cellular V2X Communication. ICC 2019 – 2019 IEEE International Conference on Communications (ICC), Shanghai, China, 2019, pp. 1–6. https://doi.org/10.1109/ICC.2019.8761878.
30.
LiX.LuL.NiW.JamalipourA.ZhangD.DuH.Federated Multi-Agent Deep Reinforcement Learning for Resource Allocation of Vehicle-to-Vehicle Communications. IEEE Transactions on Vehicular Technology, Vol. 71, No. 8, Aug.2022, pp. 8810–8824. https://doi.org/10.1109/TVT.2022.3173057.
31.
HongyuH.ZixuanW.MingC.ZhengyiL.BinL.GuoyingC.Distributed Control of a Vehicular Platoon Using Event-Triggered Communication Strategy Based on State Estimation. Journal of Transportation Engineering, Part A: Systems, Vol. 149, No. 9, 2023, p. 04023092. https://doi.org/10.1061/JTEPBS.TEENG-7756.
32.
WangJ.HuC.ZhaoJ.ZhangL.HanY.Deep Q-Network-Enabled Platoon Merging Approach for Autonomous Vehicles. Transportation Research Record, 2023. 2678(7): 17–31. https://doi.org/10.1177/03611981231203229.
33.
XuZ.JiaoX.RuS.Proximal Policy Optimization-Based Driving Control Strategy of Connected Cruise Vehicle Platoons to Improve Traffic Efficiency and Safety. Transportation Research Record. 2023. 2677(6): 58–72. https://doi.org/10.1177/03611981221144283.
34.
WangJ.HuC.ZhaoJ.ZhangL.HanY.Deep Q-Network-Based Efficient Driving Strategy for Mixed Traffic Flow with Connected and Autonomous Vehicles on Urban Expressways. Transportation Research Record. 2023. 2677(10): 324–338. https://doi.org/10.1177/03611981231161355.
35.
LiM.LiZ.XuC.LiuT.Deep Reinforcement Learning-Based Vehicle Driving Strategy to Reduce Crash Risks in Traffic Oscillations. Transportation Research Record. 2020. 2674(10): 42–54. https://doi.org/10.1177/0361198120937976.
36.
AmmourahR.TalebpourA.Deep Reinforcement Learning Approach for Automated Vehicle Mandatory Lane Changing. Transportation Research Record. 2022. 2677(2): 712–724. https://doi.org/10.1177/03611981221108377.
37.
ElSamadisyO.ShiT.SmirnovI.AbdulhaiB.Safe, Efficient, and Comfortable Reinforcement-Learning-Based Car-Following for AVs with an Analytic Safety Guarantee and Dynamic Target Speed. Transportation Research Record. 2023. 2678(1): 643–661. https://doi.org/10.1177/03611981231171899.
38.
SegataM.Safe and Efficient Communication Protocols for Platooning Control. PhD thesis. Institute of Computer Science, University of Innsbruck, Innsbruck, Austria, Feb. 2016.
39.
BaumannD.ZhuJ.MartiusG.TrimpeS.Deep Reinforcement Learning for Event-Triggered Control. 2018 IEEE Conference on Decision and Control (CDC), Miami, FL, USA, 2018, pp. 943–950. https://doi.org/10.1109/CDC.2018.8619335.
