Vehicle motion control in handling limits, including drifting control, involves highly nonlinear system dynamics and uncertainties. Conventional dynamic-model-based approaches often require extensive parameter tuning, since fixed parameters are difficult to handle dynamic environmental changes. To address this, a Model Predictive Control (MPC) framework with parameter self-adaptation via reinforcement learning (RL) is proposed. The RL agent can autonomously adjust MPC controller parameters based on its learned experiences, and is capable of online learning during closed-loop control. This framework is first validated on the high-fidelity simulation platform CarSim, showing that the algorithm can achieve stable drifting under various conditions and effectively adapt to dynamic environmental changes. Further, real-vehicle testing is conducted on a full-size B-class electric car with rear-wheel-drive and steer-by-wire systems. According to the published literature, this is the first time that RL methods are successfully trained and deployed on real-vehicle drifting tasks, outperforming the conventional MPC-based algorithm.
CaiPMeiXTaiL, et al. High-speed autonomous drifting with deep reinforcement learning. IEEE Robot Autom Lett2020; 5(2): 1247–1254.
2.
ZubovIAfanasyevIGabdullinA, et al. Autonomous drifting control in 3D car racing simulator. In: 2018 international conference on intelligent systems (IS), Funchal, Portugal, 25–27 September 2018, pp. 235–241. New York: IEEE.
3.
KolterJZPlagemannCJacksonDT, et al. A probabilistic approach to mixed open-loop and closed-loop control, with application to extreme autonomous driving. In: 2010 IEEE international conference on robotics and automation, Anchorage, AK, USA, 3–7 May 2010, pp. 839–845. New York: IEEE.
4.
HindiyehRYChristian GerdesJ.A controller framework for autonomous drifting: design, stability, and experimental validation. J Dyn Syst Meas Control2014; 136(5): 051015.
5.
GohJYGerdesJC. Simultaneous stabilization and tracking of basic automobile drifting trajectories. In: 2016 IEEE intelligent vehicles symposium (IV), Gothenburg, Sweden, 19–22 June 2016, pp. 597–602. N New York: IEEE.
6.
LiDZhangJLinS.Planning and control of drifting-based collision avoidance strategy under emergency driving conditions. Control Eng Pract2023; 139: 105625.
7.
ZhaoTYurtseverEChladnyR, et al. Collision avoidance with transitional drift control. In: 2021 IEEE international intelligent transportation systems conference (ITSC), Indianapolis, IN, USA, 19–22 September 2021, pp. 907–914. New York: IEEE.
8.
HindiyehRYGerdesJC. Equilibrium analysis of drifting vehicles for control design. In: ASME 2009 dynamic systems and control conference, dynamic systems and control conference, Hollywood, CA, USA, 12–14 October 2009, vol. 1, pp. 181–188. New York: ASME.
9.
LenzoBGoelTChristian GerdesJ.Autonomous drifting using torque vectoring: innovating active safety. IEEE Trans Intell Transp Syst2024; 25(11): 17931–17939.
10.
TianXYangSYangY, et al. A multi-layer drifting controller for all-wheel drive vehicles beyond driving limits. IEEE ASME Trans Mechatron2024; 29(2): 1229–1239.
11.
StanoPMontanaroUTaverniniD, et al. Model predictive path tracking control for automated road vehicles: a review. Annu Rev Control2023; 55: 194–236.
12.
DuoRHuCZhouX, et al. An MPC-based drifting control system for collision avoidance in autonomous vehicles. In: NakamatsuKKountchevRPatnaikS, et al. (eds) Advanced intelligent technologies for information and communication. Springer Nature Singapore, 2023, pp. 411–426.
13.
PetersonMTGoelTGerdesJC.Exploiting linear structure for precision control of highly nonlinear vehicle dynamics. IEEE Trans Intell Veh2023; 8(2): 1852–1862.
14.
WeberTPGerdesJC.Modeling and control for dynamic drifting trajectories. IEEE Trans Intell Veh2024; 9(2): 3731–3741.
15.
MeijerSBertipagliaAShyrokauB. A nonlinear model predictive control for automated drifting with a standard passenger vehicle*. In: 2024 IEEE international conference on advanced intelligent mechatronics (AIM), Boston, MA, USA, 15–19 July 2024, pp. 284–289. New York: IEEE.
16.
AcostaMKanarachosS.Teaching a vehicle to autonomously drift: a data-based approach using neural networks. Knowl Based Syst2018; 153: 12–28.
17.
KobayashiTWeberTPGerdesJC. Trajectory planning using tire thermodynamics for automated drifting. In: 2024 IEEE intelligent vehicles symposium (IV), Jeju Island, Republic of Korea, 2–5 June 2024, pp. 2103–2109. New York: IEEE.
18.
DjeumouFGohJYTopcuU, et al. Autonomous drifting with 3 minutes of data via learned tire models. In: 2023 IEEE international conference on robotics and automation (ICRA), London, UK, 29 May–2 June 2023, pp. 968–974. New York: IEEE.
19.
DingNThompsonMDallasJ, et al. Drifting with unknown tires: learning vehicle models online with neural networks and model predictive control. In: 2024 IEEE intelligent vehicles symposium (IV), Jeju Island, Republic of Korea, 2–5 June 2024, pp. 2545–2552. New York: IEEE.
20.
DombergFWembersCCPatelH, et al. Deep drifting: autonomous drifting of arbitrary trajectories using deep reinforcement learning. In: 2022 international conference on robotics and automation (ICRA), Philadelphia, PA, USA, 23–27 May 2022, pp. 7753–7759. New York: IEEE.
21.
BhattacharjeeSKabaraKDJainR, et al. Autonomous drifting RC car with reinforcement learning. Technical report, University of Hong Kong, Hong Kong, 2018.
22.
TóthSHViharosZJBárdosÁ, et al. Sim-to-real application of reinforcement learning agents for autonomous, real vehicle drifting. Vehicles2024; 6(2): 781–798.
23.
JaumannFSchusterTUnterreinerM, et al. Powerslide control with deep reinforcement learning. In: MastinuGBraghinFCheliF, et al. (eds) 16th international symposium on advanced vehicle control. Springer Nature Switzerland, 2024, pp. 862–868.
24.
ChenGZhaoXGaoZ, et al. Dynamic drifting control for general path tracking of autonomous vehicles. IEEE Trans Intell Veh2023; 8(3): 2527–2537.
25.
SongZHuaYZhuangY, et al. Vehicle steady drifting control with safety boundary constraints. Nonlinear Dyn2024; 112(20): 18235–18254.
26.
SpielbergNABrownMGerdesJC.Neural network model predictive motion control applied to automated driving with unknown friction. IEEE Trans Control Syst Technol2022; 30(5): 1934–1945.
27.
LeeTSeoDLeeJ, et al. Real-time drift-driving control for an autonomous vehicle: learning from nonlinear model predictive control via a deep neural network. Electronics2022; 11(17): 2651.
28.
CutlerMHowJP.Autonomous drifting using simulation-aided reinforcement learning. In: 2016 IEEE international conference on robotics and automation (ICRA), Stockholm, Sweden, 16–21 May 2016, pp. 5442–5448. New York: IEEE.
29.
StanoPTaverniniDMontanaroU, et al. Enhanced active safety through integrated autonomous drifting and direct yaw moment control via nonlinear model predictive control. IEEE Trans Intell Veh2024; 9(2): 4172–4190.
30.
HeLWangYGaoZ, et al. A model predictive control for steady-state drifting and tracking of electronic-two-rear-wheel drive automobiles. Preprint, In Review, 2022.
31.
DongHYuHXiJ. Real-time model predictive control for simultaneous drift and trajectory tracking of autonomous vehicles. In: 2022 6th CAA international conference on vehicular control and intelligence (CVCI), Nanjing, China, 28–30 October 2022, pp. 1–6. New York: IEEE.
32.
OrgovánLBécsiTAradiS.Autonomous drifting using reinforcement learning. Period Polytech Transp Eng2021; 49(3): 292–300.
33.
GohJYGoelTChristian GerdesJ.Toward automated vehicle control beyond the stability limits: drifting along a general path. J Dyn Syst Meas Control2019; 142: 021004.
34.
VelenisEKatzourakisDFrazzoliE, et al. Steady-state drifting stabilization of RWD vehicles. Control Eng Pract2011; 19(11): 1363–1376.
35.
RomeroAAljalboutESongY, et al. Actor-critic model predictive control: differentiable optimization meets reinforcement learning. arXiv preprint arXiv:2306.09852, 2025.
36.
ReiterRGhezziABaumgärtnerK, et al. AC4MPC: actor-critic reinforcement learning for nonlinear model predictive control. arXiv preprint arXiv:2406.03995, 2024.
37.
ZhouBHuCZengJ, et al. Adaptive learning-based model predictive control strategy for drift vehicles. Rob Auton Syst2025; 188: 104941.
38.
SchulmanJMoritzPLevineS, et al. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438, 2015.
ParkMKangY.Experimental verification of a drift controller for autonomous vehicle tracking: a circular trajectory using LQR method. Int J Control Autom Syst2021; 19(1): 404–416.
41.
JohanssonMTaghavianH.Stable MPC with maximal terminal sets and quadratic terminal costs. arXiv preprint arXiv:2406.02760, 2024.
