This paper presents a control strategy that integrates deep reinforcement learning-based active disturbance rejection control (ADRC) with deep deterministic policy gradients (DDPG) for leader-follower coordination in unmanned tracked vehicles. In the proposed framework, DDPG adaptively tunes ADRC parameters, enabling robust leader-following performance under challenging conditions such as track slippage and high-frequency measurement noise. Simulation studies on a laboratory vehicle model with varying leader velocities validate the effectiveness of the method. Compared to conventional fixed-parameter ADRC, the adaptive ADRC–DDPG controller achieves substantial performance gains, reducing the integral absolute error by up to 62%, the integral time absolute error by up to 63%, and the integral time square error by up to 88%. These results highlight the potential of the proposed approach to enhance UTV autonomy and adaptability in dynamic environments, representing a promising step toward advanced adaptive control for autonomous ground vehicles.
WeiMWangSZhengJ, et al. UGV navigation optimization aided by reinforcement learning-based path tracking. IEEE Access2018; 6: 57814–57825.
2.
VermaSEngYHKongHX, et al. Vehicle detection, tracking and behavior analysis in urban driving environments using road context. In: Proceedings of IEEE international conference on robotics and automation, 2018, pp. 1413–1420. DOI: 10.1109/ICRA.2018.8460951.
3.
KobilarovMSukhatmeGHyamsJ, et al. People tracking and following with mobile robot using an omnidirectional camera and a laser. In: Proceedings of IEEE international conference on robotics and automation, 2006, pp. 557–562. DOI: 10.1109/ROBOT.2006.1641769.
4.
CaoSJinYTrautmannT, et al. Design and experiments of autonomous path tracking based on dead reckoning. Appl Sci2022; 13(1): 317.
5.
ZouTAngelesJHassaniF.Dynamic modeling and trajectory tracking control of unmanned tracked vehicles. Robot Auton Syst2018; 110: 102–111.
6.
HanJ.From PID to active disturbance rejection control. IEEE Trans Ind Electron2009; 56(3): 900–906.
7.
ChenSXueWLinZ, et al. On active disturbance rejection control for path following of automated guided vehicle with uncertain velocities. In: Proceedings of the 2019 American Control Conference, 2019, pp. 2446–2451. DOI: 10.23919/ACC.2019.8815348.
8.
WangHZuoZWangY, et al. Composite nonlinear extended state observer and its application to unmanned ground vehicles. Control Eng Pract2021; 109: 104731.
9.
WangHZuoZXueW, et al. Switching longitudinal and lateral semi-decoupled active disturbance rejection control for unmanned ground vehicles. IEEE Trans Ind Electron2024; 71(3): 3034–3043.
10.
Ramírez-NeriaMLuviano-JuárezAMadonskiR, et al. Leader-follower ADRC strategy for omnidirectional mobile robots without time-derivatives in the tracking controller. In Proceedings of the 2023 American Control Conference, 2023, pp. 405–410. DOI: 10.23919/ACC55779.2023.10156648.
11.
Abdul-AdheemWRIbraheemIK.Decoupled control scheme for output tracking of a general industrial nonlinear mimo system using improved active disturbance rejection scheme. Alex Eng J2019; 58(4): 1145–1156.
12.
HerbstGMadonskiR.Active disturbance rejection control: from principles to practice. Birkhäuser, 2025. DOI: 10.1007/978-3-031-72687-3.
13.
GaoZ.Scaling and bandwidth-parameterization based controller tuning. In: Proceedings of the 2003 American control conference, 2003, Vol. 6. pp. 4989–4996. DOI:10.1109/ACC.2003.1242516.
14.
HerbstGHempelAJ.Half-gain observer tuning for noise reduction in discrete-time adrc. Control Eng Pract2025; 164: 106501.
15.
ZhaoPNagamuneR.Switching linear parameter-varying control with improved local performance and optimized switching surfaces. Int J Robust Nonlinear Control2018; 28(10): 3403–3421.
16.
LiYJiaMHanX, et al. Towards a comprehensive optimization of engine efficiency and emissions by coupling artificial neural network (ANN) with genetic algorithm (GA). Energy2021; 225: 120331.
17.
ZhengYTaoJSunQ, et al. DDPG-based active disturbance rejection 3d path-following control for powered parafoil under wind disturbances. Nonlinear Dyn2023; 111(12): 11205–11221.
18.
WangSZhuHWuM, et al. Active disturbance rejection decoupling control for three-degree-of-freedom six-pole active magnetic bearing based on BP neural network. IEEE Trans Appl Supercond2020; 30(4): 1–5.
19.
RomanRCPrecupREPetriuEM.Hybrid data-driven fuzzy active disturbance rejection control for tower crane systems. Eur J Control2021; 58: 373–387.
20.
ZhaoLHeWLvF, et al. Trajectory tracking control for parafoil systems based on the model-free adaptive control method. IEEE Access2020; 8: 152620–152636.
21.
YuDMaSLiuYJ, et al. Finite-time adaptive fuzzy backstepping control for quadrotor uav with stochastic disturbance. IEEE Trans Autom Sci Eng2024; 21(2): 1335–1345.
22.
KongXXiaY.Reinforcement learning-based active disturbance rejection control for nonlinear systems with disturbance. In Proceedings of the 2023 2nd conference on fully actuated system theory and applications, 2023, pp. 799–804. DOI: 10.1109/CFASTA57821.2023.10243334.
23.
ZhengYTaoJHartikainenJ, et al. DDPG based LADRC trajectory tracking control for underactuated unmanned ship under environmental disturbances. Ocean Eng2023; 271: 113667.
24.
ShinJBadgwellTALiuKH, et al. Reinforcement learning – overview of recent progress and implications for process control. Comput Chem Eng2019; 127: 282–294.
25.
NianRLiuJHuangB.A review on reinforcement learning: introduction and applications in industrial process control. Comput Chem Eng2020; 139: 106886.
26.
SilverDLeverGHeessN, et al. Deterministic policy gradient algorithms. In: Proceedings of the 31st international conference on machine learning, 2014, pp. 387–395. https://proceedings.mlr.press/v32/silver14.html
27.
ZhuMWangYPuZ, et al. Safe, efficient, and comfortable velocity control based on reinforcement learning for autonomous driving. Transp Res Part C: Emerg Technol2020; 117: 102662.
28.
ZhangYGuoLGaoB, et al. Deterministic promotion reinforcement learning applied to longitudinal velocity control for automated vehicles. IEEE Trans Vehicular Technol2020; 69(1): 338–348.
29.
LiaoJLiuTTangX, et al. Decision-making strategy on highway for autonomous vehicles using deep reinforcement learning. IEEE Access2020; 8: 177804–177814.
30.
Pérez-Gil Ó BareaRLópez-GuillénE, et al. Deep reinforcement learning based control for autonomous vehicles in Carla. Multimed Tools Appl2022; 81(3): 3553–3576.
WangYZhengCSunM, et al. Reinforcement-learning-aided adaptive control for autonomous driving with combined lateral and longitudinal dynamics. In: Proceedings of the 2023 IEEE 12th data driven control and learning systems conference, 2023, pp. 840–845. DOI: 10.1109/DDCLS58216.2023.10166569.
33.
WangYChenZSunM, et al. Enhancing active disturbance rejection design via deep reinforcement learning and its application to autonomous vehicle. Expert Syst Appl2024; 239: 122433.
34.
ZhangZRahmanAHAZulkarnainN, et al. Enhancing vehicle lateral stability: a DDPG-based active anti-roll bar control strategy. IEEE Access2024; 12: 153030–153043.
35.
CaoYNiKJiangX, et al. Path following for autonomous ground vehicle using DDPG algorithm: a reinforcement learning approach. Appl Sci2023; 13(11): 6847.
36.
SibooSBhattacharyyaANaveen RajR, et al. An empirical study of DDPG and PPO-based reinforcement learning algorithms for autonomous driving. IEEE Access2023; 11: 125094–125108.
37.
AmokraneSBLaidouniMZAdliT, et al. Active disturbance rejection control for unmanned tracked vehicles in leader–follower scenarios: discrete-time implementation and field test validation. Mechatronics2024; 97: 103114.
38.
François-LavetVHendersonPIslamR, et al. An introduction to deep reinforcement learning. Found Trends Mach Learn2018; 11(3–4): 219–354.
39.
AradiS.Survey of deep reinforcement learning for motion planning of autonomous vehicles. IEEE Trans Intell Transp Syst2022; 23(2): 740–759.
40.
LillicrapTPHuntJJPritzelA, et al. Continuous control with deep reinforcement learning. In: Proceedings of the international conference on learning representations, San Juan, Puerto Rico. http://arxiv.org/abs/1509.02971.
41.
KrizhevskyASutskeverIHintonGE.Imagenet classification with deep convolutional neural networks. Commun ACM2017; 60(6): 84–90.
42.
RobotD.All-terrain autonomous navigation robot with GPS-IMU jaguar lite user guide, https://www.DrRobot.com/ (2001, accessed 25 July 2024).
