As a major auxiliary transportation equipment in coal mines, manually operated mining electric locomotives often cause accidents due to the complex and harsh mining environment. Reinforcement learning (RL) focuses on how agents take action in the environment to maximize returns, which is helpful for achieving automatic control of mining electric locomotives. In this paper, RL is applied to autonomous control of mining electric locomotives, considering unsafe conditions such as avoidance of dynamic obstacles and maintaining a safe distance from the vehicle in front. To achieve more precise control, an improved -greedy (IEG) algorithm which can better balance the exploration and exploitation is further proposed. To verify the effectiveness of this method, a co-simulation platform for autonomous control of mining electric locomotives is built which can complete closed-loop simulation of the vehicles. The simulation results show that this method ensures the locomotives following the front vehicle safely and responding promptly in the event of sudden obstacles on the road when the vehicle in complex and uncertain coal mine environments.
WangGLiuFMengX, et al. Research and practice on intelligent coal mine construction (primary stage). Coal Science and Technology2019; 47(8): 1–36.
3.
SongYSongW.A novel dual speed-curve optimization based approach for energy-saving operation of high-speed trains. IEEE Trans Intell Transp Syst2016; 17(6): 1564–1575.
4.
QuJFengXWangQ. Real-time trajectory planning for rail transit train considering regenerative energy. In: International IEEE conference on intelligent transportation systems, 2014, pp.2738–2742.
5.
HeTXiongR.Research on multi-objective real-time optimization of automatic train operation (ato) in urban rail transit. J Shanghai Jiaotong Univ (Sci)2018; 23(2): 327–335.
6.
YangSSunX.A scheduling and control integration optimization method for regenerative braking energy utilization. MATEC Web Conf2021; 336: 02018.
7.
BinZShijunYLanfangZ, et al. Energy-efficient speed profile optimization for high-speed railway considering neutral sections. IEEE Access2021; 9: 25090–25100.
8.
WuZGaoCTangT.An optimal train speed profile planning method for induction motor traction system. Energies2021; 14(16): 5153.
9.
XunJLiuTNingB, et al. Using approximate dynamic programming to maximize regenerative energy utilization for metro. IEEE Trans Intell Transp Syst2020; 21(9): 3650–3662.
10.
XuYZhaoXWangL, et al. Optimal control of automatic train operation based on multi-scale dynamic programming. In: Proceedings of the 33rd Chinese control conference, 2014, pp.3429–3433.
11.
JordanMIMitchellTM.Machine learning: trends, perspectives, and prospects. Science2015; 349(6245): 255–260.
HardtMPriceESrebroN.Equality of opportunity in supervised learning. Adv Neural Inform Process Syst2016; 29. ISSN:1049-5258.
14.
RoyABorkarVKarandikarA, et al. Online reinforcement learning of optimal threshold policies for markov decision processes. IEEE Trans Automat Contr2022; 67(7): 3722–3729.
15.
ZhangLPengYYangW, et al. Semi-infinitely constrained markov decision processes and provably efficient reinforcement learning. IEEE Trans Pattern Anal Mach Intell2024; 46(5): 3722–3735.
16.
BotvinickMRitterSWangJX, et al. Reinforcement learning, fast and slow. Trends Cogn Sci2019; 23(5): 408–422.
17.
KaelblingLPLittmanMLMooreAW.Reinforcement learning: a survey. J Artif Intell Res1996; 4: 237–285.
18.
MnihVKavukcuogluKSilverD, et al. Human-level control through deep reinforcement learning. Nature2015; 518(7540): 529–533.
19.
DeshmukhV S S. A technique for web page ranking by applying reinforcement learning. Int J Comput Appl2016; 155(7): 1–4.
20.
JohanninkTBahlSNairA, et al. Residual reinforcement learning for robot control. In: 2019 international conference on robotics and automation, 2019, pp.6023–6029.
21.
ChenCChenXQMaF, et al. A knowledge-free path planning approach for smart ships based on reinforcement learning. Ocean Eng2019; 189: 106–299.
22.
MousaviSSSchukatMHowleyE.Traffic light control using deep policy-gradient and value-function-based reinforcement learning. IET Intell Transp Syst2017; 11(7): 417–423.
23.
YinJChenDLiL.Intelligent train operation algorithms for subway by expert system and reinforcement learning. IEEE Trans Intell Transp Syst2014; 15(6): 2561–2571.
24.
ŠemrovDMarsetičRŽuraM, et al. Reinforcement learning approach for train rescheduling on a single-track railway. Transp Res Part B: Methodol2016; 86: 250–267.
25.
ZhuLHeYYuFR, et al. Communication-based train control system performance optimization using deep reinforcement learning. IEEE Trans Vehicular Technol2017; 66(12): 10705–10717.
26.
SuttonRS.Learning to predict by the methods of temporal differences. Mach Learn1988; 3(1): 9–44.
AmhraouiEMasrourT.Expected lenient Q-learning: a fast variant of the lenient Q-learning algorithm for cooperative stochastic Markov games. Int J Mach Learn Cybern2024; 15(7): 2781–2797.
29.
ChenHLuWSongR.Statistical inference for online decision making: in a contextual bandit setting. J Am Stat Assoc2021; 116(533): 240–255.
30.
KatohMShimotaniRTokushigeK. Integrated multiagent course search to goal by epsilon-greedy learning strategy: Dual-probability approximation searching. In: IEEE international conference on systems, man, and cybernetics, 2015, pp.388–392.
31.
Vázquez-CanteliJRNagyZ.Reinforcement learning for demand response: a review of algorithms and modeling techniques. Appl Energy2019; 235: 1072–1089.
32.
RumelhartDEHintonGEWilliamsRJ.Learning representations by back-propagating errors. Nature1986; 323(6088): 533–536.
33.
NguyenTTNguyenNDNahavandiS.Deep reinforcement learning for multiagent systems: a review of challenges, solutions, and applications. IEEE Trans Cybern2020; 50(9): 3826–3839.
34.
ZhouCHuangBFräntiP.A review of motion planning algorithms for intelligent robots. J Intell Manuf2022; 33(2): 387–424.
35.
AminMOthmanM. Re-exploration of epsilon-greedy in deep reinforcement learning. In: Proceedings of the 8th international conference on robot intelligence technology and applications, 2020, pp.264–272.
