Research on combat simulation agent modelling methods combined with reinforcement learning

Abstract

Agent-based combat simulation is an important research method in the field of military science and system simulation. Behaviour decision model plays the key role in the design of combat simulation agents. The behaviour tree (BT) designed by nonplayer characters (NPCs) in the game provides an efficient and concise method for the construction of combat simulation agents and has been widely used. Because the rationality of BT construction directly affects the rationality of agent decision logic, designing a reasonable BT has become a crucial step. The design of the operational agent BT not only relies on rich tactical experience but also needs to repeatedly adjust and optimize the BT according to the operational deduction and simulation results. To avoid unreasonable BT design caused by lack of experience and eliminate the process of repeated debugging, a modelling method of a combat simulation agent that combines reinforcement learning and the BT method was proposed. This method not only makes the design process of BT more automatic but also simplifies the experience requirements of the combat simulation agent designers. Experiments show that RL-BT effectively integrates the reinforcement learning method and can endow combat simulation agents with battlefield learning ability while making independent decisions. The agent based on RL-BT for decision modelling can continuously adjust and optimize the decision process through experience accumulation, and its performance in combat simulation is significantly better than that of the agent using the original BT.