Study of Ship Path Planning in Complex Environments Based on Spatiotemporal Reinforcement Learning

Abstract

To address the challenges in ship path planning and collision avoidance in complex marine environments, this study proposes a deep reinforcement learning (DRL)-based local path planning algorithm that integrates spatiotemporal feature modeling, aiming to achieve unified ship collision avoidance and motion control. Firstly, an end-to-end state space and action space framework is established, enabling the system to directly output control commands based on perceptual information, thereby bridging the entire pipeline from perception to control. Secondly, a network architecture incorporating spatial position encoding, temporal context modeling, and an attention mechanism is designed to enhance feature modeling and decision-making capabilities in complex environments. Finally, a reward function system combining navigation objectives and collision avoidance requirements is constructed, alleviating the sparse reward problem in reinforcement learning and accelerating training convergence. Comparative experiments conducted in various typical scenarios demonstrate the superiority of the proposed algorithm with reference to path efficiency, safety, and control stability, providing robust technical support for the autonomous navigation of intelligent ships.