Mobile robot navigation based on spatio-temporal graph and intrinsic rewards

Abstract

With the wide application of service robots in dynamic and complex environments, their navigation capabilities are challenged by the complexity of pedestrian interactions and environmental uncertainties. Traditional obstacle-avoidance methods have difficulty dealing with social behaviors in crowds, while existing deep reinforcement learning solutions focus more on model improvement and neglect the optimization of exploration mechanisms. To this end, this paper proposes a robot navigation framework that fuses a spatio-temporal attention recurrent neural network (STA-RNN) with a state feature encoder (SFE). By constructing a spatio-temporal graph model, STA-RNN implicitly captures the spatio-temporal dependencies of pedestrian behavior and combines a hierarchical attention mechanism to dynamically identify critical pedestrians. SFE generates intrinsic rewards through environmental state encoding, which drives the robot to actively explore unknown areas and alleviates the sparse reward problem. The results demonstrate that the proposed model achieves the highest navigation success rate in experimental scenarios, enabling the robot to attain better social adaptability.