Robot target tracking control considering obstacle avoidance based on combination of deep reinforcement learning and PID

Abstract

When simultaneously addressing the challenges of dynamic target tracking and obstacle avoidance for robots, conventional control and control only based on reinforcement learning cannot deal with the complex scenarios effectively. The purpose of this study is to design a robot control algorithm that combines deep reinforcement learning (Soft Actor-Critic, SAC) with PID to achieve real-time tracking of a moving object and effectively avoid single or multiple obstacles. The control of the robot is divided into two key components: initially, the first joint of the 6-degree-of-freedom robot is controlled by PID algorithm, which makes the working plane (the plane coincident with the axis of the first joint and parallel to the linkage) quickly approach the target until it overlaps. Subsequently, the task of reinforcement learning is simplified to control the planar robot to track the target projection in working plane while avoiding the obstacle projection, ultimately achieve target tracking and obstacle avoidance in 3D space. The simulation and experiment results show that the proposed method has good efficiency and convergence speed. The SAC-PID strategy effectively controls the Universal-Robots UR5 to complete dynamic target tracking while accomplishing obstacle avoidance in both virtual and real-world environments.