Reinforcement learning control for nonlinear modeling of supercavitating vehicles considering memory effect

Abstract

Supercavitating vehicles can move at higher speeds than traditional underwater vehicles. However, higher speed will pose higher challenges for the design of the controller. During the controller design for supercavitating vehicles, not only traditional interferences like nonlinear dynamics, actuator saturation, and external interferences should be considered, but also cavity memory effect must be tackled. Hence, we propose a control method based on a reinforcement learning algorithm for the nonlinear model of supercavitating vehicles. First, a nonlinear model of the longitudinal motion of high-speed underwater supercavitating vehicles based on the cavity memory effect is established. Then, by considering both the present moment and the delayed states, a reinforcement learning controller of supercavitating vehicles is designed. The controller can solve the problem of nonlinearities and time-delay effects of the system while limiting the output of the actuator. Finally, the design of the reward function of the reinforcement learning controller is optimized to improve the control accuracy and reduce the oscillation of the actuator. The simulation results show that the reinforcement learning controller can resist external disturbances and parameter perturbations, and keep the system stable with strong robustness in the case of actuator saturation.