Adaptive reinforcement learning-based tuning of neural PID controllers for nonlinear systems

Abstract

This paper introduces a novel adaptive tuning strategy for a neural network-based PID (NNPID) controller that leverages reinforcement learning (RL) to enhance control performance in nonlinear systems. Unlike conventional NNPID approaches that rely on fixed or manually defined learning rates, the proposed method embeds the learning rate as part of a policy-based RL framework, enabling dynamic and autonomous adjustment during training. This adaptive mechanism allows the controller to better cope with system uncertainties, external disturbances, and nonlinear behaviors while ensuring fast and stable convergence. The RL agent is trained using a reward function designed solely from the tracking error, promoting improved trajectory tracking and reduced steady-state error. By continuously adapting the learning rate through interaction with the environment, the controller achieves superior robustness and generalization across varying operating conditions. Simulation studies and validation on a nonlinear transesterification reactor confirm that the proposed RL-NNPID outperforms traditional gradient descent methods and fixed learning rate strategies in terms of tracking accuracy, convergence speed, and disturbance rejection capability.