In this paper, a multi-agent–based reinforcement learning (RL) algorithm is proposed to solve the leveling control problem of a multi-cylinder hydraulic press with coupling phenomena. This algorithm is a model-free control algorithm, which can avoid the modeling difficulties and low efficiency caused by the complexity of the model. The control algorithm of the hydraulic press adopts Multi-Agent Soft Actor–Critic (MASAC). The concept of multi-agent is introduced to control each coupling input separately. The distributed updating method is used to realize accurate and stable control of the hydraulic press. At the same time, a reward function of the piecewise function type is proposed in this paper. Compared with common algorithms such as the quadratic reward function, this algorithm has a faster and more stable convergence effect in the whole process. Experiments show that the proposed algorithm has better convergence speed and leveling accuracy than the traditional single-agent algorithm.
AhnKKTruongDQThanhTQ, et al. (2008) Online self-tuning fuzzy proportional–integral–derivative control for hydraulic load simulator. Proceedings of the Institution of Mechanical Engineers Part I Journal of Systems & Control Engineering222(2): 81–95.
2.
AlbabaBMYildizY (2022) Driver modeling through deep reinforcement learning and behavioral game theory. IEEE Transactions on Control Systems Technology30(2): 885–892.
3.
AlMahamidFGrolingerK (2022) Autonomous unmanned aerial vehicle navigation using reinforcement learning: A systematic review. Engineering Applications of Artificial Intelligence115: 105321.
4.
BotvinickMRitterSWangJX, et al. (2019) Reinforcement learning, fast and slow. Trends in Cognitive Sciences23(5): 408–422.
5.
ChengGPanS (2008) Adaptive sliding mode control of electro-hydraulic system with nonlinear unknown parameters. Control Theory & Applications16(11): 1275–1284.
6.
DingZWuJWangY, et al. (2022) DRL-based frequency response of wind turbine generators adapting their variable regulation capabilities. IET Renewable Power Generation17(3): 495–506.
7.
DongHZhaoX (2022) Composite experience replay-based deep reinforcement learning with application in wind farm control. IEEE Transactions on Control Systems Technology30(3): 1281–1295.
8.
DongHZhaoX (2023) Data-driven wind farm control via multiplayer deep reinforcement learning. IEEE Transactions on Control Systems Technology31(3): 1468–1475.
9.
FengKYangLZHeDF, et al. (2022) A study on deep reinforcement learning-based crane scheduling model for uncertainty tasks. High Temperature Materials and Processes41(1): 469–481.
10.
HaarnojaTZhouAAbbeelP, et al. (2018a) Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. arXiv:1801.01290.
11.
HaarnojaTZhouAHartikainenK, et al. (2018b) Soft actor-critic algorithms and applications. arXiv: 1812.05905.
12.
HuaHFangYZhangX, et al. (2022) A time-optimal trajectory planning strategy for an aircraft with a suspended payload via optimization and learning approaches. IEEE Transactions on Control Systems Technology30(6): 2333–2343.
13.
JiaC (2021) An adaptive smooth second-order sliding mode repetitive control method with application to a fast periodic stamping system. Systems and Control Letters151(1): 104912.
14.
JiaCSunYDongE, et al. (2020) Driving force allocation strategy of multi-cylinder hydraulic press machine considering operation phases. In: 2020 Chinese Automation Congress (CAC), Shanghai, China, 6–8 November. New York: IEEE, pp. 657–662.
15.
JiaCSunYDuL, et al. (2023) Fault-tolerant control strategy for multicylinder hydraulic press machine based on dynamic control allocation and adjustable multiobjective optimization. Asian Journal of Control25: 3541–3558.
16.
LillicrapTPHuntJJPritzelA, et al. (2015) Continuous control with deep reinforcement learning. arXiv: 1509.02971.
17.
NguyenTTNguyenNDNahavandiS (2020) Deep reinforcement learning for multiagent systems: A review of challenges, solutions, and applications. IEEE Transactions on Cybernetics50(9): 3826–3839.
18.
ParkJKimBHanS (2023) Reinforcement learning with model-based assistance for shape control in Sendzimir rolling mills. IEEE Transactions on Control Systems Technology31: 1867–1874.
19.
SchulmanJWolskiFDhariwalP, et al. (2017) Proximal policy optimization algorithms. arXiv: 1707.06347.
20.
SharmaRGopalM (2010) Synergizing reinforcement learning and game theory—A new direction for control. Applied Soft Computing10(3): 675–688.
21.
WuCPanWStaaR, et al. (2023) Deep reinforcement learning control approach to mitigating actuator attacks. Automatica152. DOI: 10.1016/j.automatica.2023.110999.
22.
ZhouSKLeHNLuuK, et al. (2021) Deep reinforcement learning in medical imaging: A literature review. Medical Image Analysis73: 102193.
23.
ZhuKZhangT (2021) Deep reinforcement learning based mobile robot navigation: A review. Tsinghua Science and Technology26(5): 674–691.
24.
ZhuangHSunQChenZ (2021) Sliding mode control for electro-hydraulic proportional directional valve-controlled position tracking system based on an extended state observer. Asian Journal of Control23(4): 1855–1869.
