In order to solve the problem of acquiring the position of the target and tracking control of the target, this paper designs a visual servo system for manipulator position based on model predictive control. First, the initial camera position value is obtained by solving the relative position relationship between the camera and the target by the Perspective-n-Point algorithm. Second, the Levenberg–Marquardt algorithm is adopted to reduce the influence of noise and other factors on measurement accuracy. It is adopted to optimize the camera position by minimizing the reprojection error. Third, considering the motion space constraints during the movement of the manipulator, the model predictive control method is employed to solve the visual servo control problem with constraints. The monocular vision system is used to measure the position of the end of the manipulator and the target to provide feedback for the model predictive control. Finally, the simulation experiment is provided to verify the validity and effectiveness of the proposed method.
DhanarajNYoonYJMalhanR, et al. (2022) A mobile manipulator system for accurate and efficient spraying on large surfaces. Procedia Computer Science200: 1528–1539.
2.
FarkhRAljaloudK (2023) Vision navigation based PID control for line tracking robot. Intelligent Automation & Soft Computing35(1): 901–911.
3.
GuoJZhuZSunB, et al. (2021) A novel field box girder welding robot and realization of all-position welding process based on visual servoing. Journal of Manufacturing Processes63: 70–79.
4.
HillJ (1979) Real time control of a robot with a mobile camera. Proceedings of the 9th International Symposium on Industrial Robots12: 233–245.
5.
JeddiMKhoogarAR (2023) A modified eye-in-hand stereo visual control for grasping unknown objects via scara robot. IET Image Processing17(7): 2015–2031.
6.
LiLLuoJHongP, et al. (2024a) Robust adaptive robotic visual servo grasping with guaranteed field of view constraints. Actuators13: 457.
7.
LiSLiDZhangC, et al. (2020) RGB-D image processing algorithm for target recognition and pose estimation of visual servo system. Sensors20(2): 430.
8.
LiWXiongR (2021) A hybrid visual servo control method for simultaneously controlling a nonholonomic mobile and a manipulator. Frontiers of Information Technology & Electronic Engineering22(2): 141–154.
9.
LiZLaiBPanY (2024b) Image-based composite learning robot visual servoing with an uncalibrated eye-to-hand camera. IEEE/ASME Transactions on Mechatronics29(4): 2499–2509.
10.
MattingleyJBoydS (2012) CVXGEN: A code generator for embedded convex optimization. Optimization and Engineering13(1): 1–27.
11.
MicheliFBersaniMArrigoniS, et al. (2023) NMPC trajectory planner for urban autonomous driving. Vehicle system dynamics61(5): 1387–1409.
12.
NicodemusJKneiflJFehrJ, et al. (2022) Physics-informed neural networks-based model predictive control for multi-link manipulators. IFAC-PapersOnLine55(20): 331–336.
13.
Oh-haraSFujimoriA (2023) A leader-follower formation control of mobile robots by position-based visual servo method using fisheye camera. ROBOMECH Journal10(1): 30.
14.
Qing-junX (2021) Trajectory tracking algorithm of manipulator based on model predictive control. Industrial Technology Innovation8(3): 47–53.
15.
RawlingsJBMayneDQDiehlM, et al. (2017) Model predictive control: Theory, computation, and design, vol. 2. Madison, WI: Nob Hill Publishing.
16.
ShenYJiaQWangR, et al. (2023) Learning-based visual servoing for high-precision peg-in-hole assembly. Actuators12(4): 144.
17.
ShiLLiBShiW (2024) Vision-based UAV adaptive tracking control for moving targets with velocity observation. Transactions of the Institute of Measurement and Control46(12): 2255–2265.
18.
SuB (2025) Tracking control of robotic manipulator end-effector trajectory based on robust sliding mode method. PLoS ONE20(4): e0320118.
19.
SunHXueQPanM, et al. (2025a) Obstacle avoidance control for multi-USVs formation based on improved artificial potential field and model predictive control. Journal of Dalian Maritime University51(2): 39–48.
20.
SunWJinYDaiK, et al. (2025b) Flexible manipulator trajectory tracking based on an improved adaptive particle swarm optimization algorithm with fuzzy PD control. Mechanical Sciences16(1): 125–141.
21.
TangMTangKZhangY, et al. (2024) Motion/force coordinated trajectory tracking control of nonholonomic wheeled mobile robot via LMPC-AISMC strategy. Scientific Reports14(1): 18504.
22.
WangBDongJ (2025) A visual servo reinforcement learning control of uncalibrated manipulators with multi-channel gain decision. Transactions of the Institute of Measurement and Control47(2): 265–277.
23.
WangFHuJQinY, et al. (2025) Trajectory tracking control based on deep reinforcement learning for a robotic manipulator with an input deadzone. Symmetry17(2): 149.
24.
WangJ (2023) A PnP algorithm for two-dimensional pose estimation. arXiv [preprint]. DOI: 10.48550/arXiv.2312.08488.
25.
YangHWangZXiaY, et al. (2023) EMPC with adaptive APF of obstacle avoidance and trajectory tracking for autonomous electric vehicles. ISA transactions135: 438–448.
26.
YarenTKizirS (2025) Advanced SOA-NMPC controller design minimising real-time computational burden for dynamic obstacle avoidance in robotic manipulators. International Journal of Systems Science56(16): 3878–3900.
27.
YuanSGeZYangL (2023) Single-camera multi-view 6DoF pose estimation for robotic grasping. Frontiers in Neurorobotics17: 1136882.
28.
ZanonMGrosS (2020) Safe reinforcement learning using robust MPC. IEEE Transactions on Automatic Control66(8): 3638–3652.
29.
ZhuTMaoJHanL, et al. (2024) Fuzzy adaptive model predictive control for image-based visual servoing of robot manipulators with kinematic constraints. International Journal of Control, Automation and Systems22(1): 311–322.
