Fusion of residual UNet image segmentation model and digital twin for object repair system

Abstract

Traditional object repair methods usually rely on complex algorithms and a lot of manual labor, making it difficult to repair large and complex objects. In order to improve the efficiency of repairing large and complex targets and reduce the workload of object repair, an object repair model based on digital twin technology is developed. A charge-coupled device in the mechanical arm collects image data from the target object and transfers it to the real-virtual coordinate matching module. The attention mechanism is introduced to construct a multiscale residual UNet object image segmentation model for real-virtual coordinate matching to deal with the object repair trajectory planning problem. When verifying the function of the model, the loss degree, repair model parameters, and object repair error are used as evaluation indicators. The results of the model performance test revealed that the proposed multiscale residual UNet model for object image segmentation exhibited an accuracy of 0.942 while maintaining a loss value of 0.099 at steady state. Compared to the traditional UNet model, the study’s model had fewer parameters by 20.02 million and a slightly improved prediction accuracy of 0.01 on the self-built dataset. Additionally, the inclusion of the attention module enhanced the prediction accuracy by 0.02 M without adding too many parameters. The experiment demonstrated that the deviation between the predicted and actual object center coordinates was less than 1 mm, both horizontally and vertically. This sub-millimeter accuracy allowed for precise virtual-to-real alignment, which was essential for the operation of high-fidelity digital twins. It also ensured reliable robotic manipulation in demanding applications, such as precision component repair and cultural heritage restoration. Furthermore, the use of a dual-robot cooperative approach proved more effective. It was more effective in completing complex repair tasks. It increased system repair capability. It enabled omni-directional repairs previously unattainable with a single robot arm. Crucially, this dual-robot strategy reduced the time required for complex object repair to under 15 s. This represented a 40% improvement in speed over single-robot operation, while maintaining a repair pass rate of 99.2%.