GripDepthSense3DNet: A Depth-Enabled Hardness Sensing Framework in Soft Robotic Grasping

Abstract

Despite the development of numerous soft grippers designed to handle deformable objects, hardness sensing remains a challenge, yet it is essential for various applications such as product selection or sorting, assessing fruit ripeness, or food quality control. This research introduces GripDepthSense3DNet, an innovative approach integrating 3D depth sensing with machine learning for accurate hardness sensing during grasping. Leveraging a dataset comprising of depth images of diverse objects undergoing deformation, the proposed novel network is trained to capture intricate spatial–temporal deformation features from a series of depth images. GripDepthSense3DNet outperforms state-of-the-art networks, exhibiting a commendable mean absolute percentage error of 0.46% for trained shapes and hardness. Specifically, the model achieves a reduction in parameters of approximately 94.8% compared to ResNet-50, with a training time that is around 92.9% shorter on equivalent hardware. Different depth ranges and intervals were studied to eventually arrive at an optimal configuration. Through dynamic tuning, the network’s ability to seamlessly incorporate new shapes, new hardness, and even intricate arbitrary objects highlights the adaptability of the approach.