Humans can feel and grasp efficiently in the dark through tactile feedback, whereas it is still a challenging task for robots. In this research, we create a novel soft gripper named JamTac, which has high-resolution tactile perception, a large detection surface, and integrated sensing-grasping capability that can search and grasp in low-visibility environments. The gripper combines granular jamming and visuotactile perception technologies. Using the principle of refractive index matching, a refraction-free liquid-particle rationing scheme is developed, which makes the gripper itself to be an excellent tactile sensor without breaking its original grasping capability. We simultaneously acquire color and depth information inside the gripper, making it possible to sense the shape, texture, hardness, and contact force with high resolution. Experimental results demonstrate that JamTac can be a promising tool to search and grasp in situations when vision is not available.
Get full access to this article
View all access options for this article.
References
1.
JohanssonRS, FlanaganJR. Coding and use of tactile signals from the fingertips in object manipulation tasks. Nat Rev Neurosci, 2009; 10:345–359.
2.
LeeJ, HwangboJ, WellhausenL, et al.Learning quadrupedal locomotion over challenging terrain. Sci Robot, 2020; 5:eabc5986.
3.
CalandraR, OwensA, UpadhyayaM, et al., The feeling of success: Does touch sensing help predict grasp outcomes?. ArXiv Preprint, ArXiv:1710.05512 2017.
4.
LinY, SunY. Robot grasp planning based on demonstrated grasp strategies. Int J Robot Res, 2015; 34:26–42.
5.
WangZ, LiZ, WangB, et al.Robot grasp detection using multimodal deep convolutional neural networks. Adv Mech Eng, 2016; 8:1687814016668077.
6.
KaraoguzH, JensfeltP. Object detection approach for robot grasp detection. In: IEEE International Conference on Robotics and Automation. Montreal, QC, Canada: IEEE, 2019, pp. 4953–4959.
7.
XuW, ZhangH, YuanH, et al.A compliant adaptive gripper and its intrinsic force sensing method. IEEE Trans Robot, 2021; 37:1584–1603.
8.
MariaGD, NataleC, PirozziS, Force/tactile sensor for robotic applications. Sens Actuat A Phys, 2012; 175:60–72.
9.
ZouL, GeC, WangZJ, et al Novel tactile sensor technology and smart tactile sensing systems: A review. Sensors, 2017; 17:2653.
10.
WuY, LiuY, ZhouY, et al.A skin-inspired tactile sensor for smart prosthetics. Sci Robot, 2018; 3:eaat0429.
11.
MaY, LiuN, LiL, et al.A highly flexible and sensitive piezoresistive sensor based on MXene with greatly changed interlayer distances. Nat Commun, 2017; 8:1–8.
12.
LiT, LuoH, QinL, et al.Flexible capacitive tactile sensor based on micropatterned dielectric layer. Small, 2016; 12:5042–5048.
13.
DuL, ZhuX, ZheJ. An inductive sensor for real-time measurement of plantar normal and shear forces distribution. IEEE Trans Biomed Eng, 2014; 62:1316–1323.
14.
LeporaNF. Soft biomimetic optical tactile sensing with the TacTip: A review. IEEE Sens J, 2021; 21:21131–21143.
15.
KamiyamaK, KajimotoH, KawakamiN, et al. Evaluation of a vision-based tactile sensor. In: IEEE International Conference on Robotics and Automation. New Orleans, LA, USA: IEEE, 2004, pp. 1542–1547.
16.
LambetaM, ChouPW, TianS, et al.DIGIT: A novel design for a low-cost compact high-resolution tactile sensor with application to in-hand manipulation. IEEE Robot Autom Lett, 2020; 5:3838–3845.
17.
YuanW, DongS, AdelsonEH. Gelsight: High-resolution robot tactile sensors for estimating geometry and force. Sensors, 2017; 17:2762.
18.
PadmanabhaA, EbertF, TianS, et al. Omnitact: A multi-directional high-resolution touch sensor. In: IEEE International Conference on Robotics and Automation. Paris, France: IEEE, 2020, pp. 618–624.
19.
Ward-CherrierB, PestellN, CramphornL, et al.The Tactip family: Soft optical tactile sensors with 3d-printed biomimetic morphologies. Soft Robot, 2018; 5:216–227.
20.
SunH, KuchenbeckerKJ, MartiusG. A soft thumb-sized vision-based sensor with accurate all-round force perception. Nat Mach Intell, 2022; 4:135–145.
21.
Ward-CherrierB, RojasN, LeporaNF. Model-free precise in-hand manipulation with a 3d-printed tactile gripper. IEEE Robot Autom Lett, 2017; 2:2056–2063.
22.
Ward-CherrierB, CramphornL, LeporaNF. Tactile manipulation with a TacThumb integrated on the open-hand M2 gripper. IEEE Robot Autom Lett, 2016; 1:169–175.
23.
PangC, MakK, ZhangY, et al. Viko: An adaptive Gecko gripper with vision-based tactile sensor. In: IEEE International Conference on Robotics and Automation. Xi'an, China: IEEE, 2021, pp. 736–742.
24.
Van DuongL. Large-scale vision-based tactile sensing for robot links: Design, modeling, and evaluation. IEEE Trans Robot, 2020; 37:390–403.
25.
KuppuswamyN, AlspachA, UttamchandaniA, et al. Soft-bubble grippers for robust and perceptive manipulation. In: IEEE/RSJ International Conference on Intelligent Robots and Systems. Las Vegas, NV, USA: IEEE, 2020, pp. 9917–9924.
26.
KimYJ, SongH, MaengCY. Blt gripper: An adaptive gripper with active transition capability between precise pinch and compliant grasp. IEEE Robot Autom Lett, 2020; 5:5518–5525.
27.
YuanS, ShaoL, YakoCL, et al. Salisbury, design and control of roller grasper V2 for in-hand manipulation. In: IEEE/RSJ International Conference on Intelligent Robots and Systems. Las Vegas, NV, USA: IEEE, 2020, pp. 9151–9158.
28.
ShaquraM, ShammaJS. A novel gripper design for multi hand tools grasping under tight clearance constraints and external torque effect. In: IEEE International Conference on Mechatronics and Automation. Takamatsu, Japan: IEEE, 2017, pp. 840–845.
29.
AndrychowiczM, BakerB, ChociejM, et al.Learning dexterous in-hand manipulation. Int J Robot Res, 2020; 39:3–20.
30.
ChenW, XiongC, WangY. Analysis and synthesis of underactuated compliant mechanisms based on transmission properties of motion and force. IEEE Trans Robot, 2020; 36:773–788.
31.
GallowayKC, BeckerKP, PhillipsB, et al.Soft robotic grippers for biological sampling on deep reefs. Soft Robot, 2016; 3:23–33.
32.
LiH, YaoJ, ZhouP, et al.High-load soft grippers based on bionic winding effect. Soft Robot, 2019; 6:276–288.
33.
MironG, BédardB, PlanteJS. Sleeved bending actuators for soft grippers: A durable solution for high force-to-weight applications. Actuators, 2018; 7:40.
34.
MantiM, HassanT, PassettiG, et al.A bioinspired soft robotic gripper for adaptable and effective grasping. Soft Robot, 2015; 2:107–116.
35.
Al AbeachLAT, Nefti-MezianiS, DavisS. Design of a variable stiffness soft dexterous gripper. Soft Robot, 2017; 4:274–284.
36.
LiuSQ, AdelsonEH. GelSight Fin Ray: Incorporating tactile sensing into a soft compliant robotic gripper. In: IEEE International Conference on Soft Robotics. Edinburgh, United Kingdom: IEEE, 2022, pp. 925–931.
37.
ZhaoH, O'BrienK, LiS, et al.Optoelectronically innervated soft prosthetic hand via stretchable optical waveguides. Sci Robot, 2016; 1:eaai7529.
38.
TrubyRL, Della SantinaC, RusD. Distributed proprioception of 3D configuration in soft, sensorized robots via deep learning. IEEE Robot Autom Lett, 2020; 5:3299–3306.
39.
GuG, ZhangN, XuH, et al.A soft neuroprosthetic hand providing simultaneous myoelectric control and tactile feedback. Nat Biomed Eng, 2023; 7:589–598.
40.
CuiY, LiuXJ, DongX, et al.Enhancing the universality of a pneumatic gripper via continuously adjustable initial grasp postures. IEEE Trans Robot, 2021; 37:1604–1618.
41.
HussainI, MalvezziM, GanD, et al.Compliant gripper design, prototyping, and modeling using screw theory formulation. Int J Robot Res, 2021; 40:55–71.
42.
LinghuC, ZhangS, WangC, et al.Universal SMP gripper with massive and selective capabilities for multiscaled, arbitrarily shaped objects. Sci Adv, 2020; 6:eaay5120.
43.
BrownE, RodenbergN, AmendJ, et al.Universal robotic gripper based on the jamming of granular material. Proc Natl Acad Sci U S A, 2010; 107:18809–18814.
44.
AmendJR, BrownE, RodenbergN, et al.A positive pressure universal gripper based on the jamming of granular material. IEEE Trans Robot, 2012; 28:341–350.
45.
FitzgeraldSG, DelaneyGW, HowardD. A review of jamming actuation in soft robotics. Actuators, 2020; 9:104.
46.
HowardD, O'ConnorJ, BrettJ, et al. Shape, size, and fabrication effects in 3D printed granular jamming grippers. In: IEEE International Conference on Soft Robotics. New Haven, CT, USA: IEEE, 2021, pp. 458–464.
47.
AmendJ, ChengN, FakhouriS, et al.Soft robotics commercialization: Jamming grippers from research to product. Soft Robot, 2016; 3:213–222.
48.
LichtS, CollinsE, MendesML, et al.Stronger at depth: Jamming grippers as deep sea sampling tools. Soft Robot, 2017; 4:305–316.
HughesJ, IidaF. Tactile sensing applied to the universal gripper using conductive thermoplastic elastomer. Soft Robot, 2018; 5:512–526.
51.
SakumaT, DrigalskiFV, DingM, et al. A universal gripper using optical sensing to acquire tactile information and membrane deformation. In: IEEE/RSJ International Conference on Intelligent Robots and Systems. Takamatsu, Japan: IEEE, 2018, pp. 1–9.
52.
BudwigR. Refractive index matching methods for liquid flow investigations. Exp Fluids, 1994; 17:350–355.
53.
KraghH. The Lorenz-Lorentz formula: Origin and early history. Substantia, 2018; 2:7–18.
54.
ChenY, LinJ, DuX, et al. Non-destructive fruit firmness evaluation using vision-based tactile information. In: IEEE International Conference on Robotics and Automation. Philadelphia, PA, USA: IEEE, 2022.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
