Electronic skin for robotic tactile sensing has been studied extensively over the past years, yet practical applications of electronic skin for the grasping state monitoring during robotic manipulation are still limited. In this study, we present the fabrication and implementation of electronic skin sensor arrays for the detection of unstable grasping. The piezoresistive sensor arrays have the advantages of facile fabrication, fast response, and high reliability. With the tactile data from the sensor array, we propose two quantitative indicators, correlation coefficient and wavelet coefficient, to identify grasping with variable forces and slippage. Those two indicators reflect both time and frequency domain characteristics in the contact forces from the sensor array and can be obtained without large amount of calculation. We demonstrate the utility of this method under various conditions, the results indicate grasping with variable forces, and slippage can be distinguished by this method. The flexible sensor arrays are adopted for tactile sensing on a bionic hand, and the effectiveness of this method in detecting various grasping states has been verified. The electronic skin sensor array and the grasping state monitoring method are promising for applications in robotic dexterous manipulation.
Get full access to this article
View all access options for this article.
References
1.
JohanssonRS, FlanaganJR. Coding and use of tactile signals from the fingertips in object manipulation tasks. Nat Rev Neurosci, 2009; 10:345–359.
2.
YousefaH, BoukallelaM, AlthoeferbK. Tactile sensing for dexterous in-hand manipulation in robotics—a review. Robot Auton Syst, 2011; 167:171–187.
3.
TeginJ, WikanderJ. Tactile sensing in intelligent robotic manipulation—a review. Ind Robot, 2010; 32:64–70.
4.
DahiyaRS, MettaG, ValleM, et al.Tactile sensing—From humans to humanoids. In: IEEE Transactions on Robotics. IEEE,, 2010; 26:1–20.
5.
BuTZ, XiaoTX, YangZW, et al.Stretchable triboelectric–photonic smart skin for tactile and gesture sensing. Adv Mater, 2018; 30:1800066.
6.
BartolozziC, NataleL, NoriF, et al.Robots with a sense of touch. Nat Mater, 2016; 15:921–925.
7.
ChenW, KhamisH, BirznieksI, et al.Tactile sensors for friction estimation and incipient slip detection—towards dexterous robotic manipulation: a Review. IEEE Sens J, 2018; 18:9049–9064.
8.
DollarAM, HoweRD. Simple, robust autonomous grasping in unstructured environments. In: 2007 IEEE International Conference on Robotics and Automation (ICRA). IEEE,, 2007:4693–4700.
9.
JiangS, LiZ, WuH, et al.Grab and heat: highly responsive and shape adaptive soft robotic heaters for effective heating of objects of three-dimensional curvilinear surfaces. ACS Appl Mater Interfaces, 2019; 11:47476–47484.
10.
ZhaoJQ, BuTZ, ZhangXH, et al.Intrinsically stretchable organic-tribotronic-transistor for tactile sensing. Research, 2020; 2020:1398903.
11.
DeimelR, BrockO. A novel type of compliant and underactuated robotic hand for dexterous grasping. Int J Robot Res, 2016; 35:161–185.
12.
BillardA, KragicD. Trends and challenges in robot manipulation. Science, 2019; 364:eaat8414.
SuH, QiW, HuY, et al.Towards model-free tool dynamic identification and calibration using multi-layer neural network. Sensors, 2019; 19:3636.
15.
HuangX, GuoW, LiuSY, et al.Flexible mechanical metamaterials enabled electronic skin for real-time detection of unstable grasping in robotic manipulation. Adv Funct Mater, 2022; 2109109.
16.
LiuY, HeK, ChenX, et al.Nature-inspired structural materials for flexible electronic devices. Chem Rev, 2017; 117:12893–12941.
17.
HammockML, ChortosA, BaoZ, et al.25th anniversary article: the evolution of electronic skin (e-skin): a brief history, design considerations, and recent progress. Adv Mater, 2013; 25:5997–6038.
18.
LuN, KimDH. Flexible and stretchable electronics paving the way for soft robotics. Soft Robot, 2014; 1:53–62.
19.
DahiyaR. E-skin: from humanoids to humans. P IEEE, 2019; 107:247–252.
20.
ZengJH, ZhaoJQ, LiCX, et al.Triboelectric nanogenerators as active tactile stimulators for multifunctional sensing and artificial synapses. Sensors, 2022; 22:975.
21.
GongS, WangY, ChengW, et al.A wearable and highly sensitive pressure sensor with ultrathin gold nanowires. Nat Commun, 2014; 5:3132.
22.
ZhangC, LiuS, WuH, et al.A stretchable dual-mode sensor array for multifunctional robotic electronic skin. Nano Energy, 2019; 62:164–170.
23.
YaoG, WangZ, WuH, et al.Bioinspired triboelectric nanogenerators as self-powered electronic skin for robotic tactile sensing. Adv Funct Mater, 2019; 30:1907312.
24.
TangX, WuC, GanL, et al.Multilevel microstructured flexible pressure sensors with ultrahigh sensitivity and ultrawide pressure range for versatile electronic skin. Small, 2019; 15:1804559.
25.
ZhouQ, JiB, WeiY, et al.A bio-inspired cilia array as the dielectric layer for flexible capacitive pressure sensors with high sensitivity and a broad detection range. J Mater Chem A, 2019; 7:27334–27346.
26.
HeP, WuJ, PanX, et al.Anti-freezing and moisturizing conductive hydrogels for strain sensing and moist-electric generation applications. J Mater Chem A, 2020; 8:3109–3118.
27.
MuthJT, VogtDM, TrubyRL, et al.Embedded 3D printing of strain sensors within highly stretchable elastomers. Adv Mater, 2014; 26:6307–6312.
28.
GaoY, LiQ, WuR, et al.Laser direct writing of ultrahigh sensitive SiC-based strain sensor arrays on elastomer toward electronic skins. Adv Funct Mater, 2019; 29:1806786.
29.
YangY, ZhaoG, ChengX, et al.Stretchable and healable conductive elastomer based on PEDOT:PSS/natural rubber for self-powered temperature and strain sensing. Acs Appl Mater Inter, 2020; 13:14612–14624.
30.
ZhuM, LouM, DingB, et al.Highly shape adaptive fiber based electronic skin for sensitive joint motion monitoring and tactile sensing. Nano Energy, 2020; 69:104429.
31.
LeeS, FranklinS, SomeyaT, et al.Nanomesh pressure sensor for monitoring finger manipulation without sensory interference. Science, 2020; 370:966–970.
32.
WonP, ParkJJ, LeeT, et al.Stretchable and transparent kirigami conductor of nanowire percolation network for electronic skin applications. Nano Lett, 2019; 19:6087–6096.
33.
WangS, XuJ, BaoZ, et al.Skin electronics from scalable fabrication of an intrinsically stretchable transistor array. Nature, 2018; 555:83–88.
34.
LeeWW, TanYJ, TeeBCK, et al.A neuro-inspired artificial peripheral nervous system for scalable electronic skins. Sci Robot, 2019; 4:eaax2198.
35.
WangX, ZhangH, WangZ, et al.Self-powered high-resolution and pressure-sensitive triboelectric sensor matrix for real-time tactile mapping. Adv Mater, 2016; 28:2896–2903.
36.
ZhaoJQ, GuoH, PangYK, et al.Flexible organic tribotronic transistor for pressure and magnetic sensing. ACS Nano, 2017; 11:11566–11573.
37.
BoutryCM, NegreM, VardoulisO, et al.A hierarchically patterned, bioinspired e-skin able to detect the direction of applied pressure for robotics. Sci Robot, 2018; 3:eaau6914.
38.
WangY, ChenJ, MeiD. Flexible Tactile sensor array for slippage and grooved surface recognition in sliding movement. Micromachines, 2019; 10:579.
39.
SunX, SunJ, LiT, et al.Flexible tactile electronic skin sensor with 3D force detection based on porous CNTs/PDMS nanocomposites. Nano-Micro Lett. 11:57.
40.
KaboliM, ChengG. Robust tactile descriptors for discriminating objects from textural properties via artificial robotic skin. IEEE T Robot, 2018; 34:985–1003.
41.
ChuangCH, LiouYR, ChenCW. Detection system of incident slippage and friction coefficient based on a flexible tactile sensor with structural electrodes. Sensor Actuat A-Phys, 2012; 188:48–55.
42.
KaboliM, WalkerR, ChengG. Re-using prior tactile experience by robotic hands to discriminate in-hand objects via texture properties. In: IEEE International Conference on Robotics and Automation (ICRA). IEEE,, 2016:2242–2247.
43.
JamesJW, PestellN, LeporaNF. Slip detection with a biomimetic tactile sensor. In: IEEE Robotics and Automation Letters. IEEE,, 2018; 3:3340–3346.
44.
Stabilizing novel objects by learning to predict tactile slip. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE,, 2015:5065–5072.
45.
Zapata-ImpataBS, GilP, TorresF. Learning Spatio temporal tactile features with a ConvLSTM for the direction of slip detection. Sensors, 2019; 19:523.
46.
LiJ, DongS, AdelsonE. Slip detection with combined tactile and visual information. In: IEEE International Conference on Robotics and Automation (ICRA). IEEE,, 2018:7772–7777.
47.
ShirafujiS, HosodaK. Detection and prevention of slip using sensors with different properties embedded in elastic artificial skin on the basis of previous experience. 2011 15th International Conference on Advanced Robotics (ICAR). IEEE,, 2011:459–464.
48.
KappassovZ, CorralesJA, PerdereauV. Tactile sensing in dexterous robot hands—review. Robot Auton Syst, 2015; 74:195–220.
49.
SanchezJ, BouzgarrouBC, MezouarY, et al.Robotic manipulation and sensing of deformable objects in domestic and industrial applications: a survey. Int J Robot Res, 2018; 37:688–716.
50.
ChoiS, KuchenbeckerKJ. Vibrotactile display: perception, technology, and applications. P IEEE, 2013; 101:2093–2104.
51.
MallatSG. A theory for multiresolution signal decomposition: the wavelet representation. In: IEEE Transactions on Pattern Analysis and Machine Intelligence. IEEE,, 1989; 11:674–693.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
