Proprioception in soft robots is essential for enabling autonomous behaviors, allowing them to navigate and interact safely in unstructured environments. Previous sensorization-based shape reconstruction methods, which often rely on machine learning techniques, have limitations in their broad applicability for different robotic systems and environments. In this work, we present a shape reconstruction scheme enabled by sparsely distributed soft strain sensors on the surfaces of soft robots, combined with a model-based reconstruction framework. Our approach utilizes miniaturized stretchable capacitive strain sensors with large stretchability and low hysteresis, which can be easily attached to soft robot surfaces for accurate local strain measurements. These measurements are fed into an optimization algorithm with embedded mechanical constraints. Our approach can predict all deformation modes in a soft bar with a maximum displacement error of less than 4% of the bar length and accurately reconstruct the shapes of soft pneumatic grippers during grasping actions. Additional reconstructions of a bioinspired arm in complex contact scenarios further demonstrate the versatility of our approach. This shape reconstruction scheme using distributed strain sensors offers a convenient and broadly applicable solution for enhancing proprioception in soft robots.
LiuY, HouJ, LiC, et al.Intelligent soft robotic grippers for agricultural and food product handling: A brief review with a focus on design and control. Adv Intell Syst, 2023; 5(12):2300233.
5.
GallowayKC, BeckerKP, PhillipsB, et al.Soft robotic grippers for biological sampling on deep reefs. Soft Robot, 2016; 3(1):23–33.
6.
AracriS, Giorgio-SerchiF, SuariaG, et al.Soft robots for ocean exploration and offshore operations: A perspective. Soft Robot, 2021; 8(6):625–639.
7.
der MaurPA, DjambaziB, HaberthürY, et al.RoBoa: Construction and evaluation of a steerable vine robot for search and rescue applications. In: 2021 IEEE 4th International Conference on Soft Robotics (RoboSoft) New Haven, CT, USA; 2021; pp. 15–20.
DouW, ZhongG, CaoJ, et al.Soft robotic manipulators: Designs, actuation, stiffness tuning, and sensing. Adv Mater Technol, 2021; 6(9):2100018.
14.
KimSH, ChangH-S, ShihC-H, et al.A physics-Informed, vision-based method to reconstruct all deformation modes in slender bodies. In: 2022 International Conference on Robotics and Automation (ICRA). IEEE: Philadelphia, PA, USA; 2022; pp. 4810–4817.
15.
WangR, WangS, DuS, et al.Real-Time soft body 3D proprioception via deep vision-based sensing. IEEE Robot Autom Lett, 2020; 5(2):3382–3389.
16.
ScharffRBN, FangG, TianY, et al.Sensing and reconstruction of 3-D deformation on pneumatic soft robots. IEEE/ASME Trans Mechatron, 2021; 26(4):1877–1885.
17.
ScharffRBN, DoornbuschRM, DoubrovskiEL, et al.Color-based proprioception of soft actuators interacting with objects. IEEE/ASME Trans Mechatron, 2019; 24(5):1964–1973.
18.
MartinYJ, BruderD, WoodRJ. A proprioceptive method for soft robots using inertial measurement units. In: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE: Kyoto, Japan; 2022; pp. 9379–9384.
19.
HughesJ, StellaF, SantinaCD, et al.Sensing soft robot shape using IMUs: an experimental investigation. In: Experimental Robotics. (SicilianoB, LaschiC, KhatibO eds) Springer International Publishing: Floriana, Malta; 2021; pp. 543–552.
20.
MengY, FangG, YangJ, et al.Spring-IMU fusion-based proprioception for feedback control of soft manipulators. IEEE/ASME Trans Mechatron, 2024; 29(2):832–842.
21.
WuX, FuK, ZhangQ, et al.Inverse kinematics solving of a novel soft manipulator with vision and IMU data. In: 2023 International Conference on Service Robotics (ICoSR). IEEE: Shanghai, China; 2023; pp. 112–117.
22.
PeiG, StellaF, MeebedO, et al.IMU based pose reconstruction and closed-loop control for soft robotic arms. In: 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE: Abu Dhabi, United Arab Emirates; 2024; pp. 1847–1852.
23.
Van MeerbeekIM, De SaCM, ShepherdRF. Soft optoelectronic sensory foams with proprioception. Sci Robot, 2018; 3(24):eaau2489.
24.
GallowayKC, ChenY, TempletonE, et al.Fiber optic shape sensing for soft robotics. Soft Robot, 2019; 6(5):671–684.
25.
LunTLT, WangK, HoJDL, et al.Real-time surface shape sensing for soft and flexible structures using fiber bragg gratings. IEEE Robot Autom Lett, 2019; 4(2):1454–1461.
26.
AnX, CuiY, DongX, et al.Shape reconstruction of soft continuum robots via the fusion of local strains and global poses. Cell Rep Phys Sci, 2024; 5(10):102224.
27.
ZhuangW, SunG, LiH, et al.FBG based shape sensing of a silicone octopus tentacle model for soft robotics. Optik, 2018; 165:7–15.
28.
WallV, ZollerG, BrockO. A Method for sensorizing soft actuators and its application to the RBO hand 2. In: 2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE: Singapore; 2017; pp. 4965–4970.
29.
TapiaJ, KnoopE, MutnýM, et al.MakeSense: Automated sensor design for proprioceptive soft robots. Soft Robot, 2020; 7(3):332–345.
30.
CianchettiM, RendaF, LicofonteA, et al.Sensorization of continuum soft robots for reconstructing their spatial configuration. In: 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob). IEEE: Rome, Italy; 2012; pp. 634–639.
31.
WangP, XieZ, XinW, et al.Sensing expectation enables simultaneous proprioception and contact detection in an intelligent soft continuum robot. Nat Commun, 2024; 15(1):9978.
32.
SoJ, KimU, KimYB, et al.Shape estimation of soft manipulator using stretchable sensor. Cyborg Bionic Syst, 2021; 2021:9843894.
33.
StellaF, SantinaCD, HughesJ. Soft robot shape estimation with IMUs leveraging PCC kinematics for drift filtering. IEEE Robot Autom Lett, 2024; 9(2):1945–1952.
34.
LilgeS, BarfootTD, Burgner-KahrsJ. Continuum robot state estimation using Gaussian process regression on SE(3). Int J Robot Res, 2022; 41(13–14):1099–1120.
35.
CanganBG, NavarroSE, YangB, et al.Model-based disturbance estimation for a fiber-reinforced soft manipulator using orientation sensing. In: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE: Kyoto, Japan; 2022; pp. 9424–9430.
36.
ShihB, ShahD, LiJ, et al.Electronic skins and machine learning for intelligent soft robots. Sci Robot, 2020; 5(41):eaaz9239.
37.
HuD, Giorgio-SerchiF, ZhangS, et al.Stretchable e-skin and transformer enable high-resolution morphological reconstruction for soft robots. Nat Mach Intell, 2023; 5(3):261–272.
38.
TrubyRL, SantinaCD, RusD. Distributed proprioception of 3D configuration in soft, sensorized robots via deep learning. IEEE Robot Autom Lett, 2020; 5(2):3299–3306.
39.
SharmaH, Najera-FloresDA, ToddMD, et al.Lagrangian operator inference enhanced with structure-preserving machine learning for nonintrusive model reduction of mechanical systems. Comput Methods Appl Mech Eng, 2024; 423:116865.
40.
AloraJI, CenedeseM, SchmerlingE, et al.Data-driven spectral submanifold reduction for nonlinear optimal control of high-dimensional robots. In: 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE: London, United Kingdom; 2023; pp. 2627–2633.
41.
ArmaniniC, BoyerF, MathewAT, et al.Soft robots modeling: A structured overview. IEEE Trans Robot, 2023; 39(3):1728–1748.
42.
HuangX, LiuL, LinYH, et al.High-stretchability and low-hysteresis strain sensors using origami-inspired 3D mesostructures. Sci Adv, 2023; 9(34):eadh9799.
43.
ChenJ, ZhangJ, LuoZ, et al.Superelastic, sensitive, and low hysteresis flexible strain sensor based on wave-patterned liquid metal for human activity monitoring. ACS Appl Mater Interfaces, 2020; 12(19):22200–22211.
44.
KurianAS, SouriH, MohanVB, et al.Highly stretchable strain sensors based on polypyrrole-silicone rubber composites for human motion detection. Sensor Actuat A-Phys, 2020; 312:112131.
45.
XiaQ, WangS, ZhaiW, et al.Highly linear and low hysteresis porous strain sensor for wearable electronic skins. Compos Commun, 2021; 26:100809.
46.
PuJ-H, ZhaoX, ZhaX-J, et al.Multilayer structured AgNW/WPU-MXene fiber strain sensors with ultrahigh sensitivity and a wide operating range for wearable monitoring and healthcare. J Mater Chem A, 2019; 7(26):15913–15923.
47.
KoS, ChhetryA, KimD, et al.Hysteresis-free double-network hydrogel-based strain sensor for wearable smart bioelectronics. ACS Appl Mater Interfaces, 2022; 14(27):31363–31372.
48.
ZhangD, ZhangJ, WuY, et al.Liquid metal interdigitated capacitive strain sensor with normal stress insensitivity. Adv Intell Syst, 2022; 4(4):2100201.
49.
WangX, DengY, JiangP, et al.Low-hysteresis, pressure-insensitive, and transparent capacitive strain sensor for human activity monitoring. Microsyst Nanoeng, 2022; 8(1):113.
50.
CaoP-J, LiuY, AsgharW, et al.A stretchable capacitive strain sensor having adjustable elastic modulus capability for wide‐range force detection. Adv Eng Mater, 2020; 22(3):1901239.
51.
YoshidaA, WangY-F, SekineT, et al.Printed low-hysteresis stretchable strain sensor based on a self-segregating conductive composite. ACS Appl Eng Mater, 2023; 1(1):50–58.
52.
YuanL, ZhaoT, LiT, et al.Flexible, anisotropic strain sensor based on interdigital capacitance for multi-direction discrimination. Sensor Actuat A-Phys, 2023; 359:114459.
53.
SvanbergK. The method of moving asymptotes—a new method for structural optimization. Numerical Meth Engineering, 1987; 24(2):359–373.
54.
LiuL, LeeHP. A design of auxetic metamaterial with locally resonant bandgaps using topological optimization. Int J Appl Mechanics, 2023; 15(06):2350048.
55.
WuJ, LuoZ, LiH, et al.Level-set topology optimization for mechanical metamaterials under hybrid uncertainties. Comput Methods Appl Mech Eng, 2017; 319:414–441.
56.
MosadeghB, PolygerinosP, KeplingerC, et al.Pneumatic networks for soft robotics that actuate rapidly. Adv Funct Mater, 2014; 24(15):2163–2170.
57.
SouriH, BanerjeeH, JusufiA, et al.Wearable and stretchable strain sensors: Materials, sensing mechanisms, and applications. Adv Intell Syst, 2020; 2(8):2000039.
58.
FangG, TianY, YangZ-X, et al.Efficient Jacobian-based inverse kinematics with sim-to-real transfer of soft robots by learning. IEEE/ASME Trans Mechatron, 2022; 27(6):5296–5306.
59.
TianY, FangG, SuR, et al.Function based sim-to-real learning for shape control of deformable free-form surfaces. In: Robotics. Science and Systems Robotics Science and Systems Foundation: Delft, Netherlands; 2024; p. 98.
60.
GaoJ, MichelisMY, SpielbergA, et al.Sim-to-real of soft robots with learned residual physics. IEEE Robot Autom Lett, 2024; 9(10):8523–8530.
61.
WangT, HalderU, GribkovaE, et al.Modeling the Neuromuscular Control System of an Octopus Arm. 2022. Available from: https://arxiv.org/abs/2211.06767v1
62.
DagenaisP, HensmanS, HaechlerV, et al.Elephants evolved strategies reducing the biomechanical complexity of their trunk. Curr Biol, 2021; 31(21):4727–4737.
63.
HaK, ZhangW, JangH, et al.Highly sensitive capacitive pressure sensors over a wide pressure range enabled by the hybrid responses of a highly porous nanocomposite. Adv Mater, 2021; 33(48):2103320.
ZouQ, WangY, YangF. An intrinsically embedded pressure-temperature dual-mode soft sensor towards soft robotics. Sensor Actuat A-Phys, 2021; 332:113084.
66.
YuY, LiJ, SolomonSA, et al.All-printed soft human-machine interface for robotic physicochemical sensing. Sci Robot, 2022; 7(67):eabn0495.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
