Sage Journals: Discover world-class research

Abstract

Soft robots and bioinspired systems have revolutionized robot design by incorporating flexibility and deformable materials inspired by nature’s ingenious designs. Similar to many robotic applications, sensing and perception are paramount to enable soft robots to adeptly navigate the unpredictable real world, ensuring safe interactions with both humans and the environment. Despite recent progress, soft robot sensorization still faces significant challenges due to the virtual infinite degrees of freedom of the system and the need for efficient computational models capable of estimating valuable information from sensor data. In this article, we present a new model-based proprioceptive system for slender soft robots based on strain sensing and a strain-based modeling approach called Geometric Variable-Strain (GVS). We develop a flexible 2-Plate 6D strain sensor (Flex-2P6D) capable of measuring the 6 dimensions (6D) strain at specific points of the soft robot with an accuracy higher than 95%. Coupled with the GVS approach, the proposed methodology is able to directly measure the configuration variables and reconstruct complex robot shapes with very high accuracy, even in very challenging conditions. The sensors are embedded inside the soft body, which makes them also suitable for underwater operation and physical interaction with the environment. Something that we also demonstrate experimentally. We believe that our approach has the potential to be applied across a wide variety of applications, including observation and exploration missions, as well as human–robot interaction, where the states of the system are required for implementing precise closed-loop control and estimation methods.

Introduction

The observation of creatures in nature has been a profound source of inspiration for numerous researchers, leading to the development of bioinspired designs. Animals, in particular, exploit their body compliance to navigate complex environments, enabling them to accomplish tasks more efficiently. Soft robotics takes the cue from this research and stands as a growing field, aiming to mimic the physical characteristics of living organisms,¹ including physical compliance, and seamless interaction with the environment. The importance and functionality of soft robots have recently driven new advances in modeling, actuator design, and the development of control and sensory systems for this new generation of robots.^2,3 However, the development of proprioception systems to accurately estimate the state of a soft robot is still a challenging task and requires the development of new technologies and paradigms that can push the boundaries of soft robots’ abilities. Sensory systems enable animals to perceive and respond to stimuli, providing crucial information for various essential functions. Similarly, soft robots require sensory systems to navigate unpredictable environments and perform complex tasks in a more human-like or animal-like manner.⁴

Soft robot sensorization requires unconventional approaches to detect the robot’s large deformations without introducing rigid components into the system.^5–7 Soft robots are highly susceptible to deformation from external forces or internal actuation, resulting in complex and extensive deformations characterized by nonlinearities, hysteresis, and viscoelastic effects, among other factors. This highlights the crucial need for new technologies capable of detecting complex shapes with localized deformations. Earlier research has presented innovative methods to measure the configuration of soft robots maintaining a constant strain or curvature along the robot’s length, restricting the potential to attain complex shape results.^8–12 While these methods prove to be reliable, they encounter difficulties in accurately reconstructing complex shapes that involve variable deformations along the robot length. The highly deformable nature of soft robots, combined with their inherent infinite degrees of freedom, presents a challenge in accurately reconstructing their 3D morphology solely based on a limited set of measurements. In response to the essential need to capture variable strain along the robot structure, and to enable practical applications of soft robots in real-world scenarios, recent studies have been making notable advancements in the development of soft proprioception systems with an expanded set of sensors. In this direction, recent research has successfully implemented multiple sensors across various sections of the robots combined with machine learning algorithms to capture more intricate shapes.^13,14 Although these strategies have demonstrated excellent results, they require the segmentation of the soft robot into a number of elements and assume constant strain along these individual elements. Additional methods involve employing a stretch-receptive sensor network¹⁵ inside the soft robot to reconstruct the full deformation of the robot by measuring the elongation of the sensors or the introduction of stretchable shape-sensing sheets that provide orientation and strain measurements from sensor arrays.¹⁶

The current trend in soft robots estimation explores the emerging confluence of developing biologically inspired stretchable electronic skin (e-skin) and machine learning algorithms to process the sensed data.^8,13,17–20 Despite these recent advances, there remain several challenges yet to be addressed in the field of soft robots estimation. Particularly, the methods based on machine learning face a significant limitation arising from the uncertainty regarding how these algorithms will adjust or adapt to shapes that were not included in their training dataset. Regarding skin sensors, numerous challenges persist, including the enhancement of shape-sensing skins’ stretchability, improving sensor resolution for detecting small curvatures and optimizing their performance amid continuous surface contact or external factors such as temperature or humidity. Furthermore, extracting valuable information from the large quantity of data provided by e-skin sensors requires algorithms that maintain computational efficiency for real-time implementation, especially for control system implementation.^21,22

A rising trend in soft robotics involves utilizing the strain field to represent the robot’s configuration, encompassing curvature, twist, elongation, and shear. Visualizations of the six strains, ϵ ∈ℝ⁶, are provided in Figure 1.

FIG. 1.

The six strains corresponding to ϵ ∈ℝ⁶. Each strain is linked with either a pure translation or rotation along one of the three local axes within the reference frame of the robot.

In this article, we propose a model-based proprioception system designed for soft robots. Our proposed system involves embedding 6 dimension (6D) strain sensors along the robot’s body and utilizes a model-based methodology that correlates the measured strain with the 3D morphology of the system (see Fig. 2). This approach builds upon the recent Geometric Variable-Strain (GVS) model,^23–25 which describes the soft manipulator configuration using a finite set of strain bases. We propose “finite element method (FEM)-like” local strain bases where the 3D geometry can be reconstructed with the measurements of our sensors acting as the generalized coordinates of the system. This approach resembles the FEM; however, it utilizes the values of strain at element nodes. By strategically positioning the sensors at various sections of the robot, our method enables the detection of geometric variations across the entire soft body. The sensor configuration consists of two closely positioned plates embedded in the soft body. By measuring the relative displacement and rotation between these plates, we can compute the 6D strain of the system at that specific point. We refer to this as flexible 2-Plate 6D strain sensor or Flex-2P6D. While previous works in soft robot proprioception primarily focused on measuring bending, twisting, and stretching, for example, in studies^{9–11,14,20,26–33}, our sensor distinguishes itself by its ability to measure the complete 6D strain, marking a distinctive and innovative feature within this particular field (see Table 1).

FIG. 2.

Overview of the proposed methodology for soft robot shapes reconstruction with embedded Flex-2P6D. (A) Rendered view of a soft robot with embedded Flex-2P6D and actuation. (B) Development of bioinspired prototypes. (C) Steps of the proposed proprioception method for soft robots. Flex-2P6D, flexible 2-Plate 6D strain sensor.

Table 1.

State of the Art on Sensing Different Types of Deformation, Along with Their Respective Sensing Mechanism

	Type of sensing deformation	Sensing mechanism
³⁰	Bending one axis	Fiber optic sensors
²⁶	Bending two axis	Liquid metals sensors
³¹	Twisting	Fiber optic sensors
³²	Twisting and bending	Hydrogel
³³	Stretching, bending, localized pressing	Fiber-optic sensors
¹¹	Bending one axis	Graphene woven mesh
⁹	Bending one axis	Magnetic sensors
¹⁰	Twisting and bending	Fiber Bragg grating
¹⁴	Stretching and bending	Magnetic sensors
²⁸	Bending one axis	Electro-optical and resistive strain
²⁹	Bending one axis	Position and pressure sensors
²⁷	Stretching, bending, local compression	Optical, piezoresistive, microfluidic
Our work	Twisting, stretching, bending, shear	Capacitive sensor

Another significant difference from previous methodologies, such as in studies^{13–16,18,19,34}, is that our approach directly measures the generalized coordinates of the system that define the system’s state without requiring additional complex mapping. We believe that this distinctive feature positions our approach as an innovative method for soft robot sensorization, potentially influencing various other fields related to soft robotics. The strains and their numerical differentiation can serve as the states of the system to be utilized for closed-loop control schemes, dynamic observers, and other estimation methods, especially in real-time applications. Our system efficiently integrates embedded sensors and internal actuation in a compact manner, as illustrated in Figure 2A, offering numerous advantages. Embedding sensors within soft robots provides significant advantages as they are shielded from external elements, ensuring increased durability and reliability, particularly in contact with surfaces or submerged in water.^{14,15,34–36}

The efficacy of the proposed system (Flex-2P6D) is validated in different prototypes resulting in the first model-based soft robot proprioception systems capable of measuring local 6D strains. The approach is also validated in scenarios involving physical interaction with underwater environments, demonstrating its potential for exploration in underwater applications.

Methods

Design of Flex-2P6D

Capacitive sensing has gained significant attention among sensor designers thanks to its minimal power consumption, fast response, and easy manufacturing. The design of our Flex-2P6D is inspired by the sensing strategy proposed in ref³⁷, which was originally developed to measure forces and torques in the fingertip of a robot hand. Flex-2P6D consists of two closely positioned plates and employs capacitance sensing mechanisms to detect position and orientation changes between two plates (see Fig. 3A). By measuring the relative displacement and rotation between these plates, we can compute the 6D strain of the system at this specific point. One of the plates contains six embedded sensing cells to measure the 6D strain, while the other plate contains three pins and serves as the ground component. The movement of these pins within the cells generates responses in the sensing cells, enabling the measurement of deformation. The space between the two plates is filled with a thin layer of the same silicone as the soft body, acting as a dielectric. Then, six capacitors are formed between the sensing cells and the pins on the ground plate, facilitating relative movement between the two plates. Each capacitor is formed by pairing one sensing cell with an adjacent section of the pin. The sensing cells serve as the conductive plates, while the pins act as the other conductive plate, and the silicone functions as the dielectric. Thus, the combination of two sensing cells and one pin results in two distinct capacitors. Both plates are designed with a custom shape that accommodates cable passage and internal actuation through the sensor’s section within the soft robot.

FIG. 3.

Design of Flex-2P6D for 3D shape reconstruction of soft robots. (A) Exploded view of Flex-2P6D, highlighting the two plates responsible for measuring the strain of the robot at the specific points they are positioned. Additionally, a front-view photo of the Flex-2P6D assembly is provided. (B) Definition of the sensor geometric parameters. (C) Integration of Flex-2P6D inside the soft body, which involves accurately placing the sensors within the soft robot. This is achieved by inserting multiple needles through small holes in the molds at the intended locations.

We developed a mathematical model that, in conjunction with the sensing approach outlined in ref³⁷, maps the six capacitances of the sensor to the 6D strain. Every sensing cell can be considered as a parallel connection of two capacitors: a normal capacitor between planar electrodes on the pin and the hole, and a lateral capacitor between the walls of the pin and the semicircular electrodes on the hole.^37,38 The mathematical model facilitates the simulation of diverse deformations and allows for the computation of their respective capacitance responses. The combination of this model and an optimization process allows us to derive the most appropriate geometrical parameters for Flex-2P6D based on the specifications for the sensor’s dimensions and the desired range of strains in our application. Further details about the mathematical model, design process, and the final geometrical parameters of the sensor are presented in Supplementary Data S1.

Fabrication of the sensor

The proposed sensor, Flex-2P6D, can be easily developed by fabricating the two aforementioned plates individually and joining them together using the silicone layer. We fabricate the printed circuit board depicted in Supplementary Figure S4 that integrates the sensing cells, a CDC chip (AD7147, Analog Devices) responsible for capacitance readings, and the necessary connections for data transmission to the microcontroller. The sensing cells consist of three via holes designed with specific parameters. To create six individual sensing cells, a rectangular cut is made in the center of these vias. The bottom plate is made of aluminum due to its lightweight nature and excellent electrical conductivity. The silicone layer is manufactured using customized molds mimicking the same shape and dimensions of the plates, with the thickness determined through the optimization process. This layer is manufactured with the same material as the soft robot, ensuring it does not add unnecessary stiffness to the system. As a result, when the sensor is embedded, this layer seamlessly integrates with the soft material, allowing the two plates to float within the silicone structure, causing minimal influence on the stiffness of the robot. Finally, the 4 mm high sensor is precisely embedded in the soft robot at the location where strain measurements are required. Customized molds, shaped to match the final design of the manipulator, are fabricated for constructing the soft robot as illustrated in Figure 3C.

Internal actuation

Various soft actuation methods have been developed to effectively drive soft mechanisms. Cable-driven actuation stands as one of the prominent methods extensively utilized in the field of soft robotics. This particular actuation method surpasses others in its ability to minimize weight and improve the compactness of the mechanism, fully integrating it within the body. One of the proposed soft manipulators is equipped with three straight cables that extend from the base to the tip of the soft manipulator. The cables are phased 120 degrees apart to facilitate deformation in multiple directions. The cables are driven by servo motors connected to pulleys, generating pulling movements to actuate the system.

Model-based reconstruction method

The shape reconstruction method implemented in this work is based on the GVS method.^25,39 In this approach, the soft links are represented by Cosserat rods that can bend, twist, stretch, and shear (6D). Then, the geometry of the robot along its length (rotation and translation) is parameterized by the field of strains. Thanks to that, knowing the strain basis function along the rod, the rod shape can be reconstructed. We divide the soft link into finite strain elements to discretize the continuous strain field into a finite set of FEM-like bases. Traditional FEM shape functions are conventionally utilized to describe the position field, whereas in our case, we employ them to model the strain field. The generalized coordinates’ values correspond to the strains’ values at every node, precisely aligning with what our sensors measure. Then, by positioning the sensors at various points along the soft robot, the complete shape can be reconstructed. This method is recapped and specialized for the needs of the present study in Supplementary Data S1. For more comprehensive details, readers may also refer to study⁴⁰.

Results

Characterization of the sensor

Calibration of the strains involves determining the relationship between the measured output of the sensor (capacitances) and the actual strain being applied. However, directly measuring the actual strain of the system is not feasible. Supplementary Figure S5 illustrates the setup utilized for calibration and evaluation purposes of Flex-2P6D. The soft body is clamped at its base, while the upper section is monitored using a camera system facilitated by the two 3D-printed parts, manufactured at both the base and the tip of the prototype. The transformation matrix that relates the two 3D-printed parts can be measured directly from the camera system. This matrix can be transformed into the resulting 6D strain, assuming constant strain along the body (note that this assumption will not be needed for reconstruction). The mathematical expressions to obtain the 6D strains from the data recorded with the camera system are explained in Supplementary Data S1. In this setup, forces and moments are applied to the upper part of the robot, producing deformation in the soft body, consequently altering the capacitance within the sensor’s six sensing cells. A linear least-squares method was employed to correlate the changes in capacitance with the 6D strain (see Supplementary Data S1). Similar calibration processes have been used in studies^37,41.

The changes in capacitances (ΔC) obtained in the calibration process are shown in Figure 4A, while the comparison between the ground truth strain measurements and the measured values are shown in Figure 4B. Finally, to assess the performance of the sensor characterization, a series of experiments were conducted, involving five repetitions of loading and unloading cycles in each deformation direction. Figure 4C shows the results of these tests, and Table 2 depicts the errors obtained for each strain. Those results demonstrate an accuracy higher than 95% for the six strains. For further details, refer to Supplementary Movie S1, which illustrates the aforementioned experiments.

FIG. 4.

Characterization of Flex-2P6D for 3D shape reconstruction of soft robots. (A) Capacitance readings (ΔC) during the calibration process. Each reading capacitance is defined as follows: ΔC = C − C^∗, where C is the current capacitance and C^∗ is the capacitance without deformation. (B) Comparison between the ground truth strain measurements and the values obtained using obtained using Flex-2P6D. The solid lines represent the ground truth measurements, while the dotted lines depict the measurements recorded by the sensor. (C) Sensing evaluation of Flex-2P6D across the six distinct strains.

Table 2.

Results Derived from the Evaluation Tests

Individual tests	Range	Error	Error percentage (%)	Maximum error
ϵ₁(rad/m)	[−15, 15]	0.26 ± 0.17	1.75 ± 1.15	0.83
ϵ₂(rad/m)	[−12, 12]	0.35 ± 0.22	2.9 ± 1.8	1.00
ϵ₃(rad/m)c	[−12, 12]	0.11 ± 0.43	3.6 ± 2.4	1.42
ϵ₄(m/m)	[−0.1, 0.1]	0.004 ± 0.003	4 ± 3	0.0104
ϵ₅(m/m)	[−0.25, 0.25]	0.011 ± 0.06	2.52 ± 4.24	0.0287
ϵ₆(m/m)	[−0.2, 0.2]	0.007 ± 0.005	3.6 ± 2.4	0.0256

These results are presented in the format of mean ± standard deviation. The error is normalized by the range value observed in the tests. These results indicate an overall accuracy exceeding 95%.

Robot shape reconstruction using one sensor

Accurately reconstructing the shape of a soft robot is extremely challenging due to the system’s infinite degrees of freedom. There exists a compromise between the required number of strain measurements and the complexity of the shapes necessary for reconstructing the shape. We first propose a 90 mm long soft robot equipped with a single sensor possessing a circular cross-section with a diameter of 32 mm. In this setup, the strains are taken at a single point. Thus, a constant strain basis is a natural choice. The prototype underwent manual deformation for 1 min, generating a diverse set of deformations. A representative set of 3D shape reconstructions using the proposed technique is displayed in Figure 5A and Supplementary Movie S2. We demonstrated the efficacy of this methodology, particularly in simple robot geometries where strains are consistently uniform along the length, and there are no localized forces applied at intermediate points of the body. The tip position and orientation of the system are tracked with a camera system and then compared with the estimated measurements obtained with the sensor system (see Fig. 5C). The results show an accuracy of 3.7 ± 2.1 mm and 0.06 ± 0.042 rad for position and orientation errors at the tip, respectively. The position and orientation errors at the tip over time are depicted in Figure 5D. This setup yields good results for specific cases; however, it cannot accurately estimate complex shapes that involve multiple localized deformations. A video demonstrating the real-time 3D shape reconstruction is also presented in Supplementary Movie S2.

FIG. 5.

3D shape reconstruction using a 90 mm long soft robot with one sensor. (A) Snapshots of the soft robot prototype alongside the 3D reconstruction showing distinct deformations. The colors in the 3D shape reconstructions indicate the longitudinal strain experienced on the manipulator’s surface. Red signifies elongation, while blue indicates compression. (B) 6D strain measurements over the whole experiment. The instants of the snapshots are indicated in the figure by dashed lines. (C) Position and orientation of the robot’s tip. The orientation is defined using ZYX Euler angles (α,β,γ). It presents a comparison between the estimated measurements obtained using Flex-2P6D and the measurements tracked by a camera system. (D) Position and orientation errors of the tip of the robot. The orientation error is calculated by employing the logarithm of the matrix transformation ∈ SO(3) that relates the measured and estimated relative orientations. The analytical formula for this is provided in the Supplementary Methods.

Robot shape reconstruction using multiple sensors

To validate the performance of the sensing methodology on a more complex system, we designed a 165 mm long soft robot equipped with three sensors positioned at equally spaced intervals along the body. One sensor is placed at the base, another at one-third, and the last at two-thirds of the body’s length. We employed local FEM bases with two linear and one constant strain elements for the 3D reconstruction of the robot. Figure 6 represents the block diagram to reconstruct the 3D robot shape using the proposed bases. For additional details on the implementation of the GVS model using local FEM bases refer to study⁴².

FIG. 6.

A block diagram depicting the process of 3D shape reconstruction for a soft robot utilizing three sensors and employing local FEM bases. The shape of the soft robot is reconstructed based on capacitance readouts from Flex-2P6D sensors, which are strategically positioned at various points throughout the robot. Subsequently, the capacitance readouts from each sensor, denoted as ΔC ∈ℝ⁶, are converted into 6D strain at specific points ϵ ∈ℝ⁶, using the expression (22) from the Supplementary Material. The matrices involved in this expression are derived from the calibration of each sensor. The strain field along the robot’s length, ϵ(X), is then determined by considering the strain components, defined as $ϵ_{i} (X) = ϵ_{i} (0) f_{1} + ϵ_{i} (\frac{L}{3}) f_{2} + ϵ_{i} (\frac{2 L}{3}) f_{3}$ , where f₁, f₂, and f₃ represent the local strain bases as defined in equations (25a)–(25c) of Supplementary Material. Finally, the robot’s configuration is computed by recursively evaluating equation (29) from the Supplementary Material, assuming that the initial condition g(0) = g₀ is known, and that the exponential map is analytically computed using formulas (1) and (2) of the Supplementary Material, with $\hat{(.)}$ being the isomorphism from ℝ⁶ to se(3). FEM, finite element method.

We conducted a total of five experiments to evaluate the efficiency of the proposed technique. The results of the 3D shape reconstruction are shown in Figure 7 and Supplementary Movies S3 and Movie S4. In the first three experiments, the system’s accuracy in reconstructing the position and orientation of the robot’s tip was assessed to validate the accuracy of the reconstruction process (see Supplementary Fig. S6 and Fig. 7C). The sensor system demonstrates exceptional predictive capabilities, accurately capturing highly complex deformations due to the varying strain along the robot. This capability allows the system to capture localized deformations effectively. Additionally, it exhibits a high sensitivity to minor deformations and effectively senses system vibrations.

FIG. 7.

3D shape reconstruction using a 165 mm long soft robot equipped with three Flex-2P6D. (A) Snapshots of the soft robot during various experiments alongside the 3D reconstruction achieved using the proposed methodology. (B) Illustration of the strain field along the soft robot’s length, emphasizing two linear strain elements and one constant strain element. (C) Performance evaluation of the methodology across three experiments. It shows the position and orientation errors of the robot’s tip (mean ± standard deviation).

A second soft robot, similarly equipped with three sensors positioned identically, was designed, this time integrating internal actuation using three cable-driven actuators. An open-loop trajectory is defined, wherein the system executes a four-cycle rectangular trajectory. The results are depicted in Figure 8A and B. The quantitative tip position and orientation errors are illustrated in Supplementary Figure S7, displaying an accuracy of 4.3 ± 2.7 mm and 0.11 ± 0.075 rad, respectively. Additionally, the results of the 3D shape reconstruction are depicted in Supplementary Movie S5, revealing the consistency in estimation across the four cycles.

FIG. 8.

3D shape reconstruction with internal actuation. Real-world underwater proprioception experiments were performed, validating the effectiveness of our approach in underwater scenarios while physically interacting with the environment. (A) Shape reconstruction results following an open-loop rectangular trajectory using cable actuation. The system executes a four-cycle square trajectory. (B) Comparison between the ground truth measurements (solid lines) and the estimated measurements obtained with the sensors (dotted lines). (C) Comparison of the estimated robot tip trajectories between two underwater experiments employing the same open-loop policy shown in the lower part of the figure. The first experiment displays the system’s movement without interacting with the environment. In contrast, the second experiment involves placing various objects along the trajectory to facilitate interaction with them. The illustration highlights certain actions of the system during the interaction (see Supplementary Movie S6 for more details). (D) Snapshots of the soft robot during the underwater experiment while making contact with the objects alongside the 3D reconstruction.

Real-world underwater proprioception

By emulating bioinspired systems, soft robots can navigate through tightly constrained environments.^3,43 The 3D geometry reconstruction of soft robots while interacting with the environment can help these systems to estimate contacts, properties of the environment, and external forces—essential elements for enhancing their autonomy. We show the 3D shape reconstruction of the robot operating underwater and interacting with the environment. Two underwater experiments were conducted, both following the same open-loop trajectory. In the first experiment, the system moves without interacting with the environment. In contrast, the second experiment involved placing various objects in the middle of the trajectory to facilitate interaction with them. The 3D reconstruction employing the proposed methodology is depicted in Figure 8C and D, and Supplementary Movie S6. The estimated robot tip trajectories in both experiments are depicted in Figure 8C alongside the cable length actuation input. There is a noticeable difference between the two experiments, making it highly intuitive to discern the locations of the objects. This demonstrates the capability of this approach to estimate certain environmental properties—a valuable asset, particularly in autonomous exploration tasks.

Discussion

We have developed a model-based proprioception system that is capable of reconstructing the complete 3D geometric deformations of soft robots in a generalizable manner. We strategically embedded strain sensors within various sections of the soft body to measure the generalized coordinates of the system without the need for additional mappings. These coordinates, together with their derivatives, define the statics and dynamics of the robot. This attribute holds great potential for closed-loop control schemes and dynamics/kinetostatic estimation methods, particularly in real-time applications.

The developed sensor (Flex-2P6D) consists of six embedded sensing cells to measure the 6D strain. While various soft robot proprioception sensors have been previously proposed to measure different types of deformation (refer to Table 1), this sensor stands out as the first of its kind capable of measuring 6D strain at specific points. Its innovative design allows us to fully embed the sensors and internal actuation inside the soft robot, protecting it from contacts or external elements. Furthermore, Flex-2P6D offers a modular and versatile method for sensorizing soft robots. Each sensor can undergo separate calibration to establish the relation between capacitance values and the localized strains. A linear least-squares method was employed to correlate changes in capacitance with the 6D strain. While more complex regression techniques, such as neural networks, could be employed, the results demonstrate high accuracy, exceeding 95% in the operational range, even when accounting for combinations of different strains and their interference. Subsequently, the complete 3D geometry of the robot can be directly derived from the GVS model. This method contrasts with supervised learning methods,¹⁹ where the data acquisition is often expensive, time-consuming, or even impossible.

In this work, we have developed various prototypes equipped with one or three sensors to assess the effectiveness of our approach. Experiments with just one sensor in a 90 mm long soft robot demonstrated good 3D reconstruction results, particularly in simple robot geometries where strains are consistently uniform along the length. However, it revealed limitations in accurately reconstructing complex shapes due to its inability to detect changes in the strain field. We also implemented our approach on two 165 mm long soft manipulators featuring three embedded sensors positioned across different sections of the robot. These prototypes successfully detect localized deformations and reconstruct complex shapes with high accuracy influenced by both external and internal forces applied at different points. These outcomes underscore the importance of using different sensors to measure the strain at different points due to the infinite degrees of freedom inherent in soft bodies. The choice of two linear elements and one constant element for defining the local basis of the current design is not unique. Several other combinations are possible, as long as the number and location of the nodes remain the same. One can think of a single element with a quadratic basis, having nodes at X = 0, X = L/3, and X = 2L/3. In this case, the strain values at X in the range from 2L/3 to L are extrapolated. Another possibility is to divide the entire length into three piecewise constant strain elements. In this scenario, the strain field would be discontinuous. The current choice of strain basis ensures continuity and linearly interpolates the strain based on the nodal values of the first two elements (three nodes). For the last element, a constant strain value corresponding to the third node is used.

Increasing the number of sensors would likely result in improved performance in geometric reconstruction, reducing errors, especially in very complex shapes or longer manipulators. However, this improvement would come at the cost of increased system complexity. Scaling our approach to accommodate a greater number of sensors is easily achievable due to its utilization of the I2C protocol for communication. In terms of current limitations, Flex-2P6D can only be integrated in cross-sections with sizes bigger than the sensor itself. Limiting the implementation of the sensor to thinner soft robots, for example, in minimally invasive applications. For a future study, we envision the adoption of advanced technologies to miniaturize the components of the sensors to be utilized in a wider range of soft robot manipulators.

In summary, the proposed proprioception system represents a promising solution for addressing a wide spectrum of unsolved tasks. It demonstrated the ability to accurately estimate complex shapes under various conditions, including challenging scenarios involving physical interaction within underwater environments. This advancement opens up a broad range of possibilities, such as noninvasive animal monitoring and performing sampling tasks or inspection in human-made underwater structures such as oil and gas pipelines. These applications are driven by the safe interaction between soft robots and nature, ultimately aiming to minimize the impact on animals, plants, and humans. Moreover, we believe that our approach, which is based on embedded sensors, could be complemented with surface sensors, such as e-skin^8,18 or whiskers,⁴⁴ promoting the creation of hyper-redundant sensorized systems capable of perceiving multiple physical parameters. We firmly believe that this integration could significantly enhance the information gathered from the environment, playing an important role in achieving full autonomy for these systems. This is an area we intend to explore in our forthcoming studies. Additionally, our future research will prioritize the estimation of the interaction forces using the proposed sensory system.

Footnotes

Acknowledgment

The authors thank Rawan Abdelfattah for her work in helping with the realization of .

Authors’ Contributions

Conceptualization: D.F.-T., A.T.M., and F.R. Methodology: D.F.-T., Y.A.A., A.T.M., A.Y.A., and F.R. Investigation: D.F.-T., Y.A.A., A.Y.A., A.T.M., and F.R. Design and development: D.F.-T., Y.A.A., A.Y.A., and A.T.M. Experimentation: D.F.-T. and Y.A.A. Visualization: D.F.-T., Y.A.A., A.T.M., and A.Y.A. Funding acquisition: F.R. Project administration: F.R. Supervision: D.F.-T., A.T.M., and F.R. Writing—original draft: D.F.-T., A.T.M., and F.R. Writing—review and editing: D.F.-T., Y.A.A., A.T.M., A.Y.A., and F.R.

Author Disclosure Statement

No competing financial interests exist.

Funding Information

The work was supported by the U.S. Office of Naval Research Global under grant N62909-21-1-2033 and in part by the Khalifa University of Science and Technology under grants CIRA-2020-074 and RC1-2018-KUCARS.

Supplementary Materials

References

Laschi

, Mazzolai

, Cianchetti

. Soft robotics: Technologies and systems pushing the boundaries of robot abilities. Sci Robot, 2016; 1(1):eaah3690.

, Pal

, Aghakhani

, et al. Soft actuators for real-world applications. Nat Rev Mater, 2022; 7(3):235–249.

Hawkes

, Blumenschein

, Greer

, et al. A soft robot that navigates its environment through growth. Sci Robot, 2017; 2(8):eaan3028.

Rus

, Tolley

. Design, fabrication and control of soft robots. Nature, 2015; 521(7553):467–475.

Lin

, Wang

, Zhao

, et al. Recent advances in perceptive intelligence for soft robotics. Advanced Intelligent Systems, 2023; 5(5):2200329.

Case

, Yuen

, Mohammed

, et al. Sensor skins: An overview. Stretchable Bioelectronics for Medical Devices and Systems. Springer; 2016, pp. 173–191.

Amjadi

, Kyung

, Park

, et al. Stretchable, skin-mountable, and wearable strain sensors and their potential applications: A review. Adv Funct Materials, 2016; 26(11):1678–1698.

Van Meerbeek

, De Sa

, Shepherd

. Soft optoelectronic sensory foams with proprioception. Sci Robot, 2018; 3(24):eaau2489.

Mitchell

, Hurley

, Onal

. Fast probabilistic 3-D curvature proprioception with a magnetic soft sensor. In: 2021 IEEE 17th International Conference on Automation Science and Engineering (CASE). IEEE; 2021, pp. 215–220.

10.

, Yurkewich

, Patel

. Curvature, torsion, and force sensing in continuum robots using helically wrapped FBG sensors. IEEE Robot Autom Lett, 2016; 1(2):1052–1059.

11.

Tabassian

, Nguyen

, Umrao

, et al. Graphene mesh for self-sensing ionic soft actuator inspired from mechanoreceptors in human body. Adv Sci (Weinh), 2019; 6(23):1901711.

12.

Scimeca

, Hughes

, Maiolino

, et al. Model-free soft-structure reconstruction for proprioception using tactile arrays. IEEE Robot Autom Lett, 2019; 4(3):2479–2484.

13.

Truby

, Della Santina

, Rus

. Distributed proprioception of 3D configuration in soft, sensorized robots via deep learning. IEEE Robot Autom Lett, 2020; 5(2):3299–3306.

14.

Baaij

, Holkenborg

, Stölzle

, et al. Learning 3D shape proprioception for continuum soft robots with multiple magnetic sensors. Soft Matter, 2022; 19(1):44–56.

15.

Tapia

, Knoop

, Mutný

, et al. Makesense: Automated sensor design for proprioceptive soft robots. Soft Robot, 2020; 7(3):332–345.

16.

Shah

, Woodman

, Sanchez-Botero

, et al. Stretchable Shape-Sensing Sheets. Advanced Intelligent Systems, 2023; 2300343.

17.

Shih

, Shah

, Li

, et al. Electronic skins and machine learning for intelligent soft robots. Sci Robot, 2020; 5(41):eaaz9239.

18.

Cacucciolo

, Shintake

, Kuwajima

, et al. Stretchable pumps for soft machines. Nature, 2019; 572(7770):516–519.

19.

, Giorgio-Serchi

, Zhang

, et al. Stretchable e-skin and transformer enable high-resolution morphological reconstruction for soft robots. Nat Mach Intell, 2023; 5(3):261–272.

20.

Park

, Chen

, Wood

. Design and fabrication of soft artificial skin using embedded microchannels and liquid conductors. IEEE Sensors J, 2012; 12(8):2711–2718.

21.

Haggerty

, Banks

, Kamenar

, et al. Control of soft robots with inertial dynamics. Sci Robot, 2023; 8(81):eadd6864.

22.

Thuruthel

, Falotico

, Renda

, et al. Model-based reinforcement learning for closed-loop dynamic control of soft robotic manipulators. IEEE Trans Robot, 2019; 35(1):124–134.

23.

Renda

, Seneviratne

. A geometric and unified approach for modeling soft-rigid multi-body systems with lumped and distributed degrees of freedom. In: 2018 IEEE International Conference on Robotics and Automation (ICRA); 2018, pp. 1567–1574.

24.

Renda

, Armanini

, Lebastard

, et al. A geometric variable-strain approach for static modeling of soft manipulators with tendon and fluidic actuation. IEEE Robot Autom Lett, 2020; 5(3):4006–4013.

25.

Boyer

, Lebastard

, Candelier

, et al. Dynamics of continuum and soft robots: A strain parameterization based approach. IEEE Trans Robot, 2021; 37(3):847–863.

26.

Xie

, Yuan

, Liu

, et al. A proprioceptive soft tentacle gripper based on crosswise stretchable sensors. IEEE/ASME Trans Mechatron, 2020; 25(4):1841–1850.

27.

Kim

, Lee

, Hong

, et al. Heterogeneous sensing in a multifunctional soft sensor for human-robot interfaces. Sci Robot, 2020; 5(49):eabc6878.

28.

Totaro

, Mondini

, Bellacicca

, et al. Integrated simultaneous detection of tactile and bending cues for soft robotics. Soft Robot, 2017; 4(4):400–410.

29.

Yang

, Chen

. Innovative design of embedded pressure and position sensors for soft actuators. IEEE Robot Autom Lett, 2018; 3(2):656–663.

30.

Zhao

, O’Brien

, Li

, et al. Optoelectronically innervated soft prosthetic hand via stretchable optical waveguides. Sci Robot, 2016; 1(1):eaai7529.

31.

Yang

, Cooper

, Liu

, et al. Twining plant inspired pneumatic soft robotic spiral gripper with a fiber optic twisting sensor. Opt Express, 2020; 28(23):35158–35167.

32.

Wang

, Sun

, Zhao

, et al. A highly stretchable hydrogel sensor for soft robot multi-modal perception. Sensors and Actuators A: Physical, 2021; 331:113006.

33.

Bai

, Li

, Barreiros

, et al. Stretchable distributed fiber-optic sensors. Science, 2020; 370(6518):848–852.

34.

Thuruthel

, Shih

, Laschi

, et al. Soft robot perception using embedded soft sensors and recurrent neural networks. Sci Robot, 2019; 4(26):eaav1488.

35.

Tian

, Cangan

, Navarro

, et al. Multi-tap resistive sensing and FEM modeling enables shape and force estimation in soft robots. IEEE Robot Autom Lett, 2024; 9(3):2830–2837.

36.

Feliu-Talegon

, Mathew

, Alkayas

, et al. Dynamic shape estimation of tendon-driven soft manipulators via actuation readings. IEEE Robot Autom Lett, 2024; 1–8.

37.

Kim

, Jeong

, Do

, et al. Six-axis force/torque fingertip sensor for an anthropomorphic robot hand. IEEE Robot Autom Lett, 2020; 5(4):5566–5572.

38.

Morales

, Diaz

, Herrera

. Solutions of Laplace’s equation with simple boundary conditions, and their applications for capacitors with multiple symmetries. J Electro, 2015; 78:31–45.

39.

Renda

, Boyer

, Dias

, et al. Discrete cosserat approach for multisection soft manipulator dynamics. IEEE Trans Robot, 2018; 34(6):1518–1533.

40.

Mathew

, Hmida

, Armanini

, et al. Sorosim: A matlab toolbox for soft robotics based on the geometric variable-strain approach. arXiv Preprint arXiv, 2021; 210705494.

41.

Kim

, Lee

, Kim

, et al. A novel six-axis force/torque sensor for robotic applications. IEEE/ASME Trans Mechatron, 2017; 22(3):1381–1391.

42.

Mathew

, Feliu-Talegon

, Alkayas

, et al. Reduced order modeling of hybrid soft-rigid robots using global, local, and state-dependent strain parameterization. Reduced order modeling of hybrid soft-rigid robots using global, local, and state-dependent strain parameterization. Inter J Roboti Rese, 2024.

43.

Xie

, Yuan

, Liu

, et al. Octopus-inspired sensorized soft arm for environmental interaction. Sci Robot, 2023; 8(84):eadh7852.

44.

Wang

, Liu

, et al. Potential applications of whisker sensors in Marine science and engineering: A review. JMSE, 2023; 11(11):2108.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

23.85 MB

18.18 MB

16.70 MB

13.22 MB

4.03 MB

9.33 MB

5.78 MB

0.14 MB

0.27 MB

0.28 MB

0.24 MB

0.13 MB

0.99 MB

0.26 MB

0.24 MB

3.60 MB

0.00 MB

Advancing Soft Robot Proprioception Through 6D Strain Sensors Embedding

Abstract

Introduction

Methods

Design of Flex-2P6D

Fabrication of the sensor

Internal actuation

Model-based reconstruction method

Results

Characterization of the sensor

Robot shape reconstruction using one sensor

Robot shape reconstruction using multiple sensors

Real-world underwater proprioception

Discussion

Footnotes

Acknowledgment

Authors’ Contributions

Author Disclosure Statement

Funding Information

Supplementary Materials

References

Supplementary Material