Gravity-aware proactive joint-level compensation for portable soft slender robots using a single IMU and real-time simulation

Abstract

Soft slender robots with high aspect ratios are prone to passive deformation due to gravity, rendering conventional kinematics ineffective and necessitating gravity compensation. Existing solutions often rely on multiple onboard sensors, which increase system complexity and reduce generalizability. This paper presents a bio-inspired real2sim2real framework that enables real-time gravity awareness and proactive joint-level compensation for portable, cable-driven soft slender robots—using only a single inertial measurement unit (IMU) and real-time simulation. In this method, an IMU affixed to the robot base captures orientation changes during quasi-static rotations. Simultaneously, the IMU readings are streamed to a robotic simulation platform established under the SOFA Framework, where the direction of virtual gravity is dynamically updated. To counteract undesired deformation caused by rotated gravity, we use a quadratic programming (QP) solver to continuously compute the necessary joint space motions that actively negate the virtual deformations. These motions are then executed in both the virtual and physical robots, forming a real2sim2real architecture. Experiments validate the effectiveness of the framework, achieving a compensation recovery rate (correlation coefficient) exceeding 99% in static cases, and 94% in low-motion dynamic cases. This novel method enables soft robots to maintain rich state estimation and configuration consistency under varying gravity by leveraging advanced soft body simulation, thereby minimizing reliance on physical sensors. Our method offers a generalizable and scalable solution for gravity-aware control in soft robotics.

Introduction

Soft slender robots have rapidly advanced in recent years, showing great potential for surgical applications (Kwok et al., 2022; Runciman et al., 2019; Yasa et al., 2023). The slenderness and dexterity allow minimally invasive or even noninvasive access to the surgical sites, while the robot softness physically ensures irrefutable safety in human (patient)-robot interaction. When these slender robots are used, the aspect ratio, low moduli, and structural stiffness significantly affect their quasi-static configuration under gravity. Long soft robots distinctly exhibit passive deformation toward the gravitational direction (i.e., becoming saggy, see the left image in Figure 1(b)), as the gravitational potential energy prevails over their elastic energy (Xu and Simaan, 2008), or referred as the self-loading effects of gravity (Marchese et al., 2016).

Figure 1.

(a) Principle of how elephants sense gravity and head orientation through the inner ear system. The head motion alters the gelatinous-like otolithic membrane inside the macula and bends the hairs of the hair cells, generating electrical signals to the nerves. After that, the signal is transferred to the brain for processing. Our work is inspired by this physiological phenomenon. (b) Schematic diagram of the real2sim2real framework for soft slender robots. In an un-actuated state, a long-soft robot made from low-moduli materials will undergo passive deformation when its axial direction (the local “Z axis”) is misaligned with the gravitational direction (the global “−Z axis”). Our proposed framework computes closed-loop compensation joint space motions based on a single IMU and FEM-based simulation.

In most circumstances, people work on predictable, controllable soft robots that can eventually evolve into proprioceptive systems for unpredictable environments (Rich et al., 2018; Zhao et al., 2022). However, soft slender robots are highly susceptible to external forces that impact their shape. The robot’s own mass (viz., gravitational force) is a major dynamic factor to consider for its general motion tasks in 3-D space. This gives rise to an interesting tendency observed in the literature, wherein soft continuum robots are frequently oriented vertically with respect to gravity in practical implementations, likely as a means to mitigate the influence of gravitational forces. Therefore, gravity awareness and the corresponding gravity compensation mechanisms are crucial for effective motion planning and control of soft manipulators operating in diverse and dynamic environments. In the absence of sufficient stiffness, slender robots are unable to withstand gravity-induced deformations; likewise, without adequate sensing, they would fail to perceive the spatial configuration necessary for effective gravity compensation.

The field of soft robotics has presented numerous works addressing these concerns. For example, the development of stiffness variability by implementing jamming mechanisms (Cianchetti et al., 2014; Jadhav et al., 2022; Yang et al., 2021), antagonistic actuation mechanisms (Althoefer, 2018; Konda et al., 2022; Lai et al., 2022), and phase-change materials (PCMs) (Buckner et al., 2019) can make the soft slender robots substantially stiffer to against most external forces and gravity. The drawbacks are commonly acknowledged that jamming mechanisms and using PCMs would freeze the robots from major motions (Lin et al., 2024), which reduces the soft robots’ task space compliance, despite the fact that it can be improved by introducing localized stiffening (Bruder et al., 2023). Besides, both the jamming structures and the routing of PCMs complicate the mechanical design and pose challenges to robot miniaturization. An alternative approach is to utilize dynamic models that incorporate stiffness compensation or physical forces to address joint space control (Della Santina et al., 2020). Model predictive control (MPC) methods have been employed to regulate the motion of soft robots in gravitational environments. However, the presence of unknown and nonlinearly varying model parameters—such as stiffness and damping—often necessitates extensive physical experimentation (Hyatt et al., 2020), introducing ambiguity and uncertainty at the same time. A common approach to mitigate this challenge is to incorporate the robot’s perception into the control loop, which in turn requires effective robotic sensing capabilities. Taking soft robotics as an example, the sensory feedback used for MPC can be eye-to-hand vision in Bruder et al. (2020) (task space level), an electromagnetic sensor as a distal sensor in Cursi et al. (2021) (task space level), pneumatic pressure in Best et al. (2021) and Haggerty et al. (2023) (joint space level), strain sensors in Thuruthel et al. (2019) (configuration space level), optical trackers as distal/body sensors in Molnar et al. (2018) (configuration/task space level). Although they are not specifically developed solely for gravity compensation, these methods often account for the effects of gravity in the control. However, these methods are not well suited for portable applications in which the system’s pose varies over time. Such scenarios are frequently encountered, as soft robots are often deployed as subordinate modules within larger robotic systems.

To conclude, we list some typical measures for gravity compensation in rigid and soft robots in Figure 2.

Figure 2.

Rigid robots can go against gravity with the help of counterweights and springs and can compensate for the forces, including gravitational forces, through different control methods. Soft robots can withstand gravity by enhancing their own stiffness through different methods, such as the use of PCMs (e.g., low melting point alloy (LMPA) (Peters et al., 2019), thermal responsive materials (e.g., liquid crystal elastomer (LCE) (He et al., 2019)), and jamming mechanisms. Even though it is challenging, soft robots can also be actively controlled by exploiting the dynamic models (Luo et al., 2024) and physical/model-based simulation (Caasenbrood et al., 2024) with explicit predefined parameters. Model predictive control (MPC) can employ real-time sensory data for active robot control, including in soft robots (Bruder et al., 2020). Our work utilizes physical simulation and a single sensor to achieve gravity compensation online.

Specifically, we notice a striking trend of employing inertial measurement unit (IMU) sensors on soft robots, and we are convinced that it is because they are low-cost, compact, and easy to deploy. The accessibility of finite-dimensional streaming data also reduces the computational cost for real-time response with the same computing power. Recent works have been creative in sensorizing soft robots with IMU sensor(s) for more possibilities. Even though commercial IMU chips are typically used to measure angular rate or body orientation, soft robotic researchers indirectly use them to measure many other parameters through modeling and learning-based approaches, including robot configurations (shapes) (An et al., 2024; Martin et al., 2022), forces (Ang and Yeow, 2022), stiffness (Lin et al., 2019), and so on, that benefit better motion planning and control. For instance, Martin et al. (2022) proposed using multiple IMUs attached to each link of a soft arm to approximate robot configuration based on the serially chained orientations. Despite the design complication, this method does provide an inexpensive and effective solution for soft robot proprioception. To improve the sensing fidelity and deal with the occlusion problems when using only eye-to-hand vision, Bezawada et al. (2022) presented a model-based method to reconstruct a soft manipulator’s shape through IMU-vision sensory fusion with Pythagorean Hodograph curve model. In Meng et al. (2023), researchers demonstrated the possibility of using a fusion sensing system that contains on-manipulator IMUs and conductive springs to obtain the major deformation of a soft manipulator for closed-loop control with sim-to-real transfer learning methods. The sensory fusion is excluded from the geometric-based simulation, so pre-trained networks are required.

It is worth noting that, based on our inexhaustive review of literature, most IMU-equipped soft robots utilize centimeter-scale IMU chips affixed directly to the continuum body, which increases system redundancy and limits general applicability. A clear implication is that such implementations are unsuitable for soft robots operating at smaller scales. The wiring and extra payload induced by the IMU make delicate manipulation tasks challenging. In addition, for soft robots with high aspect ratios (i.e., elongated structures), multiple IMUs are typically required to provide adequate spatial resolution for accurate shape reconstruction and motion estimation. Besides, conventional learning-based methods are, in general, prototype-dependent. Although the advancement of sim-to-real transfer learning reduces the prototype dependency by exploiting redundant synthetic data (Dubied et al., 2022; Lai et al., 2024; Meng et al., 2023), we argue that using more refined closed-loop physical simulations (Ferrentino et al., 2023) could further narrow the sim-to-real discrepancy with the assistance of real-world sensing. For this reason, real-to-sim-to-real (real2sim2real or R2S2R) transfer has been recently proposed (Lim et al., 2022; Wang et al., 2023) for different sensorimotor robot control. Yet, the real2sim2real framework has not been extensively explored in soft-bodied robotic control.

Thus far, several simulation frameworks have been developed for soft continuum robots (Table 1). For practical real-world integration, an ideal framework should provide explicit representations of the robot’s shape and morphology, rather than relying on overly simplified geometries, and offer extensive flexibility for tuning physical parameters, given the inherent complexity of tangible manipulators. Among these accredited works, SOFA framework has emerged as a widely adopted finite element method-based platform for soft robot simulation (Coevoet et al., 2017; Faure et al., 2012), owing to its modular architecture and support for real-time, physics-based modeling. Its capabilities have been demonstrated in simulating self-collision (Goury et al., 2021), interactions with deformable objects (Baksic et al., 2021; Tagliabue et al., 2021), and hybrid rigid-soft robots (Adagolodjo et al., 2021). These works collectively highlight SOFA’s strengths in high-fidelity deformation modeling, modular multi-physics integration, and compatibility with learning-based methods. However, challenges remain, particularly in computational efficiency and controller integration. Despite these limitations, SOFA remains central to sim2real pipelines in soft robotics, and it is ideal for the real2sim2real deployment.

Table 1.

Overview of simulation frameworks for rigid/soft robots (inexhaustive).

Simulator	Principle	Rigid robots	Soft/continuum robots	Continuum morphology	Rigid-soft hybrid robots	Open source
Gazebo Koenig and Howard (2004)	Rigid body (.cpp/.py)	✓	✗	✗	✗	✓
MoJuCo Todorov et al. (2012)	Rigid body (.py/cpp)	✓	✗	✗	✗	✓
PyBullet + SoMo Graule et al. (2021)	Rigid body (.py)	✓	✓	✗	✓	✓
Huang et al. (2020)	Differential geometry	✗	✓	✓	✗	✗
Elastica Gazzola et al. (2018)	Cosseart rods (.py)	✗	✓	✓	✗	✓
ChainQueen Hu et al. (2019)	Particle-grid hybrid (.py)	✓	✓	✗	✗	✓
Sorotoki Caasenbrood et al. (2024)	FEM/Beam (.m)	✓	✓	✓	✗	✓
SOFA + SoftRobots Plugin Coevoet et al. (2017)	FEM/Beam (.cpp/.py)	✓	✓	✓	✓	✓

This paper reports an IMU-based real2sim2real framework that enables real-time gravity awareness and proactive joint-level compensation on portable cable-driven soft slender robots to maintain the desired robot configuration. The framework is sketched in Figure 1. A compact footprint for cable-driven soft slender robots is presented. (Real) The real-world robot’s pose is read by an IMU attached to the footprint, which is relatively static to the soft robotic manipulator. (Real2Sim) Then, the soft robot’s theoretical configuration, without any active compensation and stiffening, is computed based on the real-time IMU reading that alters the virtual gravitational field and its virtual robot in SOFA Frameworks. (Sim2Real) In the simulator, the virtual soft robot would be passively deformed due to the changing gravity. Simultaneously, necessary joint space compensation that opposes such uninvited deformation would be computed by the built-in numerical optimization in SOFA. Holistically, the resultant joint space adjustment would be responsive to the changing pose of the portable robotic system, making the robot gravity-aware. As such, a single IMU solution becomes desirable regardless of the soft robot’s dimensions, in order to reduce system complexity, minimize redundancy, and maintain applicability across various scales. It electronically mimics the vestibular apparatus of the inner ear system (Dagenais et al., 2021; Day and Fitzpatrick, 2005) that makes mammals aware of the gravity and head orientation (Figure 1(a)).

The main contributions of this paper are as follows: first, we propose a novel control framework that utilizes only a single IMU in conjunction with real-time finite element simulation to enable gravity awareness and proactive joint-level compensation for soft slender robots with high aspect ratios ( > 10) and high joint dimensionality ( ≥ 6 cables). This approach eliminates the need for multiple on-body IMUs typically required by conventional methods, thereby enabling the reconstruction and control of longer robots with minimal sensor requirements for gravity compensation. Second, we introduce and validate a real-time reconstruction pipeline for virtual gravity, leveraging low-drift IMU readings enhanced through a unified solution that streamlines active temperature regulation and real-time filtering via an Extended Kalman Filter. This approach to IMU stabilization opens new opportunities for mono-sensor-based augmented sensing capabilities in soft robotic systems. Third, we present extensive experimental validation on a compact, portable, six-degrees-of-freedom cable-driven soft slender robot. The results demonstrate the proposed framework’s accuracy in motion prediction and compensation, as well as its potential for deployment in portable applications.

Simulation deployment

Simulation environment

A robophysical ¹ simulation would be in favored in the proposed framework. In this work, the soft robotic simulation was established on the SOFA v22.06.99 using the SoftRobots plugin with Python 3.8. Since SOFA is primarily based on the finite element method (FEM) (Ferrentino et al., 2023), continuously altering the robot’s pose in a constant gravity environment will dramatically boost the computational cost. Instead, making the virtual robot standstill and changing the virtual gravity will be more efficient in terms of finite element computation. Moreover, we consider the cable constraints, which cannot be easily described through the other SOFA-ready models, such as the rod model. Here, we assume the robot has no contact with the external environment, but only constrained by the embedded cables (to be discussed in Section 2.2). To do that, the virtual gravity was first set to be $\widehat{\mathbf{G}}={[G_x,G_y,G_z]}^{\top }$ m/s² that resembles the earth gravity G = [0,0,9.81] ^⊤ m/s², subjected to $\Vert \widehat{\mathbf{G}}\Vert =\Vert \mathbf{G}\Vert$ . Given that the IMU reflects the roll-pitch-yaw configuration of the robot base, which notes I = [α,β,γ] ^⊤ , the Euler orientation of virtual gravity ${\widehat{\mathbf{G}}}_o$ becomes

${\widehat{\mathbf{G}}}_o=\underset{{\mathcal{R}}_{\text{RPY}}\left(\mathbf{I}\right)}{\underbrace{\left[\begin{array}{ccc}\hfill \mathrm{c}\alpha \mathrm{c}\beta \hfill & \hfill \mathrm{c}\alpha \mathrm{s}\beta \mathrm{s}\gamma -\mathrm{s}\alpha \mathrm{c}\gamma \hfill & \hfill \mathrm{c}\alpha \mathrm{s}\beta \mathrm{c}\gamma +\mathrm{s}\alpha \mathrm{s}\gamma \hfill \\ \hfill \mathrm{s}\alpha \mathrm{c}\beta \hfill & \hfill \mathrm{s}\alpha \mathrm{s}\beta \mathrm{s}\gamma +\mathrm{c}\alpha \mathrm{c}\gamma \hfill & \hfill \mathrm{s}\alpha \mathrm{s}\beta \mathrm{c}\gamma -\mathrm{c}\alpha \mathrm{s}\gamma \hfill \\ \hfill -\mathrm{s}\beta \hfill & \hfill \mathrm{c}\beta \mathrm{s}\gamma \hfill & \hfill \mathrm{c}\beta \mathrm{c}\gamma \hfill \end{array}\right]}}\cdot \mathbf{G}$

(1)

where ${\mathcal{R}}_{\text{RPY}}\left(\mathbf{I}\right)\in SO(3)$ is the rotation matrix based on the IMU’s reading; s and c denotes the sin() and cos(), respectively.

Proposition. Let ${\widehat{\mathbf{G}}}_o\in {\mathbb{R}}^3$ be a noisy virtual gravity vector such that $\Vert {\widehat{\mathbf{G}}}_o{\Vert }_2\ne \Vert \mathbf{G}{\Vert }_2$ , where G is the true gravity vector with known magnitude ‖G‖₂ = g. Then, the normalized vector

$\widehat{\mathbf{G}}:=\frac{g}{\Vert {\widehat{\mathbf{G}}}_o{\Vert }_2}\cdot {\widehat{\mathbf{G}}}_o$

(2)

satisfies $\Vert \widehat{\mathbf{G}}{\Vert }_2=g$ .

Proof. By definition of the Euclidean norm and scalar multiplication, we compute:

$\Vert \widehat{\mathbf{G}}{\Vert }_2={‖\frac{g}{\Vert {\widehat{\mathbf{G}}}_o{\Vert }_2}\cdot {\widehat{\mathbf{G}}}_o‖}_2.$

(3)

Using the property of norms ‖a ⋅v‖₂ = |a|⋅‖v‖₂ for any scalar $a\in \mathbb{R}$ and vector $\mathbf{v}\in {\mathbb{R}}^n$ , we get:

$\Vert \widehat{\mathbf{G}}{\Vert }_2=\left|\frac{g}{\Vert {\widehat{\mathbf{G}}}_o{\Vert }_2}\right|\cdot \Vert {\widehat{\mathbf{G}}}_o{\Vert }_2=g.$

(4)

Hence, the normalization rescales the noisy vector ${\widehat{\mathbf{G}}}_o$ to have the same magnitude as the true gravity vector G, completing the proof. ■

Remark

Normalizing the IMU readings ensures that the simulated rotating gravity can be treated as if the robot were rotating in a real gravitational environment with a constant gravity direction (toward the ground) and magnitude ‖G‖₂.

Therefore, the normalized virtual gravity should be updated as

$\widehat{\mathbf{G}}=\frac{\Vert \mathbf{G}{\Vert }_2}{\Vert {\widehat{\mathbf{G}}}_o{\Vert }_2}\cdot {\widehat{\mathbf{G}}}_o.$

(5)

The change of virtual gravity reconstructs the pose change of the portable robotic system in the simulation and reduces the cost of finite element computations.

Virtual robot modeling

In SOFA, a soft object is loaded as a 3D volumetric meshed entity, and the robotization can be completed by defining geometric constraints to formulate a “joint-link” relationship. Regarding cable-driven soft robots, the constraints include cable distribution and actuation mechanisms. Here, we used the open-source Gmsh (https://gmsh.info/) (Geuzaine and Remacle, 2009) to mesh the soft robot’s 3D model.

In this paper, we consider a classic two-segment soft robot with each segment driven by three equiangular distributed cables. The cables are not providing any extra pre-tension at the zero state. At the default (straight) configuration, the “insubstantial” cables can be geometrically defined by a number of 3D nodes with respect to (w.r.t.) a base frame {B}, which are governed by

${\mathbf{L}}_i=\left[\begin{array}{c}\hfill L_{i,1,n}\hfill \\ \hfill L_{i,2,n}\hfill \\ \hfill L_{i,3,n}\hfill \end{array}\right]=\left[\begin{array}{ccc}\hfill 0\hfill & \hfill u_{i}r\hfill & \hfill ⋮\hfill \\ \hfill u_i{r}_c\mathrm{s}\beta \hfill & \hfill -u_i{r}_c\mathrm{c}\beta \hfill & \hfill (n-1)d\hfill \\ \hfill -u_i{r}_c\mathrm{s}2\beta \hfill & \hfill -u_i{r}_c\mathrm{c}2\beta \hfill & \hfill ⋮\hfill \end{array}\right],$

(6)

where i = {1, 2} represents the proximal and distal segment, respectively; u _i = (−1)ⁱ⁻¹ denotes the holistic cable distribution’s rotation of a clockwise π between two segments; n = {1, 2, …, N _i } indicates the nth node out of N _i nodes along the slender body. The distal-most node anchors the robot entity. Since part of the cable nodes of the distal segment (L₂) pass through the proximal segment (L₁), the constraint N₂ > N₁ is valid at all times. Besides, r _c represents the radius for cable distribution, $\beta =\frac{2\pi }{3}$ is the isometric offset between the neighboring cables within the segment, and d customizes the density of cable nodes. The above parameterization descriptions are visualized by Figure 3. In addition, physical parameters for the soft materials, such as Young’s modulus, Poisson’s ratio, and total mass, can be customized in the simulator. Details are given in Table 2. The virtual robot is driven by the translation motion of the first node of each cable, and robot deformation and cable nodes given by (6) will be iteratively updated. At a quasi-static state, the robot tip’s frame {T} can be trigonometrically acquired (Lai et al., 2024) as

${\left[\begin{array}{ccc}\hfill \mathit{x}\hfill & \hfill \mathit{y}\hfill & \hfill \mathit{z}\hfill \end{array}\right]}^{\top }=\left({\mathbf{p}}_1-{\mathbf{p}}_2\right)\times \left({\mathbf{p}}_3-{\mathbf{p}}_2\right),$

(7)

where ${\mathbf{p}}_j=L_{2,j,N_2}\in {\mathbb{R}}^{3\times 1}$ is the position of the cable ends w.r.t. {B}, with the cable indexed j = {1, 2, 3}, such that the origin of {T} can be obtained by $\frac{1}{3}{\left({\mathbf{p}}_1+{\mathbf{p}}_2+{\mathbf{p}}_3\right)}^{\top }$ . It should be noted that the actuation of one segment inherently affects the other. This interaction is already accounted for in the simulation framework, and therefore, the segments are not actuated independently.

Figure 3.

(a) Parameters of the virtual two-segment soft robotic model in a cross-section view. This is to model cable tension and robot deformation. (b) Exploded view of cables’ geometric constraints (in the form of cable nodes) of a meshed robot. (c) The defined cable nodes affect the robot configuration in the simulation—reflecting real-world behavior. The example below shows the same robot as in (b), both with cable nodes (untensioned) and without cable nodes, under identical conditions including physical parameters.

Table 2.

Soft robot parameters in real-world measurement and for physical simulation.

	Notation	Value	Unit	Real	Sim.
Young’s modulus	E	0.8	MPa	✓	✓
Poisson’s ratio	ν	0.48	-	✗	✓
Rayleigh stiffness	–	0.5	–	✗	✓
Total mass	m	8.0	g	✓	✓
Total length	L	100.0	mm	✓	✓
Outer diameter	d outer	9.0	mm	✓	✓
Inner diameter	d inner	3.5	mm	✓	✓
Number of nodes	n max	100	–	✗	✓
Gravity	‖G‖	9.81	m/s2	✗	✓

In SOFA, we employ linear elasticity mode as the constitutive law. We seek to compute the coupled cable (actuator) displacements ${\mathbf{u}}_a\in {\mathbb{R}}^m$ (cable length as control input) that will drive the end-effector of a soft robot to a desired position ${\mathbf{x}}_d\in {\mathbb{R}}^3$ , while accounting for system mechanics (from FEM model), joint limits, and environmental effects like gravity and likeliness of body collision. The relationship between the highly coupled cable displacements and end-effector motion can be derived from the Schur complement (Przemieniecki, 1985):

${\mathbf{u}}_i={\mathbf{W}}_{ij}{\mathit{\lambda }}_j+{\mathbf{u}}_i^{\text{free}},$

(8)

where W _ij defines the instantaneous mechanical coupling between the boundary conditions of i and j. The boundary condition may refer to either the effector (e) or the joint actuator (a). The vector ${\mathit{\lambda }}_j\in {\mathbb{R}}^n$ represents the Lagrange multipliers, which encode the constraint forces applied to boundary j, and u _{i or j} denote the corresponding boundary responses. The superscript “free” indicates the default free state without any forces. Therefore, when considering the coupling of multiple cable joints, equation (8) can be modified as

${\mathbf{u}}_a={\mathbf{W}}_{aa}{\mathit{\lambda }}_a+{\mathbf{u}}_a^{\text{free}},$

(9)

where W _aa represents the coupling among all cable joint actuators, and λ _a denotes the force applied by the cables to the soft robot body. As such, equation (9) can be re-organized as

${\mathit{\lambda }}_a={\mathbf{W}}_{aa}^{-1}\left({\mathbf{u}}_a-{\mathbf{u}}_a^{\text{free}}\right).$

(10)

Similarly, when considering the coupling of effector and cable joints (the cable nodes would have contacts to the effector, see Figure 3(b)), and substituting λ _a using equation (10), then, equation (8) can be specified as (Bieze et al., 2018)

${\mathbf{u}}_e={\mathbf{W}}_{ea}\underset{{\mathit{\lambda }}_a}{\underbrace{{\mathbf{W}}_{aa}^{-1}\left({\mathbf{u}}_a-{\mathbf{u}}_a^{\text{free}}\right)}}+{\mathbf{u}}_e^{\text{free}}.$

(11)

Here, equation (11) illustrates a linearized relationship between the contribution of the coupled cable displacement u _a and the end-effector displacement u _e . This relationship is analogous to the form “ $\dot{\mathbf{x}}=\mathbf{J}\dot{\mathit{\theta }}$ ” in rigid-body manipulators, where ${\mathbf{W}}_{ea}{\mathbf{W}}_{aa}^{-1}$ serves a role equivalent to that of a Jacobian matrix.

QP-based inverse solving

For the run time inverse simulation, the quadratic programming with linear complementarity constraints (QPCC) based “QPInverseProblemSolver” (Coevoet et al., 2017) is used to build and solve an inverse problem set by combining end-effector and actuator constraints (refer to equation (11)). In our context, QP-based optimization is preferred over other inverse kinematics methods available in SOFA, as it offers better handling of constraints and improved robustness in scenarios involving soft body deformations. We hypothesize that the inverse solutions are correct enough to control the real-world robot we built, given the robot’s modeling and fine-tuned physical parameters. To reduce computational overhead, we modify the direction of gravity $\widehat{\mathbf{G}}$ rather than altering the entire entity’s direction. Consequently, the desired end-effector position is set to a constant spatial point of ${\mathbf{u}}_e\leftarrow {\mathbf{P}}_d=\left[\begin{array}{c}\hfill P_{dx},P_{dy},P_{dz}\hfill \end{array}\right]$ . Once the simulation is activated, the QP solver computes the displacement changes for each cable subjected to the physical settings, ensuring the robot retains its agravic (i.e., kinematically valid) configuration as $\widehat{\mathbf{G}}$ changes (orientaional only) with every simulation step.

Specifically, to determine the optimal contribution of the cable joint displacements u _a , the solver minimizes the squared Euclidean norm of the error between the computed end-effector position u _e and the desired target position x _d , offset by the free (unactuated) displacement ${\mathbf{x}}^{\text{free}}=f\left(\widehat{\mathbf{G}}\right)$ . Thus, the optimization problem can be formulated as

$\underset{{\mathbf{u}}_a}{\min }\frac{1}{2}{‖{\mathbf{u}}_e-\left({\mathbf{x}}_d-{\mathbf{x}}^{\text{free}}\right)‖}_2^2.$

(12)

Substituting the expression for u _e and omitting constant terms yields

$\underset{{\mathbf{u}}_a}{\min }\frac{1}{2}{‖{\mathbf{W}}_{ea}{\mathbf{W}}_{aa}^{-1}{\mathbf{u}}_a-\left({\mathbf{x}}_d-{\mathbf{x}}^{\text{free}}\right)‖}_2^2.$

(13)

To simplify the objective function (see Appendix I), we define

$\left\{\begin{array}{cc}\hfill \mathbf{Q}& ={\left({\mathbf{W}}_{aa}^{-1}\right)}^{\top }{\mathbf{W}}_{ea}^{\top }{\mathbf{W}}_{ea}{\mathbf{W}}_{aa}^{-1},\hfill \\ \hfill \mathbf{c}& =-{\left({\mathbf{W}}_{aa}^{-1}\right)}^{\top }{\mathbf{W}}_{ea}^{\top }\left({\mathbf{x}}_d-{\mathbf{x}}^{\text{free}}\right),\hfill \end{array}\right.$

(14)

which enables the reformulation of equation (13) into the following quadratic form:

$\begin{array}{cc}\hfill \underset{{\mathbf{u}}_a}{\min }& \frac{1}{2}{\mathbf{u}}_a^{\top }\mathbf{Q}{\mathbf{u}}_a+{\mathbf{c}}^{\top }{\mathbf{u}}_a\hfill \\ \hfill \text{s.t.}& {\mathbf{u}}_a\in [{\mathbf{u}}_{\min },{\mathbf{u}}_{\max }],\hfill \\ \hfill & {\mathit{\lambda }}_a⪰\mathbf{0}.\hfill \end{array}$

(15)

The constraint λ _a ⪰ 0 ensures that the applied force remains non-negative, as only tensile (pulling) forces are permissible. The matrices Q and c encapsulate various physical parameters that can be specified within the simulation environment (see Table 2). The objective function facilitates the computation of an optimal cable joint input—that is, active compensation—that maintains the desired end-effector position, regardless of dynamically changing orientations.

Real2Sim2Real gravity-aware architecture

The precise orientation of the robot’s base—quantified in terms of roll, pitch, and yaw—is captured by an IMU mounted on the robot’s motor frame. This tri-axial data set is then processed by a Python script that calculates the relative direction of gravity based on predefined equations. The computed gravity vector, represented as $\widehat{\mathbf{G}}$ , is subsequently transmitted to the SOFA virtual environment, such that the simulated twin robot can be simultaneously “aware” of the 3D pose of the real-world robot and gravitational direction. Subject to the set physical parameters, the soft robot in SOFA shall be passively deformed toward the changing gravity when no desired tip-reaching spots are assigned.

Within SOFA, when a desired position is assigned to the robot to reach, the gravity parameters can be utilized to determine the necessary displacement for each cable through the computation with QPCC. These cable displacements are critical for the accurate simulation of cable dynamics and are then transmitted back to the robot’s motors. By doing so, the motors are able to facilitate controlled movements of the robotic system, closely mirroring the simulated motion. This iterative feedback loop between real-world measurements and virtual simulations ensures a sync environment and robot motion from both ends. Precautious upper and lower boundaries are set (related to the desired maximum joint input for the soft robot) to protect the soft robot in case of motor malfunction or signal overload. For ease of understanding, the framework is simplified as below (see Algorithm 1).

System description

Interchangeable cable-driven soft slender robots

To verify the proposed framework, we designed a 6-DOF cable-driven soft slender robotic system to control a two-segment articulated robot. A design drawing can be found in Figure 4. The system adopts a compact, modular design for portability and manipulators’ interchangeability. As a result, soft manipulators with different lengths and stiffnesses can be conveniently detached and installed. Here, the manipulator’s design follows the dimension reported in Lai et al. (2022): each segment is 50 mm long and 9 mm in outer diameter. Therefore, this two-segment robot has an aspect ratio of over 11, which is iconic for validating our method. The segments with specific stiffness (Shore 50A) are 3D-printed from the digital mixtures of soft Agilus30 and rigid VeroClear photopolymers using a PolyJet printer (J826 Prime, Stratasys Ltd., USA), while the rigid caps are CNC-machined from acrylic glass. Fishing lines (∅0.7 mm Nylon) that are counterclockwise (as from the front) winded on the 3D-printed capstans are used as actuation cables.

Figure 4.

Mechanical design of the 6-DOF portable cable-driven soft robotic system. The manipulator can be replaced with that in different lengths and stiffnesses to test the proposed real2sim2real framework. (a) Exploded view. (b) Assembly view. An IMU can be attached to the bottom of the frame. (c) The assembled robot prototype. The soft robot naturally droops due to its gravitational potential energy being greater than its elastic potential energy.

Regarding the actuators, six brushless direct current (BLDC) gear motors (M2006 P36, RoboMaster, DJI Technology, China) are concentrically mounted on a 3-D printed frame, and each motor is connected to a capstan via a flexible coupler. The capstans are responsible for pulling and releasing cables that control the robot’s motion. Each motor is connected to a BLDC speed controller that links to the same STM32-based microcontroller (Development Board Type C, RoboMaster, DJI Technology, China) in the CAN bus protocol. The robotic system is powered by a 22.8 V LiPo battery (TB48, DJI Technology, China). The IMU-equipped microcontroller is adhesive to the back of the frame to acquire the 6-axis orientation of the portable system. According to our assembled prototype, the robotic system alone, as shown in Figure 4(b), weighs 1.03 kg (the motors contribute 0.8 kg), which is suitable for hand held purposes or mounting on most collaborative robot arms.

Filters for IMU drifts

Since the microcontroller is equipped with a high-performance built-in IMU (BMI088 chip, Bosch Sensortec GmbH, Germany), we opted for the development board itself to act as the sensing unit instead of using additional IMUs. However, just like most IMUs, our setup also encounters drift problems due to ambient temperature and impact (Grewal and Andrews, 2010). Among them, temperature drift can be quite significant if the microcontroller undergoes long time operation. To get a more stable sensing performance, we modified the IMU on the algorithmic level.

For temperature drift, we employed proportional-integral-derivative (PID) control to actively heat up the chip and let it remain at a stable temperature. Specifically, the microcontroller accesses the chip’s temperature in real-time through high-speed serial peripheral interface (SPI) communication. Then, a 15°C temperature rise would be added to the read temperature, which formulates a target temperature for the IMU heating circuit to heat up (based on the PWM signal). This technique is typically used in outdoor drone applications (ArduPilot, 2024). The temperature gap would be then minimized by the PID controller.

For drift caused by other factors such as possible unexpected impact and rapid movement, we utilized an extended Kalman filter (EKF) to observe and estimate the disturbed IMU orientation based on the previously mentioned temperature control approach. Specifically, we first defined the state vectors as $\mathbf{x}={\left[\mathbf{q},{\mathit{\omega }}^{\text{bias}}\right]}^{\top }$ , where $\mathbf{q}={\left[q_w,q_x,q_y,q_z\right]}^{\top }$ is the quaternion (Hamilton’s notation) representing the rotation of the IMU sensor, and ${\mathit{\omega }}^{\text{bias}}={\left[{\omega }_x^{\text{bias}},{\omega }_y^{\text{bias}}\right]}^{\top }$ represents the zero bias of angular velocity along the subscripted axis. Since the Z-axis (which corresponds to the inertial frame with Z-Y-X intrinsic rotation convention) is typically aligned with the direction of gravitational acceleration, its measured value is much greater than the zero drift error. Therefore, we directly assume the Z-axis zero drift to be negligible in order to simplify the analysis and reduce uncertainty. The omega operator $\mathbf{\Omega }\left(\cdot \right)$ related to the angular velocity $\mathit{\omega }={\left[{\omega }_x,{\omega }_y,{\omega }_z\right]}^{\top }$ of the IMU body is given by (Sabatelli et al., 2012):

$\mathbf{\Omega }=\left[\begin{array}{cc}\hfill 0\hfill & \hfill -{\mathit{\omega }}^{\top }\hfill \\ \hfill \mathit{\omega }\hfill & \hfill {\lfloor \mathit{\omega }\rfloor }_{\times }\hfill \end{array}\right]=\left[\begin{array}{cccc}\hfill 0\hfill & \hfill -{\omega }_x\hfill & \hfill -{\omega }_y\hfill & \hfill -{\omega }_z\hfill \\ \hfill {\omega }_x\hfill & \hfill 0\hfill & \hfill {\omega }_z\hfill & \hfill -{\omega }_y\hfill \\ \hfill {\omega }_y\hfill & \hfill -{\omega }_z\hfill & \hfill 0\hfill & \hfill {\omega }_x\hfill \\ \hfill {\omega }_z\hfill & \hfill {\omega }_y\hfill & \hfill -{\omega }_x\hfill & \hfill 0\hfill \end{array}\right],$

(16)

which fulfills $\dot{\mathbf{q}}=\frac{1}{2}\mathbf{\Omega }\left(\mathit{\omega }\right)\mathbf{q}$ . As such, the process model regarding the state vector can be defined as

$\begin{array}{cc}\hfill {\mathbf{x}}_k& =f\left({\mathbf{x}}_{k-1}\right)+{\mathbf{w}}_{k-1}\hfill \\ \hfill & =\left[\begin{array}{c}\hfill {\mathbf{q}}_{k-1}+\frac{1}{2}\left({\mathbf{\Omega }}_{k-1}-{\mathbf{\Omega }}_{k-1}^{\text{bias}}\right){\mathbf{q}}_{k-1}\mathrm{\Delta }t\hfill \\ \hfill {\mathit{\omega }}_{k-1}^{\text{bias}}\hfill \end{array}\right]+{\mathbf{w}}_{k-1}\hfill \end{array}$

(17)

where k denotes the kth iteration, Δt gives the timespan, and w represents the process noise. The observation model with additive noise can be defined as

${\mathbf{z}}_k=h\left({\mathbf{x}}_k\right)+{\mathbf{v}}_k,$

(18)

where ${\mathbf{v}}_k\sim \mathcal{N}\left({\mathbf{0}}_{3\times 1},{\mathbf{R}}_k\right)$ , with R _k being the measurement noise covariance matrix, which, along with the subsequent process noise covariance matrix Q _k , needs to be continuously updated during the filtering process.

For the update phase of the EKF, the state transition matrix is computed by the Jacobian matrix of the process function $f\left(\mathbf{x}\right)$ from the process model as ${\mathbf{F}}_k=\partial f\left({\mathbf{x}}_k\right)/\partial {\mathbf{x}}_k$ . In our setting, the normalized gravitational acceleration in the inertial coordinate is used as the reference vector, and the rotation matrix derived from quaternions is utilized to transform the inertial coordinate to the IMU body coordinate. When the IMU remains stationary, the acceleration reading along each axis of the IMU would equal the transformed gravitational acceleration. Hence, the observation function $h\left(\mathbf{x}\right)$ can be obtained by

$\begin{array}{cc}\hfill h\left({\mathbf{x}}_k\right)& ={\mathcal{R}}_{\text{quat}}\left({\mathbf{q}}_k\right)\cdot \frac{\mathbf{G}}{\Vert \mathbf{G}\Vert }=\left[\begin{array}{c}\hfill 2\left(q_{x,k}{q}_{z,k}+q_{w,k}{q}_{y,k}\right)\hfill \\ \hfill 2\left(q_{y,k}{q}_{z,k}-q_{w,k}{q}_{x,k}\right)\hfill \\ \hfill 1-2\left(q_{x,k}^2+q_{y,k}^2\right)\hfill \end{array}\right]\hfill \end{array}$

(19)

where ${\mathcal{R}}_{\text{quat}}\left(\mathbf{q}\right)\in SO(3)$ is the quaternion-derived rotation matrix. Similarly, the observation matrix H _k can be derived as ${\mathbf{H}}_k=\partial h\left({\mathbf{x}}_k\right)/\partial {\mathbf{x}}_k$ . The unbiasedness of the aforementioned observation model holds only when the IMU is at rest. When it experiences rapid movements in dynamic situations, the estimated output may exhibit significant bias. Therefore, we appended a chi-squared (χ²) test to verify and filter the observation results. A detection function is used, which is defined by, in quadratic form, ${\mathbf{r}}_k={\mathbf{e}}_k^{\top }{\mathbf{D}}_k^{-1}{\mathbf{e}}_k$ , where e _k is the observation residual between the observed and predicted results, given by ${\mathbf{e}}_k={\mathbf{z}}_k-h\left({\mathbf{x}}_k^{-}\right)$ , and D _k is the variance of its normal distribution. At this point, we could construct the complete flowchart of the EKF for our IMU orientation estimation (Figure 5). All these algorithms are burned into the microcontroller for edge computing.

Figure 5.

EKF diagram for the IMU orientation estimation against drifts. See Appendix II for detailed explanatory notes.

Real-time communication

The layout of our electronic design is given in Figure 6. The development board (microcontroller) is attached to the bottom of the frame that houses the motors, such that the 3D configuration of the whole portable system can reflect the 3D pose of the soft robot’s base. The microcontroller acts as a core electronic part of the communication among different components. It transmits IMU data to SOFA (by every 10 ms, or 100 Hz), guiding the spatial orientation of simulated gravity in real-time, while also managing the operation of individual motors. The design enhances the robot’s compactness, simplicity, and qualities, which are highly sought after in many portable systems.

Figure 6.

Electronic design of the 6-DOF robotic system. The command signals and information are transmitted through the CAN bus.

Experiments

IMU filtering

A reliable and accurate IMU output is a prerequisite for our robot control framework. To ensure this, the raw IMU data is first processed through our designed pipeline before being utilized in simulation. Therefore, we conducted ablation studies to evaluate the impact of PID-based temperature control and the EKF under both static (Figures 7(a)–(b)) and dynamic conditions (Figures 7(c)–(d)).

Figure 7.

Ablation study on IMU filtering performance. (a) Static scenario with temperature control, ablating EKF: significant drift is observed in the yaw angle over time. (b) Dynamic scenario with temperature control, ablating EKF: yaw drift becomes significant. (c) Static scenario with EKF, ablating temperature control: yaw error accumulates over time, though less severe than in the EKF-ablated case, error gradually enlarged. (d) Dynamic scenario with EKF, ablating temperature control: yaw drift is still present, but significantly reduced compared to the EKF-ablated condition.

In our experimental setup, the microcontroller, equipped with an embedded IMU, was mounted on a robotic arm and connected to a PC for data acquisition. An EM tracker was employed to provide ground truth measurements. Each ablation was conducted with at least three trials; however, only one representative trial is shown here to maintain clarity in presentation. Figures 7(a) and (c) illustrate the static scenarios. While our filtering method does not fully eliminate angular velocity jitter (approximately ±1°), it effectively smooths the angular output. However, in both static cases, the yaw angle gradually diverges from the ground truth over time. As shown in Figure 7(c), the inclusion of the EKF module significantly stabilizes the angular readings and reduces yaw drift. Nevertheless, without temperature regulation, the yaw angle still accumulates an error of approximately 20° within 30 s (i.e., a total rotation within 9 min), underscoring the importance of thermal management for long-term IMU reliability. Figures 7(b) and (d) present the dynamic scenarios, where the robotic arm was programmed to perform a repetitive 90° elbow-only motion (joint 6) at an angular velocity of ∼23°/s. The results confirm that the EKF module plays a critical role in mitigating yaw drift and enhancing signal smoothness during motion.

These findings demonstrate that our processing pipeline significantly improves the stability of IMU angular readings in both static and dynamic conditions. The above-demonstrated results emphasize the necessity of both pre-processing (temperature regulation) and intra-processing (EKF filtering) to ensure trustworthy IMU data. Without these measures, natural drift—particularly along the yaw axis for long time operation—can lead to substantial errors, with potential deviations of up to 360° within a few minutes, which would severely compromise simulation fidelity.

Physical simulation fine-tuning

There is no doubt that physically simulated robots—governed by numerous but finite physical parameters—exhibit a noticeable sim2real discrepancy (Schegg et al., 2023). That is, the behavior of virtual robots can only approximate, but not exactly replicate, their real-world counterparts. This discrepancy can be mitigated by fine-tuning the major physical parameters within the simulator, including robot geometry and material properties. The previous sections have detailed the robot modeling process, including the representation of geometrical entities using a mesh model. This subsection focuses on the parameters fine-tuning procedure conducted prior to our robot experiments.

Before initiating the fine-tuning process, we first positioned the soft robot parallel to the ground, ensuring that both the robot tip and base, each equipped with electromagnetic (EM) sensors (3D Guidance trakSTAR, NDI), were free from cable tension, allowing the robot to adopt a naturally sagging configuration. We then recorded the resting position of the robot tip relative to the base (primarily along the Z-axis w.r.t. the robot base), to serve as a ground-truth reference for real-world behavior. The subsequent step involved adjusting key physical parameters in the simulation to replicate this sagging behavior, which is to produce a simulated robot configuration that closely matches the naturally bent robot shape observed in the physical robot.

In the simulation script, we selected two key parameters for fine-tuning: Young’s modulus and Rayleigh damping. This selection is informed by our empirical experimentation, which revealed that these parameters significantly influence the behavior of the virtual robot. Although Rayleigh damping is rarely specified in commercial material datasheets, it is widely used in structural dynamics to model energy dissipation due to internal friction. Based on our observations of the soft material’s behavior and also literature (Armanini et al., 2023), we believe this damping effect is a critical factor that significantly influences the robot’s simulated dynamics and should therefore be carefully calibrated to mitigate the sim2real gap. The fine-tuning workflow is illustrated by a chart shown in Figure 8(a).

Figure 8.

Physical simulation fine-tuning. (a) Flowchart of the fine-tuning procedure. (b) Heat map summarizing the temporal tip position error across 25 combinations of Young’s modulus E_SOFA and Rayleigh damping C_SOFA. The results highlight the parameter pair that best matches the real-world robot deformation. (c) Sim-and-real discrepancies in Z-axis displacement between the simulated and real robot tip positions, illustrating the degree of matching for selected parameter pairs from (b).

Guided by the material specifications (Stratasys Ltd., 2021), we narrowed the parameter search space and conducted 25 simulation trials to identify the optimal combination of Young’s modulus and Rayleigh damping. Specifically, Young’s modulus was varied as E_SOFA = {0.50, 0.65, 0.80, 1.00, 1.20} MPa, and Rayleigh damping as C_SOFA = {0.10, 0.30, 0.50, 0.70, 1.00}. In SOFA, Rayleigh damping can be defined via the Rayleigh mass and Rayleigh stiffness parameters, both of which are set to zero by default. To simplify the tuning process, we set these two sub-parameters to equal values and used their mean as C_SOFA. To evaluate the accuracy of each simulation, we computed the temporal position error of the robot tip, defined as the product of the cumulative error over the time series and its variance:

$e_{\text{total}}=\left(\sum _{i=1}^{\mathcal{N}}{e}_i\right)\cdot \text{Var}(e_1,e_2,\dots ,e_{\mathcal{N}})$

(20)

where e _i = |e_SOFA,i − e_EM,i|.

To visualize the physical response of the robot to the tuned parameters, we produced a heat-map of e_total, as shown in Figure 8(b), to highlight the sim-and-real discrepancy. The heat map applies cubic spline interpolation using $\mathtt{s}\mathtt{c}\mathtt{i}\mathtt{p}\mathtt{y}.\mathtt{i}\mathtt{n}\mathtt{t}\mathtt{e}\mathtt{r}\mathtt{p}\mathtt{o}\mathtt{l}\mathtt{a}\mathtt{t}\mathtt{e}.\mathtt{g}\mathtt{r}\mathtt{i}\mathtt{d}\mathtt{d}\mathtt{a}\mathtt{t}\mathtt{a}$ to interpolate 25 discrete data points onto a 100-by-100 grid. The resulting grid is then rendered as a smooth surface using $\mathtt{p}\mathtt{l}\mathtt{o}\mathtt{t}\text{\_}\mathtt{s}\mathtt{u}\mathtt{r}\mathtt{f}\mathtt{a}\mathtt{c}\mathtt{e}$ . The results clearly indicate that the parameter pair, E_SOFA = 0.80 MPa and C_SOFA = 0.50, yields the closest match to the real-world robot’s sagging behavior. Figure 8(c) demonstrates some sim-and-real discrepancies along Z-axis displacement between the simulated and real robot tip positions (i.e., how much of falling) based on selected parameter pairs (i)–(iv) from Figure 8(b). In case (i), the simulation fails to converge to the EM-benchmarked position under a low modulus setting. In case (ii), the robot exhibits over-damping, characterized by high viscosity and insufficient bending. In case (iii), the robot is overly stiff with a low damping rate, resulting in apparent oscillations before stabilizing. Among all 25 experimental trials, the best-performing case achieves a satisfactory natural bending behavior by balancing Rayleigh damping and Young’s modulus—although the temporal profile does not perfectly align with the real-world scenario. It should be noted that this method allows only limited fine-tuning, depending on the granularity of the parameter selection.

Real2Sim

Our hypothesis posits that virtual gravity in simulations can accurately represent the real-world robot’s free-space deformation, and that these simulated gravity compensation solutions are applicable in real-world scenarios. To validate this hypothesis and bridge the real2sim gap, we need to first verify the reliability of our simulation outcomes. For multi-axis sensing, we employed an EM tracking system to measure the position and orientation (6-axis) of the sensors affixed to the robot’s tip and base, serving as our ground truth. According to the specifications, the accuracy for sensor positioning and orientation is 1.4 mm and 0.5°, respectively. The IMU data was integrated into the SOFA framework in real-time, running on a PC equipped with an Intel i7-12,700 KF CPU. This setup yielded a dynamically responsive virtual robot, achieving an update rate of approximately 6 frames per second.

The closeup of our experiment setup is depicted in Figure 9. We positioned the portable robotic system at various inclinations toward the ground, specifically at angles of 135°, 90°, 45°, 0°, and −45°, to examine the deformation behavior of the soft manipulator deviating from its initial straight configuration. In the experiment, we dragged the robot tip to make it straight with the assistance of EM sensors’ reading. When the tip’s pose approached (0,0,100) w.r.t. the robot base, we released it for free fall and activated the simulation at the same time. Figure 10 presents the comparison results, illustrating that the simulation, based on real2sim gravity, accurately mirrors the real-world robot’s deformation, with negligible discrepancies at the distal tip—specifically, absolute errors within 4.7 mm in axial position (representing a relative error ≤4.7% with respect to the total length of the soft robot) and 5.2° in orientation. Furthermore, the results reveal a consistent match in temporal deformations between the simulated and actual robot behaviors. These minor errors are deemed inconsequential in the overall assessment of physical performance. Possible sources of these discrepancies include measurement inaccuracies (with the measurement system having a positioning accuracy of 1.4 mm), the added weight of the sensor at the tip, sensor wiring, and assembly inaccuracies.

Figure 9.

Experiment setup for the Real2Sim verification. This picture gives an example of the robot’s free space dropping when the system is parallel to the ground (i.e., 0°, as shown in Figure 10). The tip pose can be measured by the EM sensors.

Figure 10.

The experimental outcomes, juxtaposing the tip position and orientation of the SOFA-simulated robot (dashed dark lines) with the real-world robot (solid scarlet lines) in the absence of active actuation compensation across various initial inclinations toward the ground, are presented. The 100 mm-length soft robot exhibits natural deformation attributed to its slender structure, softness, and the force of gravity. These findings demonstrate that our simulated robot, under the influence of real2sim (interactive) gravity, can closely mirror the actual system configurations and soft robot deformation with minimal discrepancies.

The traditional Euler–Bernoulli beam theory offers a framework for characterizing soft, slender, hollow robotic structures, akin to the one we introduce in this study. Nevertheless, it falls short of accurately capturing certain complexities, such as the tension forces within cables. While cable-driven continuum robots have been theoretically modeled as cantilever Kirchhoff beams (Boyer et al., 2020), the adoption of physics-based simulations facilitates a more straightforward articulation of cable constraints and the examination of asymmetric scenarios. Here, we provide a comparison of computed tip deflection between simulation-based and naive mechanics-based methods. We consider the soft robot to be a free-end hollow cylindrical cantilever beam. The beam’s weight acts as a uniformly distributed load. As such, the maximum deflection δ_max can be computed by

${\delta }_{\max }=-\frac{w{L}^4}{8E{I}_z},$

(21)

where w denotes the effective mass per robot length, and I _z is the area moment of inertia for a hollow cylindrical section associated with a specific outer and inner diameter, and as in our case, they shall be

$w=\frac{m\Vert \mathbf{G}\Vert }{L}\cos {\theta }_{\text{incline}},$

(22)

$I_z=\frac{\pi }{64}\left(d_{\text{outer}}^4-d_{\text{inner}}^4\right)$

(23)

so that leave the static deflection of the robot tip when it reaches the equilibrium. We incorporate the theoretical maximum deflection into Figure 11 for ease of evaluation. The result highlights the reliability of our interactive real2sim pipeline in terms of quantitative and temporal capabilities. Its quantitative side can be judged by the significant reduction in deflection error of at least 83% over the simple cantilever beam modeling that demonstrates a much larger deflection using the same set of soft robot parameters in Table 2. The pictures in Figure 11 show the highly matched deformed robot’s appearance between virtual and actual robots in different system configurations. It can also be observed that the deformations in the 135-degree case and 45-degree case are slightly varied. Such variation is caused by the cable allocation inside the soft robot, as the cables opposed to the bending side are always stretched to resist the bending, and the longer cable provides a longer moment arm. It can be inferred that such variation can be very different when the robot bends toward different directions; and would be enlarged on higher aspect ratio robots. Variations of a similar kind are challenging to describe when using beam theory due to the complexity, but convenient to reproduce by FEMs.

Figure 11.

The gravitational-axis deflections of the robot tip computed in SOFA simulation, COMSOL simulation, and Euler–Bernoulli beam theory method, compared with the actual deflections measured by EM sensors. Each column gives a different inclination angle θ_incline toward the ground. In the First row, the deflections of the tip caused by the robot’s own mass are described, highlighting the exceptional real2sim quantitative and temporal capabilities of SOFA. Second row visualizes the superiority of using the SOFA-based simulation over simple cantilever beam modeling, with a significant reduction in deflection error of at least 83%. Third row provides the deflection results (unit: mm) of COMSOL Multiphysics simulation under the same physical parameters that we used in SOFA. The last two rows provide a visual representation of the simulated and actual appearances of the same robot in the equilibrium state, respectively. The SOFA simulations demonstrate a higher match to the actual robot behavior.

Beyond the SoftRobots plugins that facilitate the numerical computation of inverse kinematics and dynamics for cable-driven soft robots within the SOFA framework, the implementation of reduced-order finite element modeling improves the computing speed and showcases the potential for real-time deployment. For comparison, another general (commercial) FEM-based simulator, COMSOL Multiphysics v6.2, has also been put to the test. The simulator meshes our robot model into 4,039 tetrahedrons, while we use a more finely meshed model in SOFA with 4,788 tetrahedrons. Although we could not ensure a perfect match between the two, we made an effort to keep the mesh sizes/number as close as possible to enable a fair comparison. Other physical parameters used in SOFA have been replicated in the COMSOL simulator as much as possible, except for those that are not able to be defined. As shown in Figure 11, the COMSOL-computed deformations present similarities to what we obtained from the SOFA simulator and the experiments. However, it is observed that the results obtained from SOFA demonstrate marginally higher accuracy in comparison to those from COMSOL when aligned against experimental outcomes. This discrepancy may be attributed to the incorporation of cable constraints within the SOFA framework, but not readily available in other FEM simulation software. Besides, the computational time significantly varies: while the traditional mechanics-based and SOFA-based methods compute almost instantly, the commercial software takes minutes.

Real2Sim2Real: Static cases

Based on the confirmation that the configured SOFA simulation could reflect the real-world robot under static conditions, we proceeded to apply the gravity-aware control policy derived from the simulator to the actual robot, and such application was deployed online.

A series of controlled experiments have been designed and carried out for examination. The system architecture is shown in Figure 1. We first configured the physical robot with a certain angle (either known or unknown) w.r.t. the ground. Then, the IMU that adhered to the robot frame communicates with the virtual environment by updating the real-time 3D pose, which alters the virtual gravity’s orientation. A desired target tip location was assigned in the virtual environment. For ease of evaluation, in the experiment, we assigned a target location that could result in a straight robot configuration (i.e., (0,0,100) in {B} frame). After that, the QPInverseProblemSolver computes the joint motion that could correct the soft robot’s configuration with animated visualization in SOFA, which is simultaneously executed by the real motor joints. Related results can be found in Movie S1, and its snapshots are shown in Figures 12 and 13. Figure 12(a) presents the results for a horizontally posed robot using our proposed real2sim2real framework. We can see that the slender robot sags to a bent state after it loses physical support. As the simulator is simultaneously activated, the virtual robot also falls in the same direction (see Figure 12(b)). When the gravitational and elastic energy reaches an equilibrium, the bending stops, and the compensation starts effecting, gradually correcting the bent robot back to the 3D pose it is supposed to be. The EM sensor-measured tip position in the Y and Z axis is shown in Figures 12(c)–(d). The real measurements, once again, indicate that the actual deformation can be reflected by the virtual deflection. Although the sim2real discrepancy is relatively large at the initial stage, where the deformation is unstable and highly nonlinear, it finally converges to a minimum as the real2sim2real iteration continues—all relying on a single IMU attached to the robot footprint as a source of feedback. We also highlight the necessary time for our method to dent the discrepancy. In Figures 12(c)–(d), the gray shading refers to the period when gravitational energy dominates in the simulation, while the gray-and-red shading shows the same variables in the actual measurement. Here, the red shading stands for the sim2real delay (virtual peak to reality peak) of approximately 3 seconds. The visible real-world compensation motion appears at about the 6th second. We infer that such delay is related to the hysteresis that occurs in our soft robotic structure because of its slenderness and low-elasticity 3D-printed material.

Figure 12.

(a) The soft slender robot bends toward the ground upon losing its physical support from a human hand. By employing the proposed real2sim2real-based gravity aware framework, it gradually self-corrects its 3D configuration according to the real-time computation from (b) the simulated robot in the virtual environment. (c)–(d) The EM sensor-measured tip position in the Y and Z axis. The same reference frame is used in Figure 10. The X axis is not shown as there is no significant difference in such a planar motion. The red shading refers to the sim2real delay interpreted by the virtual peak to reality peak distance (Scale bars: 10 mm).

Figure 13.

The real-world robot autonomously reverts back to the straight/desired configuration when it employs the proposed real2sim2real framework. The tilted angles are (a) $\frac{\pi }{4}$ and (b) $\frac{3\pi }{4}$ , respectively (scale bars: 10 mm).

Figure 13 shows the results of repeated experiments with different spatial poses. It can be well noticed that, due to the cable allocation, these symmetric poses ( $\frac{\pi }{4}$ vs $\frac{3\pi }{4}$ ) result in different initial configurations on the soft robot. Therefore, these two experiments are essentially irrelevant in terms of joint space compensation. The measured metrics imply similar conclusions in (1) real2sim reconstruction, (2) decaying sim2real discrepancy over time, (3) effective real2sim2real gravity awareness and compensation, and (4) an about-three-second sim2real delay. The above results are sufficient to verify our hypothesis.

To further evaluate the generalizability of our method, we replaced the robot’s distal segment (originally 50 mm in length) with a longer segment measuring 80 mm. Without additional fine-tuning, but only updating the virtual geometry and cable node constraints, we repeated the same free-fall experiment illustrated in Figure 12. The results, presented in Figures 14(a)–(d), demonstrate that our method remains effective even with a more slender robot (Slenderness of 14.44). In the final stabilized phase, the virtual robot does not maintain a perfectly straight configuration after compensation, mirroring the behavior observed in the physical robot, where the midsection exhibits a slight downward deflection. When the robot is elongated, the weight of its own body, especially in the middle, causes it to sag under gravity. The longer the segment, the greater the moment arm, which increases the bending moment and leads to more pronounced deflection. The longer robot may also introduce slack or uneven tension distribution, cause a weak midsection.

Figure 14.

(a) The soft slender robot, with a longer second segment (80 mm), bends toward the ground upon losing its physical support from a human hand. (b) The simulated robot in the virtual environment. (c)–(d) The EM sensor-measured and the SOFA-predicted tip position in the Y and Z axis. (Scale bars: 10 mm).

It would be hasten to add that, based on the results computed in SOFA and the real-world experiments, it is inherently impossible to maintain a perfectly straight robot configuration as desired (see the reference lines in Figures 12 and 13), regardless of the amount of actuation or compensation applied. Nonetheless, our results demonstrate that the simulated behavior closely matches the real-world measurements, indicating that the virtual robot (and its computed active joint-level compensation) accurately reflects the physical dynamics of the soft robotic system. Therefore, the comparison between the SOFA simulation and the real-world measurements already demonstrates the effectiveness of the proposed compensation method.

Moreover, it can be observed from Figures 12 and 13 that the Z-coordinate of the robot tip recovers more effectively than the Y-coordinate after compensation. This is because the theoretical deviation (based on the geometric relationship) in the Y-coordinate is consistently larger than that in the Z-coordinate for the same deflection angle.

Last but not least, we use the sample Pearson correlation coefficient r, as defined by equation (24), to quantify the correlation between the measured (p_meas,i) and simulated data (p_sofa,i) of the 3D tip position over time. The correlation coefficient for 0° (Figure 12), 45° (Figure 13(a)), and 135° (Figure 13(b)) are 0.9984, 0.9991, and 0.9993, respectively.

$r=\frac{\sum _{i=1}^T\left({\mathbf{p}}_{\text{meas},i}-{\overline{\mathbf{p}}}_{\text{meas},i}\right)\left({\mathbf{p}}_{\text{sofa},i}-{\overline{\mathbf{p}}}_{\text{sofa},i}\right)}{\sqrt{\sum _{i=1}^T{\left({\mathbf{p}}_{\text{meas},i}-{\overline{\mathbf{p}}}_{\text{meas},i}\right)}^2\sum _{i=1}^T{\left({\mathbf{p}}_{\text{sofa},i}-{\overline{\mathbf{p}}}_{\text{sofa},i}\right)}^2}}$

(24)

Portable Real2Sim2Real: Continuously change cases

Related results can be found in Movie S2, and its snapshots are shown in Figure 15. A total of five sequential Tool Center Points (TCPs) for the Panda arm were assigned to realize repeatable changing motion. In addition, the observed sim2real delay necessitates a low-speed portable motion. Therefore, we set the Panda arm’s linear velocity as 1% of its maximum Cartesian velocity limits.

Figure 15.

We applied our gravity-aware framework to a portable soft slender robot mounted on a moving robot arm. The robot arm performs two repeated sequential motions (each lasts for 90 seconds) defined by five TCPs. In (a), the soft robot maintains a straight configuration and achieves a straighter pose compared to the configuration without our method, as shown in (b). In (c), the soft robot demonstrates a 30-degree bend, with the measured results shown in (d). Note that the results in (b) and (d) are relative to the reference frame from the optical tracker at fixed positions.

Robot arm held

To further evaluate the framework’s performance, we tested it in cases where the system’s spatial poses are continuously changed enabled by a robot arm. In the experiment, our portable soft robotic system was mounted on a Panda arm (Franka Emika GmbH, Germany). The Panda arm was teleoperated to change the 3D pose of its end-effector (viz., our soft robotic system), and such alteration was unknown to the simulation, as the simulation only reads from the embedded IMU. Since the EM sensors’ wiring may affect the soft robot’s deformation due to the large-scale motion, we switched to position-only optical tracking (FusionTrack 250, Atracsys LLC, Switzerland). The setup can be seen in Figure 16.

Figure 16.

Experiment setup for the portable real2sim2real cases where the soft robot undergoes continuous overall pose changes in 3D space.

The results in Figure 15 verify our hypothesis and the effectiveness of our proposed framework. Regardless of the soft robot configurations (straight or bent) and the overall system configurations, the soft robot can be self-adjusted against gravity-induced undesired deformation as guided by the real-time simulation. Figures 15(c) and (d) show the soft robot’s tip positions (w.r.t. a stationary optical tracker). The results present a significant improvement in recovering the compensation strategy from the simulation, compared to the cases without amending any joint space motion compensation. Again, we use the sample Pearson correlation coefficient r to quantify the correlation between the measured and simulated data over time. The correlation coefficient for 0° as shown in Figure 15(a) and 30° as shown in Figure 15(c), improves from 0.9653 to 0.9830, and from 0.9394 to 0.9439, respectively.

Hand held

We conducted experiments to validate the applicability of our proposed method for soft slender robots in portable scenarios. The setup involved a cross-sectional two-way lumen phantom with a spherical object positioned deep within the lumen, accessible by the robot through the leaned lumens on both sides (see Figure 17). Note that the left lumen is straight, while the right one is curved. In the experiment, a volunteer was asked to hold the robot in one hand and retrieve the BluTack-glued spherical object through the lumen. Due to the long slopes of the lumens, our 100-mm long soft robot, without the proposed real2sim2real method, risked getting stuck and colliding with the lumen wall due to gravity-induced deformation. As shown in Figure 17(a) and in Movie S3, our method enables the handheld robot to reach the target object despite the slopes. This is achieved through real-time gravity awareness and autonomous configuration adjustment, facilitated by a single onboard IMU and our proposed framework. A comparison on straight configuration of the portable robot operating with and without the proposed gravity compensation method is shown in Figure 18. The results suggest that our method can be applied to handheld soft slender robots without manual task/joint space motion control. Our online real2sim2real framework allows a 6-DOF soft robot to perform arbitrary handheld tasks and can be, apparently, extended to a wider range of soft robots with a high aspect ratio and high DOF in the joint space. Possible applications include gastroscopy for patients in various postures and industrial endoscopy.

Figure 17.

Portable experiment—A volunteer was asked to hold the robot in one hand and retrieve the BluTack-glued spherical object (positioned inside the two-way phantom) through the sloped lumen. (a) The leaning lumen is straight and requires the robot to be straight. (b) The leaning lumen is curved and requires the robot to maintain the same curve shape throughout the object retrieval process.

Figure 18.

Comparison on straight configuration of the portable robot operating with and without the proposed gravity-aware proactive joint-level compensation strategy.

We extended our experiment to evaluate the applicability of the proposed method and system in medical settings. Prior studies have shown that altering patient posture during colonoscopy can enhance luminal distention and facilitate navigation (Wilson and Saunders, 2015). Although the left lateral (L) decubitus position is conventionally used, alternative orientations such as right lateral (R), prone (P), and supine (S) may offer additional advantages. A self-controlled colonoscopy robot capable of maintaining consistent performance across varying patient positions would therefore be highly beneficial. In our experimental setup, we employed a phantom colon embedded with a lesion model to simulate transrectal colonoscopy using the proposed hand-held robotic system and control framework. As illustrated in the experiment setup in Figure 19(a), an EM tracking system was used to record the robot tip position, while a micro-camera (OV6946, OmniVision) was inserted through the robot’s working channel to replicate standard endoscopic visualization. The phantom was positioned in both the supine (S, Figures 19(d)–(e)) and left lateral (L, Figures 19(e)–(f)) orientations. Prior to the procedure, an EM sensor was used to slide along the top, bottom, and lateral surfaces of the colon phantom to capture its external geometry in the form of positions relative to the transmitter frame. The results shown in Figures 19(b)–(c) illustrate the EM-tracked tip positions of the robot with and without the proposed gravity-aware joint-level compensation method. Without compensation, the robot gradually deviates from the intended shape and drifts toward the colon wall during insertion. In contrast, when the gravity-aware strategy is applied, the robot actively maintains the desired bent configuration, demonstrating improved stability and control. The results, presented in Figures 19(d)–(g) and Supplemental Movie S3, demonstrate that the robot maintained a stable configuration despite changes in body orientation and operator-induced perturbations. This stability enabled consistent visualization of the intracolon environment and lesion, underscoring the system’s robustness and potential for clinical application.

Figure 19.

Different body positions for colonoscopy. (a) A phantom colon with a lesion model was used to simulate transrectal colonoscopy with our hand-held soft robot and control framework. An EM sensor was used to measure the robot tip’s position. (b) Comparison between with and without implementing our gravity-aware (GA) joint-level compensation method in the supine position setting. The “patient” was positioned in (d–e) supine and (f–g) left lateral positions. Our method ensures consistent motion and stable imaging regardless of patient position and robot body orientation.

Discussion

The reported control framework enables real-time gravity awareness and proactive joint-level compensation on portable cable-driven soft slender robots to maintain the desired robot configuration. We identify the advantages of facilitating compact design, which requires only a single IMU at the robot’s footprint. Our method is applicable to soft robots with a high aspect ratio (slenderness) up to 14.4 with high DOFs. Moreover, our framework illustrates a robot and virtual gravity simulation method based on open-source software and a commercially available development board, which facilitates the generality practice in other highly nonlinear soft slender robots that require FEM modeling instead of traditional robotics modeling. In Table 3, we compare some related works of soft robots with some sense of gravity-aware capability and their compensation mechanisms against gravity-induced deformation. The comparison implies that researchers tend to develop active compensation strategies when the soft robots have higher DOFs, often seen in cable-driven cases for controllability and responsiveness. Compared to state-of-the-art works that use on-board and on-body IMUs for robot proprioception (Pei et al., 2024; Peng et al., 2024; Stella et al., 2023), multiple sensors (usually 1 IMU per 3 DOFs) are necessary to gain comprehensive feedback for closed-loop control. Such a restriction requires IMUs to be attached along the robot. In our method, we further reduced the use of necessary sensors (1 IMU for many DOFs) with a novel FEM simulation-based ideology, and without any on-body IMUs, which significantly reduces the wiring and enables the control of even longer and more redundant robots. As shown in Figure 20, we identify the framework’s potential to bridge the gap in previously limited DOFs that a single IMU can control (horizontal axis), as well as the limited robot slenderness when applying active gravity compensation approaches (vertical axis). Although we demonstrate only a “1 IMU for 6 DOFs” example, the soft robot configuration is, in principle and as can be readily inferred, not limited by this specific case.

Table 3.

Soft robots with gravity-aware capability and their compensation mechanisms 11.

Ref.	Actuation	Slenderness(L/douter)	Gravity Compen.	Portable	Model/simulation	Soft robot DOFs	Sensor(s)	# of Sensor(s)	Methodology
Bruder et al. (2023)	Pneumatic	∼3.5	Passive	✗	Dynamics (SoMoGym)	3	–	–	External stiffening
Shah et al. (2021)	Pneumatic	–	Passive	✓	FEM (Voxelyze)	–	–	–	Sim2real
Kim et al. (2019)	Magnetic	10–20	Passive	✗	FEM (Abaqus)	2	–	–	Materials design (use stiffer materials)
Zhou et al. (2024)	Cable	7.75	Passive	✗	Data-driven dynamics	4	–	–	Collecting real world data for learning
Pustina et al. (2022)	Generic	–	Active	–	Dynamics (numerical)	#	Torque sensors (expected)	Identical to # DOFs	PD regulator
Wang et al. (2019)	Cable	∼6	Active	✗	Kinematics (Cosserat)	4	Camera & force sensor	2	Hybrid vision-force controller
Stella et al. (2023)	Cable	7.5	Active	✗	Kinematics (PCC)	6	IMU	2	Piecewise constant curvature (PCC)-based
Pei et al. (2024)	Cable	∼8.5	Active	✗	Kinematics (PCC)	12	IMU	4	IMU integration
Peng et al. (2024)	Cable	4.97	Active	✗	Kinematics (PCC)	9	IMU	3	IMU integration
Ours	Cable	11.1–14.4	Active	✓	FEM (SOFA)	6+	IMU	1	Real2sim2real

Figure 20.

Comparing different state-of-the-art soft robots with gravity-aware capability and their compensation mechanisms based on the number of DOF(s) controlled by single sensor (IMU) and the robot’s slenderness.

Compared to other commonly used sensors for soft robots (such as fiber Bragg grating (FBG) sensors, optical/vision sensors, and EM sensors), IMUs are a relatively cost-effective option that require less installation complexity and sensor calibration. For instance, employing FBG sensors in a continuum robot requires delicate robot design to ensure sufficient internal space to accommodate the customized fragile optical fiber (Li et al., 2024), as well as the use of a laptop-sized FBG interrogator. If deformation is of interest, additional calibration using a mold with a fixed curvature is typically required for bending calibration (Hou et al., 2025). While soft waveguides are cost-effective and highly flexible, their practical implementation often requires extensive training using neural networks or a combination of numerical simulations and neural networks (Wang et al., 2024), which can hinder individual deployment and scalability. Electromagnetic tracking remains a widely adopted localization (also shape sensing (Shi et al., 2016)) technique in laboratory-based soft robots due to its non-line-of-sight capabilities and high spatial resolution. However, its deployment in slender robots often necessitates the integration of multiple tethered sensors along the body, which introduces practical limitations, including restricted maneuverability and reduced scalability, primarily due to the constrained effective workspace imposed by the EM field and the physical encumbrance of sensor wiring. In contrast, employing a single IMU in conjunction with a real2sim whole-body rendering approach simplifies system complexity while enhancing scalability.

Limitations fall in several aspects. For example, the current framework is constrained by the low-speed operation caused by the sim2real delay. However, it can be refrained by improving the computing resources. In addition, our method requires trial and error on the model’s meshing (parameterization) in the simulation beforehand. The modeling can be quite arbitrary and affects the computed efficiency and efficacy, including the robot inverse. Last but not least, our method still requires online computation with an external PC, while some applications would much benefit from a completed embedded system. Future work in offline virtual training on control policy can be explored.

Conclusions

This paper presents a real2sim2real framework designed to enable real-time gravity awareness and proactive joint-level compensation for portable cable-driven soft slender robots, ensuring they maintain the desired configuration. The core concept of the proposed framework is to leverage advanced soft body simulation to minimize the reliance on physical sensors. The framework leverages (1) real-world feedback from a single IMU attached to the robot’s base and (2) virtual-world inverse computation using a reduced-order FEM model of the soft robot imported into SOFA, facilitating closed-loop control in both real and virtual unified environments. We conducted controlled and demonstrative experiments to validate the hypothesis and assess the framework’s performance on a two-segment cable-driven soft robot. Our results indicate that a single IMU with temperature and drift filters is sufficient to reconstruct a real2sim-based soft robot under dynamic gravity conditions in SOFA, outperforming a commercial FEM simulator with controlled parameters and a simplified mechanics model. We verified our method’s efficacy, achieving a compensation recovery rate (correlation coefficient) of over 99% in static cases and 94% in dynamic cases. Additionally, we observed a time delay of approximately 3 seconds in the sim2real online deployment, which can be mitigated with a more powerful computational setup. Future work could explore external force interaction within the real2sim2real framework by incorporating additional virtualizable sensors.

Supplemental material

Supplemental material - Gravity-aware proactive joint-level compensation for portable soft slender robots using a single IMU and real-time simulation

Supplemental material for Gravity-aware proactive joint-level compensation for portable soft slender robots using a single IMU and real-time simulation by Jiewen Lai, Tian-Ao Ren, Pengfei Ye, Yanjun Liu, Jingyao Sun, Hongliang Ren in The International Journal of Robotics Research.