Sage Journals: Discover world-class research

Abstract

Dexterous in-hand manipulation offers significant potential to enhance robotic manipulator capabilities. This paper presents a sensori-motor architecture for in-hand slip-aware control, being embodied in a sensorized gripper. The gripper in our architecture features rapid closed-loop, low-level force control and is equipped with sensors capable of independently measuring contact forces and sliding velocities. Our system can quickly estimate essential object properties during pick-up using only in-hand sensing, without relying on prior object information. We introduce four distinct slippage controllers: gravity-assisted trajectory following for both rotational and linear slippage, a hinge controller that maintains the object’s orientation while the gripper rotates, and a slip-avoidance controller. The gripper is mounted on a robot arm and validated through extensive experiments involving a diverse range of objects, demonstrating the architecture’s novel capabilities for manipulating objects with flat surfaces.

Keywords

in-hand manipulation manipulation and grasping grippers sensor-based control contact modeling

Introduction

Humans have the remarkable ability to pick up unfamiliar objects and quickly understand their surface properties, such as friction and dynamics. This knowledge enables us not only to reorient objects using our arms but also to manipulate them within our hands, extending our capabilities beyond what is typically seen in traditional robotics. In this paper, we introduce a sensorized parallel gripper (see Figure 1), for in-hand slip-aware control that relies solely on in-hand sensing. This work is the first to combine both planar velocity and contact forces from independent in-hand sensing modalities for the purpose of slip-aware control in a parallel gripper, introducing new opportunities for intricate robotic manipulation. Each finger is equipped with a commercial 6-degree-of-freedom (DoF) force-torque (F/T) sensor and a custom relative velocity sensor. This hardware combination enables rapid estimation of friction and contact surface properties without the need for external sensors, thus facilitating for precise in-hand manipulation of objects with flat surfaces in both rotational and translational movements.¹

Figure 1.

Picture of the custom gripper mounted with F/T and relative velocity sensors.

Slip-aware control significantly enhances the functionality of robotic manipulators by enabling the object-end-effector relative pose to adapt during grasping, thereby extending the operational workspace. This adaptability is particularly valuable in constrained environments, where the manipulator’s movement is limited, or for intelligent human–robot interaction, enabling for instance more intuitive and safe handovers. Furthermore, in-hand slippage control opens up new opportunities for multi-arm manipulation of single objects, allowing for the repositioning of grasps without releasing the object, thereby enabling more efficient and flexible handling of larger items. Our system has been rigorously tested across a wide range of experiments, demonstrating its effectiveness and versatility.

Related work

Tactile sensors

In-hand perception and manipulation are intrinsically tied to hardware. The gripper’s structure and the number of DoF dictate its dexterity. The design of tactile sensors is closely linked to the contact properties, which play a crucial role in influencing both sliding and grasping behavior. Numerous tactile sensors have been developed, and comprehensive reviews can be found in Chen et al. (2018), Kappassov et al. (2015), Dahiya et al. (2010), Yousef et al. (2011), Chi et al. (2018), and Lyu et al. (2025). A notable example is the GelSight sensor (Yuan et al., 2017), which evolved from the GelForce sensor (Kamiyama et al., 2005). The GelForce sensor utilized two layers of markers within an elastomer to detect a 3D force field over a 2D surface. Building on this, the GelSight sensor (Yuan et al., 2017), an advancement of the predecessor described in Johnson and Adelson (2009), features a clear elastic polymer with a reflective coating. A camera and multi-angled LEDs behind a rigid transparent plate capture deformations, using colored shading to reconstruct the 3D imprint. Surface markers enable estimation of shear forces and the reconstruction of a 6-axis force vector. The GelSight sensor has been commercialized and widely adopted in research. Another commercially available sensor is the BioTac sensor (Wettels et al., 2014), which has a finger-like appearance and is capable of measuring 3-axis force, vibrations, and temperature. Another approach is detailed in Costanzo et al. (2019) and De Maria et al. (2012), introducing a tactile sensor later named SUNTouch (Costanzo et al., 2023). This sensor is based on a deformable layer with cavities containing reflectors, where LED-phototransistors estimate deformation, allowing estimation of both the 6-axis F/T and surface deformation.

Several array-like tactile sensors have been proposed to measure the deflection of nibs, as demonstrated in Khamis et al. (2018), Yao et al. (2022), Wang et al. (2019), and Huh et al. (2020). Huh et al. (2020) introduced a tactile sensor specifically designed for simple in-hand manipulations, which dynamically clusters nib measurements, allowing for either faster data acquisition or higher spatial resolution. This concept of faster measurements was further expanded in Gloumakov et al. (2024), where nib vibrations were captured to estimate sliding velocity. Alternatively, Damian et al. (2015) explored the use of ridges on a flexible sensing resistor (FSR) sensor to estimate both velocity and position of sliding objects. Earlier approaches include the work of Howe and Cutkosky (1989), who developed a tactile sensor with a rubber skin embedded with an accelerometer to detect vibrations during slippage. The TacTip sensor, originally developed by Chorley et al. (2009), has been adapted into various versions that can be easily manufactured using rapid prototyping techniques, as described in Ward-Cherrier et al. (2018). Recent studies have explored fiber-optic methods for in-hand motion tracking (Tripicchio et al., 2023; Qian et al., 2018; Lu et al., 2022), showing promise for pressure, position, and force sensing free from magnetic interference.

Our approach differs from previous work by focusing on measuring in-hand force and planar velocity using independent sensing modalities. We achieve this by equipping the gripper fingers with both F/T sensors and custom relative velocity sensors based on optical mouse sensors. Optical mouse sensors have previously been explored for in-hand object manipulation to a limited extent, prior studies include Maldonado et al. (2012), which integrated single optical mouse sensors into the fingertips of the DLR/HIT hand (Liu et al. (2008)) for slip avoidance and object recognition. Sani and Meek (2011) used a single optical mouse sensor for slip detection, and Höver and Harders (2010), which employed an optical mouse sensor to capture tool displacement.

Grippers

Parallel grippers typically possess only a single DoF of control, which limits their capability for dexterous in-hand manipulation. As a result, these grippers often rely on external factors such as gravity, external contacts, or the inherent dynamics of the object to achieve complex tasks. Given the single DoF control and the rapid onset of slip events, precise and fast control becomes essential for effective manipulation. However, commercially available grippers often lack accessible, fast low-level force control, which is critical for such tasks.

Given access to sufficient bandwidth, velocity, or position-controlled grippers can be used to indirectly control grasping force with additional sensors (Viña B. et al., 2016); however, few commercial grippers offers this functionality. One discontinued commercial option is the WSG50 gripper, as utilized in Costanzo et al. (2023), which, with custom software modifications, achieves an external control rate of 50 Hz. In contrast, custom grippers using Dynamixel servo motors have been employed in various studies (Bi et al., 2021; Gloumakov et al., 2024; Zhang et al., 2022), with a reported response time of 0.12 s (Gloumakov et al., 2024). A different example is a custom gripper integrated with force and vibration sensors, as described in Zaki et al. (2010). Additionally, Wang et al. (2016) introduced a dual-motor parallel gripper, which combines a position controller with a closed-loop force controller. Another approach, described in Piga and Natale (2023), involves an adaptive force controller for low-resolution, position-controlled parallel grippers, demonstrating sliding manipulation with external object tracking.

A two-phase finger for a parallel gripper was proposed by Chavan-Dafle et al. (2015), allowing easier reorientation by switching between free-spinning contact and firm grasp. At lower grasp forces, the finger acts as a point contact, while at higher forces, it provides a stable contact surface. Dafle et al. (2014) employed a three-finger, single-motor gripper to perform in-hand reorientation of objects using external forces.

Anthropomorphic hands represent another approach to in-hand manipulation. Vision-based object reorientation has been demonstrated using the Shadow Dexterous Hand with reinforcement learning (OpenAI et al., 2019), and an overview of related learning-based methods is provided in Weinberg et al. (2024). The RBO Hand 3, a soft hand with 16 degrees of actuation, achieves in-hand manipulation using simple controllers (Puhlmann et al., 2022; Bhatt et al., 2021). However, the mechanical complexity of anthropomorphic hands presents challenges for reliability, durability, and cost-effectiveness (Huang et al., 2025). Recent efforts to address these limitations include low-cost designs enabled by rapid prototyping (Yang et al., 2021) and actuator-reduction strategies to simplify control and reduce cost (Kontoudis et al., 2019; Odhner et al., 2014; Santina et al., 2018).

Slip control and estimation

Estimation of friction and contact properties plays a crucial role in enabling effective in-hand slip-aware control. For instance, in Costanzo et al. (2023), the authors advanced their research on planar in-hand sliding control using the SUNTouch sensor. Their approach involves estimating angular velocity based on the limit surface concept and the LuGre friction model (Åström and Canudas-de Wit, 2008), utilizing estimated friction parameters in conjunction with F/T measurements. By estimating the center of rotation (CoR), they infer the linear velocity from angular velocity estimates. The estimated CoR is constrained to be within 1.5 times the contact radius, which limits the planar velocity estimation to be primarily rotational. The friction estimation method employed in Costanzo et al. (2023) is detailed in De Maria et al. (2015). Our approach does not impose such constraints on planar velocity estimation and allows for pure linear slippage as well as rotational.

Understanding the contact properties between the gripper and the object is essential for achieving controlled slippage. In previous work, we proposed a LuGre-based planar friction model for simulating in-hand slippage (Arslan Waltersson and Karayiannidis, 2024). Xydas and Kao (1999) modeled the contact mechanics for soft finger contacts based on the limit surface concept (Goyal et al., 1989, 1991) and linear elastic contact models, originally described by Hertz (1896). Additionally, in Le et al. (2021) combined visual and haptic data to estimate friction and assign friction coefficients to material segmentation based on camera input.

In Viña B. et al. (2015) and Viña B. et al. (2016), a trajectory-following controller was introduced for controlled rotational slippage in a parallel gripper, using an external camera for object pose tracking. The optoforce tactile sensor (Tar and Cserey, 2011) was employed in Viña B. et al. (2016) alongside an adaptive controller to compensate for errors in the friction coefficient. A framework for picking up and pivoting objects from a flat surface was presented in Holladay et al. (2015). Huh et al. (2020) used a custom sensor to stand objects upright on a flat surface, and the same sensor was employed in Gloumakov et al. (2024) to estimate slipping velocities and accelerations, enabling fast in-hand linear slippage. Chen et al. (2021) employed neural networks with the BioTac sensor to estimate linear sliding velocity and implemented a closed-loop controller to maintain a consistent sliding velocity.

In other in-hand slippage research, Bi et al. (2021) utilized custom tactile sensors for swing-up control. The GelSight sensor was employed by Wang et al. (2021) to identify physical features through exploratory movements aimed at dynamically swinging up objects. Chavan-Dafle et al. (2020) used motion cones with a gripper and an external pusher to plan the reorientation and repositioning of objects to achieve a desired final pose. Hou et al. (2018) demonstrated the ability to reorient objects using two motion primitives, pivoting and rolling on a table, with a gripper featuring two-phase fingers that enable both a firm grasp and pivoting.

Contributions and overview

This paper presents an architecture for in-hand slip-aware control, featuring a high-performance sensorized gripper. The architecture facilitates in-hand slippage in both linear and rotational directions using simple controllers. These slipaware controllers enables us to extend the capabilities of parallel grippers and relax the assumption of a rigid grasp. Initial estimation of Stribeck friction parameters Stribeck (1902) and the contact radius allows for manipulation of unfamiliar non-fragile planar objects. The key contributions are:

(1) High-performance parallel gripper: Designed for in-hand slip-aware control with rapid, precise force regulation. Unlike Viña B. et al. (2016) and Costanzo et al. (2023), which use velocity-controlled grippers for force control, we develop a direct force actuation-based gripper. In contrast to Arimoto et al. (1999), which uses deformation derivatives and adaptive PI control, we employ a simple PI controller without deformation derivatives, relying on standard techniques like saturation and gain scheduling, and validate its performance for time-varying references.

(2) Planar velocity sensors: In-hand sensors that accurately measure planar sliding velocity at the contact—both linear and rotational—and can be mounted atop of F/T sensors. In contrast to prior approaches to in-hand slip-aware control (Costanzo et al., 2023; Viña B. et al., 2016) and tactile optical tracking (Maldonado et al., 2012; Sani and Meek, 2011; Höver and Harders, 2010), our novel sensor set-up enables planar slip-aware control for parallel grippers through independent in-hand measurements of contact forces and sliding velocities.

(3) In-hand estimation of contact properties: The proposed architecture, based on independent measurements of contact forces and velocities, enable rapid estimation of contact properties at the moment of object pickup—relying solely on in-hand sensing, in contrast to other approaches for in-hand sliding manipulation (De Maria et al., 2015; Viña B. et al., 2016).

(4) Slip-aware controllers: Leveraging the custom gripper, sensors, and contact estimation, this work introduces—unlike prior approaches (Costanzo et al., 2023; Viña B. et al., 2016; Gloumakov et al., 2024; Chen et al., 2021)—the first architecture demonstrating both pure linear and rotational gravity-assisted in-hand slippage. We present four simple slip-aware controllers: trajectory-following for gravity-assisted linear and rotational slip, hinge control with linear slip-avoidance, and slip-avoidance.

An overview of the system is shown in Figure 2. The gripper, equipped with F/T and velocity sensors, is mounted on a robot arm. A grasp force controller—based on the F/T sensors f⁽ⁱ⁾—takes as input the desired grasp force f_d and produce the output voltage V_t that drives the motor controller. During a brief initial exploration phase at object pick-up, contact properties are estimated based on readings from the velocity v⁽ⁱ⁾ and F/T sensors. The slip-aware controllers then use these contact estimations and sensor data to execute various slip commands. Notably, the slip-aware controllers are independent of the specific gripper dynamics, so in principle a different high-performance gripper could be used; however, the authors have not found a suitable commercial alternative.

Figure 2.

Overview of the architecture with the inner and outer control loops, the mechanical connections are marked with dashed lines and information pathways are marked as solid lines.

The paper is structured as follows: Section Gripper design presents the design of the gripper and the grasp force controller. The Velocity sensor design section details the relative velocity sensors and their calibration. The method for estimating contact properties is described in Section Estimation of contact properties. The Controlled slippage section introduces the four slip-aware controllers. Experimental results are presented and analyzed in the Results section. Finally, the Conclusions section summarizes the paper.

Gripper design

Commercial grippers are generally not designed for in-hand sliding manipulation tasks and often lack accessible low-level force control. To address this limitation, we present a high-performance, force-controlled gripper. This permits treating the force response as near-perfect in the design of the slip-aware controllers. This section details the hardware design of the gripper and the grasp force controller. The gripper features a high-performance motor and efficient field-oriented control (FOC), with the low-level FOC controller running on an ESP32-S3 microcontroller. The design incorporates commercially available components and rapid prototyping techniques, such as 3D printing.

Hardware

The gripper is designed to be quasi-direct drive for rapid force control and its overall dimensions be seen in Figure 3, while the construction and internal components are detailed in Figure 4. The electrical components are listed in Table 1. The gripper is powered by a high-torque, low Kv BLDC motor, originally designed for camera gimbals and optimized for operation under stalled conditions, for example, when grasping objects. This motor is controlled using FOC based on the SimpleFOC library (Skuric et al., 2022). An encoder, directly attached to the motor output, provides real-time feedback to the FOC algorithm, ensuring constant torque regardless of the rotor orientation. The encoder also allows for the calculation of finger position and velocity. The motor is driven by a DRV8302 motor driver, with a software-limited maximum output of 1 A at 20V. The encoder connected to the ESP32 microcontroller via a voltage level shifter, offering an effective resolution of 4096 counts per revolution using the ESP32’s hardware pulse counters. The communication between the gripper and a computer is handled over USB serial at 500 Hz, with motor angle data θ transmitted and target voltage commands V_t received. Figure 5 illustrates the communication pathways, while Table. 2 summarizes the system’s operational rates.

Figure 3.

Gripper viewed from the side with the overall dimensions.

Figure 4.

The internal components of the gripper.

Table 1.

Gripper electrical components.

Microcontroller	ESP32
Motor driver	DRV8302
Encoder	AMT103-V
Motor	GM5208-24
Level shifter	BSS138

Figure 5.

The communication inside the gripper and with the computer.

Table 2.

Loop and communication rates.

FOC	$> 1 e 4$ Hz
Force-torque sensor	1e3 Hz
Gripper serial communication	500 Hz
Grasp force controller	500 Hz
Velocity sensors	120 Hz
Slip-aware controllers	120 Hz

The gripper’s body is primarily constructed using rapid prototyping techniques. As shown in Figure 4, all white components, except the gears, are 3D-printed in PLA using an FDM printer, while the outer shell and quick connector are produced via SLS printing. The motor is coupled to a 2:1 gear ratio, effectively doubling its torque output while maintaining a quasi-direct-drive configuration. This output gear drives a belt pulley system that converts rotational motion into linear motion. Each belt pulley has 22 teeth and a diameter of 17.51 mm, resulting in 27.5 mm of linear travel per motor revolution for each finger. The gripper achieves a measured maximum grasp force of approximately 45 N. The pulley and gear shafts are mounted on rotational bearings, while the fingers move along two linear bearings and are connected to the belt on either side. The finger design allows for a maximum grasp opening of 78 mm. Belt tensioning is achieved through adjustable screws, as depicted in Figures 4 and 6. The total weight of the gripper, including the sensors, is 1.23 kg.

Figure 6.

Sensors, finger and belt tensioning mechanism.

The contact pad (see Figure 6) is flat and rigid to accommodate the velocity sensors’ sensitivity to surface proximity. Designed for easy interchangeability, the contact pad is circular with a 15 mm radius and is 3D-printed from both PLA and TPU95. The TPU95 surface increases friction compared to PLA alone. The fingers, printed in solid PLA, tend to flex under load. To counteract this, a long M3 screw runs along the inner side of each finger, preloading one side to create an adjustable counter-flex, as partially visible in Figure 6.

Grasp force control

The grasp controller is responsible for rapidly achieving a desired grasp force f_d, with the feedback f_n from the F/T sensors. We model the gripper and analyze it to motivate our grasp force controller. The controller outputs a commanded force, f_c, which is translated into a target voltage V_t for the FOC algorithm as follows:

V_{t} = - \frac{1}{k_{f}} f_{c}

(1)

where the sign of V_t determines the direction of the force: positive V_t opens the gripper, while negative V_t closes it. However, the actual grasp force exerted by the gripper may not exactly match f_c due to factors such as friction f_f, damping force from the motor’s back EMF f_v and other unmodeled disturbances δ, like motor winding temperature. The gripper’s dynamics during object grasping can be simplified and modeled as shown in Figure 7. In this model, the motor’s rotational motion is expressed as an equivalent linear motion, leading to the following gripper dynamics:

m_{1} {\ddot{x}}_{1} = f_{c} - f_{f} - f_{v} - k_{1} (x_{1} - x_{2}) - d_{1} ({\dot{x}}_{1} - {\dot{x}}_{2}) + δ

(2)

where m₁ is the mass equivalent of the motor inertia, and x₁ is the equivalent linear displacement. The forces acting on the sensors can be modeled similarly as

m_{2} {\ddot{x}}_{2} = k_{1} (x_{1} - x_{2}) + d_{1} ({\dot{x}}_{1} - {\dot{x}}_{2}) - k_{2} x_{2} - d_{2} {\dot{x}}_{2}

(3)

where x₂ represents the displacement of the finger after it has made contact with the object, with x₂ ≥ 0. The measured normal force by the F/T sensors can be modeled as

f_{n} = k_{2} x_{2} + d_{2} {\dot{x}}_{2} .

(4)

Figure 7.

Simplified model of gripper dynamics.

To analyze stability of the system with states ${[x_{1}, {\dot{x}}_{1}, x_{2}, {\dot{x}}_{2}]}^{T}$ consider the Lyapunov function candidate:

\begin{align} V & = \frac{1}{2} m_{1} {\dot{x}}_{1}^{2} + \frac{1}{2} m_{2} {\dot{x}}_{2}^{2} + \frac{1}{2} k_{1} {(x_{1} - x_{2} - \frac{f_{c} + δ}{k_{1}})}^{2} \\ + \frac{1}{2} k_{2} {(x_{2} - \frac{f_{c} + δ}{k_{2}})}^{2} \end{align}

(5)

where the first two terms represent the kinetic energy, and the last two terms represent the potential energy stored in the springs, with the equilibrium position offset by the disturbances. The time derivative of V given by (5) along the systems trajectories (2), (3) is given by

\begin{align} \dot{V} = - d_{2} {\dot{x}}_{2}^{2} - (f_{f} + f_{v}) {\dot{x}}_{1} - d_{1} {({\dot{x}}_{1} - {\dot{x}}_{2})}^{2} . \end{align}

(6)

The friction f_f and back emf f_v always act in the opposite direction of ${\dot{x}}_{1}$ , where f_v naturally dampens the system, that is, $- (f_{f} + f_{v}) {\dot{x}}_{1} \leq 0, \forall {\dot{x}}_{1}$ , and thus:

\dot{V} \leq - d_{2} {\dot{x}}_{2}^{2} - d_{1} {({\dot{x}}_{1} - {\dot{x}}_{2})}^{2} .

(7)

As V is positive definite and $\dot{V} \leq 0$ , we can readily prove, relying on LaSalle’s theorem for asymptotic stability, that ${\dot{x}}_{1} = 0$ , ${\dot{x}}_{2} = 0$ is invariant and subsequently that the equilibrium point $(k_{1} + k_{2} / k_{1} k_{2} (f_{c} + δ), 0, f_{c} + δ / k_{2}, 0)$ is asymptotically stable, thus implying f_n → f_c + δ. Indeed extensive experiments have shown that if we apply a feedforward force control f_c = f_d system is asymptotically stable but with a bias affected by disturbances. To motivate the choice of the controller we will consider a reduced state system by assuming that the dynamics of convergence to the invariant set ${\dot{x}}_{1} - {\dot{x}}_{2} \equiv 0$ is faster as compared to the dynamics of ${\dot{x}}_{2}$ . In fact, substituting (3) into (2) and setting ${\ddot{x}}_{1} = {\ddot{x}}_{2}$ we get:

(m_{1} + m_{2}) {\ddot{x}}_{2} = f_{c} - f_{f} - f_{v} - f_{n} + δ

(8)

Adding $k_{2} / d_{2} (m_{1} + m_{2}) {\dot{x}}_{2}$ in both sides of (8) we get the first-order model with state f_n:

T_{f} {\dot{f}}_{n} = f_{c} - f_{n} + Δ (t),

(9)

time constant T_f = m₁ + m₂/d₂ and disturbance input

Δ (t) = - f_{f} - f_{v} + k_{2} / d_{2} (m_{1} + m_{2}) {\dot{x}}_{2} + δ

. System (9) calls for a PI controller with a feedforward term for the desired force f_d. In particular, the proposed controller is given by

f_{c} = f_{d} + \frac{k_{P}}{η} e + k_{I} I_{c} (e)

(10)

where k_P and k_I are positive control gains for the proportional and integral gain respectively, e = f_d − f_n the force error, and η = γΔe² + 1 an error dependent scaling factor, where Δe = e(t) − e(t − t_s) for a discrete time step size t_s. The integral term I_c is given by

I_{c} = \min (\max (\int_{0}^{t} e (σ) d σ, I_{b}), - I_{b})

(11)

bounded by I_b = I_max/ηk_I to alleviate wind-up. Note that f_n is the force measured by the F/T sensors. The scaling factor η aims at reducing the overshoot by limiting the P and I terms when Δe is large. The real-world stability and performance of the controller are tested using step signals and varying sinusoidal profiles, as detailed in 7.1.

Velocity sensor design

In this section, we present the design of a planar velocity sensor, which is mounted between the F/T sensor and the contact pad, see Figure 6. This allows the overall sensor set-up to accurately measure both the planar sliding velocity and the forces using in-hand sensing. An overview of the sensor and its associated microcontroller is provided in Figure 8.

Figure 8.

Overview of velocity sensor.

Planar velocity sensor

The proposed sensor consists of three optical mouse sensors arranged 120° apart in a circular pattern, allowing it to measure linear and rotational relative velocities with redundancy. The sensor is connected via a flexible ribbon cable to an ESP32-S3 microcontroller, as shown in Figure 8. The microcontroller communicates with a computer over USB serial, see Table 2 communication rates. The sensor assembly is built on a custom PCB, depicted in Figure 9. The optical mouse sensors (PAW3205DB-TJNT), along with the lenses and LEDs, are repurposed from a Rapoo M300 mouse. The optical sensors are mounted on the back of the PCB, while the LEDs are mounted under the lenses. The lenses are press-fitted into place, and the microcontroller enclosures are mounted on the sides of the gripper, as shown in Figure 1.

Figure 9.

PCB assembly of velocity sensor.

Experimental results show that each optical sensor operates at 3200 counts per inch (CPI), providing a spatial resolution of approximately 0.008 mm. Each sensor outputs the measured displacement in the x and y directions, with the sensor frames illustrated in Figure 10. The ESP32 microcontroller is configured with one core dedicated to half-duplex communication with the optical sensors, while the other core handles filtering and USB serial communication. The filtering and communication loop runs at 120 Hz, balancing low latency with accuracy.

Figure 10.

Frames of the optical mouse sensors and the combined velocity sensor in regards to the F/T sensor. The velocities are expressed in m/s or rad/s.

The filtering process is carried out in multiple stages: First, the average velocity within a time window is calculated for the x and y directions of each sensor. Next, each velocity is processed through a calibration filter. Finally, the velocity components are fused to output the sensor’s planar velocity. The filtering rate is crucial, as a rate that’s too high results in noisy velocity measurements, while a rate that’s too low introduces unnecessary delays. Minimizing delay is essential for accurately capturing slip and stick events. Common filtering techniques, such as tracking loops, Kalman filters, or other recursive or running average filters, often result in higher delay but lower noise. The velocity $v_{k}^{(j)}$ of the optical mouse sensor j in direction k is processed through a calibration filter:

v_{c, k}^{(j)} = \frac{v_{k}^{(j)}}{b + a v_{k}^{(j)}}

(12)

where a and b are calibration parameters. The calibration process is further detailed in the Velocity Sensor Calibration subsection.

The velocity output of the planar velocity sensor is aligned with the F/T sensor, as illustrated in Figure 10. The output velocity $v = {[v_{x}, v_{y}, ω]}^{T}$ includes both linear and rotational components. The velocity from each sensor $v_{c}^{(j)} = {[v_{c, x}^{j}, v_{c, y}^{j}]}^{T}$ is incorporated into a measurement model for the planar sensor:

v_{o} = [\begin{matrix} v_{c}^{(1)} \\ v_{c}^{(2)} \\ v_{c}^{(3)} \end{matrix}] = [\begin{matrix} R (- 30 °) & d \\ R (- 150 °) & d \\ R (90 °) & d \end{matrix}] [\begin{matrix} v_{x} \\ v_{y} \\ ω \end{matrix}] = A v

(13)

where R(⋅) ∈ SO(2) is a rotation matrix, and d = [d,0]^T, with d representing the distance from the center to the point where the optical sensor measures velocity, see Figure 10. The planar velocity v is then estimated using the least squares solution.

\hat{v} = A^{+} v_{o}

(14)

where ⋅⁺ is the Moore-Penrose pseudo inverse.

There is redundancy in the calculation of v, as only two of the three optical sensors are strictly necessary. This redundancy is exploited by an outlier rejection system, which addresses the risk of one sensor being outside the object or failing to track the surface correctly. When tracking fails, optical mouse sensors typically register zero or significantly reduced velocity. The outlier rejection system operates as follows: First, the estimated velocity $\hat{v}$ is substituted into (13) to calculate the least squares error $ε = ‖ v_{o} - {\hat{v}}_{o} ‖_{2}$ . Let ɛ_t = 0.2 be a threshold. If the condition ɛ/‖v_o‖₂ + 1e − 3 > ɛ_t is met, the sensor with the lowest velocity ‖v_i‖₂ is discarded. A new matrix $\bar{A}$ and vector ${\bar{v}}_{o}$ are then constructed from the remaining two sensors. The updated estimated velocity is given by

\hat{v} = {\bar{A}}^{+} {\bar{v}}_{o} .

(15)

This approach allows for one of the three sensors to be outside the object, with minimal impact on tracking accuracy.

Velocity sensor calibration

The planar velocity sensors are calibrated and tested using the test rig shown in Figure 11. The rig features a cart that moves along linear tracks, with the sensor mounted on a rotatable shaft. Two adjustable stops are installed on the linear tracks, and the cart includes an angle selector, enabling controlled displacement during testing.

Figure 11.

Testing and calibration system for the velocity sensors. The test rig can be set up to test the sensor outlier system.

To calibrate the optical mouse sensors and determine the calibration parameters a and b from (12) for both x and y directions, the stoppers are set to allow 100 mm of linear travel for the cart. The angle selector is adjusted according to the specific sensor and direction being calibrated. Data is then collected by manually moving the cart between the stoppers at varying speeds. By knowing the displacement between the stops and calculating the average time taken, the true average velocity is obtained. The ratio between the measured and estimated velocities is then calculated and plotted across different speeds, as shown in Figure 12.

Figure 12.

Typical calibration example, we call the average true velocity v_true and the raw measured velocity v_m.

Linear regression is used to fit a line to the data, providing an initial estimate for a and b. These coefficients are then manually fine-tuned if necessary, as the automated parameter identification assumes a constant velocity, which is not always the case during manual actuation. The results before and after calibration are displayed in Figure 12. Additionally, the variable d is manually tuned by rotating the sensor 180° and adjusting d until the correct rotation is estimated.

Estimation of contact properties

This section outlines a two-step process for estimating key contact properties: the static, Coulomb, and viscous friction coefficients μ_s, μ_c, and μ_v, as well as the radius r in a rim contact model, which approximates the limit surface as an ellipsoid (Arslan Waltersson and Karayiannidis, 2024). The estimations of μ_c and r are later used in Section Controlled slippage for slip-aware control. First, a brief linear slippage exploration is performed during the pickup phase while the object rests on a surface, as illustrated in Figure 13. The process begins by grasping the object with a normal force f_g. After time t₁, a linear motion with displacement d_e downwards towards the surface is initiated, lasting for a duration of t₂. The data collected from this linear exploration is processed using Algorithm 1 to estimate the friction coefficients. Each finger is evaluated independently, which allows the method to accommodate objects with different materials on either side. Forces and velocities are measured at the contact point, eliminating the need for precise velocity tracking by the robot manipulator. The algorithm discards data points with excessive angular velocity, as these tend to skew the results toward lower friction coefficient estimates according to limit surface theory (Goyal et al., 1989).

Figure 13.

Linear and rotational exploration for estimation of contact properties.

Following the estimation of the friction coefficients, a rotational exploration phase is conducted to estimate the contact radius of an equivalent rim contact. This phase is illustrated in Figure 13. During this phase, force and velocity data are collected while the gripper rotates θ_e over a period of t_θ and then returns to its original orientation. The algorithm used to estimate the contact radius r is detailed in Algorithm 2. The estimated radius r will subsequently be used to estimate the limit surface of the contact area. Overall, the friction coefficients and contact radius can be estimated within 2 s of grasping an object.

Controlled slippage

In this section, we present four slip-aware controllers: linear slippage, rotational slippage, hinge control, and slip-avoidance, as illustrated in Figure 14. These controllers are designed to be simple and to be used as control primitives for more complex tasks. They build on top of the grasp controller from Subsection Grasp Force Control and are experimentally verified. The notation $\bar{\cdot}$ denotes the average of both sensors, expressed in the middle frame {M} of the gripper, as shown in Figure 15. For example, ${\bar{f}}_{x}$ represents the average force measured by the two F/T sensors in the x direction of the middle frame. All slip-aware controllers described here operate at $\sim 120$ Hz.

Figure 14.

Four different slip control modes.

Figure 15.

The frames of the gripper.

Slip-avoidance

The primary focus of the slip-avoidance controller is to adjust the grasping force to securely hold an object without excessive grasp force. Let ${\bar{f}}_{t}$ be the Euclidean norm of the average tangential forces from the two F/T sensors:

{\bar{f}}_{t} = ‖ {[{\bar{f}}_{x}, {\bar{f}}_{y}]}^{T} ‖_{2} .

(16)

This is used to calculate the minimum grasp force to counter the measured tangential forces:

f_{n, t, m i n} = \frac{{\bar{f}}_{t}}{{\bar{μ}}_{c} γ_{t}}

(17)

where

{\bar{μ}}_{c}

is the average estimated Coulomb friction coefficient between the two fingers. The ratio γ_t is derived from an ellipsoid approximation of the limit surface (see Figure 16), adjusting the grasp based on the measured torque versus tangential forces. Despite the limitations of the ellipsoid approximation (Arslan Waltersson and Karayiannidis, 2024), it is chosen for simplicity. The average force wrench between the sensors

\bar{f} = {[{\bar{f}}_{x}, {\bar{f}}_{y}, \bar{τ}]}^{T}

and the average rim contact radius

\bar{r}

are used to calculate γ_t as

γ_{t} = \frac{{\bar{f}}_{t} + ϵ}{‖ S^{- 1} \bar{f} ‖_{2} + ϵ}

(18)

where

S = d i a g ([1,1, \bar{r}])

and ϵ = 1e − 5 for numerical stability and feasibility. The minimum grasp force to counteract the measured torque is given by

f_{n, τ, m i n} = \frac{| \bar{τ} |}{{\bar{μ}}_{c} \bar{r} γ_{τ}}

(19)

where

γ_{τ} = \frac{| \bar{τ} | / \bar{r} + ϵ}{‖ S^{- 1} \bar{f} ‖_{2} + ϵ} .

(20)

Figure 16.

Illustration of the normalized limit surface.

The overall grasp force, considering both torque and tangential forces, includes a tunable safety margin γ_s and a velocity component to account for slippage:

f_{n, c} = γ_{s} m a x (f_{n, t, m i n}, f_{n, τ, m i n}, f_{n, s, m i n}) + k_{P, s} ‖ S \bar{v} ‖_{2}

(21)

where

\bar{v} = {[{\bar{v}}_{x}, {\bar{v}}_{y}, \bar{ω}]}^{T}

and f_n,s,min is the lower limit on the normal force. To mitigate oscillations, an exponential decay filter is applied at time t with a time step of t_h:

f_{d} (t) = \{\begin{cases} f_{n, c} (t) & i f f_{n, c} (t) > f_{d} (t - t_{h}) \\ α_{s} f_{d} (t - t_{h}) + (1 - α_{s}) f_{n, c} (t) & o t h e r w i s e \end{cases}

(22)

where f_d is the control signal to be sent to the grasp force controller described in Subsection Grasp Force Control.

Linear slip control

This controller allows the object to be repositioned within the gripper along the gravity vector. Let p_d(t) and $v_{d} (t) = {\dot{p}}_{d} (t)$ represent the desired displacement and velocity trajectories, respectively, and define a reference velocity:

v_{c} = v_{d} + K_{p, v} (p_{d} - p)

(23)

where p is the measured linear distance traveled. The value of p is obtained by integrating the tangential velocity

{\bar{v}}_{t} = ‖ {[{\bar{v}}_{x}, {\bar{v}}_{y}]}^{T} ‖_{2}

measured by the planar velocity sensors, assuming the object moves in the direction of the gravitational field.

The grasp force controller designed to follow v_c is a superposition of a PI“D” controller for tracking v_c and a force control term based on f_s (25) representing the normal force required to maintain the slip. The control law is given by

f_{n, c} = f_{s} - k_{P, l} f_{s} (v_{c} - {\bar{v}}_{t}) - k_{I, l} \int_{0}^{t} (v_{c} - {\bar{v}}_{t}) d t + k_{D, l} f_{s} {\dot{\bar{v}}}_{t}

(24)

where

{\dot{\bar{v}}}_{t}

is the time derivative of the tangential velocity, computed using the backward Euler method and applied to dampen rapid accelerations. To accommodate objects with varying masses and frictional properties, the proportional term depends on f_s. The integral term compensates for friction estimation errors and unmodeled object inertia. To manage fluctuations in the measured forces during object acceleration, f_s is updated using the following heuristic:

f_{s} (t) = \{\begin{cases} α f_{s} (t - t_{h}) + (1 - α) \frac{{\bar{f}}_{t}}{{\bar{μ}}_{c}} & {\dot{\bar{v}}}_{t} < 0.1 \\ f_{s} (t - t_{h}) & {\dot{\bar{v}}}_{t} \geq 0.1 \end{cases} .

(25)

The computed grasp force f_d for the inner loop controller is given by

f_{d} = m a x (f_{n, c}, f_{n, m i n})

(26)

where f_n,min is a lower limit on the normal force. Finally, the controller switches to the slip-avoidance controller whether the target displacement has been reached or if the trajectory time has run out and the object is not moving towards the target.

Rotational slip control

In rotational slip control, the gripper remains stationary while the object rotates within the gripper under the influence of gravity; a similar problem formulation was explored in Viña B. et al. (2016). Let θ_d(t) and $ω_{d} (t) = {\dot{θ}}_{d} (t)$ represent the desired angle and angular velocity trajectories, respectively, and the reference angular velocity:

ω_{c} = ω_{d} + k_{P, ω} (θ_{d} - θ)

(27)

where θ is the measured rotational displacement, obtained by integrating the measured angular velocity

ω_{a} = | \bar{ω} |

. Similarly to (24), the grasp force controller to follow ω_c is given by

f_{n, c} = f_{τ} - k_{P, τ} f_{τ} e_{ω} - k_{I, τ} \int_{0}^{t} e_{ω} d t + k_{D, τ} f_{τ} {\dot{ω}}_{a}

(28)

where e_ω = ω_c − ω_a, and

{\dot{ω}}_{a}

is computed using the backward Euler method. Here, f_τ represents the normal force required to maintain slip during rotation and is updated using the following heuristic:

f_{τ} (t) = \{\begin{cases} α f_{τ} (t - t_{h}) + (1 - α) \frac{| \bar{τ} |}{{\bar{μ}}_{c} \bar{r} γ_{τ}} & | \dot{\bar{ω}} | < 0.1 \\ f_{τ} (t - t_{h}) & | \dot{\bar{ω}} | \geq 0.1 \end{cases}

(29)

The grasp force sent to the inner loop controller is given by

f_{d} = m a x (f_{n, c}, \frac{{\bar{f}}_{t}}{{\bar{μ}}_{c}}, f_{n, m i n})

(30)

As with linear slip control, the controller switches to the slip-avoidance controller if either the target displacement has been reached or if the trajectory time has run out and the object is not moving towards the target.

Hinge mode

In hinge mode, the object is allowed to rotate freely within the gripper while preventing linear slippage by dynamically adjusting the grasp force. If the object’s center of gravity (CoG) is directly beneath the grasp point, the object remains stationary as the gripper rotates around it. The grasp force in hinge mode is calculated as follows:

f_{n, c} = γ_{h} \frac{{\bar{f}}_{t}}{{\bar{μ}}_{c}} + k_{P, h} {\bar{v}}_{t}

(31)

where γ_h is a tunable parameter that determines how closely the controller operates to the linear slip point. The desired grasp force f_d to be sent to the grasp force controller is computed as

f_{d} = m a x (f_{n, c}, f_{n, m i n}) .

(32)

Results

In this section, we present the experimental validation of the proposed hardware and methods. Various objects detailed in Figure 17 are used for the experiments. The dimensions and weights of these objects are listed in Table 3. Depending on the specific experiment, only a subset of these objects may be evaluated. The experiments are organized as follows:

• Subsection Gripper Performance: Validation of the grasp controller.

• Subsection Relative Velocity Sensor: Testing of the relative velocity sensor on different surfaces using the set-up shown in Figure 11.

• Subsection Estimation of Contact Properties: Evaluation of the contact estimation procedure.

• Subsection Slip-Avoidance to In-Hand Manipulation Demonstration: Testing of the slip-aware controllers.

Figure 17.

Test objects, the sponge is highly deformable, the wood object has a plywood surface and is rigid. The spectacle case has a synthetic leather material. The plastic object is a plastic covered cardboard box. The cardboard box has a paper finish.

Table 3.

Object dimensions and weight.

Object type	Dimensions (h, w, d)	Weight
Sponge	(92, 62,27) mm	6.8 g
Cardboard	(153, 70, 69) mm	160.6 g
Case (synthetic leather)	(169, 70, 51) mm	138.7 g
Plastic	(140, 81, 46) mm	84.1 g
Wood	(180, 60, 28) mm	141.3 g

The experiments were conducted using a laptop equipped with an Intel i7-1185G7 processor and 32 GB of RAM. The software and algorithms was implemented in ROS Noetic, and the UR10 robot was controlled using the Universal Robots ROS driver. The built-in task-based trajectory controllers were utilized for the robot’s movement.

Gripper performance

The parameters for the inner loop controller were tuned through experimentation, and the values used are presented in Table 4. The performance and stability of the gripper were evaluated by following a predefined force trajectory while grasping a variety of objects (see Table 3 and Figure 17). The force trajectory consisted of step and sinusoidal signals with varying amplitudes and frequencies, as illustrated in Figure 18.

Table 4.

Gripper inner loop grasp controller parameters.

Parameter	Value	Parameter	Value	Parameter	Value
k _P	2	k _I	100	I _max	10
γ	5	k _f	3.9166	t _s	2e − 3

Figure 18.

Force trajectory to test gripper performance.

The gripper’s response time was measured during the transition between a force of 5 N and 25 N, with each object tested 10 times. A representative sample from each object is shown in Figure 19. The key metrics for evaluating the step response are:

• Rise Time (t_r): The time it takes for the force to rise from 10% to 90% of the desired force change.

• Half-Rise Time (t_50%): The time taken to reach 50% of the desired force change.

• Maximum Overshoot (M_p): The peak force value above the desired value, indicating how much the force exceeded the target.

• Settling Time (t_s): The time required for the force to stabilize within an error band ±2% around the desired force, which corresponds to a range of ± 0.4 N.

Figure 19.

Step response of the gripper, one sample per object.

The statistical results for these metrics across the 10 trials for each object are summarized in Table 5. The data presented in Figure 19 and Table 5 demonstrates that the gripper can quickly achieve the desired force across various object properties. Response times were generally faster for rigid objects compared to soft, deformable ones, as the fingers of the gripper need to physically move and compress the deformable objects further to achieve the desired grasp force. The wood object is much stiffer than the gripper and provides an approximation of the control characteristics where only the gripper’s stiffness matters.

Table 5.

Step response for 5 to 25 N, mean μ and standard deviation σ.

Object type	t_r (μ, σ) (s)	t_50% (μ, σ) (s)	M_p (μ, σ) (N)	t_s (μ, σ) (s)
Sponge	(0.0345, 0.0021)	(0.0242, 0.0015)	(1.4556, 0.1166)	(0.0999, 0.0129)
Cardboard box	(0.0175, 0.0018)	(0.0137, 0.0010)	(1.8778, 0.1850)	(0.0747, 0.0088)
Case	(0.0220, 0.0012)	(0.0136, 0.0006)	(1.4398, 0.1479)	(0.0787, 0.0111)
Plastic	(0.0224, 0.0014)	(0.0157, 0.0009)	(1.6043, 0.2162)	(0.0970, 0.0034)
Wood	(0.0086, 0.0010)	(0.0093, 0.0008)	(1.7614, 0.2097)	(0.06877, 0.0154)

To evaluate the stability and performance of the gripper under time-varying target forces, the gripper was subjected to a sequence of sinusoidal signals with frequencies ranging from 1 to 64 Hz, as illustrated in Figure 20. The response was analyzed by computing the amplitude and phase for each frequency. The average results from 10 runs are presented in a Bode plot, see Figure 21. Figures 20 and 21 reveal that the magnitude increases at certain frequencies but remains bounded, indicating that stability is maintained throughout the testing range. As expected, the magnitude decreases at higher frequencies and the phase shifts accordingly. Notably, the phase exceeds 180° when the 64 Hz signal is applied. This phase shift can be attributed to the operational rates of the controller. Given that the inner loop controller operates at 500 Hz, a delay of 1/500 seconds for a 64 Hz signal would result in a phase shift of 46.08°, which would account for the shift beyond 180°. Overall, the combined analysis of the step response and Bode plot demonstrates that the inner loop controller is both stable and responsive.

Figure 20.

Tracking sinusoidal grasp force trajectory, one sample per object.

Figure 21.

Bode plot, shows an average of 10 runs for each object.

Relative velocity sensor

To estimate the accuracy of the velocity sensor in a controlled environment, four different tests were conducted using the test rig shown in Figure 11. These tests were designed to evaluate the sensor’s performance under various conditions:

(1) Linear: The sensor was moved 100 mm across the surface in a straight line along the sensor’s x-direction.

(2) Rotation: The sensor was rotated 180° clockwise without any tangential movement.

(3) Rejection: The sensor was moved 100 mm linearly along the x-direction, but at the 50 mm mark, one of the sensors was positioned outside the surface, as depicted in Figure 22.

(4) Combined: The sensor underwent both 100 mm linear movement and 180-degree clockwise rotation simultaneously.

During these experiments, the sensor’s estimated velocity was integrated to calculate the displacement, which was then compared to the known displacement from the test rig. Each experiment was repeated 10 times across the three surfaces shown in Figure 22. The results are presented in Tables 6 and 7.

Figure 22.

Three different surfaces: Wood, cardboard, and clear plastic film. The figure also shows the cut out for testing the sensor rejection system.

Table 6.

Tracking error [mm] & [deg], mean μ, and standard deviation σ.

Surface type and test	x (μ, σ)	y (μ, σ)	θ (μ, σ)
Wood (linear, 100 mm)	(−2.80, 0.81)	(−3.69, 0.58)	(0.62, 2.31)
Cardboard (linear, 100 mm)	(−2.17, 0.37)	(0.47, 0.73)	(0.76, 0.91)
Plastic (linear, 100 mm)	(−1.79, 0.38)	(0.96, 0.27)	(−1.75, 0.55)
Wood (rotational, 180 deg)	(0.55, 0.19)	(1.82, 0.09)	(0.61, 0.56)
Cardboard (rotational, 180 deg)	(−0.02, 0.22)	(0.51, 0.29)	(1.01, 0.56)
Plastic (rotational, 180 deg)	(0.80, 0.29)	(2.18, 0.52)	(−0.50, 0.79)
Wood (rejection, 100 mm)	(−2.21, 0.44)	(−3.44, 0.54)	(1.99, 0.76)
Cardboard (rejection, 100 mm)	(−1.75, 0.34)	(0.44, 0.54)	(0.50, 0.66)
Plastic (rejection, 100 mm)	(−2.15, 0.27)	(1.98, 0.35)	(−3.43, 0.63)

Table 7.

Tracking error combined motion (100 mm and 180 deg), mean μ, and standard deviation σ.

Surface type and test	Distance (μ, σ)	θ (μ, σ)
Wood	(−4.24, 1.70)	(4.64, 4.42)
Cardboard	(−2.05, 1.06)	(3.47, 1.47)
Plastic	(0.94, 1.36)	(3.65, 2.19)

The findings indicate that the sensors tracked well across all tested surfaces, with the velocity integrating to approximately a 2% displacement error over a 100 mm distance. The estimation of angular velocity typically resulted in less than 1% displacement error. Notably, the sensor’s performance was not significantly disrupted when one of the sensors was outside the object, as demonstrated by the rejection test. Additionally, the sensor was capable of accurately estimating both angular and linear velocities simultaneously.

Estimation of contact properties

The contact estimation procedure was evaluated using four different objects: wood, a spectacle case, plastic, and cardboard, see Figure 17. The gripper, mounted on a UR10 robot (as shown in Figure 23), grasped each object with a 5 N force, see Table 8 the contact exploration parameters. The procedure began with a linear exploration to estimate μ_s, μ_c and μ_v, see Figure 23. Following the linear exploration, the gripper executed the rotational exploration, see Figure 24. The estimation process, as detailed in Section Estimation of contact properties, was repeated 10 times for each object, with the mean and standard deviations presented in Table 9.

Figure 23.

Linear exploration movement for friction estimation.

Table 8.

Contact exploration parameters.

Parameter	Value	Parameter	Value	Parameter	Value
f _g	5 N	t ₁	0.2 s	t ₂	0.8 s
d _e	10 mm	θ _e	10°	t _θ	0.5 s

Figure 24.

Rotational exploration movement for contact radius estimation.

Table 9.

Estimation of contact properties, mean μ, and standard deviation σ, ⋅⁽ⁱ⁾ is for sensor i.

Object	( $μ_{s}^{(1)}$ , $μ_{c}^{(1)}$ , $μ_{v}^{(1)}$ )	r⁽¹⁾ (mm)	( $μ_{s}^{(2)}$ , $μ_{c}^{(2)}$ , $μ_{v}^{(2)}$ )	r⁽²⁾ (mm)
Cardboard (μ)	(0.565, 0.538, 0.578)	6.12	(0.540, 0.511, 0.379)	5.84
Cardboard (σ)	(0.046, 0.072, 1.584)	1.58	(0.027, 0.049, 0.908)	1.65
Case (μ)	(0.533, 0.511, 1.106)	6.31	(0.507, 0.479, 1.914)	5.04
Case (σ)	(0.025, 0.028, 1.424)	1.12	(0.068, 0.033, 1.456)	1.96
Plastic (μ)	(0.411, 0.366, 11.694)	7.64	(0.399, 0.363, 3.60)	7.40
Plastic (σ)	(0.014, 0.015, 1.884)	1.58	(0.020, 0.016, 0.647)	1.68
Wood (μ)	(0.499, 0.444, −1123)	7.92	(0.507, 0.432, −1.649)	3.91
Wood (σ)	(0.023, 0.018, 0.885)	1.49	(0.065, 0.034, 0.735)	2.94

The experiments demonstrate that the gripper and sensor combination can estimate friction coefficients and contact radius in just 2 s of exploration. The procedure relies solely on in-hand sensing, without depending on the robot’s forward kinematics. As shown in Table 9, both sensors produce similar friction coefficient estimates for the same material and object. The results reveal variations in friction coefficients across different objects, with cardboard exhibiting the highest friction and plastic the lowest. Notably, the viscous friction varies significantly between objects, and it is important to mention that the gripper’s average velocity during exploration is only 0.0125 m/s and that the viscus friction estimate might only be accurate within the explored velocity range.

The actual contact surface radius is 15 mm, and under the assumption of uniform pressure distribution, the equivalent rim contact radius would be 10 mm. However, the estimated rim contact radius is lower than the theoretical value for all objects. This discrepancy is mainly attributed to two factors: object surface deformation and finger flexion. Each object deforms differently under grasping, and since the gripper’s contact pads are rigid, the assumption of uniform pressure distribution may be weak. The finger flexion during grasping is attributed as the primary factor influencing the results. Because the contact surface is rigid, even slight misalignment due to finger flexion or other causes can significantly impact the pressure distribution, as illustrated in Figure 25. This phenomenon particularly affects rigid objects more than deformable ones, as finger flexion reduces the contact area and limits the torque that can be generated.

Figure 25.

Illustration of finger flex under higher grasp force.

Slip-avoidance

The slip-avoidance controller is designed to prevent slippage by dynamically adjusting the grasp force in response to disturbances. To evaluate its effectiveness, an object was grasped while various disturbances were introduced, as depicted in Figure 26. The most recently estimated contact properties from the experiments in Subsection Estimation of Contact Properties were used, see Table 10. The parameters for the slip-aware controllers, tuned through experimentation, are presented in Table 11.

Figure 26.

Disturbances to test the slip-avoidance controller, position 9 and 10 in the figure illustrate dropping a weight on the grasped object. The corresponding forces are presented in Figure 27.

Table 10.

Contact properties used in Subsection Slip-Avoidance to Rotational Slip Control.

Object	$μ_{c}^{(1)}$	r ⁽¹⁾	$μ_{c}^{(2)}$	r ⁽²⁾
Cardboard	0.514	4.84	0.473	6.30
Case	0.538	5.72	0.467	3.41
Plastic	0.355	7.69	0.365	5.75
Wood	0.459	7.15	0.404	1.90

Table 11.

Slip-aware control parameters.

Parameter	Value	Parameter	Value	Parameter	Value
γ _s	2	k _P,v	2	k _P,ω	2
γ _h	1.4	k _P,l	50	k _P,τ	2
α	0.95	k _I,l	1000	k _I,τ	40
f _n,min	0.9 N	k _D,l	0.1	k _D,τ	0.05
f _n,s,min	2 N	k _P,s	100	k _P,h	100

The first disturbance involved adding a 141 g weight on top of the grasped object, followed by placing the weight on either side to create a torque disturbance. The resulting forces are shown in Figure 27. When the weight was placed on the side of the object, the grasping force f_n increased to compensate for the additional torque. Subsequently, a heavier weight of 727 g was placed on the object, as illustrated in positions 4 to 6 in Figure 26. The slip-avoidance controller responded by increasing the grasp force to compensate for the added weight. When the heavier weight was placed on the corner of the object, the grasp force reached its maximum, as seen in Figure 27. Following this, a manual disturbance was introduced, as shown in positions 7 and 8 in Figure 26. Finally, to enforce slippage, the 141 g weight was dropped onto the grasped object, causing a sudden acceleration, depicted in positions 9 and 10 in Figure 26. This sudden impact resulted in a spike in the grasp force until the object decelerated. The corresponding linear displacement x, as measured by the velocity sensors, is also presented in Figure 27. The experiment demonstrates that the slip-avoidance controller effectively adjusts the grasp force in real-time to prevent slippage, even under varying and sudden disturbances.

Figure 27.

Forces and tangential velocity from the slip-avoidance controller.

Linear slip control

The linear slippage controller was evaluated through four different experiments for each of the objects listed in Table 10. The estimated contact properties and control parameters used are provided in Tables 10 and 11, respectively. The experiments were designed to assess the controller’s performance across a range of target displacements and slippage velocities:

(1) 20 mm linear displacement in 2 s.

(2) 20 mm linear displacement in 5 s.

(3) 40 mm linear displacement in 2 s.

(4) 40 mm linear displacement in 5 s.

Snapshots from these experiments are shown in Figures 28(a) and 28(b). The controller followed a trapezoidal trajectory, with both the trajectory and internal displacement estimation presented in Figure 29 from two of the tests. The results demonstrate that the linear slip controller can achieve the desired final displacement; however, it experiences the stick-slip phenomenon during trajectory following.

Figure 28.

Snapshots of linear slippage experiment with cardboard object.

Figure 29.

Linear slippage: top plot follows a 20 mm trajectory for 5 s and bottom plot shows a 40 mm trajectory for 2 s. $| \hat{x} |$ is the estimated displacement by the planar velocity sensors.

The entire slip-to-stick event can occur within 0.05 s, as shown in Figure 29. When the object transitions from sticking to slipping, the friction force shifts from static to Coulomb friction, resulting in rapid acceleration. As the rigid contact pad provides minimal damping compared to soft fingers, the object’s sliding velocity and acceleration become highly sensitive to the grasp force and contact properties. This sensitivity can cause the object to overshoot the intended trajectory upon slipping, leading the linear slip controller into a stick-slip cycle. Figure 30 illustrates two extremes in surface material behavior. The spectacle case with synthetic leather shows relatively smooth velocity transitions, with only minor adjustments to the grasp force. In contrast, the plastic object quickly overshoots the trajectory, triggering pronounced stick-slip motion, which is reflected in the grasp forces. Notably, the plastic object exhibits a larger difference between the static and Coulomb friction coefficients compared to the spectacle case (see Table 9), contributing to the pronounced stick-slip behavior observed during the experiment.

Figure 30.

Highlights the two extremes in stick and slip phenomenon between the plastic and case object, with the associated measured grasp forces.

Each experiment was conducted 10 times per object, with ground-truth displacement measured using a caliper. The statistical analysis of the results is presented in Table 12. The findings demonstrate that the proposed gripper, sensors, and controllers can accurately reposition objects through linear slippage. For plastic, cardboard, and wood objects, the tracking error in the intended setting was comparable to the measured tracking error using the test rig described in 7.2.

Table 12.

Linear slippage, distance x traveled, mean μ, and standard deviation σ.

Object	x (t = 2) (μ, σ)	$x - \hat{x} (t = 2)$ (μ, σ)	x (t = 5) (μ, σ)	$x - \hat{x} (t = 5)$ (μ, σ)
Cardboard (20 mm)	(20.85, 0.97)	(1.24, 1.04)	(20.61, 0.74)	(0.71, 0.74)
Case (20 mm)	(23.46, 1.07)	(3.67, 1.08)	(21.30, 0.63)	(1.26, 0.71)
Plastic (20 mm)	(20.40, 0.92)	(0.27, 1.05)	(20.86, 0.69)	(0.51, 0.69)
Wood (20 mm)	(20.19, 0.71)	(0.42, 0.65)	(20.59, 0.61)	(0.58, 0.44)
Cardboard (40 mm)	(42.93, 1.83)	(3.36, 1.47)	(41.67, 1.70)	(0.98, 1.52)
Case (40 mm)	(48.67, 5.68)	(8.27, 5.78)	(47.15, 4.17)	(6.93, 4.03)
Plastic (40 mm)	(41.05, 1.68)	(1.74, 1.62)	(42.35, 1.07)	(1.84, 0.73)
Wood (40 mm)	(39.44, 0.72)	(0.27, 0.83)	(40.61, 0.84)	(0.70, 0.57)

The synthetic leather case was a notable outlier, as it on average overshot the target displacement significantly further than the other objects. As shown in Table 12, this overshoot can be attributed to tracking inaccuracies, where the sensors underestimated the actual distance traveled. However, the tracking accuracy for the spectacle case improved at slower velocities, such as 20 mm in 5 s. It should be noted that during the 40 mm in 2 s experiment, there was one instance where the spectacle case fell out of the gripper.

Rotational slip control

The rotational slip controller leverages gravity to reorient the object by following specific trajectories, which were evaluated under different target displacements and velocities:

(1) 45° in 2 s: Starting from 15° off vertical.

(2) 45° in 5 s: Starting from 15° off vertical.

(3) 60° in 2 s: Starting from 60° off vertical.

(4) 60° in 5 s: Starting from 60° off vertical.

The initial and final poses of the rotational slip experiments are shown in Figures 31(a) and 31(b). The internally measured rotational displacements compared to the desired trajectories are presented in Figure 32. The trajectory following in the rotational slip experiments appears smoother compared to the linear slippage experiments; the slip-stick motion is less pronounced. This smoother response can be partially attributed to the limited control authority for generating torque. The grasp forces between the linear and rotational slippage experiments differ significantly, as seen by comparing the forces in Figures 30 and 33. Higher grasp forces cause more pronounced finger flexion, reducing the effective surface contact area, as illustrated in Figure 25, which reduces torque. This finger flexion can result in a nonlinear torque response to the grasp force. However, it is worth noting that the grasp force oscillates more in the rotational slip experiments than in the linear ones. The parameters used for the rotational controller, listed in Table 11, were tuned for trajectory following rather than smooth grasp forces. Additionally, the wooden object saturated the rotational controller and was therefore not tested further.

Figure 31.

Snapshots of rotational slippage experiment.

Figure 32.

Rotational slippage for 45°, top plot shows a 5 s trajectory, and bottom plot shows a 2 s trajectory. $| \hat{θ} |$ is the estimated displacement by the planar velocity sensors.

Figure 33.

Highlights the grasp force and trajectory following for the plastic and case object.

Each experiment was conducted 10 times for each object, with the ground-truth rotation measured with a digital level meter. The statistical results are presented in Table 13. The data indicates that the rotational slippage controller consistently overshoots the target rotation on average, primarily due to tracking errors. According to the internal rotation estimates, the controller comes close to the target, but real-world measurements typically show an overshoot of approximately 10°. During linear slippage tests, the estimated displacement was closely aligned with real-world measurements. However, two factors contribute to the discrepancies in rotational slippage: higher grasp forces and a higher likelihood of multiple optical sensors being outside the object.

Table 13.

Rotational slippage, angle θ rotated, mean μ, and standard deviation σ.

Object	θ (t = 2) (μ, σ)	$θ - \hat{θ} (t = 2)$ (μ, σ)	θ (t = 5) (μ, σ)	$θ - \hat{θ} (t = 5)$ (μ, σ)
Cardboard (45°)	(54.0, 6.3)	(8.6, 5.9)	(52.9, 8.6)	(9.5, 8.4)
Case (45°)	(50.0, 6.4)	(3.7, 6.2)	(48.7, 4.0)	(3.7, 4.4)
Plastic (45°)	(58.9, 6.3)	(13.4, 6.2)	(56.2, 4.5)	(12.6, 5.6)
Cardboard (60°)	(63.9, 3.9)	(5.3, 3.8)	(74.0, 12.7)	(16.5, 11.7)
Case (60°)	(65.1, 3.9)	(4.9, 4.1)	(63.9, 5.4)	(6.8, 7.2)
Plastic (60°)	(70.8, 3.4)	(10.5, 3.8)	(74.9, 7.6)	(17.1, 7.0)

A higher grasp force increases finger flexion, which alters the distance between the sensor and the object surface. Optical mouse sensors are generally sensitive to vertical distance, potentially impacting accuracy. The authors attribute most of the tracking error to situations where multiple optical sensors are outside the object simultaneously. The current gripper design has limited space for accommodating object rotation without causing collisions, often leading to object placements where more than one sensor could be tracking outside of the object during the motion. When only one of three optical sensor accurately tracks velocity, displacement is likely to be underestimated.

Hinge control

The hinge controller enables the gripper to rotate while keeping the CoG of the object directly beneath the grasp point, as shown in Figure 34. In the experiment, the gripper is initially rotated 30° away from the vertical axis, with the object hanging vertically from the gripper. The gripper, with hinge control activated, is then rotated 60° in the opposite direction, while the object is expected to remain in a vertical orientation.

Figure 34.

Hinge mode experiment.

The final orientation of the object relative to vertical is measured using a digital level meter, and the results are summarized in Table 14. These results demonstrate that the hinge controller effectively maintains the object’s orientation as the gripper rotates. However, it should be noted that the cardboard object fell out of the gripper once during the trials, and that the robot introduced significant shaking during the rotational motion.

Table 14.

Hinge mode, object change of orientation, mean μ, and standard deviation σ.

Object	μ (deg)	σ
Cardboard	8.4	4.9
Case	7.2	1.9
Plastic	11.5	6.8
Wood	4.8	1.5

Testing the limits

Two edge cases are evaluated: (i) the maximum speed at which an object can be repositioned within the gripper and (ii) the effect of surface curvature on performance. To simplify the analysis, only plastic objects are considered.

Case (i): the object is commanded to slide 20 mm over various target durations (see Table 15). For the target time of 1 s the performance is similar to Subsection 7.5. For target times of 0.5 s and 0.25 s, the controller overshoots the desired displacement by approximately 5 mm. However, for the shortest duration (0.1 s), the overshoot is reduced to 2.7 mm. This improvement is attributed to the absence of stick-slip effects, as the object’s maximum acceleration and velocity are lower than the commanded values, resulting in no trajectory overshoot within the specified duration. Notably, the 0.1 s trajectory takes an average of 0.17 s to complete.

Table 15.

Testing the limits with plastic object for linear slippage of 20 mm with different trajectory times and surface curvatures. Distance x traveled, mean μ, standard deviation σ, and actual time t. A fail is considered if the object falls out of the gripper.

Trajectory time	x (μ, σ, t)	Surface radius	x (μ, σ)	$\hat{x}$ μ	Fail rate
1 s	(21.87, 1.80, 0.98)	200 mm	(56.28, 7.81)	19.66	10%
0.5 s	(25.40, 4.60, 0.41)	100 mm	(N/A)	17.84	100%
0.25 s	(25.96, 3.64, 0.24)	50 mm	(N/A)	13.39	100%
0.1 s	(22.70, 1.78, 0.17)

Case (ii): While the planar velocity sensors are optimized for flat surfaces, this experiment evaluates performance degradation with increasing surface curvature. The tested objects, shown in Figure 35, all match the weight of the plastic object in Table 3. Each object is commanded to slide 20 mm in 2 s, with results summarized in Table 15. As curvature increases, the velocity tracking progressively underestimates the true velocity. This degradation is attributed to the sensor moving away from its optimal sensing distance as the surface curves. On the 200 mm radius surface, the object fell from the gripper in one out of 10 trials and exhibited significant overshoot. For the 100 mm and 50 mm radius surfaces, the object fell out in all trials, with estimated displacements decreasing further as curvature increased.

Figure 35.

Plastic objects with curved surfaces.

In-hand manipulation demonstration

The final demonstration integrates all of the proposed methods into a single cohesive performance, as shown in Figure 36. The sequence begins by grasping an object in a known pose, followed by the estimation of the contact properties. Once the object is securely grasped, the slip-avoidance controller is activated, and the object is lifted. Next, the linear slip controller is employed to move the object 30 mm downward over 3 s, after which the hinge controller is used to reorient the gripper relative to the object. The slip-avoidance controller then reorients both the gripper and the object, setting the stage for the rotational slip controller to rotate the object by 30° within 3 s.

Figure 36.

In-hand manipulation demonstration.

This demonstration was conducted 10 times using the plastic object, and successfully performing all tasks 9 times without dropping the object, achieving a 90% success rate. The demonstration highlights the capabilities of the proposed system and illustrates how gravity-assisted slip control can endow parallel jaw grippers with the ability to manipulate an object and change the gripper-object configuration.

Conclusions

In this paper, we have presented a comprehensive system for advanced in-hand slip control, featuring several key components: a parallel gripper that enables fast and precise force control, relative velocity sensors mounted on force-torque sensors that accurately measure both tangential and rotational velocities, and a fast contact property estimation process. Additionally, we introduced four slip-aware controllers: slip-avoidance, trajectory-following for linear and rotational slippage, and hinge control. The gripper, equipped with these sensors, was mounted on a UR10 robot, and our system was rigorously tested across a wide range of scenarios. The results demonstrate that using solely in-hand sensing, our approach advances the state of the art for in-hand slip control.

This study demonstrates the significant potential of the proposed hardware system. However, our immediate focus will be on enhancing the sensing capabilities and improving the physical contact properties to relax the constraint of parallel surfaces or a rigid contact pad, thereby enabling interaction with more complex object geometries. Beyond this, the research could branch into various promising directions. For instance, future work could explore dynamic movements and manipulation, smoother slippage control, online friction estimation, extend the effective reach of manipulators, slip-aware multi-arm manipulation, manipulation based on external contacts, sensor fusion with external sensors, planning for slip-aware systems, and advancing human–robot interactions.

Supplemental Material

Footnotes

ORCID iDs

Gabriel Arslan Waltersson

Yiannis Karayiannidis

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was funded by the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation. Website:

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Supplemental Material

Supplemental material for this article is available online.

Note

References

Arimoto

Han

Cheah

, et al. (1999) Extension of impedance matching to nonlinear dynamics of robotic tasks. Systems & Control Letters 36(2): 109–119. https://www.sciencedirect.com/science/article/pii/S016769119800084X

Arslan Waltersson

Karayiannidis

(2024) Planar friction modeling with lugre dynamics and limit surfaces. IEEE Transactions on Robotics 40: 3166–3180.

Åström

Canudas-de Wit

(2008) Revisiting the LuGre friction model. IEEE Control Systems Magazine 28(6): 101–114.

Bhatt

Sieler

Puhlmann

, et al. (2021) Surprisingly robust in-hand manipulation: an empirical study. In: Robotics: Science and Systems XVII, RSS2021. Robotics: Science and Systems Foundation. IEEE.

Sferrazza

D’Andrea

(2021) Zero-shot sim-to-real transfer of tactile control policies for aggressive swing-up manipulation. https://arxiv.org/abs/2101.02680

Chavan-Dafle

Mason

Staab

, et al. (2015) A two-phase gripper to reorient and grasp. In: 2015 IEEE International Conference on Automation Science and Engineering (CASE), 1249–1255. IEEE.

Chavan-Dafle

Holladay

Rodriguez

(2020) Planar in-hand manipulation via motion cones. The International Journal of Robotics Research 39(2-3): 163–182.

Chen

Khamis

Birznieks

, et al. (2018) Tactile sensors for friction estimation and incipient slip detection—toward dexterous robotic manipulation: a review. IEEE Sensors Journal 18(22): 9049–9064.

Chen

Prepscius

Lee

, et al. (2021) Tactile velocity estimation for controlled in-grasp sliding. IEEE Robotics and Automation Letters 6(2): 1614–1621.

10.

Chi

Sun

Xue

, et al. (2018) Recent progress in technologies for tactile sensors. Sensors 18(4).

11.

Chorley

Melhuish

Pipe

, et al. (2009) Development of a tactile sensor based on biologically inspired edge encoding. 2009 International Conference on Advanced Robotics. IEEE, 1–6.

12.

Costanzo

De Maria

Natale

, et al. (2019) Design and calibration of a force/tactile sensor for dexterous manipulation. Sensors 19(4). Available at: https://www.mdpi.com/1424-8220/19/4/966.

13.

Costanzo

De Maria

Natale

(2023) Detecting and controlling slip through estimation and control of the sliding velocity. Applied Sciences 13(2): 921. https://www.mdpi.com/2076-3417/13/2/921

14.

Dafle

Rodriguez

Paolini

, et al. (2014) Extrinsic dexterity: In-hand manipulation with external forces 2014 IEEE International Conference on Robotics and Automation (ICRA), 1578–1585. IEEE.

15.

Dahiya

Metta

Valle

, et al. (2010) Tactile sensing—from humans to humanoids. IEEE Transactions on Robotics 26(1): 1–20.

16.

Damian

Newton

Pfeifer

, et al. (2015) Artificial tactile sensing of position and slip speed by exploiting geometrical features. IEEE/ASME Transactions on Mechatronics 20(1): 263–274.

17.

De Maria

Natale

Pirozzi

(2012) Force/tactile sensor for robotic applications. Sensors and Actuators A: Physical 175: 60–72. https://www.sciencedirect.com/science/article/pii/S0924424711007564

18.

De Maria

Falco

Natale

, et al. (2015) Integrated force/tactile sensing: the enabling technology for slipping detection and avoidance. In: 2015 IEEE International Conference on Robotics and Automation (ICRA), 3883–3889. IEEE.

19.

Gloumakov

Huh

Stuart

(2024) Fast in-hand slip control on unfeatured objects with programmable tactile sensing. IEEE Robotics and Automation Letters 9(7): 6059–6066.

20.

Goyal

Ruina

Papadopoulos

(1989) Limit surface and moment function descriptions of planar sliding. In: 1989 IEEE International Conference on Robotics and Automation. IEEE Computer Society, 794–795.

21.

Goyal

Ruina

Papadopoulos

(1991) Planar sliding with dry friction part 1. limit surface and moment function. Wear 143(2): 307–330.

22.

Hertz

(1896) On the contact of solids—on the contact of rigid elastic solids and on hardness. Miscellaneous papers: 146–183.

23.

Holladay

Paolini

Mason

(2015) A general framework for open-loop pivoting. In: 2015 IEEE International Conference on Robotics and Automation (ICRA), 3675–3681. IEEE.

24.

Hou

Jia

Mason

(2018) Fast planning for 3d any-pose-reorienting using pivoting. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), 1631–1638. IEEE.

25.

Höver

Harders

(2010) Measuring and incorporating slip in data-driven haptic rendering. In: 2010 IEEE Haptics Symposium, 175–182. IEEE.

26.

Howe

Cutkosky

(1989) Sensing skin acceleration for slip and texture perception. Proceedings, 1989 International Conference on Robotics and Automation 1: 145–150.

27.

Huang

Fan

Duan

, et al. (2025) Human-like dexterous manipulation for anthropomorphic five-fingered hands: a review. Biomimetic Intelligence and Robotics 5(1): 100212. https://www.sciencedirect.com/science/article/pii/S2667379725000038

28.

Huh

Choi

Willcox

, et al. (2020) Dynamically reconfigurable tactile sensor for robotic manipulation. IEEE Robotics and Automation Letters 5(2): 2562–2569.

29.

Johnson

Adelson

(2009) Retrographic sensing for the measurement of surface texture and shape. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, 1070–1077. IEEE.

30.

Kamiyama

Vlack

Mizota

, et al. (2005) Vision-based sensor for real-time measuring of surface traction fields. IEEE Computer Graphics and Applications 25(1): 68–75.

31.

Kappassov

Corrales

Perdereau

(2015) Tactile sensing in dexterous robot hands — review. Robotics and Autonomous Systems 74: 195–220. https://www.sciencedirect.com/science/article/pii/S0921889015001621

32.

Khamis

Izquierdo Albero

Salerno

, et al. (2018) Papillarray: an incipient slip sensor for dexterous robotic or prosthetic manipulation – design and prototype validation. Sensors and Actuators A: Physical 270: 195–204. https://www.sciencedirect.com/science/article/pii/S0924424717313419

33.

Kontoudis

Liarokapis

Vamvoudakis

(2019) An adaptive, humanlike robot hand with selective interdigitation: towards robust grasping and dexterous, in-hand manipulation. In: 2019 IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids), 251–258. IEEE.

34.

Verdoja

Abu-Dakka

, et al. (2021) Probabilistic surface friction estimation based on visual and haptic measurements. IEEE Robotics and Automation Letters 6(2): 2838–2845.

35.

Liu

Meusel

Hirzinger

, et al. (2008) The modular multisensory dlr-hit-hand: hardware and software architecture. IEEE/ASME Transactions on Mechatronics 13(4): 461–469.

36.

(2022) Design and experimental research of robot finger sliding tactile sensor based on fbg. Sensors 22(21): 8390. https://www.mdpi.com/1424-8220/22/21/8390

37.

Lyu

Zhang

, et al. (2025) Fiber optic sensors in tactile sensing: a review. IEEE Transactions on Instrumentation and Measurement 74: 1–16.

38.

Maldonado

Alvarez-Heredia

Beetz

(2012) Improving robot manipulation through fingertip perception. In: IEEE International Conference on Intelligent Robots and Systems (IROS). IEEE. https://ai.uni-bremen.de/papers/maldonado12improving.pdf

39.

Odhner

Jentoft

Claffee

, et al. (2014) A compliant, underactuated hand for robust manipulation. The International Journal of Robotics Research 33(5): 736–752.

40.

OpenAI

Baker

Chociej

, et al. (2019) Learning dexterous in-hand manipulation. https://arxiv.org/abs/1808.00177

41.

Piga

Natale

(2023) Adaptive tactile force control in a parallel gripper with low positioning resolution. IEEE Robotics and Automation Letters 8: 1–8.

42.

Puhlmann

Harris

Brock

(2022) Rbo hand 3: a platform for soft dexterous manipulation. IEEE Transactions on Robotics 38(6): 3434–3449.

43.

Qian

Ren

, et al. (2018) Sliding sensor using fiber Bragg grating for mechanical fingers. Optics Express 26(1): 254–264. https://opg.optica.org/oe/abstract.cfm?URI=oe-26-1-254

44.

Sani

Meek

(2011) Characterizing the performance of an optical slip sensor for grip control in a prosthesis. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, 1927–1932. IEEE.

45.

Santina

Piazza

Grioli

, et al. (2018) Toward dexterous manipulation with augmented adaptive synergies: the pisa/iit softhand 2. IEEE Transactions on Robotics 34(5): 1141–1156.

46.

Skuric

Bank

Unger

, et al. (2022) Simplefoc: a field oriented control (foc) library for controlling brushless direct current (bldc) and stepper motors. Journal of Open Source Software 7(74): 4232.

47.

Stribeck

(1902) Die wesentlichen eigenschaften der gleit-und rollenlager [the key qualities of sliding and roller bearings]. Z. Vereines Seutscher Ing 46: 1432–1437. https://cir.nii.ac.jp/crid/1572824500551324032

48.

Tar

Cserey

(2011) Development of a low cost 3d optical compliant tactile force sensor. In: Advanced Intelligent Mechatronics (AIM), 2011 IEEE/ASME International Conference on, 236–240. IEEE.

49.

Tripicchio

D’Avella

Avizzano

, et al. (2023) Towards robust grasping: an analysis of in-hand object motion with fbg optical fibers as force sensing technology. Mechatronics 93: 102990. https://www.sciencedirect.com/science/article/pii/S0957415823000466

50.

Viña

BFE

Karayiannidis

Pauwels

, et al. (2015) In-hand manipulation using gravity and controlled slip. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 5636–5641. IEEE.

51.

Viña

BFE

Karayiannidis

Smith

, et al. (2016) Adaptive control for pivoting with visual and tactile feedback. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), 399–406. IEEE.

52.

Wang

Xiao

Fan

, et al. (2016) Design and grip force control of dual-motor drive electric gripper with parallel fingers. In: 2016 IEEE Information Technology, Networking, Electronic and Automation Control Conference, 696–700. IEEE.

53.

Wang

Chen

Mei

(2019) Flexible tactile sensor array for slippage and grooved surface recognition in sliding movement. Micromachines 10(9). Available at: https://www.mdpi.com/2072-666X/10/9/579.

54.

Wang

Romero

, et al. (2021) Swingbot: learning physical features from in-hand tactile exploration for dynamic swing-up manipulation. https://arxiv.org/abs/2101.11812

55.

Ward-Cherrier

Pestell

Cramphorn

, et al. (2018) The tactip family: soft optical tactile sensors with 3d-printed biomimetic morphologies. Soft Robotics 5: 216–227.

56.

Weinberg

Shirizly

Azulay

, et al. (2024) Survey of learning-based approaches for robotic in-hand manipulation. Frontiers in Robotics and AI 11: 11–2024.

57.

Wettels

Fishel

Loeb

(2014) The Human Hand as an Inspiration for Robot Hand Development. Springer International Publishing, 405–429.

58.

Xydas

Kao

(1999) Modeling of contact mechanics and friction limit surfaces for soft fingers in robotics, with experimental results. The International Journal of Robotics Research 18(9): 941–950.

59.

Yang

Wei

Ren

, et al. (2021) A low-cost linkage-spring-tendon-integrated compliant anthropomorphic robotic hand: Mcr-hand iii. Mechanism and Machine Theory 158: 104210. https://www.sciencedirect.com/science/article/pii/S0094114X20304274

60.

Yao

Jiang

, et al. (2022) Fabrication and experimental validation of a sensitive and robust tactile sensing array with a micro-structured porous dielectric layer. Micromachines 13(10): 1724. https://www.mdpi.com/2072-666X/13/10/1724

61.

Yousef

Boukallel

Althoefer

(2011) Tactile sensing for dexterous in-hand manipulation in robotics—a review. Sensors and Actuators A: Physical 167(2): 171–187. https://www.sciencedirect.com/science/article/pii/S0924424711001105

62.

Yuan

Dong

Adelson

(2017) Gelsight: High-resolution robot tactile sensors for estimating geometry and force. Sensors 17(12). Available at: https://www.mdpi.com/1424-8220/17/12/2762.

63.

Zaki

Soliman

Osama

, et al. (2010) Design and implementation of efficient intelligent robotic gripper. In: 2010 International Conference on Modelling, Identification and Control, ICMIC 2010, Vol. 9, 710–716. IEEE.

64.

Zhang

Cui

Zhang

, et al. (2022) Visuotactile feedback parallel gripper for robotic adaptive grasping. In: 2022 IEEE International Conference on Robotics and Biomimetics (ROBIO), 1855–1860. IEEE.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

Perception,control,and hardware for in-hand slip-aware object manipulation with parallel grippers

Abstract

Keywords

Introduction

Related work

Tactile sensors

Grippers

Slip control and estimation

Contributions and overview

Gripper design

Hardware

Grasp force control

Velocity sensor design

Planar velocity sensor

Velocity sensor calibration

Estimation of contact properties

Controlled slippage

Slip-avoidance

Linear slip control

Rotational slip control

Hinge mode

Results

Gripper performance

Relative velocity sensor

Estimation of contact properties

Slip-avoidance

Linear slip control

Rotational slip control

Hinge control

Testing the limits

In-hand manipulation demonstration

Conclusions

Supplemental Material

Footnotes

ORCID iDs

Funding

Declaration of conflicting interests

Supplemental Material

Note

References

Supplementary Material