Sage Journals: Discover world-class research

Abstract

This paper is concerned with compliant haptic contact and deformation of soft objects. A human soft fingertip model is considered to act as the haptic interface and is brought into contact with and deforms a discrete surface. A nonlinear constitutive law is developed in predicting normal forces and, for the haptic display of surface texture, motions along the surface are also resisted at various rates by accounting for dynamic Lund-Grenoble (LuGre) frictional forces. For the soft fingertip to apply forces over an area larger than a point, normal and frictional forces are distributed around the soft fingertip contact location on the deforming surface. The distribution is realized based on a kernel smoothing function and by a nonlinear spring-damper net around the contact point. Experiments conducted demonstrate the accuracy and effectiveness of our approach in real-time haptic rendering of a kidney surface. The resistive (interaction) forces are applied at the user fingertip bone edge. A 3-DoF parallel robotic manipulator equipped with a constraint based controller is used for the implementation. By rendering forces both in lateral and normal directions, the designed haptic interface system allows the user to realistically feel both the geometrical and mechanical (nonlinear) properties of the deforming kidney.

Keywords

Haptics Surface Deformation LuGre Friction Soft Fingertip Tissue

1. Introduction

Researchers have long been interested in the fundamental dynamic properties of contact interaction with deformable objects. In retrospect, there has long been a growing and continuous demand for accurate deformation estimation between non-rigid objects. Although discretization such as finite and boundary element methods are among the most accurate numerical algorithms for simulating contact interactions and deformation, their implementation comes with a high computation cost and requires adopting specific mitigation measures. Among these measures are condensation [1], equivalent mesh use [2] and pre-computation [3, 4]. In spite of these faster finite element techniques predicting, in most cases, nearly the same deformations as those obtained by conventional linear finite element analysis [1], they are in general not feasible for real-time interactive haptic rendering, especially when complex geometric objects made of a variety of materials are considered. Within this perspective [5–7], the mass-spring-damper (MSD) physical modelling scheme has probably been the most appropriate alternative, especially at update rate requirements of over 1 kHz [8].

In haptic rendering of deformable objects, constraint- and penalty-based methods have long been the interest of many researchers. Although much has been accomplished over the last couple of decades in terms of the direct force and/or tactile feedback controllers [9, 10], realistic and fast haptic rendering of deformable objects has not yet been a success. The main obstacle to this endeavour is the difficulty in designing a unique and complete constitutive (load-deflection) model for nonlinear dynamic contact and deformations. Researchers have therefore been interested in developing finite deformation models of various materials exhibiting nonlinear constitutive responses [11–14]. Due mainly to the complexity of characterizing the material properties under various loading conditions, proper modelling is a challenging task. For MSD systems, force distribution methods have been successfully used to achieve more realistic tissue simulations [15, 16]. However, research towards a complete accurate haptic contact and deformation system is still in development.

Realizing that most tissue interaction research makes use of simple objects and kinaesthetic feedback (force feedback via a haptic interface device), this study aims for haptic interaction modelling between a soft fingertip and a complex shaped nonlinearly viscoelastic object. This choice allows us to address the common need for developing a computationally effective, accurate, non-linear and deformable viscoelastic model [17]. To this aim, the soft fingertip is considered to be virtually attached to the end effector of a haptic device and manipulated by the user to explore the mesh model of a kidney. To achieve a realistic response [18, 19], a compliant nonlinear modelling approach is employed in the normal direction by facilitating nonlinear stiffness and damping elements exhibiting rate-dependent behaviours. In the tangential directions, a simplified form of the LuGre (Lund-Grenoble) dynamic friction model [20, 21] is incorporated to predict the tangential reaction force [22] and thus resist the motion at various rates to feel surface texture. Noting that local deformation of the kidney tissue is a function of both the normal and frictional forces, contact forces are distributed to nodes neighbouring the contact point by utilizing a proper kernel weighting function. The weighing function provides a continuous symmetric force distribution within a certain radius of influence from the contact point. The nodes falling inside the influence zone are then assigned nonlinear spring-damper elements and the forces distributed are used in determining the corresponding nodal displacements (thereby the deformed shape of the kidney tissue).

A constraint-based controller is designed for implementation of our approach for the user to haptically explore a compliant surface and feel the effects of curvature variations and friction. The controller is derived from a closest-point algorithm and allows for free motion of the soft fingertip in space before the contact and provides stable interaction with deformable surfaces without any need for force and acceleration data. The accuracy and consistency of the presented framework are proven by both the theoretical and experimental analysis. Experimental results on real-time interaction data show realistic and numerically effective interactions.

2. Haptic Display System

2.1. Surface model

Let us consider (n+1) and (m+1) data points in parametric directions $u_{1}$ and $u_{2}$ on a discrete surface. A rectangular parametric surface patch [23] may then be created by blending interpolating polynomials $p (u_{1})$ and $p (u_{2})$ in parametric directions:

p (u_{1}, u_{2}) = [\begin{matrix} U_{1} M_{n} X_{n, m} M_{m}^{T} U_{2}^{T} \\ U_{1} M_{n} Y_{n, m} M_{m}^{T} U_{2}^{T} \\ U_{1} M_{n} Z_{n, m} M_{m}^{T} U_{2}^{T} \end{matrix}] .

(1)

Here, $U_{1} = [\begin{matrix} u_{1}^{n} & \dots & u_{1}^{2} & u_{1} & 1 \end{matrix}]$ , $U_{2} = [\begin{matrix} u_{2}^{m} & \dots & u_{2}^{2} & u_{2} & 1 \end{matrix}]$ . $M_{n}$ and $M_{m}$ are the coefficient matrices and $X_{n, m}$ , $Y_{n, m}$ , $Z_{n, m}$ denote the $(n + 1) \times (m + 1)$ matrices of control vertices on the surface. In the case of the haptic of deformation simulation by the soft fingertip tissue interacting with the discrete surface, to achieve $C^{2}$ continuity, a small number of patches that share control points will be considered around the contact point. This will be accomplished via a kernel-smoothing function.

2.2. Closest point

With $p_{r} = {[\begin{matrix} x_{r} & y_{r} & z_{r} \end{matrix}]}^{T}$ as the soft fingertip edge bone (distal phalanx) or equivalently the haptic device end-effector Cartesian location,

e_{p} = p_{r} - p

(2)

represents the vector from the end-effector to the nearest location on the surface. An illustration of this set up is given in Figure 1 where a local reference frame is also described by unit vectors n, t and b in the normal, tangential and binormal directions.

The nearest location from $p_{r}$ to the surface is determined by $e_{p} \times n = 0$ or:

J^{T} e_{p} = 0,

(3)

which is solved for the local coordinates $u_{1}$ and $u_{2}$ . Here, $n = (p_{u_{1}} \times p_{u_{2}}) / ‖ p_{u_{1}} \times p_{u_{2}} ‖$ is the unit normal vector and $J = [\begin{matrix} p_{u_{1}} & p_{u_{2}} \end{matrix}]$ the 3×2 Jacobian matrix. $‖ \cdot ‖$ here and throughout denotes the vectorial 2-norm (i.e., the traditional Euclidian norm) and the subscripts indicate partial derivatives of the corresponding variable. In cases where solving equation (3) for $u_{1}$ and $u_{2}$ is difficult, it may be linearized and the parametric coordinates are then obtained by successive iterations. In the process, note that each surface patch is parameterized over a domain such as $(u_{1}, u_{2}) \in ([0, 1] \times [0, 1])$ and the closest point determined by the solution of (3) can be used to identify the nearest surface patch. The local coordinates are thereby projected inside this patch and the amount of penetration identified.

Figure 1.

Schematic representation of the haptic device, fingertip position and the surface

Remark 2.1: The first and second fundamental forms in matrix form are described by [24]:

\begin{matrix} G = J^{T} J, \\ D = - J^{T} n_{u}^{T} = [\begin{matrix} n^{T} p_{u_{1} u_{1}} & n^{T} p_{u_{1} u_{2}} \\ n^{T} p_{u_{2} u_{1}} & n^{T} p_{u_{2} u_{2}} \end{matrix}], \end{matrix}

(4)

where $u = {[\begin{matrix} u_{1} & u_{2} \end{matrix}]}^{T}$ . equation (4) implies $n_{u}^{T} = - J G^{- 1} D$ .

Remark 2.2: $S = n n^{T}$ is the projection matrix for the contact normal plane. It is symmetric, idempotent and positive semi definite with eigenvalues consisting of ones and zeroes. Obviously, $S^{2} = S$ and $S J = 0.$

2.3. Contact force model

Continuous contact dynamic models, also referred to as compliant contact models, account explicitly for the deformations of the contacting bodies. In such models, contact force in the normal direction is an explicit function of the local indentation e and its rate of change $\dot{e}$ :

f_{n} = g (e, \dot{e}) .

(5)

Most common form of this equation is the linear spring-damper model:

f_{n} = k_{1} e + b_{1} \dot{e},

(6)

where $k_{1}$ and $b_{1}$ correspond to constant stiffness and viscous friction coefficients. Although it is the simplest compliant model, the linear spring-damper model is not realistic as the material and the geometric nonlinearities are not rightfully accounted for. Most materials including the soft fingertip tissue are geometrically nonlinear as their stiffness and damping strongly depend on the displacements they undergo. Beside geometric nonlinearity, they may exhibit significant physical (material) nonlinearities.

Hertzian contact law [25] is based on a nonlinear spring along the line of contact and usually serves as a good starting point in developing realistic contact force models. One such contact model is proposed in [26] for estimating the fingertip normal forces. Normal forces registered by the user pressing steadily against a flat (transparent acrylic) plate are shown to compare well with the forces predicted by the Hertzian formulation. Although this study [26] shows the Hertzian model applies well to static, or quasi-static, elastic cases, it cannot be used for compliant dynamic interactions where the fingertip tissue interacts with deformable objects. This implies that for nonlinear viscoelastic and rate-dependant interactions between the fingertip and a deformable surface, the Hertzian contact model does not result in a near fit to the actual force-indentation curves [27]. To overcome the problems of the linear spring-damper model (6) and gain the advantages of the Hertz model, alternatives may be considered. In the model proposed by Hunt and Crossley [28], energy dissipation (hysteretic damping) is accounted for in addition to nonlinear local indentation:

f_{n} = k_{1} e^{n} + b_{1} e^{n} {\dot{e}}^{m} .

(7)

Here, e, $\dot{e}$ are positive and the constants n, m are usually set greater than or equal to 1. Although (7) is a physically consistent contact force model, it does not yield realistic fingertip indentation forces [18, 19]. Therefore, inspired by the non-linear Hunt-Crossley model, we have aimed for a new mathematical model of the nonlinear conservative contact force, giving a close fit to independent experimental data [18, 19]. Noting that the fingertip tissue model must become stiffer with increasing displacement, the contact stiffness $k_{1}$ in (7) must be relatively smaller than the Hertz law indicates. To this aim, the normal (contact) force to the fingertip is modelled by equation (6) and:

\begin{matrix} k_{1} = c_{0} \exp (c_{1} e), \\ b_{1} = c_{2} \exp (c_{3} e + c_{4} \sqrt{\dot{e}}) \end{matrix}

(8)

are used to define the nonlinear contact stiffness and damping coefficients. The constants $c_{0}, c_{1}, c_{2}, c_{3}$ and $c_{4}$ are determined experimentally for the specific contact scenario. Figure 2 (a) shows a comparison of experimental force-indentation data [29] with values predicted by Equations (6) and (8) using $c_{0} = 39.3, c_{1} = 2100, c_{2} = 1, c_{3} = 3360$ and $c_{4} = - 3.6$ . See that our model presents a near match to experimental data. It is thus expected to accurately predict the normal force at any given indentation e with varying indentation rates $\dot{e}$ .

It is clear that almost none of the materials and soft tissues in particular have the same mechanical behaviour. Therefore, realistic force-indentation formulations require insightful conceptual understanding of the interaction dynamics between the soft fingertip and the contacting surface. Because the contact force (6) exerted at the soft fingertip (distal phalanx) is dependent on the mechanical and geometrical features of the deforming object, realistic haptic surface exploration calls for a proper conceptual articulation and is not possible without prior modelling of the deforming object's characteristics. Figure 2 (b) shows the measured and calculated force-indentation data when a rigid probe is pushed against a kidney tissue surface [30]. The stiffness functional that achieves a near match to experimental data [30] is given by:

k_{2} = c_{5} \exp (c_{6} s_{2}),

(9)

where $s_{2}$ denotes the contact point indentation of the kidney and $c_{5} = 45, c_{6} = 235$ . In producing Figure 2 (b), $f_{n} = k_{2} s_{2}$ is used.

Noting that energy dissipation is an essential property of biological tissues, a stiffness only model is not preferred. Therefore, a nonlinear viscous damping coefficient $b_{2}$ is added to the formula governing kidney indentations:

b_{2} {\dot{s}}_{2} + k_{2} s_{2} = f_{n} .

(10)

Here, f_n is the normal (contact) force determined by equation (6).

Figure 2.

Normal forces; (a) soft fingertip pressed against a rigid surface; (b) kidney indentation by a rigid spherical object

In Figure 3 (a), a schematic description of the resulting 3D constitutive model is seen. It shows the soft fingertip moving towards the kidney surface, where $e_{c}$ corresponds to the fingertip tissue thickness and $e_{n} = n^{T} e_{p}$ the distance to the closest point on the surface. In Figure 3 (b), the instant of contact is depicted. Here, $k_{1}, k_{2}$ and $b_{1}, b_{2}$ describe stiffness and damping coefficients of the soft fingertip and the kidney tissues, simultaneously. Figure 3 (c), shows that as the fingertip is pressed against the kidney surface and moved at a sliding velocity $\dot{r}$ on the deforming surface, the displacements $s_{1}$ and $s_{2}$ are registered. Here, $f_{n}$ and $f_{f}$ represent the normal and the frictional contact forces presumed to act at the soft fingertip bone edge due to the fact that human perception of touch is accomplished through the bones [31]. Noting that the relative distance $(s_{1} - s_{2})$ is equal to the fingertip indention $e = (e_{c} - e_{n})$ , it follows from $e = n^{T} e$ that

e = e_{c} n - e_{p} .

(11)

Then, taking the time derivative of equation (11) and using Remarks 2.1 and 2.2 yield [24]:

\dot{e} = J_{e} {\dot{p}}_{r},

(12)

where:

J_{e} = e_{c} n_{u}^{T} {(G - e_{n} D)}^{- 1} J^{T} - S .

(13)

Property 2.1: By Remark 2.2, $S J_{e} = - S$ and $J_{e}^{- T} S J_{e}^{- 1} = S$ .

The normal force $f_{n}$ in Figure 3 (c) is expressed by:

f_{n} = f_{n} n,

(14)

where f_n is computed by equation (6) using $\dot{e} = n^{T} \dot{e}$ and equation (8).

Figure 3.

Constitutive model

2.4. Dynamic friction model

Because the classical stick-slip (static Coulomb friction) model is unrealistic [22] and friction is a highly complex nonlinear phenomenon, there is a huge number of dynamic friction models in literature [20–22, 32–34]. From them, we have decided to use the LuGre model [20, 21, 34] as it is simple to implement and accounts not only for the break-away force and the viscous friction, but also for the Stribeck effect. To reduce complexity and guarantee continuity of the model, the variations in the sliding velocity $v (t)$ are assumed negligible. The LuGre friction force model is then given by:

{\hat{f}}_{f} = f_{n} [μ_{c} + (μ_{s} - μ_{c}) \exp (- \frac{v^{2}}{v_{s}^{2}})] + f_{v} v,

(15)

where $μ_{c}$ , $μ_{s}$ and $f_{v}$ represent the coefficients of Coulomb, stiction and viscous friction. To take into consideration the normal force influence on the Stribeck velocity $v_{s}$ :

v_{s} = c_{7} {[1 + c_{8} \exp (- f_{n})]}^{- 1},

(16)

is used, where the design parameters $c_{7}$ and $c_{8}$ are constants. To provide the user with a better perception of the friction force, low amplitude vibrations are eliminated by [35]:

f_{f} = {\hat{f}}_{f} \tanh (\frac{v}{c_{9}}) t,

(17)

where $c_{9}$ is another design constant.

Using Equations (15) and (17), the friction force is governed by:

f_{f} = {\begin{matrix} 0, & v = 0 \\ a \dot{r}, & v > 0 \end{matrix}

(18)

where $a = \frac{1}{v} f_{n} \tanh (\frac{v}{c_{9}}) [μ_{c} + (μ_{s} - μ_{c}) \exp (- \frac{v^{2}}{v_{s}^{2}})] + f_{v} \tanh (\frac{v}{c_{9}})$ is positive real.

Remark 2.3: When the soft fingertip starts to move along a curve $r (t)$ over the deforming surface, sliding motion with $v (t) = \dot{r} (t)$ is resisted by friction in the tangential direction, $t = \dot{r} (t) / v (t) ⊥ n .$ Sliding velocity of the fingertip $\dot{r} = v t$ with $v > 0$ , is determined by:

\dot{r} = (I - S) {\dot{p}}_{r} .

(19)

2.5. Deformation distribution

Because a point force is idealized and a finite contact area always exists between the contacting bodies, an influence zone of the fingertip contact on the deforming kidney surface is assumed and used for distribution of the contact forces. As shown in Figure 4 (a) the influence zone is described with the distance $l_{0}$ from the contact point $p$ . Here, $p_{j}^{0}$ denotes the location of jth control point before deformation and $l_{j}$ is the distance from the contact point $p$ to the jth control point $p_{j}$ inside the influence zone. To distribute the fingertip contact forces throughout the control points (nodes) located inside the influence zone, a proper kernel function is needed. This function serves to smooth the force distribution on the surface and ensures that the distributed forces decay down to a negligibly small value as the distance increases away from the contact point.

Referring to Figure 4 (b), the force distributed at the jth control point is defined by:

f_{j} = - w_{j} (f_{n} + f_{f}),

(20)

where $w_{j}$ is a proper weighing function, which is chosen as:

w_{j} = {(1 - \frac{1 - \cos (l_{j} / l_{0})}{ε_{0}})}^{n},

(21)

where $ε_{0} = (1 - \cos (1))$ , the constants n and $l_{0}$ are dependent on several variables (such as the surface geometry, material and the force applied) and can be determined experimentally. With the force $f_{j}$ acting at jth control point, control point is relocated at:

p_{j} = p_{j}^{0} + h_{j},

(22)

where, with T as the sampling time, $h_{j}$ can be calculated by:

\begin{matrix} h_{j} (t) = h_{j} (t - T) + \frac{d h_{j} (t - T)}{d t} T, \\ {\dot{h}}_{j} (t) = \frac{f_{j} (t - T) - k_{2} h_{j} (t)}{b_{2}} . \end{matrix}

(23)

Figure 4.

Force distribution; (a) influence zone, (b) control point constitutive model

2.6. Haptic device controller

A 3-DoF parallel manipulator (Novint Falcon 3D touch device [36]) is used for testing the presented approach. The dynamics of the manipulator are given by:

H (q) \ddot{q} + C (q, \dot{q}) \dot{q} = τ - J_{r}^{T} f_{h},

(24)

where $q$ is the $3 \times 1$ joint angle vector, $J_{r}$ the Jacobian matrix, $f_{h}$ the end-effector force/torque and $τ$ the joint actuator torque vector. $H (q)$ represents the inertia matrix and $C (q, \dot{q}) \dot{q}$ the vector of Coriolis and centrifugal terms. Gravitational and frictional effects are independently compensated.

Property 2.2: The matrix $C (q, \dot{q})$ and the time derivative of the positive definite symmetric inertia matrix $H (q)$ satisfy [37]:

\dot{q} [0.5 \dot{H} (q) - C (q, \dot{q})] {\dot{q}}^{T} = 0, \forall q, \dot{q} \in ℝ^{3} .

(25)

2.6.1. Free-space motion

A simple proportional-derivative (PD) controller [38] is used during free-space motion. While $e = p_{r d} - p_{r}$ describes the position error, the control torque is computed from:

τ = J_{r}^{T} (k_{p} e - k_{d} {\dot{p}}_{r} + f_{h}),

(26)

where $k_{p}$ and $k_{d}$ are positive constants. Because forces are not measured, $f_{h} ≅ (k_{p} e - k_{d} {\dot{p}}_{r})$ is assumed and the desired end-effector position $p_{r d}$ is set slightly ahead of the current position in the headed direction, $p_{r d} = p_{r} + T {\dot{p}}_{r}$ .

Proposition 2.1: Consider the robot dynamic model (24) together with the control law (26), then the closed loop system is globally asymptotically stable.

Proof:

To prove the stability of this equation, $V (e, \dot{q}) = 0.5 {\dot{q}}^{T} H \dot{q}$ is chosen as the Lyapunov candidate function. Clearly, $V > 0$ except at the equilibrium ( $e = 0, \dot{q} = 0$ ). Using Equations (24), (25) and (26), ${\dot{p}}_{r} = J_{r} \dot{q}$ and $e = T {\dot{p}}_{r}$ , the time derivative of the Lyapunov function becomes:

\dot{V} = {\dot{p}}_{r}^{T} (k_{p} T - k_{d}) {\dot{p}}_{r},

(27)

which is locally negative semi-definite as long as $k_{d}$ is chosen greater than $k_{p} T$ . With $\dot{V} = 0$ or equivalently by ${\dot{p}}_{r} = \dot{q} = \ddot{q} = 0$ , Equations (24) and (26) yield $e = 0$ . Thus, we conclude that, by LaSalle's invariance principle, the system is asymptotically stable for $k_{d} > k_{p} T$ .

2.6.2. Contact motion

As the soft fingertip touches the virtual surface, a constraint based controller similar to that used in [24] takes over:

τ = J_{r}^{T} [- (S + ε I) J_{e}^{- 1} f_{n} + ε f_{n} + (S - I) f_{f}] .

(28)

Here, ε is a small positive real number, $f_{n}$ and $f_{f}$ are the normal and frictional force vectors computed by Equations (14) and (18). The projection matrix $S$ is as described in Remark 2.2 and $(S - I) f_{f}$ lies in the tangential direction.

Proposition 2.2: Consider the robot dynamic model (24) together with the control law (28) and assume that $f_{h} = ε f_{n}$ , then the closed loop system is globally asymptotically stable.

Proof:

Let us consider the following Lyapunov function candidate function:

\begin{matrix} \begin{matrix} V (e, \dot{q}) = 0.5 {\dot{q}}^{T} H \dot{q} + g (e), \end{matrix} \\ g (e) = \int^{​} {\dot{p}}_{r}^{T} (S + ε I) J_{e}^{- 1} k_{1} e d t . \end{matrix}

(29)

Noting that ${\dot{p}}_{r} d t = J_{e}^{- 1} d e$ and using Property 2.1, one may easily show that, for all $e \neq 0$ :

\begin{matrix} g (e) = k_{1} \int^{​} e^{T} J_{e}^{- T} (S + ε I) {\dot{p}}_{r} d t \\ = k_{1} \int^{​} e^{T} (J_{e}^{- T} S + ε J_{e}^{- T}) J_{e}^{- 1} d e \\ = k_{1} \int^{​} e^{T} (J_{e}^{- T} S J_{e}^{- 1} + ε J_{e}^{- T} J_{e}^{- 1}) d e \\ = k_{1} \int^{​} e^{T} (S + ε J_{e}^{- T} J_{e}^{- 1}) d e > 0 \end{matrix}

(30)

since $(S + ε J_{e}^{- T} J_{e}^{- 1}) > 0$ . Thus, equation (29) is a proper candidate for a Lyapunov function, i.e., $V > 0$ or all $e \neq 0$ and $V (0, 0) = 0$ . Using Equations (24), (25), (28) and ${\dot{p}}_{r} = J_{r} \dot{q}$ , the time derivative of the Lyapunov function becomes:

\dot{V} = - {\dot{p}}_{r}^{T} (S + ε I) b_{1} {\dot{p}}_{r} + {\dot{p}}_{r}^{T} (S - I) f_{f} .

(31)

Because equation (19) indicates that $f_{f} = 0$ for $v = 0$ , it is necessary to see if equation (29) is negative semi definite for $\forall v > 0$ . For this case, using Equations (18) and (19), equation (31) becomes:

\begin{matrix} \dot{V} = - {\dot{p}}_{r}^{T} (S + ε I) b_{1} {\dot{p}}_{r} + a {\dot{p}}_{r}^{T} (S - I) \dot{r} \\ = - {\dot{p}}_{r}^{T} (S + ε I) b_{1} {\dot{p}}_{r} + a {\dot{p}}_{r}^{T} (S - I) (I - S) {\dot{p}}_{r} \\ = - {\dot{p}}_{r}^{T} (S + ε I) b_{1} {\dot{p}}_{r} + a {\dot{p}}_{r}^{T} (S - I) {\dot{p}}_{r} \\ \leq - (ε b_{1} + a) {\dot{p}}_{r}^{2} . \end{matrix}

(32)

This result and the fact that $(ε b_{1} + a)$ is always positive implies that the equilibrium point $(q = 0, \dot{q} = 0)$ is locally stable in the sense of Lyapunov. To proceed further, more analysis is required in proving the asymptotic stability by the LaSalle's invariant set theorem. To demonstrate that the only invariant set contained in the set $V = 0$ is the equilibrium, note first that $\dot{V} = 0$ implies the case where ${\dot{p}}_{r} = \dot{e} = \dot{r} = \dot{q} = \ddot{q} = 0$ . It then follows from Equations (24) and (28) that this is the case when:

- (S + ε I) J_{e}^{- 1} f_{n} + (S - I) f_{f} = 0.

(33)

Finally, Equations (14) and (18) together with (19) are substituted into this equation and $e = 0$ is obtained. This concludes our proof.

3. Experimental Results

Figure 5 (a) shows a kidney surface mesh model used in implementing our approach. This choice is partly due to the fact that kidney tissue behaves in a viscoelastic manner and is thus a proper choice for the proposed algorithm. The model used is described by equation (1) and consists of non-uniform rational basis spline (NURBS) [23] patches of degree 3 (created by 3D raw data approximation). Using the model in Figure 5 (a), a singularity-free smooth kidney surface mesh [39] is then generated via Lagrange interpolation, as seen in Figure 5 (b). Figure 5 (c) shows the motion trajectory of the fingertip bone edge (haptic device gripper) at various instances of time.

Figure 5.

Kidney surface mesh and the soft fingertip motion trajectory; (a) surface mesh model; (b) Lagrange interpolated surface mesh; (c) motion trajectory

A Novint Falcon robotic manipulator is chosen as the experimental test-bed (depicted in Figure 6). This particular low-cost commercial force-feedback device is a 3DoF parallel mechanism with a gripper at the end effector. It has around 1 kHz update rate, 60 µm position resolution and 9 N continuous force display capability. Due to its limited resolutions, the device is not ideal for display and haptic perception of small forces. In running this device, a component-based open-source haptic API (Haptik Library [40]) under MATLAB software is used on a Windows 8 run computer (Intel 3.2 GHz Core i5 CPU, 4 GB Ram). A MATLAB script reads the Novint Falcon encoder signals and calculates the controller output signals. These signals are subsequently sent to the device to provide force feedback to the user. Noting that the haptic touch perception threshold is in the range of 10–400 Hz and human fingers cannot discriminate two consecutive force input signals beyond 320 Hz [41, 42], an update rate of 1 kHz [8] or at least twice the perception bandwidth is required (by the Nyquist-Shannon sampling theorem). Experimental data collected in this study show that the update rates registered are in 878–1265 Hz range and the mean is at 1025 Hz.

Figure 6.

Experimental-set up

In minimizing the adverse effects of low velocity resolutions, an eight-point moving average filter is used for high frequency noise reduction and signal smoothing. This decision is based on our empirical trials and the fact that moving average filters [43] are simple to implement and computationally inexpensive. Experimental data show that the cut-off (3 dB attenuation) frequency of this low-pass filter remain in the 110–160 Hz range, well above the 5–10 Hz force exertion and the 20–30 Hz kinaesthetic sensing bandwidths of hands and fingers [41].

Experiments are carried out after careful assignment of system constants. By noting that tissue thickness from the fingertip bone edge to the initial contact point is between 3 and 5 mm [44], $e_{c} = 3$ mm is set. Next, $c_{0} = 39.3, c_{1} = 2100$ , $c_{2} = 1$ , $c_{3} = 3360$ , $c_{4} = - 3.6$ , $c_{5} = 45$ , $c_{6} = 235$ are determined by nonlinear regression analysis of the experimental data in [29, 30]. With these values, Equations (8) and (9) present an accurate mathematical fit to the available data. Because no data in the literature are available for characterizing the interaction behaviour between a human fingertip and a kidney tissue, the remaining constants are selected so as to achieve a realistic haptic rendering experience. By realizing an acceptable resemblance to the case of a robotic finger sliding on a mouse pad [45], Coulomb and stiction coefficients are set to $μ_{c} = 0.30$ and $μ_{s} = 0.35$ . Then, $c_{7} = 0.02$ , $c_{8} = 10$ and $c_{9} = 0.06$ are assigned for a chatter-free frictional behaviour. Finally, $b_{2} = 0.5 \sqrt{k_{2}}, f_{v} = 0.2 Ns / m$ are selected for the viscous damping coefficients, and $ε = 0.0001$ is used in equation (28).

The motion starts in free space and the user pushes the device gripper (soft fingertip) to initiate contact with the kidney surface. Although real-time discontinuous switching from the PD controller to the constraint based controller have the potential to cause undesirable effects, no plausible contact chattering was observed in our implementations. With the PD gains set at $k_{p} = 500$ and $k_{d} = 12$ , both controllers have produced small and nearly equal torques. Bilateral controller transitions were thus smooth. Undesirable transients (discontinuities) were noticeable only at high impact velocities of the fingertip with the kidney. In our tests, nominal values of the fingertip speeds were in the 0–0.05 m/s range. Small control-chattering behaviour was witnessed only with a really fast fingertip hit against the kidney tissue.

In Figure 7 (a), time-varying deformations $s_{1}$ and $s_{2}$ of the soft fingertip tissue and the kidney surface are presented. The data indicate that the fingertip contact with the kidney surface is initiated at around $t = 0.13$ sec of the experiment. Before this instant, $s_{1} < 0, s_{2} = 0$ and the fingertip is in approaching motion to the deformable surface. Following the simultaneous deformations of the fingertip and the kidney surface, the fingertip is pulled away from the surface at about 2.77 sec of the experiment. Note that while $s_{1} < 0$ , it takes a short time for the kidney tissue deformation to nearly die out. At about $t = 3.07$ sec, the fingertip reinitiates contact with the kidney surface and joint deformations take place. Next, following another no-contact region between $t = 4.71$ sec and 5.3 sec of the experiment, the user fingertip moves back again to contact with the deforming kidney surface. Though a low pass smoothing filter was adopted to minimize the effects of poor resolutions, low amplitude vibrations could not be totally eliminated. These vibrations are clearly visible in Figure 7 (b), where the time history for $e = s_{1} - s_{2}$ is plotted.

Figure 7.

Deflections; (a) s₁ for the soft fingertip and s₂ for the kidney tissue; (b) (s₁-s₂)

Figure 8 illustrates the kidney tissue deformation distribution at various instances of time. The colour bar to the right indicates the colour legend for deformation values between the maximum (red) 10 mm and minimum (blue) zero. Deformations observed experimentally around the contact point are the result of the nonlinear constitutive relations with an influence zone-based distribution of the normal and the frictional contact forces. The deforming tissue, as a result, looks and feels realistic.

Figure 8.

Kidney tissue surface deformations at various instants

The general nature of the normal force data is nicely illustrated in Figure 9 (a) for the indenter (soft fingertip) and in Figure 9 (b) for the indented material (kidney tissue surface). Note in this figure that experimental data shows a reasonably favourable agreement with the constitutive model predictions (see Figure 2). For small indentations, Figure 9 (a) shows that the user feels the effects of a contact force discrimination threshold. Whereas at increasing indentations, the experimental data agree better quantitatively with the constitutive model. The loading-unloading cycle (hysteresis) in Figure 9 (b) is a clear indication of the behaviour of typical viscoelastic materials. The fingertip tissue model is shown to exhibit substantial hysteretic losses under local cyclic loading and become stiffer with increasing displacement.

Figure 9.

Normal force data; (a) f_n vs. e; (b) f_n vs. s₂

Contact forces and the tangential (sliding) velocity data are shown in Figure 10. It should be clear in Figure 10 (a) that the normal and the friction forces f_n and f_f are much larger than the force component perpendicular to the friction force f_b. The tangential force consists mainly of the LuGre frictional forces and the force f_b is not zero probably due to limited resolution specifications of the haptic device, uncompensated dynamics and sensory noise-induced effects. Realizing that $e = e_{c} - e_{n}$ is a measure of the normal force, the normal force variations in Figure 10 (a) show correlation with the data in Figure 7 (b).

Figure 10 illustrates how the friction force varies proportional to the normal force and the sliding speed. In analysing the data, rectangular sub-regions A and B are chosen and zoomed-in views are shown in Figure 10 (a) and 10 (b). When Regions A are examined, one sees that the fingertip sliding speed on the kidney surface is nearly constant and that the friction force varies proportional to the normal force. In Regions B, as the normal force is almost steady, there is an increase in the friction force as the soft fingertip sliding speed is raised. These observations present a matching trend with the LuGre friction model. Achieving better results is obviously dependent on acquiring a better haptic interface device. More complex interactive deformation simulations with more advanced dynamic friction models may then be possible.

Figure 10.

Contact forces and the tangential velocity data (a) normal and frictional forces, f_n and f_f; (b) sliding velocity of the virtual finger on the surface

4. Conclusions and Future Work

In this paper we propose and demonstrate the use of a soft-fingertip model in interactive haptic rendering of deformable surfaces. In developing a realistic haptic display environment, an improved physically based nonlinear three-dimensional constitutive model is developed for studying frictional contact and deformations of the soft-fingertip with mesh model of viscoelastic surfaces. For an accurate characterization of the friction force, a simplified form of the LuGre dynamic friction model is adopted where the tangential force is a function of both the sliding velocity and the normal force.

A novel type switching control scheme is developed for testing the proposed approach. The control strategy serves as a general framework for stable haptic rendering of frictional contact between the soft fingertip and the deformable surface. In the unconstrained space, a simple PD type controller allows the user to move the soft fingertip freely before it is brought into frictional contact with the surface. For the constrained motion, a closest point-based algorithm is employed together with a new type nonlinear deformation model. The deformation model accounts for both geometric and viscoelastic material nonlinearities and the contact forces are distributed locally around the contact location by incorporating an influence zone. Deformations with an area larger than a point are thus achieved by the soft fingertip moving on the viscoelastic discrete surface. A constraint-based controller is then employed to provide the user with a fast and accurate kinaesthetic stimulus as both the finger and the surface deform. The novelty of our approach lies not only in the design of the switching type control scheme but also in the fact that it could easily be adopted for use in other contact and deformation models.

A 3DoF parallel mechanism is used for implementing and validating the proposed approach. The soft fingertip interaction with a kidney tissue surface mesh is studied experimentally and the proposed analytical approach is validated. The results present accurate (realistic) interactions between the soft fingertip and the biological tissue. A fast, smooth and compliant haptic perception is achieved.

The dynamic set up presented points to a direction of research where more sophisticated compliant contact and deformation models are possible. Hence, the focus of our future research is concerned with developing an experimental platform for studying tactile pressure mapping and stress-strain distributions in biological tissues. The data collected will be used in developing more realistic frictional compliant contact models for use in haptic simulations and subjective evaluation tests. The contact model will be derived using a time- and rate-dependent modification of Hertz theory to capture the effects of both geometrical and material nonlinearities (such as the variations in the non-planar contact surface and the modulus of elasticity). In the process, a non-linear energy-minimization approach with a penalty based finite element formulation may be used, subject to volumetric constraints.

Footnotes

Nomenclature

References

Bro-Nielsen

Finite element modeling in surgery simulation. Proceedings of the IEEE. 1998;86 (3): 490–503. DOI: 10.1109/5.662874.

Downes

Goktekin

Tendick

Adaptive nonlinear finite elements for deformable body simulation using dynamic progressive meshes. Comput Graphics Forum. 2001;20 (3):349–358. DOI: 10.1111/1467–8659.00527.

James

Fatahalian

Precomputing interactive dynamic deformable scenes. ACM Transactions on Graphics. 2003;22 (3):879–887. DOI: 10.1145/882262.882359.

Peterlik

Matyska

Filipovic

Haptic interaction with complex models based on precomputations. In: Zadeh

, editor. Advances in Haptics, InTech; 2010. p. 333–356. DOI: 10.5772/8699.

Payandeh

Azouz

Finite elements, mass-spring-damper systems and haptic rendering. In: Proceedings 2001 IEEE International Symposium on Computational Intelligence in Robotics and Automation; Alberta (Canada). 2001. p. 224–229. DOI: 10.1109/CIRA.2001.1013201.

Mahvash

A novel approach for modeling separation forces between deformable bodies. IEEE Transactions on Information Technology in Biomedicine. 2006;10 (3):618–626. DOI: 10.1109/TITB. 2006.872040.

Wang

Devarajan

Improved 2D mass-spring-damper model with unstructured triangular meshes. The Visual Computer. 2008;24 (1):57–75. DOI: 10.1007/s00371-007-0179-7.

Bolanowski

Gescheider

Verrillo

Checkosky

. Four channels mediate the mechanical aspects of touch. Journal of the Acoustical Society of America. 1988;84 (5):1680–1694. DOI: 10.1121/1.397184.

Adams

Hannaford

Control law design for haptic interfaces to virtual reality. IEEE Transactions on Control Systems Technology. 2002;10 (1): 3–13. DOI: 10.1109/87.974333.

10.

Puangmali

Althoefer

Seneviratne

Murphy

Dasgupta

. State-of-the-art in force and tactile sensing for minimally invasive surgery. IEEE Sensors Journal. 2008;8 (4):371–380. DOI: 10.1109/JSEN.2008.917481.

11.

d'Aulignac

Balaniuk

Laugier

A haptic interface for a virtual exam of the human thigh In: Proceedings IEEE International Conference on Robotics and Automation; San Fransisco (ICRA'00); CA (USA). 2000. Vol. 3, p. 2452–2457. DOI: 10.1109/ROBOT.2000.846395.

12.

Kim

Park

Kwon

. Real-time haptic rendering of a high-resolution volumetric deformable object in a collaborative virtual environment. Advanced Robotics. 2005;19 (9):951–975. DOI: 10.1163/156855305774307022.

13.

del-Castillo

Basañez

Gil

Modeling nonlinear viscoelastic behavior under large deformations. International Journal of Non-Linear Mechanics. 2013;57:154–162. DOI: 10.1016/j.ijnonlinmec.2013.07.001.

14.

Pourhosseini

Azimirad

Kazemi

A new fast nonlinear modeling of soft tissue for surgical simulation. Journal of Robotic Surgery. 2014;8 (2): 141–148. DOI: 10.1007/s11701-013-0444-x.

15.

James

Pai

. A unified treatment of elastostatic contact simulation for real time haptics. Haptics-e. 2001;2 (1):1–13.

16.

Luo

Xiao

Contact and deformation modeling for interactive environments. IEEE Transactions on Robotics. 2007;23 (3):416–430. DOI: 10.1109/TRO. 2007.895058.

17.

Dogan

Celebi

. Real-time deformation simulation of non-linear viscoelastic soft tissues. SIMULATION: Transactions of The Society for Modeling and Simulation International. 2011;87 (3):179–187. DOI: 10.1177/0037549710364532.

18.

Serina

Mote

Rempel

Force response of the fingertip pulp to repeated compression: Effects of loading rate, loading angle and anthropometry. Journal of Biomechanics. 1997;30 (10):1035–1040. DOI: 10.1016/S0021–9290 (97)00065–1.

19.

Perez

Cirio

Hernandez

Garre

Otaduy

. Strain limiting for soft finger contact simulation In: IEEE World Haptics Conference; Daejeon (Korea). 2013. p. 79–84. DOI: 10.1109/WHC. 2013.6548388.

20.

De Wit

Olsson

Astrom

Lischinsky

A new model for control of systems with friction. IEEE Transactions on Automatic Control. 1995;40 (3): 419–425. DOI: 10.1109/9.376053.

21.

Song

Liu

Althoefer

Nanayakkara

Seneviratne

. Efficient break-away friction ratio and slip prediction based on haptic surface exploration. IEEE Transactions on Robotics. 2014;30 (1):203–219. DOI: 10.1109/TRO. 2013.2279630.

22.

Armstrong-Helouvry

Dupont

De Wit

. A survey of models, analysis tools and compesation methods for the control of machines with friction. Automatica. 1994;30 (7):1083–1138. DOI: 10.1016/0005–1098 (94)90209–7.

23.

Salomon

Curves and Surfaces for Computer Graphics. Springer Science & Business Media, 2007.

24.

Zafer

Constraint-based haptic rendering of a parametric surface. Journal of Systems and Control Engineering. 2007;221 (3):507–517. DOI: 10.1243/09596518JSCE297.

25.

Popov

. Contact Mechanics and Friction. Berlin: Springer Science & Business Media, 2010. p. 60–61.

26.

Akihito

Tsuji

Kurita

Fingertip force estimation based on the deformation of the fingertip In: Haptic Interaction (Lecture Notes in Electrical Engineering, vol. 277). New York, Springer. 2015. p. 201–205.

27.

Tomlinson

Lewis

Carre

. The effect of normal force and roughness on friction in human finger contact, Wear. 2009;267 (5–8): 1311–1318. DOI: 10.1016/j.wear.2008.12.084.

28.

Hunt

Crossley

. Coefficient of restitution interpreted as damping in vibroimpact. Journal of Applied Mechanics. 1975;42 (2):440–445. DOI: 10.1115/1.3423596.

29.

Pawluk

Howe

. Dynamic lumped element response of the human fingerpad. Journal of Biomechanical Engineering. 1999;121 (2):178–183. DOI: 10.1115/1.2835100.

30.

Liu

Song

Seneviratne

Althoefer

Rolling indentation probe for tissue abnormality identification during minimally invasive surgery. IEEE Transactions on Robotics. 2011;27 (3):450–460. DOI: 10.1109/TRO.2011.2127210.

31.

Hirai

Toward a platform of human-like fingertip model in haptic environment for studying sliding tactile mechanism In: Robotics: Science and Systems (RSS IX); Berlin (Germany). 2013.

32.

Swevers

Al-Bender

Ganseman

Prajogo

An integrated friction model structure with improved presliding behaviour for accurate friction compensation. IEEE Transactions on Automatic Control. 2000;45 (4):675–686. DOI: 10.1109/9.847103.

33.

Lampaert

Al-Bender

Swevers

A generalized Maxwell-slip friction model appropriate for control purposes In: Proceedings IEEE International Conference on Physics and Control; Saint- Petersburg (Russia). 2003. Vol. 4, p. 1170–1177. DOI: 10.1109/PHYCON.2003.1237071.

34.

Hayward

Armstrong

Altpeter

Dupont

. Discrete-time elasto-plastic friction estimation. IEEE Transactions on Control Systems Technology. 2009;17 (3):688–696. DOI: 10.1109/TCST. 2008.2001710.

35.

Specker

Buchholz

Dietmayer

A new approach of dynamic friction modelling for simulation and observation In: Proceedings of the 19th World Congress of The International Federation of Automatic Control (IFAC); Cape Town (South Africa). 2014. p. 4523–4528.

36.

Samur

Performance metrics for haptic interfaces In: Springer Series on Touch and Haptic Systems, London: Springer Science & Business Media; 2012. p. 11–15.

37.

Sciavicco

Siciliano

Modeling and Control of Robot Manipulators. Springer Science & Business Media, 2012.

38.

Zafer

Yilmaz

Haptic exploration of highly curved compliant surfaces. Journal of Systems and Control Engineering. 2015;229 (10):949–956. DOI: 10.1177/0959651815598257.

39.

Mann

Adjusting control points to achieve continuity. Computer Aided Geometric Design. 2002;19 (7):589–602. DOI: 10.1016/S0167–8396 (02)00147–4.

40.

De Pascale

Prattichizzo

The Haptik Library. IEEE Robotics & Automation Magazine. 2007;14 (4): 64–75. DOI: 10.1109/M-RA.2007.905747.

41.

Shimoga

. A survey of perceptual feedback issues in dexterous telemanipulation: Part I. Finger force feedback In: Proceedings of the IEEE Virtual Reality Annual International Symposium; Los Angeles (USA). 2003. p. 263–270. DOI: 10.1109/VRAIS.1993.380770.

42.

Hoggan

Brewster

Nonspeech auditory and crossmodal output. In: The Human-Computer Interaction Handbook - Fundamentals, Evolving Technologies and Emerging Applications (Ed. Jacko

). 3rd Edition. CRC Press. 2012. p. 211–236.

43.

Smith

. Digital Signal Processing: A Practical Guide for Engineers and Scientists. 3rd Edition. Burlington MA. Newnes. 2002. p. 277–284.

44.

Dandekar

Raju

Srinivasan

. 3-D finite-element models of human and monkey fingertips to investigate the mechanics of tactile sense. Journal of Biomechanical Engineering. 2003;125 (5):682–691. DOI: 10.1115/1.1613673.

45.

Song

Liu

Bimbo

Althoefer

Seneviratne

. Object surface classificaiton based on friction properties for intelligent robotic hands In: IEEE World Automation Congress; Puerto Vallarta. 2012. p. 1–5.

Frictional Compliant Haptic Contact and Deformation of Soft Objects

Abstract

Keywords

1. Introduction

2. Haptic Display System

2.1. Surface model

2.2. Closest point

2.3. Contact force model

2.4. Dynamic friction model

2.5. Deformation distribution

2.6. Haptic device controller

2.6.1. Free-space motion

2.6.2. Contact motion

3. Experimental Results

4. Conclusions and Future Work

Footnotes

Nomenclature

References