Sage Journals: Discover world-class research

Abstract

Replication of the human sense of touch would be highly advantageous for robots or prostheses as it would allow an agile and dexterous interaction with the environment. The article presents an approach for the integration of a micro-electromechanical system sensing skin with 144 tactile sensors on a soft, human-sized artificial fingertip. The sensing technology consists of thin, 1D sensing strips which are wrapped around the soft and curved fingertip. The sensing strips include 0.5 mm diameter capacitive sensors which measure touch, vibrations, and strain at a resolution of 1 sensor/mm². The method allows to leverage the advantages of sensing skins over other tactile sensing technologies while showing a solution to integrate such skins on a soft three-dimensional body. The adaptable sensing characteristics are dominated by the thickness of a spray coated silicone layer, encapsulating the sensors in a sturdy material. We characterized the static and dynamic sensing capabilities of the encapsulated taxels up to skin thicknesses of 600 μm. Taxels with 600 μm skin layers have a sensitivity of 6 fF/mN, corresponding to an ∼5 times higher sensitivity than a human finger if combined with the developed electronics. They can detect vibrations in the full tested range of 0–600 Hz. The softness of a human finger was measured to build an artificial sensing finger of similar conformity. Miniaturized readout electronics allow the readout of the full finger with 220 Hz, which enables the observation of touch and slipping events on the artificial finger, as well as the estimation of the contact force. Slipping events can be observed as vibrations registered by single sensors, whereas the contact force can be extracted by averaging sensor array readouts. We verified the sturdiness of the sensing technology by testing single coated sensors on a chip, as well as the completely integrated sensing fingertip by applying 15 N for 10,000 times. Qualitative datasets show the response of the fingertip to the touch of various objects. The focus of this article is the development of the sensing hardware and the basic characterization of the sensing performance.

Introduction

The tactile sense gives humans remarkable capabilities to explore objects or interact with them. Replicating those sensing capabilities could improve the dexterity of robots while interacting with different objects^1,2 or give prostheses an enhanced sense of touch.³ The human tactile receptors and their density depend strongly on the location on our skin.⁴ The skin in the fingertip comprises of four different tactile cells, which are capable of sensing static and dynamic touch in a range of 1–10 kPa (Meissner and Merkel endings), detect strain in the skin (Ruffini endings), as well as perceive vibrations of up to 500 Hz (Pacini endings). The density of touch sensing cells is the highest with about 1 sensor/mm², followed by a lower density of strain and vibration sensing cells.^4,5 Other important features of a human fingertip are its softness enabling conformity, as well as the interaction with delicate objects, its relatively small dimensions, as well as its sturdiness to mechanical damage. Table 1 summarizes the criteria an artificial fingertip would need to fulfil to replicate those capabilities.

Table 1.

Overview on Specifications to Build an Artificial Fingertip with Human Like Capabilities, Sensing Information Based on Refs.^4,5,13,2⁷

Criterium	Range/goal	Additional information	Camera-based systems			Magnetic flux	Sensing skins
Criterium	Range/goal	Additional information	BioTac ⁶	DIGIT ⁷	Sun et al. ¹⁰	Tomo et al. ¹⁸	Oh et al. ¹²	Boutry et al. ¹³	Chun et al. ¹⁶	This work
Touch sensing	1–10 kPa		No specific data	→	0.03 N error for F_Z, detection limit not specified	No specific data	Tested	→	Feasible, clear metrics missing	Tested, adjustable through skin thickness
Vibration sensing	0–500 Hz		Possible with one vibration sensor	Not reported, deformation possibly too small to be measured		Not investigated	→	→	Possible through triboelectric layer	Tested
Shear/strain sensing		Not quantified	Shear sensing possible	No data available	Shear sensing possible, 0.03 N estimation error	Shear sensing possible	→	→	No shear sensing	Shear sensing qualitatively tested
Spatial resolution sensors	1 sensor/mm²	for touch sensing, lower for vibration/strain	Lower, 19 electrodes and 1 vibration sensor	High resolution possible for touch sensing.		Lower, 24 sensors on fingertip	Tested down to 0.1 mm taxel size	Lower, sensor size 1.5 mm	Possible, sensor size 1 mm	1 sensor/mm² tested for networks of 144 taxels
Size and shape fingertip and its sensing area	Human like	Approximate values:L = 25 mm, W = 15 mm, H = 10 mm	Size of two human digits, sensing area limited to silicone bubble.	Planar surface for measurement, fingertips rather bulky.	Finger-like shape, larger than human thumb. Full area can be observed.	Discrete sensors integrated in larger fingertip	No integration tested	→	→	Integrated on 3D curved area of human-sized fingertip
Softness	Human like	Compare data in Figure 3.	Similar softness probable	Gel is soft, no data available	Finger with soft and stiff regions, no data available.	Integrated on soft finger, stiffness unknown.	Not tested	→	→	Integrated on configurable human-like fingertip
Robustness	Resistant to wear	No standards	No metrics reported	→	→	→	400,000 interactions reported, no details	30,000 iterations with max 70 kPa applied	No metrics reported	Tested for 10,000 iterations with 15 N, some degradation observable
Electrical interface	Available, fast, small	Allows sensing capabilities	Available	→	→	Interface existing	Matrix readout presented	Not investigated	Interface existing, matrix readout unclear.	Matrix readout presented with 220 Hz sampling frequency
Additional comments			Electrolyte filled bubble prone to drifting, costly	Commercially available	Internal grid probably results in measurement irregularities	Possibly influenced by ferromagnetic materials or EM fields

EM, electromagnetic.

Various technical solutions have been developed to allow an artificial sense of touch, either in an integrated form as a fingertip sensor or as sensing skins. Table 1 compares a selection of prominent examples. Some reached market-readiness like the BioTac finger developed by SynTouch,⁶ which includes an array of 19 electrodes for touch measurement, as well as one pressure sensor for vibration measurement. Camera-based sensing systems are studied in several approaches like DIGIT,⁷ GelSight,⁸ by Sferrazza et al.⁹ or a thumb sized model proposed by Sun et al.¹⁰ Several groups are working on advanced sensing skins with higher spatial resolution, more sensing capabilities, or higher sensitivities^5,11; a small overview is listed here. Oh et al.¹² used ZnO based piezoelectric thin film transistor arrays for force and shear force measurements, while Boutry et al.¹³ addressed this task with a grid of carbon nanotube based electrodes. Lee et al.¹⁴ built arrays of direction-sensitive tactile sensors using four electrodes below a polydimethylsiloxane bump, while Liang et al.¹⁵ used a similar approach with an interposer layer consisting of truncated pyramids for higher sensitivity. Chun et al.¹⁶ combined two sensing layers (resistive graphene sensing array and triboelectric sensing layer) for touch and vibration measurements with high spatial resolution. Further work presents the integration of sensors in a fingertip like Schmitz et al.¹⁷ who covered a finger with 12 capacitive sensors or Tomo et al.¹⁸ who used 24 commercial magnetometer chips on a fingertip.

The existing solutions have some drawbacks, which are summarized as follows: (1) the BioTac finger⁶ lacks mainly spatial resolution, (2) camera-based sensing systems^7–10 are limited in vibration detection (very small deformations not measurable, and high frame rates needed) and the minimum size of the sensing system, and (3) sensing skins^12–16 were not yet shown to be integrated in a finger-like, soft, and curved 3D object or characterized thereafter. The overall advantage of sensing skins is the better adaptivity to small bodies due to their small form factor and their good, reported sensing capabilities.

The scope of this work is to investigate the integration of a sensing skin in an artificial fingertip, which can cover all criteria presented in Table 1. We recently presented an approach on how to integrate dense arrays of capacitive sensors directly on a flexible substrate which is structured in thin sensing fibers¹⁹ and showed a mechanical and electromechanical characterization of those bare tactile sensors (taxels). The thin flexible fibers allow an easier integration into a curved 3D object than previously reported sensing skins. In Weichart et al.²⁰ we proved that the sensing array was free of cross-talk between sensors, as well as the 3D sensing capabilities of the sensors. The developed taxels were however only characterized on-chip, fragile and unpackaged, not integrated in a 3D object and only tested with very small forces of 30 mN. This work presents an approach for the packaging of fragile micro-electromechanical system (MEMS) structures with a novel silicone spray coating process and the characterization of the influence of such a protection layer. We investigate the transfer of the sensing arrays from the silicon carrier chip onto an artificial fingertip with human-like shape and softness and present an approach on how to readout such a sensing array in an efficient manner, as well as a series of experiments performed on the artificial fingertip. Those experiments investigate the highly sensitive response of the artificial fingertip to touch, vibration, and shear events, as well as the long-term stability to moderate forces of 15 N.

The main contribution of this work focuses on sensor technology development and the characterization of the sensing fingers with simplified stimuli like a needle probe and a rectangular stamp. We do not focus on evaluating the sensing finger response to various objects and how this might induce mechanical cross talk between sensors. We expect that such further studies would be more efficiently implemented through learning algorithms, which is potential future work.

Concept

A concept drawing of the proposed artificial fingertip is shown in Figure 1A, which loosely resembles the structure of a human fingertip.²¹ It consists of five major components (in brackets—human equivalent) as follows: (1) the bone (finger bone), (2) the very soft filler material of the pulp (fatty tissue), (3) the sturdy bulk being considerably stiffer than the pulp (dermis), (4) the sensing array with the potentially integrated circuits (tactile nerves and cells), as well as (5) the skin layer for protecting the sensors with similar stiffness as the bulk (epidermis). The main ideas behind this approach are as follows: (1) the usage of 1D fibers allows an easy integration of taxels in a 3D body. The fibers in this work are a simple array of straight fibers, but arbitrary shapes or splitting fibers are possible to cover 3D bodies more efficiently. (2) The sensing fibers are placed in a sturdy outer layer of silicone which surrounds the soft inner pulp. Hereby a soft finger can be built using a thin, robust silicone layer which achieves softness through its low thickness, resembling the sturdy dermis skin layer in a human fingertip. This is advantageous over a homogenous body, which would need to consist of much softer material (with often worse material properties) to achieve similar mechanical characteristics.

FIG. 1.

Overview concept. (A) Conceptual drawing of the artificial finger with its major components. (B) The fabricated sensing arrays on a 30 × 30 mm silicon chip. (C) A tactile sensor (taxel) with its main structural elements, adapted from Weichart et al.¹⁹

Weichart et al.¹⁹ presented the basics of the sensing technology, and an adapted image of a tactile sensor (taxel) is shown in Figure 1C; the architecture of the sensing arrays was however modified as shown in Figure 1B. The taxel basically resembles a parallel plate capacitor with a static electrode as the first capacitor plate and the deformable metallic membrane as the second capacitor plate. The gap distance changes if the membrane is deformed due to application of mechanical forces, resulting in a change of capacitance.

One electrode (1E) taxels allow the measurement of z-axis static and dynamic forces, while three electrodes (3E) additionally allow a shear direction estimation in x and y direction as differential signals between the electrodes include information about the tilting direction of the suspended membrane. The calculation of the compression and shear components is done using a matrix multiplication introduced in Supplementary 1 in Supplementary Data S1. Such taxels, however, need three times as many electrodes, reducing the overall amount of taxels possible in the array. The two types of taxels can be combined. All sensor networks have 12 × 12 electrodes, which were used to build two types of sensing arrays presented in Figure 1B: (1) device left (DL): 12 × 12 sensors with only 1E sensors. (2) Device right (DR): 12 × 6 sensors with alternating 1E and 3E sensors (an average of two electrodes per sensor results in half the number of total sensors).

Results

Skin layer process and static characterization

We chose silicone spray coating as the sensor packaging process due to its capabilities of covering the fragile MEMS by sequentially building up a sturdy, self-stabilizing layer. A fine spray mist needs to be achieved to avoid the collapse of the membranes during the coating process. Additional advantages were the conformal coating capability of 3D objects, as well as good average thickness control. We could achieve closed and conformal layers down to 5 ± 0.5 μm thickness. The sprayed skin consisted of Nusil MED-4905 silicone diluted with n-Hexane to sufficiently reduce the viscosity. This enabled spraying using a modified 3D printer with an attached air-atomizing spray nozzle. Details are described in Supplementary 2. Nusil MED-4905 was used due to its resilience to brittle fracture and less drift with aging compared to more widely used materials like Sylgard 184.^22,23

The sensor arrays were characterized in their static and dynamic behavior with the characterization tool presented in Figure 2A while being kept on the silicon carrier chip (compare Fig. 1B). The static characterization was done by compressing the taxel with the z-axis stage while measuring the capacitance change in the taxel and the applied force with a force probe (range 0–100 mN). We used six sensing arrays (3 × DL, 3 × DR) from three chips.

FIG. 2.

Characterization of the taxels embedded in a soft silicone skin. (A) Testing setup with a close-up of the measurement area and its characterization probes. (B1–4) Static characterization of the taxel sensitivity with a varying skin thickness. (C1–2) Dynamic characterization of the taxels, showing their transfer characteristics in dependence of frequency and silicone layer thickness. (D1–2) Taxel hysteresis and endurance test for two taxels with a 220 μm thick silicone coating.

The taxels have a high sensitivity region (R1) for small forces and an extended low sensitivity region (R2) for larger forces as shown in Figure 2B1.¹⁹ The goal of this study was to investigate the influence of the skin thickness on R1 and R2 by iteratively measuring the force-capacitance (FC) characteristics and spraying another layer of silicone on the sensor arrays. Figure 2B2 shows the averaged curves of chip 3 (1 × DL, 1 × DR) of all working taxels for an increasing skin thickness. The averaging of all curves results in the artifact of a more rounded transition between R1 and R2. Figure 2B3 and B4 shows the extracted sensitivities (fF/mN) of R1 and R2 for all three chips; error bars are shown representatively for the two arrays of chip 3 (black). Figure 2B3 depicts a decline of R1, as the increasing skin thickness influences the deformation of the thin metal membrane in the taxel. The decline for R2 is less prominent, as this region corresponds to small deformations in the polymer studs (see Weichart et al.¹⁹), which are less dominated by deformations in the upper silicone layer.

Dynamic taxel characterization

We characterized the same sensing arrays after each iteration of silicone application for their dynamic sensing capabilities by applying a forced vibration with a piezo actuator added to the z-axis of the characterization setup (Fig. 2A). Four 1E taxels per array were excited between 1 and 600 Hz, and three measurement iterations per taxel were performed. The taxel below the probe needle was read out during the vibration, the capacitance converted to a displacement, and the resulting sinusoidal movement analyzed with a Fourier transform to extract the measured amplitude $A_{m e a s}$ at the excitation frequency. The procedure is explained in more detail in Supplementary 3.

Figure 2C1 shows the average amplitude $A_{m e a s}$ versus the applied amplitude $A_{a p p l i e d}$ for an increasing silicone thickness for the taxels of chip 1—DR. The piezo actuator, its controller, and the setup define the transmission curve for the uncoated taxel, as the taxel can completely follow the forced vibration. This was verified with measurements in a laser doppler vibrometer (LDV): (1) the transmission curve of the piezo actuator was comparable to the one of the uncoated taxel. (2) The first eigenfrequency of the uncoated taxels was between 19.3 ± 2.9 kHz for 12 measured taxels with a theoretical eigenfrequency of $f = \sqrt{\frac{k}{4 π^{2} m}} = 17.7 k H z .$

Considering the large spectral range between the first eigenfrequency and the observed range of 1–600 Hz, a flat transmission curve is expected. Some observations from Figure 2C1 are as follows: (1) the transmission of the vibration is damped by applying a thicker silicone layer; thus thicker silicone layers reduce the sensitivity. (2) The transmission curve with a linear decline observed for the uncoated taxels shifts toward a bent transmission curve for coated taxels with a maximum transmission at around 200 Hz. Lower frequencies are therefore damped slightly more, which is explained with the viscoelastic characteristics of the silicone. (3) For higher frequencies the coated taxels showed no declines compared to the uncoated taxels. The full necessary bandwidth of 0–500 Hz therefore can be well measured by the coated taxels. Figure 2C2 shows the average of each transmission curve for the four measured sensing arrays versus the silicone layer thickness. The decrease in sensitivity for thicker silicone layers is similar as in the static experiments (Fig. 2B3).

Taxel hysteresis and endurance

The viscoelastic properties of the silicone encapsulation result in a hysteresis observable upon the compression and release of the taxel. The stability of this hysteresis was investigated in a 10,000 cycles compression and release test for 220 μm thick silicone coatings, and the data is plotted in Figure 2D1. Previous tests on uncoated taxels¹⁹ showed no hysteresis, why it can be associated to the viscoelastic properties of the coating. The hysteresis remains stable over the tested period. The test was repeated on two similar taxels, and the sensitivities R1 and R2 were extracted for each compression cycle displayed in Figure 2D2. No degradation can be observed for R1 over 10,000 cycles, while R2 shows a short run-in period but stabilizes later. The hysteresis is expected to be unavoidable for the soft material integration; the observed effects are however repeatable.

Design of a finger bulk

In this study, we present how a human finger was measured as a reference and a simplified silicone (Dragon Skin 20, Smooth-on) body was designed to replicate the mechanics for small applied forces. A left index finger was taken as a reference, and the softness of the finger was measured along the X and Y coordinates as displayed in Figure 3A with the same characterization setup as shown in Figure 2A. Supplementary 4 summarizes further information, including Supplementary Movie S1 of how the finger was probed with the needle, a representative force-displacement curve, and data S1 with all measurement data. Figure 3B shows the measured stiffness between 10 and 70 mN along the X and Y direction of the index finger. The following observations can be made: (1) the finger stiffness is not symmetric along the X or Y direction for the tested index finger, independent of a forward or reverse order of the measurements. The finger is stiffer toward the joint (Y direction) and on the outer side of the index finger (X direction). (2) An average stiffness of 113 N/m in the force range of 10–70 mN could be measured for the indentation with the 0.5 mm wide cylindrical probe needle.

FIG. 3.

Replication of human finger mechanics. (A) Testing of a human index finger in the characterization setup. (B) Finger stiffness over different locations on the human finger. (C) Crosscut drawing of the artificial finger used for the replication of the mechanics.

A simplified finger was built consisting of a 3D printed backbone and a Dragonskin 20 silicone structure. The very soft characteristics of the human finger could be achieved by a 1.4 mm thin silicone structure (crosscut in Fig. 3C) with an air cavity, which is similar to the sturdy dermis and very soft finger pulp in a human finger.²¹ The stiffness of the finger was evaluated by simulation (142 N/m) and validated with four fabricated fingers (144 ± 11.5 N/m). The artificial finger is slightly stiffer than the human finger as even thinner silicone structures were less reliable due to trapped air bubbles and the limited positioning accuracy of the 3D printed molds. The sensing arrays, which are later integrated on the artificial finger, will modify its bending stiffness to a small degree. The effect is however deemed to be negligible as the fibers are only 15 μm thick.

Simulation of sensing behavior and finger integration studies

This section presents the simulation results for the coated taxels on a chip and compares them with the previously presented data. The validated models are then used for prognoses. The simulations in this section were divided in two main cases executed on the same model as shown in Figure 4A: in the (1) taxel on chip case the plane directly below the taxel and the surrounding skin is fixed, corresponding to the measurements in the static characterization. In (2) taxel on finger the fixed boundary condition is moved to the bottom of the simplified silicone finger structure that was validated in the previous section. Supplementaries 5 and 6 contain further information on the performed simulations.

FIG. 4.

Taxel simulation. (A) Simulation model for simulating the taxel response to different stimuli, either with a fixed plane directly below the taxel (right—“taxel on chip”) or at the bottom of the silicone finger (left—“taxel on finger”). (B) Measurement results compared to simulation data (taxel on chip—2.4 MPa) and exploration of the behavior upon implementation of the sensors on the soft finger.

The thickness of the skin $h_{s k i n}$ on top of the taxels was varied to evaluate the taxel sensitivity. Figure 4B shows the simulation results of some relevant cases compared to the experimental data provided in Figure 2B3. The experimental data correlate to the curve taxel on chip—2.4 MPa, this being the Young's modulus showing a good fit to the measured data. The simulations showed that a reduction of the Young's modulus to 1/3 (see curve taxel on chip—0.8 MPa) would result in a two times higher sensitivity for all thicknesses. Softer skins would therefore allow thicker skin layers, which might be valuable for a more robust overall design.

The most important comparison is made between the two cases taxel on chip and taxel on finger. The inset in Figure 4B displays the sensitivity ratio $S_{R} = R 1_{T a x e l O n F i n g e r} ∕ R 1_{T a x e l O n C h i p}$ for different skin thicknesses. For a thin skin the difference between the two cases is marginal while it gets more significant for thicker layers. This behavior is intuitive considering the ratio of the silicone thickness below and above the taxels. A more detailed interpretation of the results based on basic mechanical equations can be found in Supplementary 5. These results allow the transfer of the statistical information gathered for the taxels on chip to taxels on finger.

Readout electronics and measurement resolution

All measurements in this document were conducted with the readout electronics presented in this section. Figure 5A shows the overall concept of the readout electronics with the three major components as follows: (1) the sensing array, (2) the printed circuit board (PCB) for signal routing and amplification, which comes in two versions: either as the stationary, larger PCB connected to the probe card for the chip readout as visible in Figure 2A or as the smaller PCB directly integrated on the sensing finger as shown in Figure 6, and (3) the Eclypse Z7 board (Digilent). We used the field programmable gate array to implement an in-phase and quadrature mixer,²⁴ which is described in Supplementary 7. Figure 5B shows the average measured resolution²⁰

FIG. 5.

Readout electronics for the sensing array and their characteristics. (A) Concept drawing of all components. (B) Average root mean square noise level of all sensors for different sensor update frequencies. (C) Drift of the first four sensors in an array over 1 h measurement time.

FIG. 6.

Artificial fingertip. (A) Artificial fingertip in comparison to the human reference fingertip. (B) Backside with the miniaturized printed circuit board for analog signal routing.

R M S_{N o i s e} = \sqrt{\frac{1}{N} \sum_{n = 1}^{N} {(C_{n} - \bar{C})}^{2}}

in relation to the sensor update frequency (1.25 kHz was used for single taxel characterization, 31 kHz for finger readout experiments). This information can be combined with the measured sensitivities between 220 and 660 μm (Fig. 2B3) times the sensitivity ratio S_R (Fig. 4B) to calculate the force resolution of the taxels on the artificial finger (Supplementary Table S1). Choosing a 31 kHz update rate (∼217 Hz for a 144-sensor array) a skin thickness between 440 and 660 μm results in a force resolution of ∼0.1–0.2 mN per taxel.

Another important criterion is a low drift of the sensors and readout electronics, which is often observed in resistive sensor types and was one of the main reasons to aim for capacitive taxels in this work. Figure 5C shows the drift of four taxels over 1 h measurement time with relatively low fluctuations of −0.5 to 2.5 fF.

Integrated finger and interaction

In this study, we demonstrate the integrated fingertip and studies about the interaction with objects. Figure 6 shows the front and backside of the fingertip compared to the human reference finger. The compact size suggests easy integration with a humanoid robotic gripper.

Figure 7 summarizes several experiments, which were performed on integrated fingers. The fingers were coated with 440 μm of silicone. Figure 7A depicts the single taxel response with the 0.5 mm wide probe needle pointing on one taxel, repeated 3 × in a row. The characteristic curve shows similar behavior as on the flat chip (compare Fig. 2B1). The simulation results in Figure 4 predict an 18% lower sensitivity for the sensor integrated on the artificial fingertip compared to on-chip for a 440 μm coating, which is plausible from the measured data. A statistical analysis of all taxels on the fingertip proved to be very difficult due to limitations of the existing setup, as well as the difficult aiming of the probe needle onto the integrated taxels. The measured response to the pointy probe needle is sensitive to the alignment of needle tip and the taxel below.

FIG. 7.

Experiments performed on an artificial fingertip. (A) Single taxel characteristics after the integration on a soft finger. (B) Experimental setup for the manipulation of the sensing fingers with a 20 × 10 mm stamp. (C) Raw data of a touch and slide experiment with a 3D printed stamp, showing the taxel responses over time with their X, Y, and Z components. (D) Raw taxel data for a similar experiment performed with a stamp of same shape but with a very smooth surface. (E) Average of all sensor capacitances for applying forces in the range of 0–15 N for 10,000 iterations. (F) Real-time response of the fingertip to the touch of different objects (extracted from Supplementary Movie S2).

We used a 20 × 10 mm wide stamp (Fig. 7B) to apply different stimuli on the fingers. The stamp is connected to a 6-axis force probe (resolution ±150 mN). Supplementary 8 contains the narrated Supplementary Movie S2 which explains the setup in more detail and gives further details to all experiments presented in this section.

We first studied the response of the sensing finger to different touch and slide experiments, which consist of three phases as follows: (1) a compression of the finger with 2 N plus 4 s waiting, (2) slide over the finger for 10 mm plus 4 s waiting, and (3) release of the finger. Figure 7C displays the raw sensor data of this experiment, X, Y, and Z components are calculated as explained in Supplementary 1. Supplementary Movie S2 additionally shows a synchronized video of the experiment, as well as a video of the pressure and direction plots during this experiment; data S2 contains the raw data for all experiments, including repetitional runs for further analysis.

This experiment allows important observations in all three periods: (1) the force is applied within 1 s, and the taxels first in touch react with a sharp increase in capacitance, easily distinguishable as touch. The center of highest pressure moves from the top of the fingertip to the front of the fingertip during the finger compression as observable in Supplementary Movie S2. The offset in X and Y directions results from the tilted compression of the taxels in the downwards movement. (2) In the sliding period the sensors show a dynamic response due to the initial slip. Periodic oscillations are visible in the stable sliding period, which are related to the surface roughness of the 3D printed stamp. An overall decrease of the signal in X direction is also observable for the sliding in -X direction, and oscillations are stronger in that direction than in the Y direction, which allows the observation of the slip direction. (3) In the release phase the sensor signal immediately decreases. A relatively long time period is however needed for a complete recovery of the deformation due to viscoelastic effects of the silicone body and skin.

Figure 7D shows a similar experiment (only the Z data, X and Y data are additionally presented in Supplementary 8) with a stamp of the same shape but a very smooth surface, which was achieved with a surface-leveling coating on the 3D printed stamp. Pictures of both stamps are additionally visible in Supplementary 8. Additional important observations are as follows: (1) in the compression and wait periods the signal is very similar, as those static periods mainly depend on the shape of the stamp itself. (2) In the slide segment the finger reacts differently. The initial pressure decay is also observable for the very smooth stamp, allowing the observation of the onset of slip. During the sliding no periodic oscillations are observable, which originated from the surface roughness. This kind of dynamic information should therefore allow the distinction of different surface roughness.

Figure 7E summarizes the results of an endurance test performed on the finger. The finger was compressed for 10,000 times with a force of ≥15 N. During those compressions the average capacitance of all taxels was measured and is plotted over the applied force in Figure 7E1. The curves show that a rough estimation of the force is possible with this average capacitance; there are, however, some degradation effects in the current finger version, which result in a slightly increasing signal. The capacitances at 5 and 15 N were extracted for each compression and are plotted in Figure 7E2. The slight increase in signal is observable here as well. At 2000 and 7000 iterations the measurement was interrupted, allowing the finger to completely form back, resulting in a slightly different signal. Supplementary Movie S2 shows video footage of the finger compression, as well as a performed touch and slip experiment on the finger before and after the 10,000 compressions, which shows that the finger basically has the same response before and after the compression cycle.

Supplementary Movie S2 additionally shows a sequence of various objects being pressed on a fingertip and the life output of the taxels; two sequences are presented in Figure 7F1 and F2. The different shape of the objects is recognizable in the pressure pattern, which should allow simple object recognition, as in this case for the sharp edge of the scissor or the round shape of the ring spanner. The video additionally contains the compression with different polymer stamps or a soft human finger. All bodies, independent of the material, give a pressure signal.

Discussion and Conclusion

In this work we showed our findings for a novel tactile sensing technology, based on taxel arrays on a flexible substrate, embedded in a soft fingertip with human-like mechanical characteristics. The fiber-like structure of our sensing arrays allowed us to cover a 3D curved, soft fingertip with polyimide fibers of limited stretchability. In addition, our proposed technology can cover an arbitrary object, either by changing the shape and length of the sensing fibers or by superposing several sensing arrays, as the thickness of one flexible substrate is in the range of 10–20 μm.

We proved the sensing capabilities of the taxels to static and dynamic touch when integrated in a soft material and supported the findings with simulations. An important design element is the skin thickness, which allows an easy tailoring of system sensitivity and/or robustness. We observed that a skin thickness of 440–660 μm results in a force resolution 5–10 times smaller than for a human finger (sensing range 1–10 kPa⁵ and sensor density 1 sensor/mm²). Thinner skin layers are unfavorable as they would result in a less robust fingertip. Thicker skin layers will result in a more robust taxel packaging and a generally longer expected lifetime. The presented simulations showed that skin thickness of about 1 mm should still allow human detection limits; even thicker skin layers would need to be compensated with more sensitive electronics. The taxels have shear sensing capabilities when three electrodes are integrated beneath one suspended membrane. A matrix multiplication allows to extract compression and shear components, which however would need to be better quantified in additional experiments. The low drift of the sensing system can also guarantee stable long-term measurements.

The bulk finger integration is an important step to prove the applicability of the sensing technology. The complete soft integration of the taxels modifies their characteristics, and single sensor data ceases to be as relevant compared to the whole sensor array in the artificial fingertip. A soft, human-like finger bulk is expected to be advantageous in real applications as follows: (1) the finger surface is more adaptable to different shapes and increases the contact area, allowing better object manipulation and (2) a softer bulk should make the sensing finger more robust, as the bulk can more easily compensate for punctual forces or impacts. We showed the measurements of some characteristic use cases, like touching and sliding over the finger with stamps of different surface roughness and how the taxels give distinctive signals which depend on the shape of the object, which should allow simple object recognition, as well as its surface roughness, which should allow surface classification. Most advanced tactile sensing technologies use neural networks for feature extraction.^8,10,25 We expect that neural networks would allow us to extract several relevant features as well. In addition, such data-driven approaches are expected to also show higher robustness to existing measurement imperfections (viscoelastic effects and sensor response variations).

A force of 15 N was applied for 10,000 times as an initial test of finger robustness. Those robustness tests will need to be extended according to the needs of the application of the sensing fingers, which are not yet defined. The test showed a reasonable robustness of the developed finger to moderate forces, especially compared to state-of-the-art technologies. We expect to improve the robustness of the taxels by applying thicker skin layers of up to 1 mm and an improved design of the finger bulk. The data additionally allowed a rough estimation of the contact force, which however depends on the previous compression cycles of the fingertip.

Table 1 summarizes the achievements of this work. It is possible to conclude from the experiments that the integrated capacitive taxels can track similar mechanical stimuli as a human can detect in its finger skin. Mechanical contact can be detected at small applied forces due to the high sensitivity of the membrane-based taxels to deformations, comparable to Meissner endings and Merkel endings.²⁶ The function of Pacinian endings to sense vibration can be registered by the same taxels by sampling them at high enough speed as shown in Figure 7C, allowing the detection of slippage. The capability of Ruffini endings to measure strain or tangential forces is possible by adding three electrodes in some of the taxels, giving directional force information. The location of the types of taxels could be adjusted according to the needs of a sensing fingertip. Combining those functionalities with the 0.5 mm cell size of the taxels and the possibility to fully integrate them in a soft, 3D curved fingertip result in a promising approach for a highly versatile tactile sensing technology, which can withstand moderate forces over a longer test period while still being highly sensitive.

Footnotes

Acknowledgments

The authors thank Cosmin Roman for his conceptual support. Furthermore, the authors thank the students D. Wojcikiewicz, L. Kehrbein, T. Groenveld, D. Zürcher, M. Hüppin, F. Mähr, C. Cavelti, and A. Caviezel who supported the outcomes of this document in their theses with improving the measurement/spray coating setups, the execution and analysis of measurements, as well as simulations. We greatly appreciated the help and support of the Cleanroom Operations Teams of the Binnig and Rohrer Nanotechnology Center (BRNC) and ETH FIRST-CLA.

Authors' Contributions

J.W. and C.H. conceived the project. J.W. designed the hardware and experiments, performed the fabrication, as well as hardware/software development, and analyzed all results. P.S. executed the development of the hardware/software of the readout electronics. F.B.C. consulted on soft material processing, and T.B. consulted on the readout electronics. J.W. wrote the article, and all authors contributed in the revisioning.

Author Disclosure Statement

J.W. and C.H., together with Cosmin Roman, are inventors on a PCT patent application related to this work filed by ETH Zurich (no. 1001/4067849, filed on March 31, 2021).

Funding Information

This work was supported by internal funds of ETH Zurich.

Supplementary Material

References

Billard

, Kragic

Trends and challenges in robot manipulation. Science (80−) 2019;364(6446).

Harmon

LD.

Automated tactile sensing. Int J Rob Res, 1982:3–32.

George

, Kluger

, Davis

, et al. Biomimetic sensory feedback through peripheral nerve stimulation improves dexterous use of a bionic hand. Sci Robot, 2019; 4(32):1–12.

Johansson

, Flanagan

. Coding and use of tactile signals from the fingertips in object manipulation tasks. Nat Rev Neurosci, 2009; 10(5):345–359.

Dahiya

, Metta

, Valle

, et al. Tactile sensing-from humans to humanoids. IEEE Trans Robot [Internet], 2010; 26(1):1–20.

Fishel

, Loeb

. Bayesian exploration for intelligent identification of textures. Front Neurorobot, 2012; 6:1–20.

Lambeta

, Chou

, Tian

, et al. DIGIT: A novel design for a low-cost compact high-resolution tactile sensor with application to in-hand manipulation. IEEE Robot Autom Lett, 2020; 5(3):3838–3845.

Yuan

, Dong

, Adelson

. GelSight: High-resolution robot tactile sensors for estimating geometry and force. Sensors (Switzerland), 2017;17(12).

Sferrazza

, D'Andrea

. Sim-to-real for high-resolution optical tactile sensing: From images to three-dimensional contact force distributions. Soft Robot, 2022; 9(5):926–937.

10.

Sun

, Kuchenbecker

, Martius

. A soft thumb-sized vision-based sensor with accurate all-round force perception. Nat Mach Intell, 2022; 4(2):135–145.

11.

Yousef

, Boukallel

, Althoefer

. Tactile sensing for dexterous in-hand manipulation in robotics—A review. Sensors Actuators A Phys, 2011; 167(2):171–187.

12.

, Yi

, Yip

, et al. Scalable tactile sensor arrays on flexible substrates with high spatiotemporal resolution enabling slip and grip for closed-loop robotics. Sci Adv, 2020; 6(46):1–15.

13.

Boutry

, Jorda

, Vardoulis

, et al. A hierarchically patterned, bioinspired e-skin able to detect the direction of applied pressure for robotics. Sci Robot, 2018; 3(24):eaau6914.

14.

Lee

, Chung

, Chang

, Il, et al. Normal and shear force measurement using a flexible polymer tactile sensor with embedded multiple capacitors. J Microelectromechanical Syst, 2008; 17(4):934–942.

15.

Liang

, Wang

, Mei

, et al. Flexible capacitive tactile sensor array with truncated pyramids as dielectric layer for three-axis force measurement. J Microelectromechanical Syst, 2015; 24(5):1–10.

16.

Chun

, Son

, Kim

, et al. Self-powered pressure- and vibration-sensitive tactile sensors for learning technique-based neural finger skin. Nano Lett [Internet], 2019; 19(5):3305–3312.

17.

Schmitz

, Maiolino

, Maggiali

, et al. Methods and technologies for the implementation of large-scale robot tactile sensors. IEEE Trans Robot, 2011; 27(3):389–400.

18.

Tomo

, Schmitz

, Wong

, et al. Covering a robot fingertip with uSkin: A soft electronic skin with distributed 3-axis force sensitive elements for robot hands. IEEE Robot Autom Lett, 2018; 3(1):124–131.

19.

Weichart

, Roman

, Hierold

. Tactile sensing with scalable capacitive sensor arrays on flexible substrates. J Microelectromechanical Syst, 2021; 30(6):915–929.

20.

Weichart

, Ott

, Burger

, et al. Towards artificial robotic skin: Highly sensitive flexible tactile sensing arrays with 3D sensing capabilities. In: IEEE MEMS. IEEE; 2022; pp. 67–70.

21.

Xie

, Lu

, Yoon

, et al. Intensity variation normalization for finger vein recognition using guided filter based singe scale retinex. Sensors (Switzerland), 2015; 15(7):17089–17105.

22.

Hopf

, Bernardi

, Menze

, et al. Experimental and theoretical analyses of the age-dependent large-strain behavior of Sylgard 184 (10: 1) silicone elastomer. J Mech Behav Biomed Mater, 2016; 60:425–437.

23.

Bernardi

, Hopf

, Sibilio

, et al. On the cyclic deformation behavior, fracture properties and cytotoxicity of silicone-based elastomers for biomedical applications. Polym Test [Internet], 2017; 60:117–123.

24.

Schilcher

RF applications in digital signal processing. CAS 2007—Cern Accel Sch Digit Signal Process Proc, 2008; 249–283.

25.

, Fishel

, Yamamoto

, et al. Use of tactile feedback to control exploratory movements to characterize object compliance. Front Neurorobot, 2012; 6(JULY):1–9.

26.

Dahiya

, Valle

. Robotic Tactile Sensing: Technologies and System. Vol. 1. Springer: Dordrecht; 2013.

27.

Dellon

, Mourey

, Dellon

. Human pressure perception values for constant and moving one- and two-point discrimination. Plast Reconstr Surg [Internet], 1992; 90(1):112–117.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

3.74 MB

89.72 MB

1.72 MB

0.00 MB

Artificial Fingertip with Embedded Fiber-Shaped Sensing Arrays for High Resolution Tactile Sensing

Abstract

Introduction

Concept

Results

Skin layer process and static characterization

Dynamic taxel characterization

Taxel hysteresis and endurance

Design of a finger bulk

Simulation of sensing behavior and finger integration studies

Readout electronics and measurement resolution

Integrated finger and interaction

Discussion and Conclusion

Footnotes

Acknowledgments

Authors' Contributions

Author Disclosure Statement

Funding Information

Supplementary Material

References

Supplementary Material