A preliminary study on characterisation of finger interface kinetics using a pressure and shear sensor system

Abstract

Background:

Our hands constantly handle objects throughout our lives, where a crucial component of this interaction is the detection of grasping (pressure) and slipping (shear) of the object. While there have been a large amount of studies using pressure sensors for grasping detection, synchronised pressure and shear detection at the finger/object interface is still needed.

Objectives:

This study aims to assess the feasibility of a sensor system designed to detect both pressure and shear at the fingertip/object interface via a single subject test.

Study design:

Descriptive study, proof of concept.

Methods:

One healthy subject participated in the study and was asked to perform a single finger test protocol and a simple hand test protocol. The corresponding multidirectional loads at the fingertip/object interface were measured in real time using a pressure and shear sensor system.

Results:

Results from the finger test protocol show peak values of up to approximately 50 kPa (5 N) and 30 kPa (3 N) of pressure for each test, respectively. Results from the hand test protocol show a pressure and shear profile that shows a large increase in grip force during the initial grasping of the object, with a peak pressure of approximately 50 kPa (5 N). The pressure and shear profile demonstrates that the load is not evenly distributed across all digits.

Conclusion:

This study provides evidence that the reported sensor system has sufficient resolution, dynamic response and load capability to capture biomechanical information during basic protocols and hand-grasping tasks.

Clinical relevance

The presented sensor system could be potentially used as a tool for measuring and evaluating hand function and could be incorporated into a prosthetic hand as a tactile feedback system.

Background

Human hands display a vast number of movements, gestures, poses and fine motor skills, while constantly interacting with the environment and objects around us. This dexterity is dependent on our ability to sense the environment through touch. Our sense of touch is provided by mechanoreceptors in the glabrous skin (i.e. the smooth skin side of the fingertip), with the highest density of these being at the fingertips, ¹ providing us with an incredibly sensitive array of organic sensors. These provide both somatosensory and proprioceptive feedback, enabling us to perform complex tasks and handle objects in a natural way. Control of finger/object interaction relies on real-time sensing of both pressure and shear, referring to the forces normal and tangential to the fingertip/object interface, respectively, which exist during these tasks.^2,3 Moreover, the transduction of shear stresses is critical for slip detection and maintaining a stable grasp on an object. ⁴ Measurement of shear forces would also allow grip compensation before the object slips. The ability to measure dynamic changes in both pressure and shear would facilitate an in-depth understanding of the kinetics at the fingertip/object interface. ⁵ Such sensing capability, in combination with clinical hand-function assessments, ⁶ could potentially provide a more objective approach and thus aid upper limb rehabilitation in, for example, stroke patients and amputees. Recognising the long standing need, many works have been reported to evaluate multidirectional forces at the finger/object interface during object or weight handling tasks. This includes the design and exploitation of custom built simulators or handles, where large rigid force transducers were employed at five finger digits.^7–10 However, rigid sensors can affect the contact and interaction between the fingertip and object and thus not suitable for practical hand applications. Vogt et al. ¹¹ reported a soft multiaxis sensor based on microfluidic principles. However, these sensors are much larger (i.e. 50 mm × 60 mm × 7 mm) than the area of a fingertip and also require complex fabrication procedures. No participant-based results were reported using these microfluidic sensors. ¹¹ Thus, there is an unmet need to develop a fingertip interface sensor system which not only can measure multidirectional forces during object handling but also utilises thin and flexible sensor units to allow direct applications at finger and object interface during dynamic hand actions.

This article presents a novel pressure and shear sensor system, which is capable of measuring three-directional forces. The primary aim of this article is to study the feasibility of this sensor system to detect multidirectional mechanical interaction at the fingertip/object interface during daily actions such as a single finger touching a surface and a simple hand action. In particular, for the hand test protocol, we adopted a simple grasp-and-lift hand action using a cube-shaped object. Similar hand actions have been previously reported in evaluating prosthetic hand function using the Southampton Hand Assessment Procedure (SHAP). ⁶

Method

Subjects

A healthy volunteer (male, 28 years) with no known hand-function deficiencies, took part in an experiment to provide preliminary hand testing data for this study. This study was approved by the University of Southampton Ethics and Research Governance Committee, with submission ID: 20847.

Apparatus

The sensor system comprises a flexible capacitive-based sensor of dimensions of 10 mm × 10 mm in area and a thickness of 1 mm. The sensor output is captured by a small data acquisition device, and data is transmitted wirelessly via Bluetooth, as shown in Figure 1(a). Sensors were placed at the fingertips to collect real-time pressure and shear data during two specific test protocols. Sensor system design details and calibration are detailed elsewhere.^12,13

Figure 1.

(a) Sensor system overview and sensor dimensions. (b) Sensor attached to the right-hand index finger, showing directions of pressure, +S_x and +S_y for the finger test actions. (c) Sensors attached to the right hand, showing directions of pressure, +S_x and +S_y for the thumb sensor and the other four fingers, respectively, for the hand test action, that is, grasping and lifting the block.

Procedures

First, a finger test was designed and implemented by mounting a single sensor to the face of the index finger on the subject’s right hand, shown in Figure 1(b). The subject was then asked to repeatedly perform a press–drag–lift action and a press–push–lift action on a hard surface, respectively, at the subject’s natural speed and a self-selected force. These one-finger protocols were designed to provide an initial insight into the pressure and shear ranges at the finger/object interface during very basic tasks. For instance, a press–drag–lift action could occur when a finger interacts with a touch screen or feeling the texture of a surface. A press–push–lift action could occur when a finger operates a laptop touchpad or pressing a light switch. For the press–drag–lift action, the finger was dragged in the lateral direction, that is, dragged from the subject’s left to right, aligning with the positive shear X (+S_x) direction as shown in Figure 1(b). For the press–push–lift action, the finger was pushed in the longitudinal direction, that is, pushing away from the subject, aligning with the positive shear Y (+S_y) direction shown in Figure 1(b).

The hand test protocol involved mounting five sensors to the faces of all five fingertips of the subject’s right hand. The subject was asked to repeatedly perform a simple grasp–lift–hold–put-down action on a square block which was approximately 0.8 kg in weight and measured 60 mm× 60 mm × 75 mm, as shown in Figure 1(c). This activity was chosen as it utilises the tripod/power grip, one of the most common actions our hands perform on a daily basis. ⁶

Data analysis

Synchronous pressure (P) and shear (S_x and S_y) data from all these tests was collected and analysed as a function of time. Standard deviation (SD) values were calculated over five cycles for the peak values of pressure, S_x and S_y, respectively.

Results

One-finger test protocols

Figure 2(a) and (b) shows the typical pressure and shear response as a function of time during the one-finger press–drag–lift and press–push–lift actions, respectively. The insets show the respective directions of finger movement aligning with those for +S_x and +S_y.

Figure 2.

Pressure, S_x and S_y measured at the index fingertip when performing (a) press–drag–lift test and (b) press–push–lift test.

For Figure 2(a), initial contact was established at t ≈ 0.2 s, followed by the press phase, the drag phase and the lift phase, as highlighted in Figure 2(a). During the press phase there is a sharp increase in pressure shown, with S_x and S_y showing a gradual increase. Pressure, S_x and S_y increase to a peak value of approximately 50, 32 and 18 kPa in the drag phase, respectively. Pressure, S_x and S_y decreased during the lift phase, with pressure showing a sharper decline than S_x and S_y. Over five repeated actions, the SD values for peak pressure, S_x and S_y are 7.6, 3.6 and 3.8 kPa, respectively.

For the press–push–lift action in Figure 2(b), during the press phase, there is an increase in pressure and S_y, while S_x increased less rapidly. During the push phase, pressure, S_x and S_y increased to approximately 30, 10 and 24 kPa, respectively. S_y shows fluctuations of up to approximately 5 kPa during the push phase. During the lift phase, pressure, S_x and S_y decreased. At t ≈ 1.2 s, finger contact ceased and pressure, S_x and S_y return to baseline values. Over five repeated actions, the SD values for peak pressure, S_x and S_y are 4.9, 2.7 and 3.3 kPa, respectively.

Hand test protocol

Figure 3(a)–(c) shows the pressure, S_x and S_y response as a function of time for the grasp–lift–hold–put-down action, respectively. In particular, Figure 3(a) shows that the contact was initiated at t ≈ 0.8 s, followed by the grasp–lift phase, hold phase and the put-down phase, as highlighted in Figure 3. It was observed that pressure obtained from the thumb, index and ring fingers increased at the point of contact, while pressure from the middle and little fingers only started to show at t ≈ 1.2 s and t ≈ 1.4 s, respectively. Throughout the hold phase, pressure from the index, middle, ring and little fingers stabilised while pressure from the thumb decreased from approximately 20 to 10 kPa. Pressure from all fingers show a decrease until the release of the object at t ≈ 6.3 s.

Figure 3.

(a) Pressure, (b) transverse shear (S_x) and (c) longitudinal shear (S_y) measured at all five fingertip surfaces, when the grasp–lift–hold–put-down action was performed.

Figure 3(b) shows that S_x obtained from the index finger started to increase at the point of contact and increased to approximately 4 kPa, while S_x from the thumb and middle fingers started to increase later at t ≈ 1.25 s to approximately 6 and 1.5 kPa, respectively. The ring and little fingers show negligible change during the grasp–lift phase. Throughout the hold phase, S_x from the thumb, index and middle fingers remained relatively stable, while S_x from the ring and little fingers increased to approximately 1 and 2 kPa, at t ≈ 2 s.

Figure 3(c) shows that S_y obtained from the thumb changed from approximately −4 kPa at t ≈ 1.2 s to 5 kPa at t ≈ 1.7 s. S_x obtained from the index and middle fingers increased to approximately 6 and 7 kPa, respectively. S_x from the middle finger shows a change of approximately 5 kPa of at t ≈ 1.2 s, to −4 kPa at t ≈ 1.75 s. S_x from the little finger shows negligible change during the grasp–lift phase. During the hold phase, S_x at the little finger shows a small change of approximately 1 kPa at t ≈ 2.5 s, while S_x outputs from all other fingers remained relatively stable. In the put-down phase, peak S_y values of 13 and 12 kPa are shown for the thumb and ring fingers, respectively. The index and middle fingers show peak S_y values of approximately −2 and −6 kPa, before S_y from all fingers decreased to baseline at t ≈ 6.3 s. The total contact time is approximately 5.5 s. Over five cycles, the peak pressure SD values are 3.4, 11.7, 12.2, 6.9 and 6.0 kPa for the thumb, index, middle, ring and little fingers, respectively. Peak SD values for S_x are 4.0, 2.5, 3.0, 2.1 and 1.4 kPa. Peak SD values for S_y are 4.8, 6.6, 0.8, 2.3 and 3.9 kPa.

Discussion

One-finger test protocols

In order to assess the sensor system response, resolution and the timing, output from a single sensor during a press–push–lift and press–drag–lift has been analysed. The profiles and peak values displayed in Figure 2 align with values exhibited by Su et al., ¹⁴ where approximately 10 N of F_z (pressure) and 5 N of F_x and F_y (shear) were observed during a press–drag–lift test, conducted on a force plate. This indicates that the sensor system is capable of measuring dynamic changes in pressure and shear at the fingertip/object interface, with minimal delay. The resolution of the sensor system is approximately 0.5 kPa (approximately 0.05 N) for pressure and 0.25 kPa (approximately 0.025 N) for shear measurements when calibrated up to approximately 100 kPa (10 N) and 25 kPa (2.5 N), respectively. ²

Hand protocol

The hand protocol was used to examine the sensor systems’ capability to capture synchronised data from five fingers and therefore capture the inter-digit coordination during a simple grasping task. Figure 3(a) shows that, during the initial grasping phase, peak pressure values reached approximately 50 and 45 kPa at the thumb and index fingers, respectively, equating to approximately 5 N of normal force and 4.5 N of tangential shear force. These values align with values reported by Landsmeer, ¹⁵ where approximately 5 N of normal force was observed at the index, middle and ring fingers and approximately 4–6 N of tangential force was observed across all digits, when the hand was lifting an object equating to 10.4 N. Furthermore, pressure peaks occur at 0.8 and 0.6 s after initial contact at t ≈ 0.8 s. Subsequently, this grip force reduced to approximately 20 and 5 kPa for the thumb and index fingers, respectively, at t ≈ 1.9 s and sustained at that level throughout the hold phase. The ‘over-estimation’ of the required grasping force at initial contact has also been observed by others^16,17 and demonstrates the sensor systems’ capability to capture synchronised biomechanical information. During the grasp–lift phase, S_x from the thumb showed a sharp increase, as shown in Figure 3(b), and remained almost constant at approximately 5 kPa, while S_x from the remaining fingers showed comparatively lower values, up to 2 kPa, throughout the hold phase. This may suggest that, in comparison with the index, middle, ring and little fingers, the thumb sustained a large amount of pressure and shear forces, while securing the object. The importance of the thumb for object handling has also been previously reported.^15,18 During the grasp–lift–hold–put-down action, the majority of shear expected was in +S_x as shown in Figure 3(b); however, shear in +S_y direction was observed with a peak value seen at the thumb interface of approximately 12 kPa, as shown in Figure 3(c). The coexistence of S_x and S_y indicates that although this particular hand action was to grasp and then lift the object in the S_x direction, at each fingertip, there was a combination of multidirectional forces, reflecting the complex and dynamic interactions between fingers and the object.

In order to study the time difference and coordination among different fingers during the grasp–lift–hold–put-down action, results have been analysed from the thumb, index and middle fingers, respectively, because the results in Figure 3 indicate that these three digits are more dominant for this action. Contact between the subject’s hand and the object was initiated by the index finger at initial contact, when pressure obtained from the index finger started to increase, as shown in Figure 3(a). Initial contact by the index finger has been commonly observed and reported. ¹⁹ It was only after the subject has begun to lift the object that the middle finger was recruited to help grasp, thus the increase in pressure to 6 kPa. This grasp adjustment during this task has been previously observed by Flanagan et al. ¹⁶ Fluctuations in shear were observed at the thumb, index and middle fingers, which could be attributed to the subject stabilising the block during the put-down part of the action.

Conclusion

A dynamic, real-time pressure and shear sensor system for the fingertip/object interface has been deployed in preliminary tests using a healthy subject, as a proof of concept. Both the one-finger and the hand test protocols were conducted to perform a finger press–drag–lift and press–push–lift tests on a surface, as well as a hand grasp–lift–hold–put-down action on an object, with a view to measuring pressure, transverse shear and longitudinal shear at the fingertip/object interface during these typical hand tasks. The ability to measure real-time pressure and shear at the fingertip/object interface could lead to a deeper understanding of the finger and hand kinetics during daily hand tasks. This biomechanical information could also be used to provide an objective hand-function assessment tool, which could be potentially exploited to assist upper limb rehabilitation. It is worth noting that the primary aim of this study was to demonstrate the feasibility of the developed interface sensor system for fingertip/object applications. As such, proof-of-concept tests with a single subject have been presented. This leads to inevitable limitations for the assessment of fingertip-and-hand biomechanics.

The presented results suggest that the sensor system is sensitive enough to detect dynamic changes in pressure and shear at the fingertip/object interface during basic object handling actions. The results are encouraging and suggest that the sensor system could be potentially used for applications, such as providing tactile feedback for prosthetic hand users, and to facilitate biomechanical studies as a research tool in combination with existing hand-functional assessments, for example, SHAP. ⁶

For future work, this study would benefit from tests with a variety of daily hand actions, such as lateral grip actions, inclusion of objects with different weight and shape, as well as tests involving multiple participants including those with hand deficiencies and users of prosthetic hands. This would give an insight into biomechanical interaction at the fingertip/object interface during daily hand actions.