Neurodata Tracker: Software for computational assessment of hand motor skills based on optical motion capture in a virtual environment

Abstract

Objectives

Deficits affecting hand motor skills negatively impact the quality of life of patients. The NeuroData Tracker platform has been developed for the objective and precise evaluation of hand motor deficits. We describe the design and development of the platform and analyse the technological feasibility and usability in a relevant clinical setting.

Methods

A software application was developed in Unity (C#) to obtain kinematic data from hand movement tracking by a portable device with two cameras and three infrared sensors (leap motion®). Four exercises were implemented: (a) wrist flexion-extension (b) finger-grip opening-closing (c) finger spread (d) fist opening-closing. The most representative kinematic parameters were selected for each exercise. A script in Python was integrated in the platform to transform real-time kinematic data into relevant information for the clinician. The application was tested in a pilot study comparing the data provided by the tool from ten healthy subjects without any motor impairment and ten patients diagnosed with a stroke with mild to moderate hand motor deficit.

Results

The NeuroData Tracker allowed the parameterization of kinematics of hand movement and the issuance of a report with the results. The comparison of the data obtained suggests the feasibility of the tool for detecting differences between patients and healthy subjects.

Conclusions

This new platform based on optical motion capturing provides objective measurement of hand movement allowing quantification of motor deficits. These findings require further validation of the tool in larger trials to verify its usefulness in the clinical setting.

Keywords

Optical motion capture parameterization virtual environment hand motor deficit stroke neurorehabilitation

Introduction

The functional deficits of the hand due to various diseases have a highly negative impact on the patient's abilities and quality of life.¹ Adequate hand functionality depends not only on strength, but also on fine and precision motor skills, tactile sensitivity and proprioception, which are interrelated and affect movement in different ways and degrees.² These functionalities’ constraints are sometimes difficult to assess with precision. Usually, hand function is assessed using clinical scales, that rely on the clinician's qualitative perception.³ An accurate quantitative measurement would be of great interest, especially in cases of mild deficits, that may be underestimated and may not receive adequate rehabilitation therapy.⁴ The development of innovative solutions for objective measurement of hand function is therefore crucial to aid in the design and monitoring of personalized rehabilitation programmes.⁵

Technological development has facilitated the analysis of movement through the creation of tools that allow its detection and parameterization, acquiring kinematic data for its analysis. However, most of the systems currently used rely on information obtained through sensors placed on the limb or from devices that combine robotics and exoskeletons (Amadeo®, Tyromotion GmbH^6,7 (Austria); Kinarm products®, Kinarm^8,9 (Canada); Sinfonía®,¹⁰ Maestro®,¹¹ Crescendo®,¹² Gloreha^13,14 (Italy), among others). However, these devices are mainly focused in rehabilitation and may add some other functionalities that require complex and big equipment in the majority of the cases. Therefore, they do not meet the characteristics of simplicity and accessibility necessary for their general use in routine clinical practice.

Nonetheless, in recent years, new image recognition devices have been developed to generate data for computational analysis of motion without the use of any type of marker. This technology has the further advantage of versatility and adaptability, requiring minimal equipment (computers, tablets, or smartphones in common use) and, therefore, offering the possibility of serving a larger number of patients in any clinical or even domestic environment. Within this group of optical motion capture devices, the Kinect® (Microsoft, USA) has been validated for measuring motor deficits associated with neurological diseases.^15,16 However, this application only allows to track large movements, involving the entire limb or large segments of the body, but does not have the ability to discriminate slight and fine movements.¹⁷ Therefore, it is not useful to accurately track hand movements. More recently, sensible optic tracking systems such as the Leap Motion Controller® (Ultraleap)¹⁸ capable of tracking movements of small objects and particularly of the hands are available.^19,20

In this study, we present the technical development of an ad-hoc platform (NeuroData Tracker) to parameterize the kinematics of hand movement using Leap Motion® and to evaluate its feasibility to objectively measure the movement. A secondary objective was to test the capability of the platform to detect motor deficits in a pilot study, comparing kinematic information from patients with hand motor impairment after a stroke with healthy subjects.

Materials and methods

Hand movement assessment platform

The platform (Figure 1) is composed of custom-built software, described in detail later on, developed for the parameterization of the kinematic data of the movement of the hands, captured using a commercial device based on optical motion tracking (i.e., Leap Motion, Ultraleap®); a self-designed support structure for the arms; and a PC, which serves as a station to run the software.

Figure 1.

Hand movement assessment platform.

This custom-built software also provides the graphical interface necessary for the interaction between the optical motion capture device and the PC.

Leap motion

The Leap Motion Controller® is an optical tracking module that captures the movements of the hands with unparalleled accuracy. The Leap Motion Controller is capable of tracking hands within a three-dimensional (3D) interactive zone. It has an effective measurement range that extends from 10 to 60 cm, extending from the device in a 140 × 120° typical field of view.²¹ It is based on two 640 × 240-pixel near-infrared cameras, separated 40 mm apart and three LEDs spaced on either side and between the cameras to prevent overlaps.²² These cameras operate in the 850 ± 25 nm spectral range and at a sample frequency of 120 Hz. The device is connected directly to a computer via USB 2.0 and provides a three-dimensional real-time representation of hands and their movements in the PĆs screen, thus allowing feedback for the patient.

This optical motion capture device has been chosen for its ease of use and portability since it does not require the use of markers or any other type of sensor, its small dimensions (80 × 30 mm) and light weight (32 g) makes it optimal to be used in any clinical environment or even at patient´s home.

The Leap motion controller provides a calibration method through its own control panel. This calibration consists of some predefined movements of the device associated with a final score that has to exceed 80 points to consider the device calibrated. This process is performed the first time the device is used and whenever the clinician in charge of supervising the test notices any anomaly in data acquisition.

The Ultraleap Tracking System analyzes each pair of stereo images obtained to detect the presence, posture, and motion of hands. Therefore, it is the device itself that synchronizes both images.²³ These synchronized data are transmitted as a LEAP_TRACKING_EVENT() structure. Then, it analyzes the images and fits an internal model of the human hand to the observed scene. In the case that parts of a hand are out of range, the software estimates the data based on the internal interpolation method.

These functions are provided by Ultraleap's API and are directly implemented into our custom-built software.

Support structure for the arms

A complementary support structure for the arms of the user was designed for two purposes: (a) to constantly maintain the hands in an optimal position for the Leap Motion's field of vision and (b) to help the patient maintain a fixed posture, preventing arms from falling, considering the possibility of proximal weakness of upper limb in certain patients (Figures 2 and 3). This provides homogeneity of the measurements obtained, facilitates data processing and eliminates low-frequency noise due to involuntary arm movements.

Figure 2.

Technical draft of the arm's platform and final assembly. (1) Optical motion capture placement; (2) upper part of the structure; (3) middle part of the structure; (4) lower part of the structure; (5) central part of the structure; (6) arm resting place.

Figure 3.

Experimental set up.

Battery of exercises for the movement assessment

A total of four exercises were selected to test the ability of the tool to quantify the movement tracked by the device (Figures 3 and 4): (a) Wrist flexion–extension, starting and ending in the horizontal plane. (b) Finger grip between thumb and index with the remaining three fingers spread apart. (c) Finger adduction and abduction in the horizontal plane. (d) Fist opening and closure. Each exercise should be performed repeatedly for 5 s.

Figure 4.

Exercises for the hand movement assessment implemented in the software NeuroData Tracker. Wrist flexo – extension, finger grip, finger separation, fist opening and closure.

Development of the software NeuroData Tracker

The NeuroData Tracker software (Copyright identifier: 2210102294056, 10 Oct 2022) was developed using Unity 3D (Unity Technologies)²⁴ to parameterize the kinematics of movements tracked by the leap motion. The requirements fulfilment process for the development of the software involved discovering, analyzing, documenting, and verifying the services to be provided by the platform: its functions, limitations, and constraints. Figure 5 summarizes the main functionalities of the application. The software is composed of three distinguished blocks: (1) Data extraction and storage; (2) Data processing and analysis; (3) Report of results generation.

Figure 5.

Neurodata tracker flow diagram and functionality blocks.

Data extraction and storage

According to the datasheet of the optical motion capture device, the Leap Motion Controller is designed to track hand and finger movements accurately and reliably at a wide range of speeds (assuring an accurate of tracking hand and finger movements at speeds of up to 1 m/s, with a max sampling frequency of 120 Hz). However, the sampling frequency of the system is set at 50 Hz as the user must perform the exercise at a slow and controlled pace. This sampling frequency is optimized to collect the essential data without losing information.

We selected the following variables to be collected for further processing: Fingertip position, middle point of the palm position and velocity vector of the palm (Figure 6).

Figure 6.

Graphical representation of the vectors which define: (a) Orientation of the hand (palmNormal and Direction) and (b) position and the general direction in which a finger is pointing (fingerTip and Direction).

The software extracts Cartesian coordinates of each variable, at a sample frequency of 50 Hz, from the optical capture device along the execution of the exercises previously described. These raw data consist of twenty-four kinematic variables for the fingers and palms of each hand plus the variable of ‘Time’ common to both, which are stored locally in a .csv file, one for each exercise performed. These variables are listed in Table 1.

Table 1.

Raw variables obtained from leap motion.

	Kinematic variables
	Left hand	Right hand
	Time	Time
Fingertip position	leftThumbTipPosition_X	rightThumbTipPosition_X
	leftThumbTipPosition_Y	rightThumbTipPosition_Y
	leftThumbTipPosition_Z	rightThumbTipPosition_Z
	leftIndexTipPosition_X	rightIndexTipPosition_X
	leftIndexTipPosition_Y	rightIndexTipPosition_Y
	leftIndexTipPosition_Z	rightIndexTipPosition_Z
	leftMiddleTipPosition_X	rightMiddleTipPosition_X
	leftMiddleTipPosition_Y	rightMiddleTipPosition_Y
	leftMiddleTipPosition_Z	rightMiddleTipPosition_Z
	leftRingTipPosition_X	rightRingTipPosition_X
	leftRingTipPosition_Y	rightRingTipPosition_Y
	leftRingTipPosition_Z	rightRingTipPosition_Z
	leftPinkyTipPosition_X	rightPinkyTipPosition_X
	leftPinkyTipPosition_Y	rightPinkyTipPosition_Y
	leftPinkyTipPosition_Z	rightPinkyTipPosition_Z
Palm position	leftPalmPosition_X	rightPalmPosition_X
	leftPalmPosition_Y	rightPalmPosition_Y
	leftPalmPosition_Z	rightPalmPosition_Z
Palm velocity	leftPalmVelocity_X	rightPalmVelocity_X
	leftPalmVelocity_Y	rightPalmVelocity_Y
	leftPalmVelocity_Z	rightPalmVelocity_Z
Normal vector	leftPalmNormal_X	rightPalmNormal_X
	leftPalmNormal_Y	rightPalmNormal_Y
	leftPalmNormal_Z	rightPalmNormal_Z

Data processing

The kinematic data obtained from each exercise are processed to obtain the final parameters which will be further analysed using a custom-built Python script, implemented directly into the platform.

The parameters of interest obtained (Table 2) are structured as follows:

Exercise 1. Wrist flexion – extension (Figure 7(a) and (b)): The maximum angle of wrist flexion is the parameter of interest for this exercise. The unit vector normal to the palm of the hand, obtained by the variable ‘PalmNormal’, is processed to get this parameter. The normal vector is divided into its Cartesians components, which are normalized in [−1, 1]. According to the coordinate system of the device, the value of the z-component is 0 when the palms are fully stretched. Furthermore, a perfectly horizontal position would be given by values of the z and y components of 0 and −1 respectively. The z-component will take higher values as the wrist flexes and lower values as it extends.

Exercise 2. Finger grip (Figure 7(c)–(f)): Both the subject's ability to successfully perform the gripper between thumb and forefinger and the way the exercise is achieved are considered of relevance. For this reason, the position of the tip of each finger is processed in two different analyses: (1) The minimum and maximum Euclidean distance between the thumb and the forefinger is calculated in the space domain. (2) The movement of the remaining three fingers during the execution of the exercise is studied both in a plane (x–z) and in a space domain. For this purpose, the Convex Hull of each finger trajectory is calculated for the entire exercise. This allows to calculate the total perimeter of the movement area of each fingertip studied in the x–z plane and the total volume occupied by each fingertip during the exercise.

Exercise 3. Finger adduction – abduction (Figure 7(g) and (h)): The processing of this exercise is carried out in a plane (x–z) domain, since the entire exercise is performed in the x–z plane. The range of movement is calculated based on each fingertip position. The maximum distances reached between individual fingers are considered.

Exercise 4. Fist opening and closure (Figure 7(i) and (j)): The position of the tip of each finger in the x–z plane and in the domain of space is studied. The range of movement of each finger and the maximum and minimum perimeter in the x–z plane are calculated. In the space domain, the absolute maximum and minimum values of the volume of the Convex Hull of the values of the position of each fingertip at each time instant.

Figure 7.

Processing plots for each exercise. Plots (a and b) Normal palm vector. Z component. Plots (c and d) x–z plane thumb and index grip movement. The rest of the fingers are represented in grey. Plots (e and f) Convex Hull of the fingers not involved on the grip. Plots (g and h) x–z plane finger separation. Plots (i and j) x–z plane fist opening and closure.

Table 2.

Variables of interest to be analyzed for each exercise.

Exercise	Variable
1. Wrist Flexion – Extension	1.1 Maximum amplitude from the horizontal
2. Grip opening and closure	2.1 Maximum grip distance
	2.2 Thumb range
	2.3 Index range
	2.4 Ring perimeter
3. Finger Adduction – Abduction	3.1 Thumb range
	3.2 Index range
	3.3 Ring range
	3.4 Pinky range
4. Fist opening and closure	4.1 Maximum perimeter
	4.2 Thumb range

Interface and functionalities

As aforementioned, the NeuroData Tracker platform includes a carefully designed graphical interface in which the user can easily register, set data and configuration parameters, switch screens, and obtain reports of the execution of exercises (Figure 8).

Figure 8.

Software interface. (a) Menu screen, (b) configuration screen, (c) preparation screen, (d) confirmation screen.

From the start menu screen (Figure 8(a)) there is the option to start a new study or to retrieve reports of previous studies. When starting a new study, the user is driven to the configuration screen (Figure 8(b)), in which three selection fields appear: (a) The duration of the exercises; (b) a drop-down menu that shows two options: patient or control. It has been decided to provide both options since this application is intended both for the development of research projects and for clinical use; (c) the identification field.

Once the study is initiated, a preparation screen is displayed for each exercise (Figure 8(c)). It includes (a) the title of the exercise; (b) the avatar of the subject´s hands provided by Leap Motion; (c) brief instructions on the correct performance of the exercise; (d) a gif showing the movement that the user must perform.

Pressing the “ready” button, the recording of movement starts. While the timer is playing, kinematic data are collected. Once the test is finished, a confirmation screen appears (Figure 8(d)). Data are analyzed in real time and a report of the study is automatically generated. A loading screen appears once the process is completed, and the system gives the user the option of either viewing the report or concluding the study.

Once a study is completed the user is always redirected to the start menu screen.

The application is fully automated to prevent the user from accessing screens or functions that could compromise data extraction and storage. In addition, each screen is provided with the possibility of a manual confirmation of the correct performance of the exercise to avoid errors.

In compliance with the General Data Protection Law, the application never identifies patients by any personal data (no data about age, race, sex, name, medical history number, etc. is stored).

Report of results generation

The Neuro Data Tracker provides real-time feedback on exercise performance to the users. The report includes (Figure 9): (a) A summary of the results of the execution of the four exercises by each hand is displayed in first place (Figure 9(a)). By means of a colour code (green: normal, red: altered) a first impression of the deficit can be obtained. (2) The following pages (Figure 9(b)) contain each of the previously mentioned parameters of interest placed on graphs. For exercise one, a graph representing the normal vector to the palm as a function of time is added. (3) Finally, if more than one evaluation has been performed, the system generates evolutionary graphs with the most relevant parameters along the different dates of the evaluation for each exercise (Figure 9(c)).

Figure 9.

Report of results. (a) Summary page, (b) First exercise analysis, (c) Evolutionary page.

Pilot study

To assess the capability of the software to measure and analyze kinematic data, a pilot case-control study was conducted. Cases (n = 10) were clinically stable acute stroke patients, within the first 7 days, suffering from mild functional impairment of one of the hands. Controls (n = 10) were volunteers with no history of stroke nor any motor impairment. Subjects with aphasia, cognitive impairment, confusional syndrome, complete plegia or severe paresis, or other conditions that would impede the adequate performance of the tasks were not considered for the study. All subjects were recruited at the Department of Neurology and Stroke Centre of the University Hospital La Paz in Madrid, Spain. The study was conducted according to ethical standards of good clinical practice and to the protection of personal data requirements. All participants signed informed consent.

Stroke severity of patients was assessed using the National Institute of Health Stroke Scale score²⁵ (NIHSS) and the hand motor deficit using the Fugl Meyer Assessment²⁶ (Upper Extremity, Section C) (FMA-UE). Daniel's scale²⁷ was used to assess the strength of the muscles involved in the exercises. Patients performed the test immediately after the clinical evaluation.

The parameters described in the data processing section are shown for patients and controls.

Statistical analyses

The parameters shown in the data processing section are described for patients and controls as media ± standard deviation to illustrate the technical feasibility of the tool for obtaining kinematic data and to differentially measure various motor performances. Formal comparisons are not done due to the small sample size. The correlations between the most representative kinematic variable of each exercise and clinical scales scores are analysed using the Pearson's correlation coefficient (R). The entire analysis process has been carried out through Python, with the assistance of the following libraries: Numpy, Pandas, Scipy, and Statistics.

Results

Study subject characteristics

Characteristics of the patients are summarized in Table 3. Among controls, the median (range) age was 48 (21–60), and eight of them were women.

Table 3.

Clinical characteristics of the patients in the pilot study.

Patient	Age	Sex	Stroke subtype	NIHSS	FMA – UE	Daniel's scale (Symptomatic Hand)
1	79	M	Territorial cerebral infarction	1	13	5, 4, 5, 4
2	64	M	Territorial cerebral infarction	4	10	5, 4, 5, 4
3	83	F	Territorial cerebral infarction	2	14	5, 5, 5, 4
4	70	F	Territorial cerebral infarction	0	12	5, 5, 5, 4
5	74	F	Territorial cerebral infarction	0	14	5, 5, 5, 5
6	67	F	Territorial cerebral infarction	0	14	5, 5, 5, 5
7	74	M	Lacunar cerebral infarction	0	11	4, 4, 5, 4
8	53	M	Territorial cerebral infarction	1	12	5, 3, 4, 4
9	77	F	Territorial cerebral infarction	7	14	5, 5, 5, 5
10	56	M	Cerebral haemorrhage	1	8	4, 4, 4, 3

F: female; M: male; NIHSS: National Institute of Health Stroke Scale score, FMA-UE: Fugl Mayer Assessment – Upper Extremity score. Scores for Daniel's Scale refer to the selected exercises in the following order: wrist extension, finger grip, fist closure, and finger separation.

Kinematics results

Kinematic parameters obtained from the performance of the exercises by the patients and the controls are summarized in Table 4.

Table 4.

Results of performance of each exercise in the pilot study.

	Patients		Controls
	Symptomatic hand	Asymptomatic hand	Controls
Wrist Flexo – extension
Amplitude from the horizontal plane (°)	63.31 ± 5.13	62.35 ± 8.82	70.56 ± 10.66
Finger grip
Max grip distance (mm)	103.37 ± 8.94	103.25 ± 9.27	107.29 ± 13.11
Thumb range (mm)	43.60 ± 10.55	46.74 ± 10.66	49.99 ± 12.24
Index range (mm)	59.66 ± 17.38	65.21 ± 15.96	55.82 ± 10.69
Ring perimeter (mm)	127.35 ± 80.89	151.47 ± 85.76	88.66 ± 33.38
Finger separation
Thumb range (mm)	47.93 ± 18.17	46.83 ± 9.61	55.39 ± 10.28
Index range (mm)	44.33 ± 18.66	42.89 ± 16.34	37.15 ± 13.74
Ring range (mm)	71.73 ± 21.35	73.10 ± 24.49	70.28 ± 22.43
Pinky range (mm)	87.72 ± 43.09	79.15 ± 22.51	81.55 ± 20.30
Fist open and closure
Max perimeter (mm)	337.19 ± 58.51	315.44 ± 42.86	366.99 ± 35.26
Thumb range (mm)	64.17 ± 18.98	63.03 ± 14.20	75.57 ± 20.49

Most patients had mild deficits as per the clinical scales scores with many of them showing normal examination of hand motor function (i.e., 14 points in the FMA-UE). However, differences between patients and controls could be observed in most of the parameters obtained from the four exercises. Besides, data from the symptomatic and asymptomatic hand of the patients were similar, which may indicate some degree of motor dysfunction also in the asymptomatic hand in stroke patients.

Some specific parameters were of particular interest. In exercise 2, the perimeter of the involuntary movement of the ring finger while performing the thumb-index gripper, seems to be of relevance, with larger random movement in patients than in controls, which may indicate that, as a result of the motor dysfunction, patients have less control of the involuntary movements of the rest of the fingers or that involvement of all fingers is needed to successfully perform the gripper. The fist opening and closing exercise appears to be the most discriminating. Consistent differences were observed in the two parameters analyzed. On the contrary, finger separation results to be the least discriminative exercise, since no differences were found between patients and controls, except for the thumb range parameter. This exercise was included in the workout because of the capability of the device to track accurately any movement performed in the horizontal plane so that any slight difference might be detected with precision. However, the range of finger movement during the execution of this exercise is small and this may be the reason why differences between cases and controls were not found.

The correlation analysis showed poor correlations between clinical scales scores and kinematic data, which might be explained by the small sample size, but also because most patients showed normal or nearly normal scores in the clinical scales.

Discussion

This work describes the design and development of a tool capable of parameterizing kinematic data of hand movement through the optical capturing of motion. Based on this parameterization the tool is able to report the measures of the relevant components of movement that allows objective evaluation of motor performance. The tool has been tested in a relevant use case (i.e., stroke patients with mild to moderate hand motor deficit) in a pilot study that shows the ability of the tool to detect differences between stroke patients and healthy subjects with no motor impairment. Even in patients that were scored as normal by the usual clinical scales, the tool was able to detect motor deficit as they showed poorer performance compared with controls. Hand functional impairment has a highly negative impact on patient´s activities of daily living and on their quality of life, causing high healthcare and social costs. In the case of stroke patients, for instance, upper limb motor recovery is considered among the top 10 research priorities agreed upon by patients, caregivers, and health professionals.²⁸ An accurate assessment and quantification of motor deficits is crucial to optimize and personalize rehabilitation programmes³ and therefore, objective evaluation of hand motor performance is of key importance.

Advances in the development of optical motion capture technologies and new tracking algorithms have achieved adequate accuracy to measure movements that require high precision without the need for complex tools or devices that are too cumbersome or expensive.²⁹ These technologies have been used also for rehabilitation purposes and virtual reality-assisted rehabilitation programmes have been developed.^30,31 Particularly Leap motion® has been used with this purpose.^20,32,33 The NeuroData Tracker platform goes one step forward, by allowing the parameterization and analysis of the kinematic data retrieved from the tracked movement. This is important since the accurate assessment and quantification of motor deficits are crucial to determine the degree of disability and to establish individualized rehabilitation programmes, as well as to detect evolutionary changes that allow the optimization of therapies. The main advantages of the tool are its simplicity, portability, low cost and use of accessible hardware, which allows generalization of its use in any environment, at the bedside or even at home.

The pilot study suggests that the most sensitive kinematic variables appear to be the maximum amplitude from the horizontal plane for wrist flexo-extension, maximum gripper distance, index finger range and ring finger perimeter for finger-grip; thumb range, index finger range and little finger range for finger separation and maximum perimeter and thumb range for fist opening and closure. However, these results must be considered with caution since this is a pilot study including only a few patients and there is the possibility that those specific components of movement related to the kinematic variables with less discriminative capacity were not affected in this group of patients and therefore, not detected by the tool. It is of interest that the analysis of kinematics shows certain parameters that are not usually explored in the clinical setting when evaluating hand movements, that appear to be of relevance. These are derived from the involuntary movements of fingers not directly involved in the performance of the exercise being explored, such as the perimeter of the trajectory of the ring finger while performing the finger grip. Also, the tool seems to be able to detect deficits of the theoretically unaffected hand in stroke patients compared with controls. These findings need to be verified in larger trials and their clinical significance should also be confirmed. In fact, there is a biological basis that may explain such involvement, through the ipsilateral motor pathway of the pyramidal tract, but they support the ability of this tool to objectively quantify hand movements and to detect subtle changes that go unnoticed on clinical examination.

A considerable dispersion of kinematic data can be observed not only in patients, where it may be related to variability in functional deficit but also in healthy subjects. This may be a reflection of interpersonal variability in movement execution related to age, sex, or other factors and also of the sensitivity and the ability of the tool to detect this variation. However, this may add difficulties in determining the parameters of normality that are key to grade the deficit. This limitation can be overcome by analysing large cohorts of patients and by implementing algorithms for machine learning that may aid in the automatic classification of motor deficits.

It is, therefore, necessary to increase the database and to carry out clinical trials to validate the tool in a clinical setting, both to verify its usefulness for the accurate diagnosis of the deficit and in therapeutic decision-making.

Conclusion

In this paper, we propose a new software for kinematic analysis of hand movement tracked using optical technology as a tool with potential utility for quantifying motor deficit, particularly mild deficits that conventional scales may not be able to discriminate. These promising results deserve clinical validation in a larger trial.

Footnotes

Acknowledgments

The authors thank Morote Traducciones for editorial assistance.

Contributorship

MAdeL and AG conceived the study and obtained ethical approval. JR-P, LC, DL, FF, BF, EA, DH, and MAdeL were involved in patient recruitment. AG, DL, BL, and FF participated in the design and technical development of the platform; MAdeL, AG, and DL analysed data. DL wrote the first draft of the manuscript. ED-T revised critically the article. All authors reviewed and edited the manuscript and approved the final version of the manuscript.

Data availability statement

The datasets generated and analysed during the current study are available in the Figshare repository.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

This study was approved by the Ethics Committee of the University Hospital La Paz (Approval number PI – 4776).

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Invictus Plus network under grant RD16/0019/0005 and the RICORS network under grant RD21/0006/0012, Spanish Ministry of Health-Carlos III Health Institute (ISCIII) and the Next Generation EU funds (Recovery and Resilience Plan).

Informed consent

All participants signed an informed consent.

Guarantor

MAdeL and AG.

ORCID iD

María Alonso de Leciñana

Blanca Larraga-García

David Hernández

References

Katan

Luft

. Global burden of stroke. Semin Neurol 2018; 38: 208–211.

Dan

Nair

Bajaj

, et al. Rehabilitation of motor function after stroke: a multiple systematic review focused on techniques to stimulate upper extremity recovery. Front Human Neurosci|www.frontiersin.org 2016; 10: 442.

Kwakkel

van Wegen

Burridge

, et al. Standardized measurement of quality of upper limb movement after stroke: consensus-based core recommendations from the second stroke recovery and rehabilitation roundtable. J World Stroke Organization 2019; 14: 783–791.

Marsh

Lawrence

Gottesman

, et al. The NIH stroke scale has limited utility in accurate daily monitoring of neurologic Status. Neurohospitalist 2016; 6: 97–101.

Bernhardt

Kwakkel

Lannin

, et al. Consensus statements from the stroke recovery and rehabilitation roundtable standardized measurement of sensorimotor recovery in stroke trials: consensus-based core recommendations from the stroke recovery and rehabilitation roundtable*. Neurorehabil Neural Repair 2017; 31: 784–792.

Stein

Bishop

Gillen

, et al. A pilot study of robotic-assisted exercise for hand weakness after stroke. IEEE Int Conf Rehabil Robot 2011; 90: 887–894.

Stein

Bishop

Gillen

, et al. Robot-assisted exercise for hand weakness after stroke: a pilot study. Am J Phys Med Rehabil 2011; 90: 887–894.

Little

Emery

Black

, et al. Test-retest reliability of KINARM robot sensorimotor and cognitive assessment: in pediatric ice hockey players. J Neuroeng Rehabil 2015; 12: 78.

Mochizuki

Centen

Resnick

, et al. Movement kinematics and proprioception in post-stroke spasticity: assessment using the Kinarm robotic exoskeleton. J Neuroeng Rehabil 2019; 16: 146.

10.

Bressi

Santacaterina

Cricenti

, et al. Robotic-assisted hand therapy with gloreha sinfonia for the improvement of hand function after pediatric stroke: a case report. Applied Sciences 2022; 12: 4206.

11.

Yun

Dancausse

Esmatloo

, et al. Maestro: an EMG-driven assistive hand exoskeleton for spinal cord injury patients. In: 2017 IEEE international conference on robotics and automation (ICRA), 2017, pp.2904–2910.

12.

Lee

Kuo

Lin

, et al. Effects of robot-assisted rehabilitation on hand function of people with stroke: a randomized, crossover-controlled, assessor-blinded study. Am J Occup Ther 2021; 75: 1–11.

13.

Borboni

Mor

Faglia

. Gloreha – hand robotic rehabilitation: design, mechanical model and experiments. J Dyn Sys Meas Control 2016; 138: 111003

14.

Milia

Peccini

de Salvo

, et al. Rehabilitation with robotic glove (Gloreha) in poststroke patients. J Digit Med 2019; 5: 62–67.

15.

Bakhti

KKA

Laffont

Muthalib

, et al. Kinect-based assessment of proximal arm non-use after a stroke. J Neuroeng Rehabil 2018; 15: 1–12.

16.

Gutiérrez Zúñiga

Alonso de Leciñana

Díez

, et al. A new software for quantifying motor deficit after stroke: a case–control feasibility pilot study. J Front Neurol 2021; 12: 603619.

17.

Faity

Mottet

Froger

. Validity and reliability of kinect v2 for quantifying upper body kinematics during seated reaching. J Sensors (Basel) 2022; 22: 27–35.

18.

Leap Motion Controller TM , https://www.ultraleap.com/product/vr-developer-mount/.

19.

Păvăloiu

I-B

. Leap motion technology in learning. In: Edu world 7th international conference, 2017, pp.1025–1031.

20.

Aguilera-Rubio

Alguacil-Diego

Mallo-López

, et al. Use of the leap motion controller® system in the rehabilitation of the upper limb in stroke. A systematic review. J Stroke Cerebrovasc Dis 2022; 31: 106–174.

21.

Gonçalves

Souza

Carbone

. Analysis of the leap motion controller’s performance in measuring wrist rehabilitation tasks using an industrial robot arm reference. Sensors 2022; 22: 4880.

22.

Mohandes

Aliyu

Deriche

. Arabic sign language recognition using the leap motion controller. In: 2014 IEEE 23rd international symposium on industrial electronics (ISIE), Istanbul, Turkey, 2014, pp.960–965.

23.

Pongpanitanont

Charoensuk

. Leap motion ®: signal preservation and medical training system. In: The 7th 2014 biomedical engineering international conference, Fukuoka, Japan, 2014, pp.1–4.

24.

Buyuksalih

Bayburt

Buyuksalih

, et al. 3D modelling and visualization based on the unity game engine – advantages and challenges. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences 2017; 4: 161–166.

25.

Brott

Adams

Olinger

, et al. Measurements of acute cerebral infarction: a clinical examination scale. Stroke 1989; 20: 864–870.

26.

Fugl Meyer

Jaasko

Leyman

. The post stroke hemiplegic patient. I. A method for evaluation of physical performance. Scand J Rehabil Med 1975; 7: 13–31.

27.

Daniels

Williams

Worthingham

. Muscle testing: techniques of manual examination. Philadelphia: W.B. Saunders, 1946.

28.

Pollock

St George

Fenton

, et al. Top 10 research priorities relating to life after stroke–consensus from stroke survivors, caregivers, and health professionals. Int J Stroke 2014; 9: 313–320.

29.

Ballester

Antenucci

Maier

, et al. Estimating upper-extremity function from kinematics in stroke patients following goal-oriented computer-based training. J Neuroeng Rehabil 2021; 18: 186.

30.

Aminov

Rogers

Middleton

, et al. What do randomized controlled trials say about virtual rehabilitation in stroke? A systematic literature review and meta-analysis of upper-limb and cognitive outcomes. J Neuroeng Rehabil 2018; 15: 29.

31.

Mekbib

Han

Zhang

, et al. Virtual reality therapy for upper limb rehabilitation in patients with stroke: a meta-analysis of randomized clinical trials. Brain Inj 2020; 34: 456–465.

32.

Ögün

Kurul

Yaşar

, et al. Effect of leap motion-based 3D immersive virtual reality usage on upper extremity function in ischemic stroke patients. Arq Neuropsiquiatr 2019; 77: 681–688.

33.

Rehman

Ullah

Khan

, et al. Fingertip gestures recognition using leap motion and camera for interaction with virtual environment. Electronics (Switzerland) 2020; 9: 1–20.