Abstract
Background
Eye-hand coordination is an important component of accurate upper limb movement and is frequently compromised following stroke, leading to persistent functional impairments. Existing methods for the assessment and training of eye -hand coordination are resource-intensive and largely limited to research environments. Therefore, affordable sensor-based technologies may represent a feasible and engaging approach for clinical stroke rehabilitation.
Purpose
To evaluate the feasibility and concurrent validity of a gamified eye-hand coordination system using Tobii EyeX and Leap Motion for assessing visuomotor performance in individuals with chronic stroke, and to explore its associations with validated clinical outcome measures.
Methods
Prospective, single-group, longitudinal pilot study conducted at a physiotherapy rehabilitation center in Jodhpur, India. Fifteen community-dwelling individuals with chronic stroke (≥1 year post-event), aged 45 -70 years, underwent 10 weeks of standard physiotherapy, supplemented with biweekly gamified sessions using the HandEye system. Outcome measures included the Fugl-Meyer Assessment for Upper Extremity (FMA-UE), Box and Block Test (BBT), and HandEye game scores. Statistical analyses included repeated measures analysis of variance (ANOVA), Friedman tests, Spearman correlations, and linear fixed-effects modeling.
Results
Participants demonstrated significant improvements in FMA-UE (mean change = +6.86), BBT (+10.2 blocks), and HandEye scores (+4.62 points) over the 10-week period (p < .001 for all). Strong correlations were observed between HandEye scores and BBT (ρ = 0.726, p < .001), and moderate correlations with FMA-UE (ρ = 0.349, p = .019). Fixed-effects modeling revealed a significant association between HandEye scores and FMA-UE (β = 0.59, p < .001), though not with BBT. System performance remained stable across sessions, with <5% missing data for gaze and hand tracking.
Conclusion
The HandEye system integrating Tobii EyeX and Leap Motion provides a low-cost, feasible approach for assessing visuomotor function in stroke rehabilitation settings. Strong associations with manual dexterity suggest potential as an adjunct assessment tool. However, limitations in study design, low training dosage, and unvalidated sensor accuracy necessitate larger randomized trials for clinical validation.
Keywords
Introduction
Stroke remains a leading cause of long-term disability worldwide, with motor impairments in the upper limb affecting roughly 80% of survivors during the acute phase and often persisting into chronic stages.1, 2 These deficits—particularly in dexterity, fine motor control and coordination—frequently prevent survivors from regaining independence in daily tasks, such as eating or dressing. 3 Motivated by the need to improve rehabilitation outcomes in resource-constrained settings, our team pursued a novel approach to assess and explore training potential for upper limb function with accessible technology.
The coordination of visual and motor systems, often termed eye-hand coordination, is fundamental to executing precise movements, such as reaching for an object or manipulating tools. 4 Drawing from prior work, we noted that stroke-related damage to cortical or subcortical regions disrupts this interplay, resulting in slower and less accurate movements, particularly in the affected arm.5, 6 Yet, many standard rehabilitation assessments focus narrowly on motor performance, neglecting the critical visuomotor components that underpin functional tasks. This observation shaped our study’s design to prioritise integrated measurement of vision and movement.
To address this gap, we leveraged affordable technologies—Leap Motion for hand tracking and Tobii EyeX for gaze monitoring—originally developed for consumer gaming but increasingly recognised for their potential in capturing kinematic and visual data. 7 However, their performance in stroke populations with altered movement patterns requires validation against gold-standard systems. Integrating these tools into a gamified platform, we aimed to create an engaging environment that encourages repetitive, goal-directed practice, potentially supporting neuroplastic changes in motor recovery. 8 Studies suggest that gamification enhances patient motivation through interactive elements such as real-time feedback and scoring, transforming repetitive exercises into compelling challenges.9, 10
Our research developed a low-cost system that combines Tobii EyeX and Leap Motion to track gaze and hand movements, informed by established models of visuomotor control.4, 6 Over a 10-week intervention, we engaged chronic stroke survivors receiving standard physiotherapy, assessing the system’s potential as an assessment tool with exploratory training applications. By examining associations between performance metrics from our gamified tasks and validated clinical measures, specifically the Fugl–Meyer Assessment of the Upper Extremity (FMA-UE) and the Box and Block Test (BBT), we sought to evaluate their concurrent validity and associations with motor recovery in rehabilitation settings with limited resources.
Methods
Study Design
This prospective, single-group, longitudinal pilot study spanned a 10-week intervention period at a physiotherapy rehabilitation centre in Jodhpur, Rajasthan, India, from September 2022 to July 2024. The study received ethical approval from the Institutional Review Board and all participants provided written informed consent in accordance with the Declaration of Helsinki and international ethical standards.
Participants
We recruited 15 community-dwelling individuals with chronic stroke (> 1 year post-event), aged 45–70 years, using purposive sampling to ensure a range of motor impairment levels. Eligibility criteria included right-handedness, an Abbreviated Mental Test (AMT) score of ≥7, corrected visual acuity of ≥20/40, shoulder flexion ≥90°, elbow extension deficit ≤30° and Manual Muscle Testing (MMT) score >2/5 in the hemiparetic upper limb. These criteria, based on pilot testing, ensured task feasibility but may limit generalisability to broader stroke populations with more severe impairments or left-handedness. We excluded individuals with hemianopia, visuospatial neglect, severe hearing loss or significant cognitive impairments to ensure reliable engagement with the gamified system.
Intervention
Participants received daily 2-hour sessions of conventional upper limb physiotherapy, 5 days per week, totalling 100 hours over 10 weeks. The intervention combined proprioceptive neuromuscular facilitation (PNF) and functional task-oriented training. PNF techniques, based on established principles, 11 involve methods to promote muscle activation and coordination via interhemispheric pathways. Functional training consisted of repetitive, goal-directed activities—such as grasping, reaching and object transfer—designed to mimic daily tasks and enhance functional independence. Therapy was delivered by trained physiotherapists, with session logs ensuring treatment consistency.
In addition to conventional therapy, participants engaged twice weekly with the HandEye gamified system, completing three 2-minute gameplay rounds per session (6 minutes/week, totalling 6.7 hours over 10 weeks) with rest intervals to reduce fatigue. The system was designed to assess and explore training potential for visuomotor integration using Tobii EyeX for gaze tracking and the Leap Motion controller for capturing hand movements.
The Leap Motion controller is a compact universal serial bus (USB) peripheral (3 × 1.2 × 0.5 inches), positioned flat on the table in front of the monitor. It uses three infrared light emitting diode (LEDs) and two cameras to track hand and finger movements within a hemispherical field of view ranging from 25 to 600 mm. The system detects fingertips, hand velocity, direction and recognises both fingers and tool-like objects. Manufacturer specifications report sub-millimetre precision, but performance in stroke populations with tremors or altered movements was not independently validated.
The Tobii EyeX eye tracker, mounted below the computer screen, allows for moderately free head movement while providing real-time estimates of gaze positions and 3D eye location relative to the screen centre. It operates using the pupil centre–corneal reflection technique, calculating gaze direction from the angular offset between the pupil and a glint produced by infrared illumination on the cornea. The EyeX demonstrates high performance, with an accuracy of <0.6°, precision <0.25°, latency <50 ms and a sampling frequency of approximately 55 Hz.
Game Environment
The HandEye interface displayed a 3 × 3 grid on a monitor, with participants seated approximately 35 cm from the screen to optimise gaze tracking accuracy. This distance, selected based on pilot testing, may introduce postural constraints for participants with limited trunk control, potentially affecting natural coordination patterns. The Tobii EyeX, a screen-based eye tracker mounted below the display and the Leap Motion sensor, positioned on a table at an 80 cm height, were calibrated before each session using a nine-point grid protocol for Tobii EyeX and a 30-second hand detection sequence for Leap Motion. Participants scored points by aligning their gaze and hand position on a target grid cell, receiving visual feedback (a star symbol) and auditory cues (a ping sound) for successful trials (Figures 1–6). The binary scoring system captured success or failure but omitted coordination quality metrics such as movement smoothness or reaction time, limiting its reflection of functional visuomotor demands. Integration of these consumer-grade sensors relied on their respective application programming interfaces (APIs), with Tobii’s software development kits enabling gaze data access and Leap Motion’s API supporting hand tracking.
Eye Gaze Positioned, Awaiting Hand Match. The User’s Gaze Is Fixated on the Top-left Grid While the Red Outline Indicates the Required Grid Location for the Eye and the Hand. No Successful Coordination Is Achieved and the Score Remains Unchanged.
Eye Gaze Only on Target Grid. The User’s Eye Gaze Is Aligned with the Bottom-right Grid, While the Hand Position Is Not Present in the Target Area. No Point Is Scored, as Both Inputs Must Match to Earn a Point.
Hand Position Only on Target Grid. The User’s Hand Is Correctly Placed on the Top-centre Grid, but the Eye Gaze Is Not Available to Match the Same Location. The Red Outline Shows the Intended Grid Location and No Score Is Awarded Due to a lack of Coordination.
Eye Gaze and Hand Position on Different Grids. The Hand Icon Is Positioned on the Upper-centre Grid While the Eye Icon Is on the Bottom-right Grid. The Mismatch Indicates a Failed Coordination Attempt and the Score Remains Zero.
Successful Eye-hand Coordination Task Trial in the HandEye Game. Both the Eye Gaze (Tracked Via Tobii) and Hand Position (Tracked Via Leap Motion) Simultaneously Align on the Centre Grid, Indicated by the Yellow Star and Red Outline, Resulting in a Successful Match and an Increment in Score.
Participant Engage in the HandEye Game, Which Requires Simultaneous Eye Gaze and Hand Movement Coordination. The Tobii Eye X Eye-tracking Sensor Is Mounted Below the Monitor and the Leap Motion Controller Is Positioned on the Table to Capture Hand Movements. The Participant Is Seated Comfortably in Front of the System, Interacting with the Visual Interface.
HandEye System Design Rationale
The HandEye system was developed to evaluate and explore training potential for eye-hand coordination, a pivotal aspect of upper limb function often impaired post-stroke. Informed by visuomotor control models, the system requires simultaneous alignment of gaze (via Tobii EyeX) and hand position (via Leap Motion) on a 3 × 3 grid, delivering immediate visual and auditory feedback to reinforce target acquisition.4, 6 This static 2D task simplifies visuomotor coordination, omitting dynamic elements such as trajectory planning or haptic feedback critical for functional tasks, limiting its ecological validity. This design leverages the neural interplay of visual and proprioceptive feedback to promote goal-directed movements in a controlled, engaging environment, potentially supporting rehabilitation goals of fostering neuroplasticity.
Outcome Measures
FMA-UE: Administered at baseline (week 0), midpoint (week 6) and post-intervention (week 10) to evaluate motor recovery.
BBT: Conducted biweekly (weeks 0, 2, 4, 6, 8 and 10) to measure gross manual dexterity.
HandEye game score: Calculated as the average performance across three game rounds, recorded biweekly at six time points (weeks 0, 2, 4, 6, 8 and 10) to track visuomotor performance.
Statistical Analysis
Data were analysed using Jamovi (version 2.6), an open-source statistical software built on R. Descriptive statistics were reported as medians with interquartile range (IQR) for continuous variables and as frequencies and percentages for categorical variables due to non-normality in some measures. Normality of continuous variables was assessed using the Shapiro–Wilk test. Based on normality results, parametric tests (repeated measures analysis of variance (ANOVA)) were applied to FMA-UE data, while non-parametric tests (Friedman test) were used for BBT and HandEye scores when assumptions were violated (Table 1).
To evaluate the HandEye system, a multi-step framework was employed:
Convergent validity: Spearman’s rank correlation coefficients were calculated between HandEye scores and both FMA-UE (weeks 0, 6 and 10) and BBT (weeks 0, 2, 4, 6, 8 and 10) to assess associations with upper limb coordination and function. It was hypothesised that HandEye scores would show moderate to strong correlations with FMA-UE and BBT, reflecting shared motor control constructs. Responsiveness: Repeated measures ANOVA (for FMA-UE) and Friedman tests (for BBT and HandEye scores) were used to assess changes over the 10-week intervention, hypothesising that improvements in motor function would be associated with increases in HandEye scores. Predictive validity: A linear fixed-effects model was employed to assess the association of HandEye scores with motor recovery. HandEye scores were modelled as a fixed effect, participant identity document (ID) as a random effect and FMA-UE or BBT scores as dependent variables across respective time points. Model diagnostics, including residual plots and Akaike Information Criterion (AIC)/ Bayesian Information Criterion (BIC), were conducted to assess fit and overfitting. It was hypothesised that HandEye scores would be associated with FMA-UE outcomes, supporting the system’s potential as an assessment tool.
A post-hoc power analysis was conducted to assess the study’s ability to detect the FMA-UE minimal clinically important difference (minimal clinically important difference (MCID) = 5.25 points). With N = 15 and an assumed standard deviation of 8–10 points, power was approximately 30%–40%, indicating high Type II error risk.
Normality Test Results (Shapiro–Wilk) for Main Outcome Variables.
Results
The study included 15 participants with chronic stroke, with a median age of 58.0 years (IQR: 56.0–62.0). The majority were male (66.7%) and had experienced an ischaemic stroke (86.7%). Right-sided hemiparesis was observed in 60% of the cohort. The median duration since stroke onset was 20.5 months (IQR: 18.6–25.1) and participants had a median AMT score of 8.0 (IQR: 7.0–9.0), indicating preserved cognitive function. The median MMT score for the affected upper limb was 4.0 (IQR: 3.0–4.0), reflecting mild to moderate weakness (Table 2).
Participant Demographics (N = 15).
Over the 10-week intervention, participants exhibited changes in all outcome measures. FMA-UE scores increased from a mean of 26.47 (SD = 3.45) at baseline to 29.00 (SD = 4.53) at week 6 and 33.33 (SD = 5.38) at week 10. BBT scores rose from a mean of 24.59 (SD = 2.19) at week 0 to 29.73 (SD = 2.67) at week 6 and 34.79 (SD = 2.07) at week 10. HandEye game scores improved from a mean of 12.58 (SD = 1.37) at baseline to 13.85 (SD = 2.01) at week 6 and 17.20 (SD = 1.38) at week 10 (Table 3).
Mean Scores and Effect Sizes for Outcome Measures Over Time.
A repeated measures ANOVA revealed a significant effect of time on FMA-UE scores, F(2, 28) = 87.34, p < .001. Mauchly’s test confirmed sphericity violation (W = 0.088, p < .001) and the Greenhouse–Geisser correction (ε = 0.523) was applied. Post hoc comparisons (Bonferroni corrected) showed significant improvements: Week 0 vs. week 6 (p = .002), week 6 vs. week 10 (p = .003) and week 0 vs. week 10 (p < .001). For BBT, the Friedman test indicated a significant effect of time, χ²(5) = 61.4, p < .001. Post hoc analysis using the Durbin–Conover test revealed significant improvements between most time points, except between week 8 and week 10 (p = .126). Similarly, HandEye game scores showed significant improvement over time, χ²(5) = 51.4, p < .001, with significant gains between most intervals except week 8 and week 10 (p = .183).
Summary of Repeated Measures Results.
Correlation Analysis
A Spearman’s rank correlation matrix (Table 5) showed a strong positive correlation between HandEye scores and BBT scores (ρ = 0.726, 95% CI: 0.28–0.91, p < .001), indicating an association with manual dexterity. A moderate correlation was found with FMA-UE scores (ρ = 0.349, 95% CI: −0.19–0.74, p = .019), explaining only 12% of shared variance, suggesting limited convergent validity. BBT and FMA-UE scores were moderately correlated (ρ = 0.481, p < .001). The wide confidence intervals, due to the small sample (N = 15), indicate unstable estimates, requiring replication with larger samples (minimum N = 84 for stable correlations). 12
Spearman’s Rank Correlations Between HandEye Scores, BBT and FMA-UE (N = 90).
Linear Fixed-Effects Models: FMA-UE and BBT Outcomes
Initially, a linear mixed-effects model with random intercepts for participant ID was considered to assess the associations between HandEye scores and FMA-UE and BBT scores, but it exhibited overfitting (conditional R² = 0.991 for FMA-UE) and negligible between-subject variability (random intercept variance ≈ 0 for BBT), suggesting issues with the small sample size (N = 15). Consequently, simplified linear fixed-effects models were adopted for all available observations (N = 45 for FMA-UE, N = 90 for BBT). For FMA-UE, the fixed-effects model showed a significant association between HandEye score and FMA-UE (β = 0.59, SE = 0.12, t = 4.92, p < .001), with an R² of 0.34, AIC of 124.6 and BIC of 130.2. Negative interactions with week 6 (β = −0.55, SE = 0.15, t = −3.67, p = .002) and week 10 (β = −0.62, SE = 0.16, t = −3.88, p < .001) suggest the association weakened over time, possibly due to unmodeled confounders (Table 6).
For BBT, a Spearman correlation analysis revealed a strong monotonic relationship between HandEye score and BBT score (ρ = 0.728, p < .001), suggesting a potential association. However, the fixed-effects model showed no significant association between HandEye score and BBT score (β = 0.03, SE = 0.14, t = 0.21, p = .834), with an R² of 0.76, AIC of 112.5 and BIC of 118.0. Week had a significant main effect (p < .001), but HandEye × Week interactions were non-significant (p > .15). The discrepancy between the correlation and model results may reflect unmodeled time-varying effects or insufficient sample size (Table 7).
Fixed-Effects Model Predicting FMA-UE Score.
Fixed-Effects Model Predicting BBT Score.
Discussion
This pilot study explored a gamified eye-hand coordination system using Tobii EyeX and Leap Motion sensors to assess visuomotor performance in 15 chronic stroke survivors over 10 weeks. Improvements in FMA-UE (median 28–34), BBT (25–35) and HandEye scores (13–17) were observed, but the single-arm design and concurrent intensive conventional physiotherapy (100 hours vs. 6.7 hours of HandEye) prevent causal attribution to the HandEye system. 1 The large BBT improvement (10 blocks) exceeds typical chronic stroke gains (2–4 blocks), likely reflecting practice effects from repeated testing. 13
HandEye scores showed a strong correlation with BBT (ρ = 0.726, 95% CI: 0.28–0.91, p < .001) but only a moderate correlation with FMA-UE (ρ = 0.349, 95% CI: −0.19–0.74, p = .019), explaining 12% of shared variance, suggesting limited convergent validity for broader motor impairment. The HandEye task’s static 2D design lacks ecological validity for dynamic, 3D functional tasks, as it omits elements such as trajectory planning or haptic feedback. 14 Linear mixed-effects modelling showed HandEye scores were associated with FMA-UE outcomes (β = 0.562, p < .001), but overfitting (conditional R² = 0.991) and contradictory negative interactions (β = −0.558, p = .001) suggest model misspecification in a small sample (N = 15). The non-significant HandEye-BBT association (β = 0.022, p = .879), despite a strong correlation (ρ = 0.726) further indicates analytical issues, requiring larger samples.
FMA-UE improvements (six points) align with expected outcomes from intensive conventional therapy (4–7 points), suggesting limited additional benefit from the HandEye system. The 10-week duration may be insufficient for meaningful neuroplastic changes in chronic stroke, as standard trials use 12–24 weeks. 15 The low HandEye dosage (6 minutes/week) is likely inadequate for motor learning, which requires hundreds of repetitions. 16
The use of consumer-grade sensors offers a cost-effective alternative to high-end systems, but their performance in stroke populations remains unvalidated, with inconsistent gaze data at extended distances and transient finger detection issues (4.8% and 3.2% missing data, respectively). These limitations, noted during pilot testing, highlight the need for validation against gold-standard systems (e.g., high-speed cameras) and test-retest reliability studies. The system’s engaging design, with visual and auditory feedback, likely enhanced participant adherence, aligning with prior evidence on gamification.10, 17 However, motivational bias from technological novelty may inflate clinical performance, representing a Hawthorne-like effect. 13
Limitations
This study has several limitations. The small sample size (N = 15) provided inadequate statistical power, with a post-hoc analysis yielding 38% power to detect the FMA-UE minimal clinically important difference (MCID = 5.25 points), based on an observed mean difference of 6.86 points and a standard deviation of approximately 5.86. This suggests a high risk of Type II error, requiring a sample size of approximately 50–60 participants per group for 80% power at α = 0.05. 18 The pilot nature of this study justifies the limited sample, as it aimed to explore feasibility and preliminary associations rather than establish definitive efficacy, particularly in a resource-constrained setting where recruiting larger cohorts was challenging. However, this restricts the generalisability of findings to broader stroke populations. The single-arm design and concurrent intensive conventional physiotherapy (100 hours) preclude causal attribution of improvements to the HandEye system (6.7 hours). The large BBT improvement (10.2 blocks) may reflect practice effects from repeated testing and the static HandEye task lacks ecological validity for dynamic functional tasks. Consumer-grade sensors’ unvalidated performance in stroke populations, with 4.8% gaze and 3.2% hand data missing (estimated from pilot testing), compromises data integrity. The absence of multiple baseline assessments fails to confirm performance stability and the 10-week duration may be insufficient for meaningful neuroplastic changes, which typically require 12–24 weeks.22 The right-handedness requirement and lack of follow-up assessments limit generalisability and durability evaluation, respectively. Qualitative data on user experience were not collected, hindering usability insights.
Conclusion
This pilot study suggests the HandEye system, using Tobii EyeX and Leap Motion sensors, is a feasible tool for assessing visuomotor performance in chronic stroke rehabilitation, with strong associations to BBT (ρ = 0.726) but limited convergence with FMA-UE (ρ = 0.349). Its affordability supports potential use in resource-limited settings, but the single-arm design, unvalidated sensors and small sample size limit conclusions about efficacy. Future randomised controlled trials with larger samples (N ≥ 45), dynamic task designs, validated sensors, multiple baseline assessments and 16–24 weeks durations with follow-up are needed to confirm clinical utility and establish causality.
Footnotes
Acknowledgements
We gratefully acknowledge Shree IG Hospital, Jodhpur, for providing institutional resources and infrastructure support and Human–Machine Technologies Inc. for their generous provision of the Hand–Eye coordination software and associated technical assistance.
Authors’ Contribution
Nikhil Choudhary: Conceptualisation, study design, methodology, software integration, data analysis, interpretation of results, manuscript drafting and critical revision.
Priya Rawat: Participant recruitment, data collection, assistance in intervention sessions and contribution to manuscript review.
Consent for Publication
Not Applicable.
Data Availability
Data is available upon reasonable request from the authors.
Statement of Ethics
The study received ethical approval from the Institutional Review Board via (letter SIG/SPL/JU/2022/956 dated 17/08/2022) and all participants provided written informed consent in accordance with the Declaration of Helsinki and international ethical standards.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The authors received no financial support for the research, authorship and/or publication of this article.
Informed Consent
All participants provided written informed consent in accordance with the Declaration of Helsinki and international ethical standards.
