Sage Journals: Discover world-class research

Abstract

Objective

Conventional occupational therapy for stroke rehabilitation often relies on pegboards and paper-based tasks, which lack digital, objective, and gamified assessment capabilities. To address this gap, we developed a gamified digital randomized target pressing task (RTPT) system as a user-centered digital platform for assessing attention and hand function, and evaluated its reliability, validity, and usability in patients with stroke. A previous pilot study was conducted to refine the system design and assessment protocol.

Methods

A total of 33 stroke patients completed the RTPT alongside established measures, including the Box and Block Test (BBT), Loewenstein Occupational Therapy Cognitive Assessment (LOTCA), and Chu's Attention Test (CAT). Twenty-six participants repeated the RTPT to evaluate test–retest reliability. Usability and acceptance were assessed in 30 patients and 16 healthcare professionals using the System Usability Scale (SUS) and Technology Acceptance Model (TAM), capturing both user engagement and perceived system usefulness.

Results

Performance with the non-paretic hand (sound side correct count (SCC)) correlated strongly with attention scores (CAT, r = .649, p < .001) and visual perception/construction (LOTCA). Paretic hand performance (affected side correct count (ACC)) correlated with manual dexterity (BBT, r = .67, p < .001) and functional abilities in self-care. Test–retest reliability was high (intraclass correlation coefficients: SCC = 0.937, ACC = 0.804). SUS scores indicated above-average usability (median 86.3), with strong internal consistency (Cronbach's α: TAM = .959) and high SUS–TAM correlations. Participants reported high engagement and satisfaction with the digital platform.

Conclusion

The RTPT provides a reliable, valid, and user-friendly digital assessment of attention and hand function in stroke rehabilitation. By integrating cognitive and motor evaluation within a gamified digital platform, it demonstrates potential as a tool to support individualized therapy planning, monitor rehabilitation progress, and enhance patient engagement in clinical practice.

Keywords

Digital health stroke rehabilitation gamified assessment cognition hand function usability machine learning

Introduction

Stroke remains one of the leading causes of long-term disability worldwide, affecting approximately 94 million people, or about 1.1% of the global population.^1,2 Post-stroke cognitive impairment and dementia contribute substantially to morbidity, with up to 40% of patients experiencing cognitive decline within the first year after stroke.³ Moreover, around 10% present with pre-existing dementia, another 10% develop new-onset dementia after a first stroke, and over one-third experience cognitive deterioration following recurrent events.⁴ In parallel, upper limb impairment occurs in 50–80% of acute stroke survivors and often persists in half of patients during the chronic phase.⁵ These combined motor and cognitive deficits highlight the need for integrated systems that can objectively assess and monitor both cognitive and motor recovery.

Conventional rehabilitation tools such as pegboards and paper-based tasks are widely used to support cognitive training, visual perception (VP), visuomotor construction (VC), and hand function, but they provide only limited quantitative feedback and lack digital, interactive, and user-centered assessment capabilities. Compared with existing digital rehabilitation tools, the randomized target pressing task (RTPT) system not only incorporates features such as gamification, real-time measurement, and long-term monitoring, but also innovatively integrates both cognitive and hand function assessment. To address the limitations of conventional tools, our team developed this gamified digital RTPT (Taiwan ROC Invention Patent No. I859077) as a digital health-oriented platform for stroke rehabilitation. It features a timed, interactive, game-like interface designed to enhance patient engagement, attention, and adherence to therapeutic activities. The system can detect, train, and evaluate patient performance while capturing session data to support adaptive, individualized therapy planning.^1,2 A demonstration video of the testing procedure is available at the following link: https://cloudhd.cgmh.org.tw/navigate/a/#/s/DC87C8CEFA8243B8AFD16104B74140976BL

This study aimed to evaluate the reliability, validity, and usability of the RTPT system as a user-centered digital assessment platform for stroke rehabilitation, with a focus on quantifying cognitive performance and hand function in a gamified, interactive format. We also explored the potential of AI-driven analytics and performance data to inform individualized rehabilitation planning and adaptive therapy. We hypothesized that, in hemiparetic patients, performance of the non-paretic upper limb during RTPT primarily reflects cognitive domains such as attention, VP, and VC, whereas performance of the paretic upper limb reflects motor function alongside cognitive demands. Differences between the limbs were proposed as an objective digital indicator of impaired hand function. Finally, given the gamified and digitalized design with clinical applicability, the RTPT was expected to demonstrate high usability, engagement, and acceptance among both patients and healthcare professionals.

Method

Study design

This pilot and feasibility study aimed to evaluate the usability and clinical applicability of the RTPT system in stroke rehabilitation. Given the exploratory nature of the study and its focus on individualized assessment rather than efficacy testing in a large clinical trial, no standardized reporting protocol (e.g., CONSORT or SPIRIT) was applied. All study procedures, measurement methods, and analyses are reported in detail to ensure transparency and reproducibility.

Following this framework, the study employed a prospective, multi-phase observational design consisting of three sequential phases to evaluate the clinical validity, reliability, and usability of the RTPT system in a rehabilitation setting. Phase 1 focused on clinical validity testing, Phase 2 examined test–retest reliability, and Phase 3 assessed system usability from both patient and healthcare professional perspectives.

Participants

The study was conducted at the Department of Physical Medicine and Rehabilitation, New Taipei Municipal Tucheng Hospital (Chang Gung Medical Foundation), a regional rehabilitation center. Participants were recruited through physician referral, based on clinical evaluation and suitability for engagement with the digital RTPTs. Patients were invited to participate and informed consent was obtained, respecting their willingness to join the study. Participants were required to understand and follow at least a three-step verbal command and demonstrate sufficient cooperation to engage with the digital RTPTs. Eligibility also required a minimum hand grip muscle strength of Grade 2/5 and Brunnstrom Stage IV or V in both the proximal and distal segments of the affected upper limb. Patients were excluded if they had severe cognitive impairment, severe aphasia preventing task comprehension, uncontrolled medical conditions, or other neurological or musculoskeletal disorders that could interfere with upper limb task performance.

Outcome measures and assessment tools

Functional Assessment: Functional status was evaluated using the Barthel Index (BI), which assesses independence in activities of daily living. The BI score ranges from 0 to 100, with higher scores indicating greater independence, and has been validated for use in stroke populations.^3,4 Normative data from previous stroke studies were used for reference. Consistent with the study's focus on upper-limb and hand function, only the feeding (0–10) and grooming (0–5) subscales were included in the analysis. Dressing was excluded because it involves additional factors, such as lower-limb coordination and balance, which were outside the scope of this study. The total BI score and manual muscle testing (MMT) of shoulder flexion and hand grip in the affected upper extremity were also recorded to complement the functional assessment.

RTPT System Measures: The gamified digital RTPT system was used to assess cognitive and hand function performance through a timed, interactive, game-like interface designed to enhance patient engagement, attention, and adherence to therapeutic activities. Each session was set to a fixed duration of five minutes, based on prior usability testing indicating that this duration provides sufficient measurement sensitivity while minimizing fatigue and maintaining attention in stroke patients. The system enables real-time capture of performance data to support adaptive and individualized therapy planning.

Ethics

This study was approved by the Institutional Review Board of Chang Gung Memorial Hospital (IRB No. 202301197A3C601), and all participants provided written informed consent in accordance with the Declaration of Helsinki.

Baseline assessment and study procedure

At baseline, all participants underwent a standardized assessment protocol conducted by trained rehabilitation professionals. Baseline evaluations included anthropometric measurements, MMT (using the Medical Research Council scale) of shoulder flexion and hand grip strength, Brunnstrom stage classification of the affected upper limb, and functional assessment using the BI (total score and sub scores for feeding and grooming).

To minimize the potential influence of fatigue on performance-based outcomes, baseline clinical assessments were completed prior to the RTPT. Adequate rest periods were provided between assessments when necessary, and all RTPTs were administered in a single session following completion of baseline measurements. The assessment sequence was kept consistent across participants to ensure comparability of results.

A total of 33 patients with hemiparetic stroke and 16 healthcare professionals were enrolled. The sample size was determined based on feasibility considerations and prior psychometric and usability studies in stroke rehabilitation, which commonly include sample sizes ranging from 20 to 40 participants for exploratory validation and reliability analyses. Given the exploratory nature of this study and its primary aim to examine initial validity, reliability, and usability of the RTPT system, a formal a priori sample size or power calculation was not performed.

The study protocol comprised three sequential phases designed to evaluate the RTPT system as a user-centered digital health platform, focusing on its clinical validity, reliability, and usability in rehabilitation settings.

Phase 1: Validity testing

In the first phase, each patient completed the RTPT alongside established reference measures, including the Box and Block Test (BBT), Loewenstein Occupational Therapy Cognitive Assessment (LOTCA), and Chu's Attention Test (CAT), administered during two visits within a one-week interval. All RTPT sessions were conducted on the digital platform, which automatically captured response accuracy, timing, and task completion metrics in real time. These digital records were then analyzed to model individual cognitive and motor performance trends, supporting the potential for AI-driven personalized rehabilitation recommendations.

Phase 2: Test–retest reliability

In the second phase, 26 of the original 33 stroke participants completed a test–retest assessment using the RTPT system across two sessions, scheduled during routine follow-up visits with an interval of 3–7 days. This interval was chosen to minimize learning and fatigue effects while ensuring stable clinical conditions.

The RTPT system automatically recorded high-resolution performance data, which were used to examine intra-individual variability and test–retest reliability. Intraclass correlation coefficients (ICCs) and Spearman correlation coefficients were calculated to quantify measurement stability.

Phase 3: Usability assessment

Formal usability evaluation of the RTPT digital platform was conducted with 30 stroke patients and 16 healthcare professionals, including five physiatrists, six occupational therapists, and five certified caregivers, using the System Usability Scale (SUS) and Technology Acceptance Model (TAM) questionnaires.

Prior to the formal usability assessment, a pretesting phase was conducted from February 1–28, 2025, involving five stroke patients and three rehabilitation professionals. This pretest aimed solely to identify usability and ergonomic issues and guide system refinement; data from this phase were not included in the formal analysis.

Of the 33 patients initially enrolled in Phase 1, 26 participated in Phase 2. Four of the seven patients who did not complete Phase 2 due to scheduling conflicts were able to participate in Phase 3, resulting in a final usability sample of 30 patients. This phase captured both quantitative and qualitative user perspectives on the digital interface, gamified task design, and overall system acceptability, informing potential refinement of AI-driven personalization and adaptive features within the RTPT platform.

SUS

The SUS consists of 10 items rated on a 5-point Likert scale (1 = strongly disagree to 5 = strongly agree), yielding a total score ranging from 0 to 100, with higher scores indicating better perceived usability. The SUS has demonstrated high reliability and validity across a wide range of healthcare applications, including digital health applications, electronic health record systems, and stroke rehabilitation technologies.^5,6 In digital health research, a score of 68 is commonly used as a benchmark for acceptable usability, with scores above this threshold considered above average. Moreover, the SUS is the most frequently used standardized usability questionnaire in stroke-related studies, and its applicability is supported by recent scoping reviews of usability assessment tools in neurological rehabilitation.^7,8

RTPT system

The RTPT system, developed by our team as a tablet-based digital assistive assessment tool, is implemented on a customized interface with an ergonomic stylus (Figure 1).

Figure 1.

Randomized target pressing task (RTPT) system.

Once the task begins, four color-coded circular targets are presented sequentially in random order and at randomized screen locations. Participants are instructed to select the corresponding color-coded cylindrical stick and maintain stable contact with the target to register a valid response. Upon successful contact, the next target appears with a newly randomized color and location, creating a gamified, interactive experience designed to maintain attention and engagement.

Each testing session was administered once using the non-paretic (sound) upper limb and once using the paretic (affected) upper limb. During each five-minute session, the digital platform automatically recorded response accuracy, response time, and total attempts, eliminating the need for manual scoring. The total testing time per participant was therefore 10 min (2 sessions × 5 min).

For each session, the system extracted the following digital performance metrics:

Single-Choice correct count (SCC)

Single-Choice total count (STC)

Sound side correct ratio (SCR = SCC/STC)

Affected side correct count (ACC)

Affected side total count (ATC)

Affected side correct ratio (ACR = ACC/ATC)

All performance data were stored in the digital platform for real-time analysis, enabling objective, reproducible, and quantitative assessment of cognitive and motor function in stroke patients. These rich longitudinal datasets can be further processed to model individual performance trajectories, predict recovery patterns, and generate personalized rehabilitation recommendations. The system thus supports adaptive therapy planning, real-time feedback, and the potential for AI-driven optimization of rehabilitation strategies over repeated sessions, emphasizing clinical applicability and patient-centered digital intervention.

CAT

The CAT is a standardized paper-and-pencil assessment designed to evaluate attention functions and has been widely applied in clinical, psychiatric, and neurological settings. It has demonstrated excellent test–retest reliability.⁹ In the present study, the CAT was used as a reference standard to validate the cognitive assessment component of the RTPT digital platform. During the formal 10-min testing session, participants counted asterisks embedded in 100 randomized character strings. Cognitive performance metrics included total attempted items (T), correct responses (S = T − E), and error rate (ER = E/T), consistent with the previously defined abbreviations. These measures provided a benchmark for evaluating the RTPT's ability to capture attention-related cognitive processes within a digital, gamified assessment environment.

LOTCA

The LOTCA is a standardized, performance-based tool assessing fundamental cognitive abilities relevant to daily functioning, including orientation, visual–spatial perception, visuomotor organization, and thinking operations. It has demonstrated good reliability and validity.¹⁰ In this study, the LOTCA served as a reference standard for evaluating visuospatial and visuomotor functions in comparison with the RTPT digital platform. Based on our study hypothesis, only the VP and VC domains were analyzed, with domain scores calculated as the average of the respective item scores. These measures provided a benchmark for validating the RTPT's ability to digitally capture visuospatial and visuomotor performance in stroke rehabilitation.

BBT

The BBT is a standardized tool for assessing unilateral gross manual dexterity and is widely used in neurological and rehabilitation populations. It demonstrates high inter-rater and test–retest reliability.¹¹ In the present study, participants performed the BBT with each hand separately, with scores recorded for the non-paretic (SS) and paretic (SA) hands. The BBT served as a reference standard to validate the hand function component of the RTPT digital assessment platform, providing a benchmark for comparison of manual dexterity performance captured through the gamified, interactive digital task. A demonstration video of the BBT testing procedure is available at the following link: https://cloudhd.cgmh.org.tw/navigate/a/#/s/2D96A00DCEFF4CD48101DA1ADE91C6D66BL

Usability assessment – TAM and SUS

Usability of the RTPT digital platform was evaluated to assess its real-world applicability in stroke rehabilitation, identify potential operational challenges, and guide iterative system refinement.¹² Pretesting involved five stroke patients and three rehabilitation professionals, while the formal assessment included 30 stroke patients and 16 healthcare professionals. Stroke patients’ educational backgrounds ranged from elementary to graduate levels, reflecting diverse user characteristics, and healthcare professionals—including occupational therapists, physiatrists, and certified caregivers—each had over five years of clinical experience.

System usability was assessed using two validated instruments: the TAM and the SUS,¹³ as recommended by Davis (1989)and Tullis and Stetson (2004),¹⁴ respectively. The TAM evaluates perceived usefulness (PU) and perceived ease of use (PEU), which influence users’ attitudes and intention to engage with digital systems.^15,16 In this study, the TAM questionnaire employed a 7-point Likert scale (1 = strongly disagree; 7 = strongly agree), with all items positively worded so that higher scores indicate more favorable perceptions of the system. To maintain scoring consistency with the SUS and reduce missing data, participants were instructed to select the midpoint (4) if uncertain. These usability measures provided quantitative and user-centered evidence for the RTPT's acceptance, engagement potential, and suitability as a digital rehabilitation assessment tool.

Both the TAM and SUS were administered to capture comprehensive user perceptions of the RTPT digital platform, including usability, learnability, and overall acceptance among both patients and healthcare professionals. Following SUS administration, participants also rated learning task readiness and overall experience, providing additional insight into engagement and perceived ease of integrating the digital platform into rehabilitation activities.

The SUS is recognized for its sensitivity to detect usability differences even with small sample sizes,^17,18 making a sample of 16 healthcare professionals sufficient for robust, user-centered evaluation. In the SUS, items Q1, Q3, Q5, Q7, and Q9 are positively worded, whereas items Q2, Q4, Q6, Q8, and Q10 are negatively worded and were reverse-scored prior to calculation using the standard formula:

Recoded score = 5 - original score

(1)

This scoring procedure ensures that higher SUS scores consistently reflect more favorable perceptions of system usability. During usability testing, participants interacted with the RTPT system under the supervision of a researcher, with any difficulties or questions recorded in real time to capture practical usability issues.

Primary outcomes captured quantitative measures of usability, ease of use, and technology acceptance through the TAM and SUS questionnaires.^15,16 Secondary outcomes included observational notes and voluntary open-ended feedback, providing qualitative insights into user experience and engagement. After hands-on interaction, participants completed both questionnaires, offering complementary qualitative and quantitative data to inform system refinement and potential integration of the RTPT digital platform into clinical rehabilitation.

Statistics

All statistical analyses were conducted using R (version 4.4.1; RStudio, version 2014.12.0)^19,20 and IBM SPSS Statistics 22. Continuous variables were expressed as median values with interquartile ranges (first and third quartiles).

To evaluate the validity of the RTPT digital assessment platform, Spearman correlation analyses were performed comparing RTPT parameters with BBT, CAT, and LOTCA scores. Partial correlation and correlation network analyses were additionally applied to explore potential direct and indirect associations among cognitive and motor measures, highlighting relationships relevant to stroke rehabilitation.

Test–retest reliability of the RTPT was examined using both Spearman correlation analysis and the ICC, calculated using a two-way mixed-effects model with absolute agreement, providing an indicator of measurement stability for repeated digital assessments.

Internal consistency of the TAM and SUS instruments was assessed using Cronbach's alpha, with values ≥0.70 considered acceptable for scale reliability.^21,22 System usability was further evaluated by comparing SUS scores to the established benchmark of 68, with higher scores indicating above-average usability.²¹ Scores above this threshold suggest that participants generally perceived the RTPT digital platform as easy to use and satisfactory, providing a practical reference for user-centered design in digital rehabilitation interventions.

Finally, a correlation matrix was generated for all instruments assessing perceived usability to examine concurrent validity. Significant positive correlations among these measures would indicate that the instruments capture a common underlying construct, supporting their applicability for evaluating usability and technology acceptance in the context of clinical rehabilitation.

Result

A total of 33 stroke patients were included in the study (median age: 67 years; 58% with left-sided hemiparesis). Baseline demographic and clinical characteristics for all participants are presented in Table 1 (N = 33).

Table 1.

Basic demographic and clinical characteristics of participants (N = 33).

Variable/characteristic	Value (n=33)
Male, n (%)	20 (61%)
Female, n (%)	13 (39%)
Age, years	67 (58, 72)
Height, cm	165 (156, 170)
Body weight, kg	62 (54, 70)
Stroke etiology
Infarction, n (%)	12 (36%)
Hemorrhage, n (%)	21 (64%)
Stroke lesion location, n (%)
Cerebral cortex	13 (39%)
Basal ganglia	5 (15%)
Thalamus	4 (12%)
Brainstem and cerebellum, n (%)	11 (33%)
Side of hemiparesis, n (%)
Left	19 (58%)
Right,	14 (42%)
Comorbidity, n (%)
Hypertension	22 (67%)
Diabetes	13 (39%)
hyperlipidemia	3 (9%)
Coronary artery disease	6 (18%)
Barthel index
Feeding score	10 (10, 10)
Grooming score	5 (3.8, 5)
Total score	90 (65, 100)
Manual Muscle Testing of the affected side (n = 25)
Upper proximal (shoulder flexion), n (%)
Grade 5	4 (16)
Grade 4	14 (56)
Grade 3	7 (28)
Upper distal (finger flexion), n (%)
Grade 5	4 (16)
Grade 4	13 (52)
Grade 3	8 (32)

Note: Data were expressed as median (first quartile, third quartile) or count (%). Manual muscle strength was assessed by Medical Research Council scale, 0 (no contraction) to 5 (normal strength). MMT data were available for 25 participants with complete shoulder and finger flexion measurements.

Upper limb motor function was assessed using MMT in participants who were able to complete the evaluation. Among these, MMT grades ranged from 3 to 5 for shoulder and finger flexion, indicating mild-to-moderate impairment. MMT data were available for 25 participants for shoulder flexion and 25 participants for finger flexion. Missing data were attributable to fatigue, pain, or inability to cooperate during assessment.

Validity

Validation results for the RTPT digital assessment platform are summarized in Table 2. For cognitive assessment, performance on the non-paretic side (SCC, STC, and SCR) was analyzed to minimize the confounding effects of motor impairment. These measures demonstrated significant correlations with CAT correct responses (S) and LOTCA subtests for VP and VC (Table S1). Variance inflation factors were all below 5, indicating minimal multicollinearity among predictors (Table S2).

Table 2.

Performance on attention and hand function assessments.

Test/subtest	Unit	Median (first, third quartile)
RTPT
SCC	Count	94 (70, 110)
STC	Count	113 (87, 128)
SCR	–	0.88 (0.74, 0.91)
ACC	Count	44 (33, 56)
ATC	Count	74 (60, 95)
ACR	Count	0.59 (0.47, 0.79)
STC—ATC	Count	30 (7, 48)
SCC—ACC	Count	45 (19, 60)
Chu's Attention Test
Total items (T)	Count	36 (29, 56)
Errors (E)	Count	2 (1, 2)
Correct responses (S)	Count	37 (26, 53)
Error rate (R)	-	3.70 (1.72, 8.33)
LOTCA
VP	Score	3.7 (3.7, 4.0)
VC	Score	4.5 (3.6, 4.8)
BBT
SS	Count	42 (37, 53)
SA	Count	25 (18, 31)
SS-SA	Count	17 (11, 25)

Note: Data are expressed as median (first quartile, third quartile) or count (% as applicable). RTPT: Randomized Target Pressing Task System; SCC: Single choice correct; STC: single target correct; SCR: single choice reaction; ACC: alternate choice correct; ATC: alternate target correct; ACR: alternate choice reaction; LOTCA: Loewenstein Occupational Therapy Cognitive Assessment; VP: visual perception; VC: visual cognition; BBT: Box and Block Test; SS: strong side; SA: affected side.

Correlation network analysis (Figure 2(a)) revealed that SCC was more strongly associated with S, VP, and VC than STC or SCR, reflecting the sensitivity of the RTPT digital platform in capturing attention and visuomotor cognitive domains. Partial correlation analysis (Figure 2(b)) further highlighted a particularly strong association between SCC and S, as illustrated in Figure 2(c), supporting the platform's validity for cognitive assessment in stroke rehabilitation.

Figure 2.

(a–c) Correlation network and partial correlation analysis of RTPT cognitive metrics.

For hand function, ACC performance on the affected limb was strongly correlated with BBT scores (r = .670) and SS-SA performance (r = .754) (Figure 3).

Figure 3.

Correlation of ACC performance on the affected limb with hand function. Panel A: Correlation between ACC and BBT scores (r = .670). Panel B: Correlation between ACC and SS-SA scores (r = .754).

ACC was also positively associated with hand grip strength and functional measures, including feeding and grooming sub scores and the total Barthel Index (Table 3).

Table 3.

Correlation matrix of RTPT with Box and Block Test (BBT), self-care abilities, and muscle strength

Variable/statistic	ACC	ACR	BI	Feeding	Grooming	MMT-HG
r	.670**	.392*	.446**	.465**	.498**	.370*
p	<.001	.024	.009	.006	.003	.034

Note: Only statistically significant correlations (p < .05) between RTPT measures and clinical outcomes are presented here. Complete correlation results, including non-significant and derived measures (ATC, STC-ATC, SCC-ACC), are provided in Supplementary Table S3. r = Spearman's rank correlation coefficient; p = p–value, two–tailed; *P < .05, **P < .01.

Reliability

Test–retest reliability of RTPT digital assessment performance is presented in Table 4. ICCs were 0.937 for SCC, 0.804 for ACC, and 0.933 for SCC–ACC, all exceeding the commonly accepted threshold of 0.75 for clinical measurements.²³ Values above 0.8–0.9 indicate good to excellent reliability, confirming the RTPT platform's stability in capturing both cognitive and motor performance across repeated sessions.²⁴ Spearman's rank correlation coefficients were consistent with these findings (SCC = 0.880, ACC = 0.713, SCC–ACC = 0.861), further supporting the reproducibility and robustness of RTPT measures for clinical and research applications in stroke rehabilitation.

Table 4.

Test–retest reliability of RTPT (N = 26).

Variable	ICC	95% CI lower	95% CI upper	Spearman rho	p
SCC	0.940	0.865	0.973	0.880	<.001
STC	0.917	0.814	0.963	0.871	<.001
SCR	0.843	0.649	0.929	0.616	.001
ACC	0.804	0.568	0.912	0.713	<.001
ATC	0.689	0.321	0.859	0.566	.003
ACR	0.882	0.735	0.947	0.736	<.001
STC—ATC	0.845	0.657	0.930	0730	<.001
SCC—ACC	0.934	0.854	0.970	0.861	<.001

Note: ICC: intraclass correlation coefficient; rho: Spearman's rank correlation coefficient.

Usability assessment

Pretest

Pretesting of the RTPT digital platform was conducted with five stroke patients and three rehabilitation professionals to identify usability and ergonomic issues prior to the formal evaluation.

SUS scores ranged from 70.0 to 100.0, with a median of 81.8 (IQR: 75.9–91.5), exceeding the benchmark of 68 and indicating usability in the “Good” to “Excellent” range. Subscale analysis demonstrated that usability items (Q1–Q8) had a median of 79.0 (IQR: 72.1–89.4), whereas learnability items (Q9–Q10) were higher (median 100.0; IQR: 89.3–100.0), suggesting that the system was not only highly usable but also particularly easy to learn (Table 5). Observations from this pretest informed refinements to the system prior to the formal usability evaluation.

Table 5.

SUS scores: total, subscale, and item-level data.

Phase	Sample size (patient/healthcare)	Total SUS score	Usability score	Learning score
Pretest	5/3	81.8 (75.9, 91.5)	79.0 (72.1, 89.4)	100.0 (89.3, 100.0)
Formal	30/16	86.3 (81.1, 91.8)	85.7 (78.6, 89.7)	100.0 (96.0, 100.0)
Item-level SUS scores (formal test)
Item	Content			Score
Q1	I think I would like to use this tool frequently.			3.3 (2.6, 3.3)
Q2	I found the tool unnecessarily complex.			3.3 (2.6,4.0)
Q3	I thought the tool was easy to use.			4.0 (3.3,4.0)
Q4	I think that I would need the support of a technical person to be able to use this system.			4.0 (4.0,4.0)
Q5	I found the various functions in this tool were well integrated.			3.3 (2.6,3.8)
Q6	I thought there was too much inconsistency in this tool.			3.6 (3.6,4.0)
Q7	I would imagine that most people would learn to use this tool very quickly.			4.0 (3.3,4.0)
Q8	I found the tool very cumbersome to use.			4.0 (4.0,4.0)
Q9	I felt very confident using the tool.			4.0 (2.6,4.0)
Q10	I needed to learn a lot of things before I could get going with this tool.			4.0 (4.0,4.0)

Note: Data were expressed as median (first quartile, third quartile). SUS: System Usability Scale.

Formal test

During formal usability testing of the RTPT digital platform, the median SUS score increased to 86.3 (IQR: 81.1–91.8), compared with the pretest median of 81.8, indicating further improvement in perceived usability (Table 5). According to standardized SUS interpretation criteria,²⁵ this score corresponds to a rating between “Good” and “Excellent,” reflecting high acceptance and favorable perception among both patients and healthcare professionals.

Subscale analysis demonstrated that usability items (Q1–Q8) had a median of 85.7 (IQR: 78.6–89.7), whereas learnability items (Q9–Q10) achieved a median of 100.0 (IQR: 96–100), indicating that the system was particularly easy to learn in addition to being highly usable. Average item scores ranged from 2.90 to 3.83. After reverse scoring, Q8 (“I found the tool very cumbersome to use”) scored highest, indicating strong disagreement, while Q1 (“I think I would like to use this tool frequently”) scored lowest, highlighting areas for potential engagement enhancement.

Internal consistency was moderate for SUS (Cronbach's α = .634) and high for TAM (Cronbach's α = .959), with subscale reliability of 0.835 (PEU) and 0.969 (PU), demonstrating robust instrument reliability for evaluating technology acceptance. Concurrent validity between SUS and TAM was strong, with correlation coefficients ranging from 0.568 to 0.952 (Table 6), supporting the applicability of these measures for assessing usability, learnability, and acceptance of digital rehabilitation platforms in clinical practice.

Table 6.

Correlation table of all the instruments assessing perceived usability.

Construct		SUS	TAM	PU	PEU	LTR	OExp
SUS	r	—
	p
TAM	r	.644**	—
	p	<.001
PU	r	.530**	.800**	—
	p	<.001	<.001
PEU	r	.231	.602**	.510**	—
	p	.123	<.001	<.001
LTR	r	.517**	.786**	.568**	.381**	—
	p	<.001	<.001	<.001	<.001
OExp	r	.874**	.920**	.753**	.506**	.730**	—
	p	<.001	<.001	<.001	<.001	<.001

Note: r = Spearman's rank correlation coefficient; p = p-value, two-tailed. PEU: perceived ease of use; PU: perceived usefulness; LTR: learning task readiness; OExp: overall experience; SUS: System Usability Scale; TAM: Technology Acceptance Model.

**p < .01.

During Phase 1 pretesting, several usability and ergonomic issues were observed, including excessive muscle tone, wheelchair–table mismatch, occasional sensor misrecognition, and unstable alignment of the assistive frame. These observations guided subsequent adjustments in the formal testing phases to optimize task performance and ensure reliable data capture. Detailed observations and solutions are provided in Supplemental Table S4.

Discussion

Conventional occupational therapy tasks, such as vertical pegboard sorting with color cues, often provide limited quantitative feedback and minimal engagement, with stroke survivors reporting feelings of boredom and unproductive time during non-therapy periods, as highlighted by Kenah et al. (2003), which may reduce patient attention and motivation.²⁶ The RTPT digital platform integrates a timed, gamified, and interactive interface, designed to enhance patient engagement, adherence, and motivation during stroke rehabilitation. Furthermore, the system captures detailed, session-by-session digital performance data, which can be leveraged with machine learning algorithms to predict individual patient performance trajectories and support personalized, data-driven rehabilitation strategies. Similar to the effects reported by Sora Park et al. (2024) in robot-assisted gait training for children with brain injury, where variations in training parameters influenced exercise intensity and muscle activity, the RTPT platform provides targeted, adaptive interventions that may optimize motor outcomes in stroke rehabilitation.²⁷ Taken together, evidence from both stroke and non-stroke populations supports the relevance of adaptive, data-driven digital rehabilitation approaches for improving engagement and motor performance.

In this study, non-paretic hand performance, indexed by SCC, demonstrated significant associations with attentional performance and visuospatial/visuomotor domains, as measured by CAT and the VP and VC subtests of the LOTCA. The strongest relationship was observed for attention-related measures, underscoring the RTPT platform's sensitivity in assessing cognitive function beyond motor impairment. ACC was also significantly correlated with paretic-hand dexterity, hand grip strength, self-care abilities (feeding and grooming), and overall functional independence as reflected by the total Barthel Index score, supporting the platform's capability to capture clinically meaningful motor and functional outcomes.

These findings are consistent with prior evidence suggesting that digitally mediated and telerehabilitation-based interventions provide net benefits for rehabilitation populations, particularly by supporting cognitive engagement alongside motor recovery. For example, Torpil et al. (2023) reported that technology-assisted telerehabilitation interventions yielded favorable cognitive and functional outcomes, indicating that well-designed digital rehabilitation approaches can be more beneficial than detrimental when integrated into clinical care.²⁸ Similarly, Pearce et al. (2023), in a scoping review of digital health interventions in rehabilitation, found that most studies focused on key implementation outcomes, including acceptability, compatibility with clinical contexts, and the delivered dose of digital interventions. In line with these observations, the significant associations between RTPT-derived performance metrics and established clinical measures observed in the present study suggest that the RTPT platform addresses critical implementation determinants, supporting its feasibility, clinical relevance, and translational potential in stroke rehabilitation settings.²⁹

Test–retest reliability was good to excellent for both SCC and ACC, indicating that RTPT provides stable and reproducible measurements across repeated assessments. Usability evaluation using SUS and TAM demonstrated high acceptance, ease of use, and satisfaction among both patients and healthcare professionals, supporting the feasibility of clinical integration and potential for routine adoption in digital stroke rehabilitation programs.

ACC serves as a quantitative indicator of motor impairment in hemiplegic patients, whereas SCC reflects cognitive deficits contributing to impaired upper-limb function. The current RTPT digital task primarily assessed completion of color-based stimuli within a fixed time frame, emphasizing attentional demands and visuomotor coordination. Furthermore, the platform captures detailed, session-by-session digital performance data, which can be analyzed using machine learning algorithms to model individual patient performance trajectories and guide personalized, data-driven rehabilitation strategies.

These findings align with Moulaei et al. (2022), who reported that digital rehabilitation technologies—including virtual reality systems, smart wearables, and robotic devices—are most frequently applied for the evaluation and measurement of upper-limb function in telerehabilitation contexts. Their review highlighted benefits such as improvements in musculoskeletal function, increased patient motivation, enhanced adherence to rehabilitation exercises, and greater participation in the therapeutic process. Importantly, the RTPT platform extends these advantages by incorporating adaptive learning features that adjust task difficulty based on individual performance, thereby delivering personalized interventions that address both motor and cognitive domains. By combining real-time digital measurement with adaptive, data-driven task modulation, RTPT not only retains the established strengths of digital rehabilitation tools but also represents a more innovative and progressive approach to individualized stroke rehabilitations.^30,31

The RTPT digital platform's ability was assessed across multiple dimensions, reflecting both interface interaction quality and user acceptance of the system's rehabilitation purpose.^25,26 In digital health interventions, usability encompasses not only the effectiveness and efficiency of human computer interaction but also the attitudinal acceptance and perceived value of the system by end-users. Interface usability and attitudinal acceptance are distinct but complementary constructs, and their integration provides a more holistic evaluation of digital rehabilitation tools.

In this study, the relationship between these dimensions was examined by comparing the validated SUS with the core constructs of the TAM, including PU and PEU. Additionally, the RTPT platform captures detailed, session-by-session interaction data, which can be analyzed using machine learning algorithms to optimize interface design, adapt task difficulty, and enhance user experience in real time. The significant correlations observed between SUS and TAM scores suggest a strong convergence between experiential usability and attitudinal acceptance, supporting the RTPT platform as a user-centered and clinically relevant tool for hemiparetic stroke rehabilitation.³² This observation is consistent with prior telerehabilitation studies, such as the SIDERA^B system proposed by Rossetto et al. (2023), which demonstrated that well-designed digital rehabilitation platforms can effectively support the transition of rehabilitation services from clinic-based to home- and community-based settings. Notably, Rossetto et al. reported that user experience was partially associated with patient-specific characteristics, indicating that individual differences should be considered as design parameters rather than barriers to adoption.

In line with these findings, the high level of perceived usability of the RTPT platform observed in the present study is particularly noteworthy given the characteristics of the stroke population, including advanced age and heterogeneous functional profiles, further supporting the feasibility and acceptability of RTPT across diverse user groups.³³

Results from SUS and TAM assessments indicate high user acceptance and satisfaction with the RTPT digital platform. Participants demonstrated strong adherence, engagement, and perceived value, suggesting that the gamified, interactive design may enhance motivation during stroke rehabilitation. Reliability analyses confirmed acceptable internal consistency and construct validity for both instruments.^34,35 Furthermore, the digital RTPT platform continuously collects detailed performance and interaction data, which can be analyzed using machine learning techniques to adapt task difficulty, provide personalized feedback, and optimize usability for individual patients in real time. Although the sample size was relatively small (n < 30), no significant differences were observed between SUS and TAM median scores, implying that both instruments capture overlapping but complementary aspects of perceived usability. Future larger-scale studies are warranted to evaluate whether these measures can be considered functionally equivalent for assessing digital health interventions in clinical rehabilitation settings.

Beyond its role as a digital assessment tool for hand function and attention, the RTPT platform may function as an interactive rehabilitation system. Its gamified, digital design enables systematic tracking of performance across sessions, supporting individualized feedback and adaptive training. This design aligns with prior findings by Wen et al. (2025), which emphasized that high-quality, personalized feedback and psychological support are critical determinants of rehabilitation adherence in elderly stroke patients. Collected performance data can be analyzed using machine learning algorithms to personalize task difficulty, provide predictive insights into recovery trajectories, and optimize intervention strategies for each patient. Based on Davidson's neuroplasticity framework,³⁶ repeated stimulation of surviving neurons in patients with brain injury may strengthen cortical pathways and facilitate functional recovery. By engaging the affected limb in repetitive, structured, and gamified tasks, RTPT may promote activation of perilesional neurons, support neural reorganization, and enhance upper-limb functional recovery.³⁷ While Wen et al. reported that physiological and psychological factors exert largely independent effects on rehabilitation adherence, this finding does not contradict the design rationale of RTPT. Rather, the RTPT system integrates these dimensions within a single digital environment, not to imply statistical interaction, but to concurrently address multiple independent determinants of rehabilitation outcomes, including motor performance, cognitive load, attention, and emotional engagement. In this sense, RTPT can be viewed as an applied translation of Wen et al.'s findings into system design, operationalizing psychological support not only through explicit feedback, but also through system-level features that foster positive emotional experiences, sustained motivation, and long-term adherence.³⁸

Feasibility in clinical settings

The RTPT system is designed to be easily integrated into clinical rehabilitation workflows. The hardware components including a tablet, touch-sensitive target board, and simple sensors are relatively inexpensive compared with conventional rehabilitation equipment, making the system cost-effective within typical healthcare budgets. The operation requires minimal specialist knowledge: trained physiotherapists or occupational therapists can administer the tasks after a brief orientation, without the need for additional engineering expertise. Early observations indicate that the system can be feasibly deployed in outpatient or inpatient rehabilitation settings, with minimal disruption to standard care routines. Future studies will further investigate scalability, cost-effectiveness, and integration across diverse clinical environments.

Limitations

This study has several limitations. The assessment tools used BBT, LOTCA, and CAT focus on specific domains of hand function or cognition and may not fully capture overall functional ability; their performance could also be influenced by participant fatigue or the interval between test–retest sessions. Some cognitive measures may require cultural or linguistic adaptation for broader applicability.

Furthermore, participants were selected among stroke patients with relatively good functional status, intact cognition, and pending discharge to ensure that they could successfully complete the digital RTPTs and provide meaningful feedback during early feasibility testing. Consistently, the median Barthel Index total score was 90 (IQR: 65–100), with feeding and grooming sub scores of 10 (10, 10) and 5 (3.8, 5), respectively, reflecting a relatively mildly affected sample. While this selection was appropriate for initial testing of system functionality and usability, it may limit the generalizability of the findings to more severely affected stroke patients.

Finally, additional factors influencing digital therapeutics such as interface design, interactivity, digital literacy, and variability in patient functional abilities—warrant further investigation to inform the development of adaptive, personalized rehabilitation platforms. Beyond usability and functional assessment, the RTPT system also shows potential for prognostic applications, where future studies could explore how task performance metrics might inform adaptive training plans, predict recovery trajectories, and guide therapy intensity or focus, thereby supporting more personalized rehabilitation strategies.

Conclusion

The RTPT, a novel digital rehabilitation assessment platform, demonstrated strong validity and reliability for evaluating cognitive and motor function in stroke patients. Performance of the non-paretic hand primarily reflected attention, whereas paretic-hand performance was associated with manual dexterity, grip strength, and self-care abilities. Its digital platform enables detailed data collection to support machine learning–driven, individualized rehabilitation, emphasizing potential applications in both hospital- and home-based settings. Overall, RTPT represents a promising tool for integrated assessment and personalized intervention in stroke rehabilitation.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076261425481 - Supplemental material for Psychometric properties and usability of a gamified digital task system for assessing attention and hand function in stroke rehabilitation

Supplemental material, sj-docx-1-dhj-10.1177_20552076261425481 for Psychometric properties and usability of a gamified digital task system for assessing attention and hand function in stroke rehabilitation by Shu-Chun Huang, Chieh-Yu Li, Watson Hua-Sheng Tseng, Ching-Yi Wu, Lan-Yan Yang, Chiu-Wen Chen and Shih-Ying Chien in DIGITAL HEALTH

Footnotes

Abbreviations

Acknowledgments

We gratefully acknowledge the support of the Chang Gung Medical Foundation and Chang Gung University for their assistance in the prototype design and implementation of this research project. We also sincerely thank all the healthcare professionals who contributed to this study, as well as the participants who volunteered their time without compensation. Their involvement made it possible to advance innovation and progress in the field of medical design.

ORCID iD

Shih-Ying Chien

Ethics and consent to participate

This study was approved by the Institutional Review Board of Chang Gung Memorial Hospital (IRB No. 202301197A3C601). All participants provided written informed consent.

Consent for publication

All authors have read and approved the final version of the manuscript and consent to its publication in the Journal of Medical Systems.

Author contributions

HSC and CSY contributed equally to this work and should be considered co-first authors.

Conceptualization: HSC, CSY, WCY, CCW;

Methodology: HSC, WCY; YLY;

Software: HYC, Watson THS; Validation: HCC, LCY;

formal analysis: HSC, CSY, HYC, Watson THS; YLY; investigation: HSC, HCC, LCY; resources: HSC, CSY, WCY, LCY

data curation: HSC, LCY, CCW; writing-original draft and preparation: HSC, CSY, LCY; writing-review and editing: HSC, CSY ; visualization: HSC, CSY, YLY; supervision: HSC, CSY; project administration: HSC, CSY, LCY, CCW; funding acquisition: HSC, CSY

All authors have read and agreed to the published version of the manuscript.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Chang Gung Medical Foundation, National Science and Technology Council, Chang Gung University, (grant number CMRPVVP0211, CORPVVN0071, 114-2221-E-182-048, 114-2314-B-182A-066, QCRPDZ0008).

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data availability statement

The data sets generated during and/or analyzed during this study are not publicly available due to participant protection rights, but are available from the corresponding author upon reasonable request.

Supplemental material

Supplemental material for this article is available online.

References

Yasufuku

Collaborators

GSRF

. Global, regional, and national burden of stroke and its risk factors, 1990–2021: a systematic analysis for the global burden of disease study 2021. Lancet Neurol 2024; 23: 973–1003.

Feigin

, et al. World stroke organization: global stroke fact sheet 2025. Int J Stroke 2025; 20: 132–144.

Chen

Zhang

Jiang

. Barthel Index score at admission to predict 1-month and 1-year prognosis in inpatients aged≥ 75 years with multimorbidity. Int J Geriatr Psychiatry 2025; 40: e70147.

Wang

Y-C

, et al. Comparison of responsiveness of the barthel Index and modified barthel Index in patients with stroke. Disabil Rehabil 2023; 45: 1097–1102.

Ouédraogo

, et al. Tools and methods for assessing the usability and related aspects of usability of extended reality and telerehabilitation technologies in stroke rehabilitation: a scoping review. J Neuroeng Rehabil 2025; 23: 44.

Hsieh

Y-W

, et al. Development and user experience of an innovative multi-mode stroke rehabilitation system for the arm and hand for patients with stroke. Sci Rep 2022; 12: 1868.

Hyzy

, et al. System usability scale benchmarking for digital health apps: meta-analysis. JMIR Mhealth Uhealth 2022; 10: e37290.

Gomes

Ratwani

. Evaluating improvements and shortcomings in clinician satisfaction with electronic health record usability. JAMA Network Open 2019; 2: e1916651–e1916651.

李柏森 , et al., Test-retest reliability and minimal detectable change of Chu's attention test in persons with chronic schizophrenia. 台灣精神醫學, 2014; 28: iv+ 46–54.

10.

Katz

, et al. Loewenstein occupational therapy cognitive assessment (LOTCA) battery for brain-injured patients: reliability and validity. Am J Occup Ther 1989; 43: 184–192.

11.

Desrosiers

, et al. Validation of the box and block test as a measure of dexterity of elderly people: reliability, validity, and norms studies. Arch Phys Med Rehabil 1994; 75: 751–755.

12.

Harte

, et al. A human-centered design methodology to enhance the usability, human factors, and user experience of connected health systems: a three-phase methodology. JMIR Hum Fact 2017; 4: e5443.

13.

Davis

FD.

Technology acceptance model: TAM. Al-Suqri, MN, Al-Aufi, AS: Inf Seeking Behav Technol Adoption 1989; 205: 5.

14.

Tullis

Stetson

. A comparison of questionnaires for assessing website usability. In Usability professional association conference. Minneapolis, USA. 2004.

15.

Moslehpour

, et al. E-purchase intention of Taiwanese consumers: sustainable mediation of perceived usefulness and perceived ease of use. Sustainability 2018; 10: 234.

16.

Pal

Vanijja

. Perceived usability evaluation of Microsoft Teams as an online learning platform during COVID-19 using system usability scale and technology acceptance model in India. Child Youth Serv Rev 2020; 119: 105535.

17.

Lewis

. Usability: lessons learned… and yet to be learned. Int J Hum Comput Interact 2014; 30: 663–684.

18.

Lewis

. The system usability scale: past, present, and future. Int J Hum–Computer Interact 2018; 34: 577–590.

19.

R Development Core Team.

. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing, 2022.

20.

Allaire

. RStudio: integrated development environment for R. Boston, MA 2012; 770: 165–171.

21.

Yusof

, et al. Measurement model of employability skills using confirmatory factor analysis. Proce-Soc Behav Sci 2012; 56: 348–356.

22.

Hair

JF.

Multivariate data analysis. 2009.

23.

Portney

Watkins

. Foundations of clinical research: applications to practice. Upper Saddle River, NJ: Pearson/Prentice Hall.Vol. 892. 2009.

24.

Liljequist

Elfving

Skavberg Roaldsen

. Intraclass correlation–A discussion and demonstration of basic features. PloS One 2019; 14: e0219854.

25.

Lewis

Sauro

. Item benchmarks for the system usability scale. J Usability Stud 2018; 13: 158–167.

26.

Kenah

, et al. “Wasting time”: a qualitative study of stroke survivors’ experiences of boredom in non-therapy time during inpatient rehabilitation. Disabil Rehabil, 2024; 46: 2799–2807.

27.

Park

, et al. The changes of muscle activity by robot-assisted gait training methods in children with brain injury: a pilot test. Arch Phys Med Rehabil 2024; 105: e167.

28.

Torpil

Pekçetin

. The effectiveness of cognitive rehabilitation intervention with the telerehabilitation method for amnestic mild cognitive impairment: a feasibility randomized controlled trial. J Telemed Telecare 2025; 31: 320–327.

29.

Pearce

, et al. Implementation of digital health interventions in rehabilitation: a scoping review. Clin Rehabil 2023; 37: 1533–1551.

30.

Moulaei

, et al. Telerehabilitation for upper limb disabilities: a scoping review on functions, outcomes, and evaluation methods. Arch Public Health 2022; 80: 196.

31.

Piga

, et al. Telemedicine applied to kinesiotherapy for hand dysfunction in patients with systemic sclerosis and rheumatoid arthritis: recovery of movement and telemonitoring technology. J Rheumatol 2014; 41: 1324–1333.

32.

Balcombe

De Leo

. Human-computer interaction in digital mental health. In: Informatics. Basel, Switzerland: MDPI, 2022.

33.

Cheah

W.-H.

, et al., Mobile technology in medicine: development and validation of an adapted system usability scale (SUS) questionnaire and modified technology acceptance model (TAM) to evaluate user experience and acceptability of a mobile application in MRI safety screening. Indian J Radiol Imag 2023; 33: 036–045.

34.

Chien

S-Y

. A usability evaluation framework for a mobile application in supporting home-based rehabilitation for stroke patients: a qualitative study. Digital Health 2025; 11: 20552076251340183.

35.

Chien

S-Y

, et al. Exploring the impact of an interactive electronic pegboard on manual dexterity and cognitive skills of patients with stroke: preliminary analysis. JMIR Formative Research 2024; 8: e55481.

36.

Zilles

. Neuronal plasticity as an adaptive property of the central nervous system. Ann Anatomy-Anatomischer Anzeiger 1992; 174: 383–391.

37.

Nudo

. Adaptive plasticity in motor cortex: implications for rehabilitation after brain injury. J Rehab Med-Supplements 2003; 41: 7–10.

38.

Wen

, et al. Enhancing long-term adherence in elderly stroke rehabilitation through a digital health approach based on multimodal feedback and personalized intervention. Sci Rep 2025; 15: 14190.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.04 MB