Sage Journals: Discover world-class research

Abstract

Background. Robots designed for rehabilitation of the upper extremity after stroke facilitate high rates of repetition during practice of movements and record precise kinematic data, providing a method to investigate motor recovery profiles over time. Objective. To determine how motor recovery profiles during robotic interventions provide insight into improving clinical gains. Methods. A convenience sample (n = 22), from a larger randomized control trial, was taken of chronic stroke participants completing 12 sessions of arm therapy. One group received 60 minutes of robotic therapy (Robot only) and the other group received 45 minutes on the robot plus 15 minutes of translation-to-task practice (Robot + TTT). Movement time was assessed using the robot without powered assistance. Analyses (ANOVA, random coefficient modeling [RCM] with 2-term exponential function) were completed to investigate changes across the intervention, between sessions, and within a session. Results. Significant improvement (P < .05) in movement time across the intervention (pre vs post) was similar between the groups but there were group differences for changes between and within sessions (P < .05). The 2-term exponential function revealed a fast and slow component of learning that described performance across consecutive blocks. The RCM identified individuals who were above or below the marginal model. Conclusions. The expanded analyses indicated that changes across time can occur in different ways but achieve similar goals and may be influenced by individual factors such as initial movement time. These findings will guide decisions regarding treatment planning based on rates of motor relearning during upper extremity stroke robotic interventions.

Keywords

stroke rehabilitation robotics upper extremity

Introduction

Analyzing the effects of a stroke rehabilitation intervention is typically achieved through comparing post- to pretraining values on outcome variables, but are there other clinically relevant analyses? Motor learning research has provided additional methods of interest. One relatively new method involves the continuous tracking of recovery profiles that can provide insight into rates of change of motor performance during an intervention.^1-5 Here we investigate the continuous tracking of recovery using upper extremity robotic training as the intervention.

Robotic interventions have generally been classified as having beneficial effects for survivors of stroke, particularly for the arm.^6-8 While some of these studies have demonstrated similar outcomes to concentrated conventional therapy,^9-11 robots offer an unparalleled ability to deliver repetitive movements at high rates compared to conventional therapy. One such robot is the MIT-MANUS, which, in addition to providing a progressive target reaching training paradigm, also generates kinematic data through digital movement encoders that may provide insight into how the quantity of this robotic intervention affects motor recovery during the course of robot-aided therapy. It does this by requiring robot-unassisted evaluations periodically during the training that can be used to investigate the continuous time-course of the unassisted kinematic data generated by the MIT-MANUS across training.⁵ By examining changes during an intervention, we can assess when the maximum benefit from robot training is gained and related questions regarding dosage and individual differences in recovery profiles.

The purpose of the study was to investigate how the kinematic data, specifically movement time, obtained during a MIT-MANUS robotic intervention, could characterize motor performance changes. Given our study sample, a parallel, but secondary, purpose was to assess whether a within-session regimen of 45 minutes of repetitive reaching training followed by 15 minutes of transition-to-task practice produced the same timeline of kinematic changes as a regimen of 60 minutes of repetitive robotic training. For both groups, we explored whether changes occurred within versus between sessions, and also when the progress of movement times changed, that is, reached a plateau. Better understanding of the time-course of improvements during an intervention will help guide clinical practice and prescription of robotic interventions for stroke survivors, as well as future clinical trials.

Methods

Participants

A convenience sample of 22 consecutively enrolled participants in a study of chronic stroke rehabilitation were included in this exploratory study. Participant demographics are included in Table 1. Participants met inclusion criteria if they were at least 6 months poststroke, had a Fugl-Meyer score between 7 and 38, and had adequate arm mobility to move a robotic manipulandum to target locations. The Joint University of Maryland/Baltimore Veterans Affairs Medical Center Institutional Review Board and the MIT Committee on the Use of Human Subjects as Experimental Subjects approved study procedures. All participants provided written informed consent.

Table 1.

Demographics.

Subject	Group	Age	Gender	Type of Stroke	Affected Side	FM
1	Robot	37	Male	Ischemic	Right	27
2	Robot	70	Male	Ischemic	Right	35
3	Robot	53	Male	Ischemic	Right	33
4	Robot	41	Male	Ischemic	Right	26
5	Robot	50	Female	Ischemic	Right	26
6	Robot	62	Female	Ischemic	Left	34
7	Robot	79	Female	Ischemic	Right	14
8	Robot	55	Female	Hemorrhagic	Right	22
9	Robot	68	Male	Hemorrhagic	Left	27
10	Robot	57	Female	Hemorrhagic	Right	22
11	Robot	61	Male	Ischemic	Right	21
Average (SD)		57 (±12)				26 (±6)
12	TTT	52	Male	Ischemic	Right	20
13	TTT	67	Male	Hemorrhagic	Right	34
14	TTT	50	Male	Hemorrhagic	Right	21
15	TTT	50	Male	Ischemic	Left	34
16	TTT	76	Male	Ischemic	Left	23
17	TTT	58	Male	Hemorrhagic	Right	22
18	TTT	34	Male	Ischemic	Left	19
19	TTT	66	Male	Hemorrhagic	Left	14
20	TTT	70	Female	Ischemic	Left	14
21	TTT	48	Male	Ischemic	Left	22
22	TTT	48	Male	Ischemic	Left	31
Average (SD)		56 (±12)				23 (±7)

Abbreviations: FM, Fugl-Meyer score; SD, standard deviation; TTT, translation-to-task practice.

Procedure

As part of a larger 12-week randomized-control trial, participants completed 3 days of therapy per week for 4 weeks. Each session consisted of 60 minutes of therapy; one group (n = 11) received 60 minutes of robotic training (Robot only) and the other group (n = 11) received 45 minutes of robotic training followed by 15 minutes of transition-to-task practice (Robot + TTT). All of the robotic therapy was on the InMotion2 Arm robot (Interactive Motion Technologies, Cambridge, MA). Participants sat comfortably at a table with their stroke-affected arm resting in a molded cradle with the hand around the manipulandum handle. Participants moved the handle across a horizontal plane to control the position of a cursor in a 2D workspace to hit 8 equidistantly spaced targets around a circle. During the therapy, the robots were in the “active-assist” mode such that the robot provided assistance as necessary if the participant was unable to reach a target. The therapy consisted of 320 movement repetitions per block, and the robot graded the task difficulty by changing the time allocated to complete the reaching movement and the movement guidance by altering the amount of wall-stiffness during the treatment.

The TTT practice consisted of 15 minutes of therapist-guided practice of patient-specific tasks that involved use of the whole arm in 3D space. These functionally based activities were performed in a sitting position and included weight bearing activities to promote stabilization and tabletop activities to promote bilateral and unilateral dexterity. Examples of tasks included lifting a cup, reaching with a brush, and wiping a countertop. Activity progression was based on objective measures such as independence with lifting, distance of reach, and number of repetitions completed in the set activity time. A stop watch was used to time each of the 2 prescribed TTT activities for 7.5 minutes of task training; manual assistance was provided as necessary. The same therapist provided all of the robotic and the TTT sessions.

Data Analysis

Before the start of the initial training block and after each block of 320 movements, an evaluation was completed without robot assistance in the center-out task to 8 targets. The robot recorded the position of the cursor at a sampling frequency of 200 Hz, and a customized MATLAB program was used to process the data offline. Data were filtered with a Butterworth low pass filter (7 Hz cutoff). Movement time was defined as the amount of time elapsed to reach within 1 cm of the target. An average movement time for completed reaches at each evaluation was calculated. The number of blocks varied individually since time, not repetition, was held constant, which resulted in some instances when a participant did not conclude the intervention time with an evaluation. In these cases, the previous evaluation was used as the last block. The number of blocks also varied across groups given the 45- versus 60-minute schedule on the robot. For all participants, a minimum of 2 evaluations per session were completed, which indicates a minimum of at least 320 repetitions per session.

A repeated measures 2 × 2 ANOVA (time [pre, post] by group [Robot only, Robot + TTT]) was used to determine pre-post changes and differences between groups. To look more closely at when and how the kinematic performance changes were occurring, we employed 2 strategies. First, we assessed whether changes tended to occur during a treatment session (online gains) or between sessions (offline gains).^12,13 Online gains were calculated as the rate of change within a session, and offline gains were calculated as the absolute change between sessions. Online and offline gains were compared using a one-way ANOVA to determine differences between groups. Second, we assessed the recovery profiles across blocks with random coefficient modeling (RCM). This analysis focused on evaluating the overall time spent on the robot such that the blocks of completed intervention on the robot were numbered sequentially for each participant over the entire study of 12 sessions. Our initial visual inspection revealed that there was a rapid reduction in movement time followed by a gradual improvement in movement time at a slower rate. The RCM facilitates the analysis of repeated measures in which the degree of change may not be linear;¹⁴ motor learning literature suggests that such a learning process is best fitted by a 2-term exponential function.^1-4 As such, we used a 2-term exponential function with random effects. The RCM statistically identifies both individuals that have different learning rates and whether group differences exist between the marginal models. This step accounts for between-subject variability by representing the performance trajectory when all individuals are identical, that is, when R_(i,j) = 0.^15,16 As part of the model development, a likelihood ratio test confirmed that the 2-term exponential function with random effects (ie, RCM on a 2-term exponential function) fit the data better compared to the 1-term exponential function with random effects ( $χ_{d f = 5}^{2} = 29$ , P < .0001). In addition, a likelihood ratio test suggests the RCM (Equation 1) rather than a 2-term exponential function with only fixed effects is a better fit of the data ( $χ_{d f = 3}^{2} = 1592$ , P < .000001).

The model is as follows:

M T_{i, t} = (β_{I} + γ_{I} \cdot g_{i} + R_{i, 3}) + (β_{1} + γ_{1} \cdot g_{i}) \cdot e^{[(β_{2} + γ_{2} \cdot g_{i} + R_{i, 1}) \cdot t]} + (β_{3} + γ_{3} \cdot g_{i}) \cdot e^{[(β_{4} + γ_{4} \cdot g_{i} + R_{i, 2}) \cdot t]} + e r r o r_{i, t}

where $M T_{i, t}$ is the movement time at block t for individual i; β_I, β₁, β₂, β₃, β₄ represent the fixed effects for the Robot only group; γ_I, γ₁, γ₂, γ₃, γ₄ are the adjustment to β₁, β₂, β₃, β₄ and thus represent the fixed effects for the Robot + TTT group; g = 0 or 1 for the Robot only or Robot + TTT group; R_i,₁, R_i,₂, R_i,₃, are the random effects; and $e r r o r_{i, t} ~ N (μ = 0, δ^{2})$ is the residual at block t for individual i. All 3 random effects were assumed to follow normal distributions, $R_{i, j} ~ N (μ_{j} = 0, δ_{j}^{2}),$ j = 1, 2, 3. Specifically, β_I represents the plateau of movement time (when $t \to \infty$ ). β₂ and β₄ are the decay rates of the 2 exponential terms in Equation 1. Our analyses revealed that β₄ had a larger magnitude than β₂, suggesting that β₄ represents the decay rate of the fast learning component while β₂ represents the decay rate of the slow learning component. Correspondingly, β₂ and β₃ are the coefficients of the slow and fast learning components, respectively. The summation of β_I, β₁, and β₃ characterizes the movement time at t = 0. Meanwhile, the magnitudes of β₁ and β₃ also affect the learning rate because the change of movement time (ie, the derivative of Equation 1) relies on β₁ and β₃. The γ parameters represent adjustments to the corresponding β parameters. Thus, the analysis facilitates differentiation between groups if the γ parameters are statistically different (P < .05). Additionally, the analysis on the parameters R allows for statistical comparisons between participants with regard to whether an individual participant does or does not show the same learning profile as the group. For example, participants may respond faster to the intervention whereas others may respond slower to the intervention. This statistical approach controls for large between-subject variability and can identify which participant displays a different course of change in motor performance over time.^15,16 This will be valuable in evaluating clinical trials where clinical gains are often modest. To facilitate how the results from the RCM relate to initial motor function in survivors of stroke, we conducted a Pearson’s correlation between the generated random coefficients for each individual with initial FM scores and movement time.

Results

The 2 groups were similar at the start of the intervention for age (P = .8) and level of impairment (P = .3). See Table 1. Movement times significantly decreased over the course of the intervention, F = 35.4, P < .001, and this was not different between groups (Robot only and Robot + TTT), F = 1.9, P = .18, as depicted in Figure 1A. The average movement time per block over the course of the intervention is illustrated in Figure 1B with the size of the data point indicating the number of individuals within the average. The Robot only group typically completed more blocks of robotic training compared to the Robot + TTT group. The groups were then compared to determine if changes within a session (online gains) or between sessions (offline gains) were different. The groups had significantly different rates of change within a session, which was quantified as the slope of change. The Robot + TTT group had a significantly greater rate of movement time improvement during the treatment session (average slope = 0.05) compared to the Robot only group (average slope = 0.004, P = .002). The absolute changes between sessions were significantly different (P = .001) with the Robot only group having a small improvement in movement time between sessions (0.08 seconds), whereas the Robot + TTT group had on average 0.2 seconds slower movement times at the start of the following session.

Figure 1.

(A) Changes in movement time were similar between the 2 groups between the start and end of the 4-week intervention. (B) The average for each block on training separated by sessions over the course of the intervention.

The results of the RCM are depicted in Figures 2 and 3. The variances $δ_{j}^{2},$ j = 1, 2, 3, of random effects $R_{i, j} ~ N (μ_{j} = 0, δ_{j}^{2}),$ j = 1, 2, 3, were all significantly larger than zero (P < .05 for all $δ_{j}^{2},$ j = 1, 2, 3). This result confirmed that all 3 random effects are necessary to model the data set. The marginal models for the groups are depicted in Figure 2A. Two different components of learning were included in the 2-term exponential model to account for the nonlinearity of changes in movement time over the intervention. Both components were found to significantly contribute to the change of movement time (β₂ and β₄ are significant). The initial “fast” component of the trajectory profiles were significantly different between the 2 groups (see Table 2 for parameter estimates). The slow component of learning, however, had the same magnitude between the 2 groups. This can be observed in Figure 2A, where there was a transition between the fast and slow components. This transition was more apparent in the Robot + TTT group, which tended to occur between 10 and 30 blocks (or approximately 4 and 6 sessions). The transition is not as pronounced for the Robot only group.

Figure 2.

(A) Group effects of the exponential analysis and random coefficient modeling prediction curves. There was a significant difference between groups in the fast component of learning but the slow component was similar. (B) Individual predictive curves for the Robot only group. (C) Individual curves for the Robot + TTT group. TTT, translation-to-task practice.

Figure 3.

Exemplar data from both groups illustrating the individual data points that were used for the analysis and how the prediction curves are similar or different than the group prediction.

Table 2.

Parameter Estimates for the Nonlinear Model^a.

Parameter	Estimate	t Value	p Value
β_I	1.8	14.7	<.0001^b
γ_I	0.15	0.8	.4
β₁	0.9	9.1	<.0001^b
γ₁	−0.3	−1.9	.07
β₂	−0.0326	−4.0	.0009
γ₂	0.0092	0.7	.5
β₃	1.2	5.7	<.0001^c
γ₃	0.7	2.7	.01^c
β₄	−0.5	−2.9	.01
γ₄	−0.12	−0.5	.7

Bolded parameters (β₂ and β₄) indicate that the 2 components of learning (slow and fast) significantly contributed to changes in movement time.

Indicates that the component significantly contributed to the model but was not different between groups.

Indicates that a significant difference between the Robot only and Robot + TTT groups was detected for the fast component of learning.

One unique feature of the RCM is that it can control for high intersubject variability and identify individual subjects who are significantly different from the group. These results are provided in Table 3 for the 3 random coefficients that were included in the model including slow component, fast component, and initial movement time. Four subjects had significant deviations for the slow component of learning, whereas 9 subjects had deviations for the fast component of learning. Three subjects had significantly faster initial movement times (2.4, 2.5, and 1.7), whereas 1 subject had a significantly slower initial movement time (5.9 seconds), which were taken into account by the model. Figure 2B and C illustrates the variance in individual learning curves for each subject in relation to the group marginal models. The fast component and the initial MT coefficients positively correlated to the movement time (both were r = .7, P < .01). All other correlations were not significant (P > .05).

Table 3.

Individual Differences^a.

Subject	FM	Group	Slow Component	Fast Component	Initial MT
1	27	Robot	0.0004	−0.13	−0.23
2	35	Robot	−0.013	−0.04	−0.27
3	33	Robot	0.026^b	−0.4	0.20
4	26	Robot	0.02	0.5^b	0.26
5	26	Robot	0.005	0.5^b	0.16
6	34	Robot	−0.03	−0.6	−0.33^b
7	14	Robot	−0.01	0.4^b	−0.13
8	22	Robot	0.025^b	0.5^b	−0.04
9	27	Robot	−0.02	−0.2	0.05
10	22	Robot	0.02	0.4	0.14
11	21	Robot	0.01	−0.1	−0.11
12	20	Robot + TTT	−0.002	−0.13	0.16
13	34	Robot + TTT	−0.006	0.025	0.03
14	21	Robot + TTT	0.03^b	0.6^b	−0.25
15	34	Robot + TTT	−0.01	−0.5	−0.33^b
16	23	Robot + TTT	−0.2	−0.98^b	−0.64^b
17	22	Robot + TTT	0.025^b	−0.3	−0.32
18	19	Robot + TTT	0.019	0.59^b	0.37
20	14	Robot + TTT	0.002	0.2	0.12
21	22	Robot + TTT	−0.007	0.6^b	0.47^b
22	31	Robot + TTT	0.01	0.66^b	0.13

Abbreviations: FM, Fugl-Meyer score; TTT, translation-to-task practice; RCM, random coefficient modeling.

Subject 19 was not included in the RCM because there were not enough blocks completed per session.

Indicates which subjects were significantly different from the group. A positive value indicated a decreased learning rate (worse than the group) and a negative value indicated an increased rate of learning.

Figure 3 depicts individual prediction curves plotted against the marginal model. The top 2 plots illustrate 2 subjects whose individual curve does not differ from the marginal model. The middle 2 plots depict smaller fast learning components compared to the marginal model. The bottom 2 plots depict how a faster initial movement time generally just shifted the prediction curve lower, yet these 2 subjects did not have different fast or slow learning components, that is, the lines have the same shape.

Discussion

Profiles of movement time during unassisted multidirectional reaching, recorded by a robotic device, were investigated during a 4-week intervention on a shoulder-elbow robot with chronic stroke survivors. Improvements in movement times were similar after 12 sessions for both groups; the expanded analysis, however, determined that these gains were achieved differently. Generally, the Robot + TTT group had a greater rate of improvement during the session, which was not maintained between sessions, whereas the Robot only group had smaller gains within a session, but had a small gain in performance between sessions. Across all blocks an RCM with a 2-term exponential model demonstrated that 2 components of learning (fast and slow components) significantly contributed to the learning profiles. The RCM analysis also identified participants who were significantly different from the group, and the coefficients of the fast component were related to initial movement time. Taken together, these are important considerations for future rehabilitation studies because of the implications for dosing studies and opportunities to employ sophisticated statistical analyses in rehabilitation interventions.

The results of the initial pre-post analysis suggested that changes in movement time were not different between groups. These changes are similar to previous pre-post kinematic reports.¹⁷ However, a premature conclusion would be that gains were achieved similarly across all participants. The between- and within-session analysis provided an approach to investigate when changes were occurring and may provide an opportunity to investigate characteristics related to an intervention. In this proof-of-concept analysis, the between-session data suggested that the Robot only group may have an advantage in consolidating their gains between sessions. This advantage could be because the longer session on the robot led to fatigue, causing slower movement times at the end of the session. Alternatively, the TTT component of the intervention could have interfered with consolidation between sessions for the combination group because more complex, integrated movements were practiced. The latter explanation is more likely since it explains the greater within-session learning by the Robot + TTT group such that they start each session slower than their last performance but quickly regain and slightly exceed their previous session’s last value.

We wanted to obtain a detailed picture of the changes in movement time over the intervention by investigating the rates of performance changes with the RCM analysis. This analysis demonstrated that 2 components were significantly contributing to changes in movement time over the course of the intervention. There was a significant fast component of learning that occurred early during the intervention, which was followed by a slower component of learning. This phenomenon of rapid learning tapering off has been previously demonstrated,³ yet the current study applied this contemporary approach to data collected within an extended intervention. The clinical implication from this analysis revealed a critical period where performance changes were slower. This effect was apparent in both intervention groups, and was slightly more pronounced in the Robot + TTT group. Most of the gains were attained during the first 20 blocks of robot training (each block includes 320 movements) with a subsequent diminished rate of gain. Each participant would have completed 20 blocks of training during different sessions given that the time in the robot therapy was held constant rather than number of repetitions. Clinically, this suggests that the first 20 blocks of cumulative training (about 5-6 sessions) promotes a faster rate of gain, which is followed by slower improvements over the additional sessions. This result is similar to Volpe and colleagues who observed the biggest improvement in the first 9 sessions.¹⁸ However, it is in contrast to those of Kahn and colleagues,¹⁹ who suggested improvements of speed from robotic training were gradual and continuous over the course of the intervention. The difference in our results may be due to the type of robotic training used, the fact that they had a very low number of movements per session (80 movements vs a range of 320 to 960 in ours), the fact that they did not use a combination training but tested robot alone versus task-practice alone, the density of their assessments (less than ours), and the overall time frame (longer than ours). It may also be due to our use of the RCM analysis. We observed a change in the rate of improvement that would not be detected with a linear model. This highlights the importance of using different statistical approaches and the potential limitations of using a linear model over repeated sessions. The results from the current study suggest that this shift in fast to slow gains is an opportunity to explore and study different dosing strategies with robotic interventions and highlights the importance of continuous monitoring of performance. For example, the robot could require greater challenges as changes in motor performance slowed. The decreased rate of gains and differences in learning between subjects over the course of the intervention may be possible explanations for why clinical gains from robot studies are often limited. Analyses like the RCM, with a 2-term exponential function, offer rehabilitation scientists new avenues for analyzing data from studies that have measures that are repeatedly collected (eg, movement time on a certain task).

One clear advantage to the RCM approach is that it allows for the detection of individuals whose pattern of performance over time differs from that of the group. As such, this analysis controls for intersubject variability, which is critical for populations such as stroke that have inherent variability. The traditional option to control for variability in clinical trials is to have strict inclusion/exclusion criteria, but this is statistically inefficient, increases recruitment burden, and decreases external validity.^20,21 More novel approaches implement appropriate statistical analyses that can control for inherent variability, as was done in the current study. The RCM identified a number of participants in each group who exhibited similar profiles over time whereas a few had different patterns (see Tables 2 and 3). The individual curves suggest that robotic treatment should be evaluated during the first 20 blocks of training to establish if patients are following the typical trajectory, as some patients may have a slow or no response at all to treatment (example participant 22 in Figure 2C) or are responding more quickly (see Figure 2). Statistical identification of subjects is a significant step for clinical research and generates new avenues for determining the factors that influence response to treatment. Further developing these performance profiles provides clinicians with better options to modify treatment plans, which should increase effectiveness of robotic treatments.

Finally, we explored the relationships between the generated coefficients from the RCM and individual initial motor severity (FM scores and movement time). The fast phase and initial MT coefficients were significantly related to the movement time (seconds). This analysis suggests that an individual with a slower movement times at the start of the intervention had decreased fast learning rates without achieving a strong plateau. For example, see Subjects 4, 5, and 8 in Figure 2B. These relationships and relationships with functional change and final performance need to be explored more fully with a robotic dosing related study.

Conclusions

By using sophisticated statistical analyses to more fully understand how performance changes over time, we demonstrated different rates of learning during the intervention (fast and slow components) that were not apparent with a more traditional pre-post statistical approach. The RCM highlighted a critical period within the intervention where motor performance appeared to reach a plateau and potential differences between groups in terms of when improvements become consolidated. The RCM approach also allowed for analysis at the individual level, which can assist to identify individuals who may not be responding to the intervention like other participants. These areas of discussion provide avenues for implementation with future robotic intervention studies and have implications for the clinical use of robotic training.

Footnotes

Acknowledgements

The authors would like to thank the participants involved with this study. They would also like to thank Patti McCarthy, OTR, Jaime Lush, Brooke David, and Harrison Segall, who assisted with data collection and analysis.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by the Veterans Affairs Merit Award (B6935R) to CTB. CLM was supported by NIDRR, UMANRRT program (H133P100014), and American Heart Association (14CRP18150008).

References

King

Harring

Oliveira

Clark

. Statistically characterizing intra- and inter-individual variability in children with developmental coordination disorder. Res Dev Disabil. 2011;32:1388-1398.

King

Kagerer

Contreras-Vidal

Clark

. Evidence for multisensory spatial-to-motor transformations in aiming movements of children. J Neurophysiol. 2009;101:315-322.

Krakauer

Ghez

Ghilardi

. Adaptation to visuomotor transformations: consolidation, interference, and forgetting. J Neurosci. 2005;25:473-478.

Krakauer

Pine

Ghilardi

Ghez

. Learning of visuomotor transformations for vectorial planning of reaching trajectories. J Neurosci. 2000;20:8916-8924.

Palazzolo

. Robotic technology to aid and assess recovery and learning in stroke patients. http://dspace.mit.edu/handle/1721.1/33918. Published 2005. Accessed November 23, 2015.

Kwakkel

Kollen

Krebs

. Effects of robot-assisted therapy on upper limb recovery after stroke: a systematic review. Neurorehabil Neural Repair. 2008;22:111-121.

Miller

Murray

Richards

. Comprehensive overview of nursing and interdisciplinary rehabilitation care of the stroke patient: a scientific statement from the American Heart Association. Stroke. 2010;41:2402-2448.

Prange

Jannink

MJA

Groothuis-Oudshoorn

CGM

Hermens

Ijzerman

. Systematic review of the effect of robot-aided therapy on recovery of the hemiparetic arm after stroke. J Rehabil Res Dev. 2006;43:171-183.

Conroy

Whitall

Dipietro

. Effect of gravity on robot-assisted motor training after chronic stroke: a randomized trial. Arch Phys Med Rehabil. 2011;92:1754-1761.

10.

Guarino

Richards

. Robot-assisted therapy for long-term upper-limb impairment after stroke. New Engl J Med. 2010;362:1772-1783.

11.

Volpe

Lynch

Rykman-Berland

. Intensive sensorimotor arm training mediated by therapist or robot improves hemiparesis in patients with chronic stroke. Neurorehabil Neural Repair. 2008;22:305-310.

12.

Robertson

Pascual-Leone

Miall

. Current concepts in procedural consolidation. Nat Rev Neurosci. 2004;5:576-582.

13.

Censor

Sagi

Cohen

. Common mechanisms of human perceptual and motor learning. Nat Rev Neurosci. 2012;13:658-664.

14.

Cudeck

Harring

. Analysis of nonlinear patterns of change with random coefficient models. Annu Rev Psychol. 2007;58:615-637.

15.

Weiss

. Modeling Longitudinal Data. Berlin, Germany: Springer Science & Business Media; 2005.

16.

Fitzmaurice

Laird

Ware

. Applied Longitudinal Analysis. New York, NY: John Wiley; 2012.

17.

Finley

Fasoli

Dipietro

Ohlhoff

MacClellan

Meister

. Short-duration robotic therapy in stroke patients with severe upper-limb motor impairment. J Rehabil Res Dev. 2005;42:683-692.

18.

Volpe

Huerta

Zipse

. Robotic devices as therapeutic and diagnostic tools for stroke recovery. Arch Neurol. 2009;66:1086-1090.

19.

Kahn

Zygman

Rymer

Reinkensmeyer

. Robot-assisted reaching exercise promotes arm movement recovery in chronic hemiparetic stroke: a randomized controlled pilot study. J Neuroeng Rehabil. 2006;3:12.

20.

Maas

AIR

Menon

Lingsma

Pineda

Sandel

Manley

. Re-orientation of clinical research in traumatic brain injury: report of an international workshop on comparative effectiveness research. J Neurotrauma. 2012;29:32-46.

21.