Sage Journals: Discover world-class research

Abstract

Background. The Microsoft Kinect presents a simple, inexpensive, and portable method of examining the independent components of the Timed Up and Go (TUG) without any intrusion on the patient. Objective. This study examined the reliability of these measures, and whether they improved prediction of performance on common clinical tests. Methods. Thirty individuals with stroke completed 4 clinical assessments, including the TUG, 10-m walk test (10MWT), Step Test, and Functional Reach test on 2 testing occasions. The TUG was assessed using the Kinect to determine 7 different functional components. Test–retest reliability was assessed using intraclass correlation coefficient (ICC), redundancy using Spearman’s correlation, and score prediction on the clinical tests using multiple regression. Results. All Kinect-TUG variables possessed excellent reliability (ICC_(2,k) > 0.90) except trunk flexion angle (ICC = 0.73). Trunk flexion angle and first step length were nonredundant with total TUG time. When predicting 10MWT and Step Test scores, adding step length into regression models comprising age and total TUG time improved model performance by 7% (P <.01) and 6% (P =.03), respectively. Specifically, an interquartile range increase in first step length (0.19 m) was associated with a 0.15 m/s faster gait speed and 1.8 more repetitions on the Step Test. These effect sizes were comparable to our minimal detectable change scores of 0.17 m/s for gait speed and 1.71 repetitions for the Step Test. Conclusions. Using the Kinect to independently assess the multiple components of the TUG may provide reliable and clinically useful information. This could enable efficient and information-rich large-scale assessments of physical deficits following stroke.

Keywords

kinematic balance stroke gait gaming

Introduction

The accurate assessment of physical deficits to evaluate function and falls risk following stroke is of considerable importance. In a clinical setting, gait and mobility limitations following stroke are commonly assessed using tests such as the 10-meter walk test (10MWT), and the Timed Up and Go (TUG).^1,2 These tests typically provide a single outcome measure, which represents performance of a specific component of physical function. These tests are therefore often performed as part of a testing battery to gain a more holistic view of a patient’s physical function. While performing multiple tests provides valuable information, it does increase the time and cost of the assessment, and adds to patient burden.

In contrast, research has shown that analyzing the movement strategies used by people with stroke as they perform functional tasks may provide additional and independent information that cannot be derived from standard single-outcome clinical tests. Instrumenting the TUG and individually assessing the various components of movement has previously been implemented in multiple studies.^2-5 Accelerometers³ and video analysis² during the TUG have been employed to assess the kinematic components of movement. For example, the use of inertial sensors during the TUG to perform a kinematic analysis of the individual components of the test has been reported as a reliable and sensitive measure in Parkinson’s patients.⁴ Previous work has reported that the turn time component of the TUG test, assessed using 3-dimensional (3D) motion analysis systems, is significantly longer in both people with stroke⁶ and healthy older adults⁷ who have experienced a fall. Similarly, research using accelerometers has demonstrated that fallers perform the sit to stand component of the TUG test with a greater range of lower back acceleration.⁵ Indeed, a detailed kinematic analysis of the constituents of the TUG test may be useful given that it is comprised of balance, gait speed, lower extremity muscle strength, and agility-related activities. However, despite these benefits of a kinematic analysis, tools such as inertial sensors or 3D motion analysis systems can require extensive participant preparation time and the use of skin mounted sensors or markers, which can be cumbersome and intrusive.

A system that may be capable of providing clinically useful information regarding the performance of independent components of movement during the TUG test is the Microsoft Kinect (Microsoft, Redmond, WA) motion-sensing camera. The Kinect, which does not require any subject preparation or attachment of sensors or markers to the patient, has been validated in healthy participants for the assessment of kinematic variables such as step length, gait speed, and trunk angle when compared with 3D motion analysis systems.^8-10 Using this system in a clinical setting to examine the different aspects of the TUG, such as trunk flexion angle during standing from the chair, first step length, gait speed, and time taken to turn, may provide additional information about movement strategies and physical deficits that are overlooked when simply recording the time taken to complete the test. Consequently, this study, in people affected by stroke, aimed to determine the (a) test–retest reliability of the variables provided via the Kinect system during the TUG; (b) additional and complementary information provided by these measures, including their redundancy with the time taken to complete the TUG; and (c) extent to which the addition of Kinect-TUG variables improved prediction of performance on other common clinical assessments of physical function compared to standard TUG time. It was hypothesized that (a) the Kinect-TUG variables would exhibit excellent test–retest reliability; (b) some variables would be nonredundant with total TUG time, providing additional and complementary information regarding task performance, and (c) the addition of Kinect-TUG variables combined with total TUG time would strengthen prediction models for performance on other clinical tests.

Methods

Participants

Thirty participants were recruited following stroke between April 2012 and July 2013 from the Community Therapy Service at The Royal Melbourne Hospital. To be eligible for the study participants must have been diagnosed with a noncerebellar ischemic or hemorrhagic stroke more than 3 months prior to recruitment, be attending physiotherapy for balance or mobility issues, be able to stand unsupported for greater than 30 seconds, and have a Mini-Mental State Examination score ≥20. Participants were excluded if they had severe apraxia, severe dysphasia, or any other medical condition that may affect their balance ability (eg, severe arthritis, progressive neurological disorders, or visual impairment). This research was given ethical approval from the institutional ethics committees. All participants provided written informed consent prior to testing.

Protocol

Participants completed 2 testing sessions at the hospital, separated by 1 week. The assessments conducted during each of the testing sessions included, in order, the 10MWT, TUG, Step Test (ST), and Functional Reach (FR) test, and were undertaken in accordance with previously published protocols.^1,11-13 All assessments were performed twice on each of the 2 days, with the best trial used for analysis, except the ST, which was only performed once on each day. The same standardized procedure was followed during each testing session, in descending order of intensity to reduce fatigue. Participants wore shoes and could use gait aids and/or ankle-foot orthoses for the 10MWT and TUG only, and were instructed to wear tight-fitting clothing.

Instrumented Assessment of the Timed Up and Go

A Microsoft Xbox360 Kinect camera was used during the 2 trials of the TUG test to obtain spatiotemporal and kinematic information from the participant. The Kinect integrates information from video and depth-sensing cameras to create a 3D representation of the field of view.¹⁴ An artificial intelligence algorithm provided freely by Microsoft is then used to automatically locate and track the joint centers and major anatomical landmarks of the body.¹⁵ This enables the camera to provide information on the 3D movement of the participant in close to real time.

The Kinect was calibrated using a custom written software program prior to each testing session using a protocol described previously.⁸ The Kinect is capable of detecting human motion between approximately 0.8 m and 3.8 m from the camera. As the TUG test requires space for the chair and for a 3-m walk and turn, the limitations of the field of view of the camera required it to be placed off-center from the starting point of the TUG test. This enabled the collection of data during the standing and walking movements occurring in a region extending from the chair to approximately 1 m from the camera. This setup is outlined in Figure 1.

Figure 1.

Positioning of the Microsoft Kinect in relation to the starting point of the Timed Up and Go test. The solid gray lines represent the field of view of the Kinect camera. The participant movement during the timed up and go test is represented by the dashed line.

Data Analysis

Custom-written LabVIEW 2009 software (National Instruments, Austin, TX) and the official Microsoft Kinect Software Development Kit (SDK; Beta 2 Release—.dll version 1.0.0.45) were used to acquire and analyze the Kinect data recorded during the TUG test. Seven significant events throughout the fastest TUG test were identified and used to determine 7 kinematic movement variables. These events included the start of the test, standing, the first step, the first stride, when the participant was 2 m from the camera facing toward it, when the participant was 2 m from the camera facing away from it, and the end of the test. All variables were determined by visual inspection of depth sensor images and 3D plots of the skeletal tracking anatomical landmarks and the graphed displacement and velocity of the skeletal tracking data. The start of the test was identified as the frame prior to the detection of trunk movement, and standing was achieved when the shoulder center reached peak height as shown on the vertical axis of the graphed skeletal tracking data. The first step and first stride were identified from the heel strike phase of gait, evident by a plateau in ankle displacement and velocity on the anterior–posterior axis of the skeletal tracking data. The events 2 m toward and away from the camera were recognized by the first breach of the trunk past the 2 m distance from the camera threshold, and return of the trunk within the 2 m threshold, respectively. Finally, the end of the test was deemed the cessation of the posterior trunk movement during the sitting movement, representing the return of the trunk to the back of the chair. These key events were used to determine the seven main Kinect-TUG variables, including peak trunk flexion angle during standing, peak trunk flexion angular velocity during standing, first step length, first stride length, gait speed, turn time (deemed the time between crossings of the 2m threshold), and total TUG time.

Statistical Analysis

To assess the test–retest reliability of the various clinical and Kinect-TUG variables, intraclass correlation coefficients (ICC_2,k) were used as indices of relative reliability, and these coefficients were calculated in a 2-way analysis of variance based on absolute agreement. Absolute reliability was represented by (a) the standard error of measurement (SEM), which was derived from the square root of the mean square error term from the respective repeated analysis of variance; (b) the minimum detectable change (MDC), which was derived by multiplying the point estimate of the SEM, the square root of 2 and the z-score of 1.96 (95% confidence level); and (c) the test–retest percentage error, which was computed by expressing the standard deviation (SD) of the test–retest difference scores (bias) as a percentage of the mean measure of the 2 testing sessions (100 × [2SD of bias]/([Meant_Test+Mean_Retest]/2)).¹⁶

To examine the associations between the clinical tests and the combination of Kinect-TUG variables, we used separate multivariable regression models that included age as a covariate. To minimize model overfitting and to avoid multicollinearity from correlated variables, we a priori specified that Kinect-TUG variables which were strongly related to total TUG time (Spearman ρs ≥ 0.80) would be deemed redundant and excluded from the model. To further account for potential model overfitting, the regression coefficients in all multivariable models were estimated using penalized maximum likelihood methods.¹⁷ Given that our various Kinect-TUG variables were measured on different scales and units, we scaled the regression coefficients of all variables by their interquartile ranges such that each coefficient may be interpreted as comparing participants with relatively high (75th percentile) versus low (25th percentile) scores. Apart from allowing meaningful between-predictor comparisons, this scaling also represented a more clinically meaningful distinction than the conventional one-unit change in predictor values.¹⁸ Finally, to assess the incremental predictive value of the inclusion of additional Kinect-TUG variables over the standard measure total TUG time, we calculated incremental R² values from nested models with and without the addition of the Kinect-TUG variables. All statistical analyses were done with R software, version 3.0.1 (R Foundation, Vienna, Austria).

Results

Participant characteristics are described in Table 1. Ten participants used walking aids (single point walking stick = 7, 4-point walking frame = 2, forearm crutch = 1). No participant required physical assistance from the assessor during the tests. The total TUG time derived from the stopwatch and Kinect showed excellent association (ρ = 0.99, P < .001), validating the Kinect for this measure. The results for the test–retest reliability analysis are provided in Table 2. With the exception of trunk flexion angle (ICC = 0.73), all clinical and Kinect-TUG variables demonstrated excellent reliability, with ICCs >0.90.

Table 1.

Participants’ Characteristics.^a

Descriptor	Value
Age (years)	68 ± 15
Height (cm)	166.7 ± 9.4
Body mass (kg)	72.5 ± 11.9
Gender (n = male)	21
Mini-Mental State Examination score (/30)	27.1 ± 2.7
Lesion type (% infarct)	73
Lesion side (% right)	63
Time since stroke (months)	21 ± 19
Comorbidity (%)
Hypertension	70
Dyslipidaemia	47
Type 2 diabetes mellitus	27

Values are reported as mean ± standard deviation unless otherwise indicated.

Table 2.

Test–Retest Reliability Measures for Clinical and Kinect-Derived Variables.

Variables	Test (Mean ± SD	Retest (Mean ± SD	ICC (95% CI)	SEM	MDC	PE (%)
Clinical tests
10MWT (m/s)	0.97 ± 0.35	0.95 ± 0.33	0.98 (0.97-0.99)	0.06	0.17	13
Step Test (n)	9.33 ± 4.86	9.63 ± 4.97	0.99 (0.98-1.00)^a	0.59	1.71	13
Forward Reach Test (cm)	28.50 ± 7.37	29.17 ± 7.61	0.96 (0.91-0.98)	2.18	6.32	16
Stopwatch TUG (s)	17.21 ± 9.89	16.85 ± 9.51	0.99 (0.99-1.00)	1.05	3.03	13
Kinect-derived variables
Trunk flexion (deg)	51.00 ± 6.01	50.67 ± 7.72	0.73 (0.42-0.87)	4.56	13.20	18
Flexion angle velocity (deg/s)	30.50 ± 16.25	31.47 ± 17.20	0.93 (0.85-0.96)	6.26	18.10	41
Step length (m)	0.37 ± 0.17	0.39 ± 0.19	0.94 (0.88-0.97)	0.06	0.16	31
Stride length (m)	0.82 ± 0.35	0.82 ± 0.38	0.98 (0.95-0.99)	0.08	0.23	20
Gait speed (m/s)	0.82 ± 0.30	0.82 ± 0.31	0.97 (0.94-0.99)	0.08	0.22	19
Turning time (s)	8.75 ± 4.17	8.74 ± 4.41	0.99 (0.97-0.99)	0.64	1.85	15
Total time (s)	18.70 ± 10.48	18.02 ± 10.90	0.99 (0.97-0.99)	1.68	4.85	19

Abbreviations: SD, standard deviation; ICC, intraclass correlation coefficient; CI, confidence interval; SEM, standard error of measurement; MDC, minimum detectable change; PE, percentage error; 10MWT, 10-meter walk test; TUG, Timed Up and Go.

Denotes ICC_(2,1), all other analyses ICC_(2,2).

The relationship between the components of the TUG test is reported in Table 3, with the exception of trunk flexion angle velocity, which, with an unacceptably high (41%) percentage error between tests, was excluded from further analysis.¹⁶ The total TUG time was strongly correlated to all Kinect-TUG variables with the exception of trunk flexion angle (ρ = −0.23, P > .05) and step length (ρ = 0.70, P < .001).

Table 3.

Redundancy Analysis of the Kinect-Derived Timed Up and Go Variables Using Spearman’s Correlation Matrix.

	Trunk Flexion	Step Length	Stride Length	Gait Speed	Turning Time
Step length	0.29
Stride length	0.31	0.91*
Gait speed	0.18	0.67*	0.83*
Turning time	−0.20	−0.72*	0.83*	0.98*
Total time	−0.23	−0.70*	−0.82*	−0.94*	−0.98*

Significant at P < .001.

The results of the regression models predicting clinic-based performance measures are provided in Table 4. Based on our a priori defined variable reduction strategies, only 3 Kinect-TUG variables (total TUG time, step length, and trunk flexion angle) were included in regression models with age as a covariate. Across all models, age and trunk flexion angle were statistically non-significant, whilst total TUG time was a significant predictor. Step length significantly contributed to the association with gait speed and the ST, but not with the FR. Specifically, the addition of this variable increased the R² values by 7% and 6% in the gait speed and ST models, respectively. An increase in first step length of 0.19 m (equivalent to its interquartile range) was associated with 0.15 m/s (95% confidence interval = 0.07-0.23 m/s; P < .001) faster gait speed and 1.79 (95% confidence interval = 0.24-3.34; P < .001) more steps on the ST. Noteworthy, these observed effect sizes were comparable to their corresponding minimal detectable change scores of 0.17 m/s for gait speed and 1.7 steps for the ST (Table 2).

Table 4.

Multivariable Regression Associations Between Kinect-Derived Timed Up and Go Measures and Clinical Performance Assessments.

	Low	High	Adjusted Difference^a (95% CI)	P Value
10MWT (R² = .86)
Age	60	78	0.01 (−0.06 to 0.07)	.81
Total time	11.9	23.4	−0.26 (−0.33 to −0.19)	<.001*
Step length	0.25	0.44	0.15 (0.07 to 0.23)	<.01*
Flexion angle	48.5	55.1	−0.01 (−0.07 to 0.05)	.75
Forward reach (R² = .53)
Age	60	78	−1.59 (−3.86 to 0.68)	.16
Total time	11.9	23.4	−3.73 (−6.01 to −1.46)	<.01*
Step length	0.25	0.44	2.04 (−0.56 to 4.64)	.12
Flexion angle	48.5	55.1	−0.01 (−2.09 to 2.08)	.99
Step test (R² = .73)
Age	60	78	0.88 (–0.42 to 2.18)	.17
Total time	11.9	23.4	−3.98 (−5.3 to −2.66)	<.001*
Step length	0.25	0.44	1.79 (0.24 to 3.34)	.03*
Flexion angle	48.5	55.1	−0.99 (–2.18 to 0.19)	.10

Abbreviations: CI, confidence interval; 10MWT, 10-m walk test.

The adjusted difference indicates the difference in clinic-based performance measures between the 75th versus the 25th percentile for each variable. For example, other variables being equal, participants with a step length at the 75th percentile (0.44 m) would have, on average, a 0.15 m/s faster 10MWT score than participants scoring at the 25th percentile (0.25 m).

Significant association at P < .05.

Discussion

This study investigated the potential clinical efficacy of instrumenting the TUG using a Kinect by examining the reliability, redundancy, and additive benefits of the derived outcome measures. All measures other than trunk flexion angle were found to possess excellent (ICC > 0.90) reliability. While many of the Kinect-TUG variables were found to be redundant with total TUG time, first step length and trunk flexion angle provided independent and complementary information regarding TUG performance. The incremental predictive value of first step length beyond total TUG time was statistically and clinically significant in models predicting gait speed and ST performance. In general, these results support our hypotheses.

Of the clinical tests, gait speed during the 10MWT had the strongest association (R² = .86) with the Kinect-TUG variables. This finding is consistent with previous research which reported a strong relationship between the 10MWT and TUG in people following stroke.¹⁹ In contrast, the clinical test with the weakest association (R² = .53) with Kinect-TUG variables was the FR, and the addition to the model of step length and trunk flexion angle during standing did not significantly improve the relationship. This reflects that the FR is composed of actions that have the least similarities to the individual components of the TUG test. This is intuitive given that the TUG test comprised primarily aspects of gait, lower extremity strength and agility, compared with the FR which focuses on dynamic, yet slow movement speed, postural control at the limits of stability over a stationary base of support.¹³

In regard to reliability, with the exception of trunk flexion angle (ICC > 0.73) all Kinect-TUG variables possessed excellent test–retest reliability (ICC range = 0.93-0.99). This is similar to the findings for the clinical tests (ICC range = 0.96-0.99), with these values and our MDC scores comparable to previous studies assessing these clinical tests in a stroke population.^19-22 It must be noted, however, that some of the percentage error values for the Kinect-TUG variables were relatively high, particularly trunk velocity. This is likely because of 1 of 3 primary reasons: (a) poor automatic location of the anatomical landmarks of the trunk in the seated position, which leads to errors in the calculation of angular displacement and velocity; (b) variability in trunk movement patterns when rising from the chair; and/or (c) difficulty identifying the beginning and end of the standing movement. Reduced reliability for trunk flexion velocity is consistent with previous findings,⁴ which have shown lower reliability for time taken to stand than other TUG components. This may have been due to the defining criteria used to determine standing, as many participants had not reached peak shoulder center height prior to taking their first step. Furthermore, stand time may also be affected by the wide range of strategies used to perform this action, or by factors such as clothing which may influence results.²³

The results of this study indicate that instrumenting the TUG using the Kinect provides potentially useful additional information that may be sensitive to change in performance over time or in response to therapeutic interventions. However, as previously stated this is not the only method of instrumenting the TUG. The potential for superior accuracy and precision of accelerometer, gyroscope, or marker-based video analysis systems in comparison with the Kinect may be beneficial, particularly for examining the kinematics of the turning movement; however, these systems often require the use of multiple body mounted sensors to be worn throughout the test. In contrast, the Kinect motion sensing camera provides a means for a clinically feasible, inexpensive kinematic analysis of physical function that requires no subject preparation and little clinician setup and analysis time, and therefore affects minimally. To support this statement regarding minimal clinician analysis time, these variables could potentially be easily automated and instantly reported on trial completion. The entire test could also be performed in a hands-free manner, with voice commands given via the Kinect’s in-built 3D microphone array to commence and terminate data collection. With respect to the inability to quantify the kinematics of the turn during the TUG, although this may be important in some populations evidence indicates that the time taken to turn is more important than kinematic or axial segment coordination for assessing falls risk in people with stroke.⁶ Additionally, future iterations of the Kinect or similar unobtrusive motion sensing technology are likely to possess improved accuracy, which may increase their efficacy for physical function assessment. Given the strong test–retest reliability, future research should explore the responsiveness of the instrumented TUG to change over time in people with stroke to aid in planning and evaluation of treatment.

A study limitation is that our clinical tests did not encompass specific aspects of physical function such as maximal strength, or other functional tasks where falls may occur such as stair descent. Future studies may wish to examine the association between the variables derived from different components of the TUG with clinical assessments that may be more specific to that aspect of function. For example, the association between trunk flexion angle during standing may be more strongly associated with a test such as the repeated sit-to-stand than the assessments performed in this study.

In conclusion, our results suggest that instrumenting the TUG using a Kinect can provide reliable measures of a range of distinct components of the test. These additional variables could provide unique and important information about a patient’s physical function when compared with the commonly reported time taken to complete. Given the low cost and widespread availability of the Kinect system, in the future this tool may provide a means for conducting large scale clinical assessments of physical function and potential falls risk following stroke.

Footnotes

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Ross Clark may release the software used in this study in the future.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was partially supported by a National Stroke Foundation Research Honours Grant.

References

Bernhardt

Ellis

Denisenko

Hill

. Changes in balance and locomotion measures during rehabilitation following stroke. Physiother Res Int. 1998;3:109-122.

Faria

Teixeira-Salmela

Silva

Nadeau

. Expanded Timed Up and Go test with subjects with stroke: reliability and comparisons with matched healthy controls. Arch Phys Med Rehabil. 2012;93:1034-1038.

Greene

O’Donovan

Romero-Ortuno

Cogan

Scanaill

Kenny

. Quantitative falls risk assessment using the timed up and go test. IEEE Trans Biomed Eng. 2010;57:2918-2926.

Salarian

Horak

Zampieri

Carlson-Kuhta

Nutt

Aminian

. iTUG, a sensitive and reliable measure of mobility. IEEE Trans Neural Syst Rehabil Eng. 2010;18:303-310.

Weiss

Herman

Plotnik

Brozgol

Giladi

Hausdorff

. An instrumented timed up and go: the added value of an accelerometer for identifying fall risk in idiopathic fallers. Physiol Meas. 2011;32:2003-2018.

Hollands

Zietz

Miles Wing

Wright

Van Vliet

. Kinematics of turning 180° during the Timed Up and Go in stroke survivors with and without falls history. Neurorehabil Neural Repair. 2010;24:358-367.

Dite

Temple

. Development of a clinical measure of turning for older adults. Am J Phys Med Rehabil. 2002;81:857-866.

Clark

Pua

Y-H

Fortin

. Validity of the Microsoft Kinect for assessment of postural control. Gait Posture. 2012;36:372-377.

Stone

Skubic

. Evaluation of an inexpensive depth camera for in-home gait assessment. J Ambient Intell Smart Environ. 2011;3:349-361.

10.

Clark

Bower

Mentiplay

Paterson

Pua

. Concurrent validity of the Microsoft Kinect for assessment of spatiotemporal gait variables. J Biomech. 2013;46:2722-2725.

11.

Thrane

Joakimsen

Thornquist

. The association between timed up and go test and history of falls: the Tromso study. BMC Geriatr. 2007;7:1.

12.

Mercer

Freburger

Chang

Purser

. Step test scores are related to measures of activity and participation in the first 6 months after stroke. Phys Ther. 2009;89:1061-1071.

13.

Duncan

Weiner

Chandler

Studenski

. Functional reach: a new clinical measure of balance. J Gerontol. 1990;45:192-197.

14.

Menna

Remondino

Battisti

Nocerino

. Geometric investigation of a gaming active device. Proceedings of Videometrics, Range Imaging and Applications XI, SPIE Optical Metrology, Vol 80850G-1-15. Bellingham, WA: ETATS-UNIS, Society of Photo-Optical Instrumentation Engineers; 2011.

15.

Shotton

Fitzgibbon

Cook

. Real-time human pose recognition in parts from single depth images. Proc IEEE Conf Comput Vis Pattern Recogn. 2011;2:1297-1304.

16.

Critchley

. A meta-analysis of studies using bias and precision statistics to compare cardiac output measurement techniques. J Clin Monit Comput. 1999;15:85-91.

17.

Moons

Donders

Steyerberg

Harrell

. Penalized maximum likelihood estimation to directly adjust diagnostic and prognostic prediction models for overoptimism: a clinical example. J Clin Epidemiol. 2004;57:1262-1270.

18.

Harrell

Jr . Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis. New York, NY: Springer; 2001.

19.

Flansbjer

U-B

Holmbäck

Downham

Patten

Lexell

. Reliability of gait performance tests in men and women with hemiparesis after stroke. J Rehabil Med. 2005;37:75-82.

20.

Perera

Mody

Woodman

Studenski

. Meaningful change and responsiveness in common physical performance measures in older adults. J Am Geriatr Soc. 2006;54:743-749.

21.

Hong

Goh

Chua

. Reliability and validity of step test scores in subjects with chronic stroke. Arch Phys Med Rehabil. 2012;93:1065-1071.

22.

Outermans

van Peppen

Wittink

Takken

Kwakkel

. Effects of a high-intensity task-oriented training on gait performance early after stroke: a pilot study. Clin Rehabil. 2010;24:979-987.

23.

Millington

Myklebust

Shambes

. Biomechanical analysis of the sit-to-stand motion in elderly persons. Arch Phys Med Rehabil. 1992;73:609-617.