Abstract
Introduction
Visual biofeedback of lower extremity kinematics has the potential to enhance retraining of pathological gait patterns. We describe a system that uses wearable inertial measurement units to provide kinematic feedback on error measures generated during periods of gait in which the knee is predominantly extended (‘extension period’) and flexed (‘flexion period’).
Methods
We describe the principles of operation of the system, a validation study on the inertial measurement unit derived knee flexion angle on which the system is based, and a feasibility study to assess the ability of a child with cerebral palsy to modify a gait deviation (decreased swing phase knee flexion) in response to the feedback.
Results
The validation study demonstrated strong convergent validity with an independent measurement of knee flexion angle. The gait pattern observed during training with the system exhibited increased flexion in the flexion period with maintenance of appropriate extension in the extension period.
Conclusions
Inertial measurement units can provide robust feedback during gait training. A child with cerebral palsy was able to interpret the novel two phase visual feedback and respond with rapid gait adaptation in a single training session. With further development, the system has the potential to support clinical retraining of deviated gait patterns.
Introduction
Cerebral Palsy (CP) is the most common cause of motor disability in childhood. 1 A wide range of interventions including physical therapy (e.g. serial casting, strength training, functional gait retraining assisted by physical therapists), pharmacological intervention (e.g. baclofen, botulinum neurotoxin), and surgical intervention (e.g. selective dorsal rhizotomy, tendon lengthening) are used to improve mobility in children with CP. 2 Some gait deviations in children with CP can persist and be recalcitrant to retraining even after pharmacologic or surgical management of spasticity and contracture. The potential for computational biofeedback modalities to augment gait retraining have been explored,3–8 in recognition of the consistency and objectivity of feedback provided.
Several studies have used optical motion analysis to provide measurement data for such biofeedback. Compared with this modality, inertial measurement unit (IMU) based biofeedback offers advantages for potential interventions outside of the laboratory and larger potential capture volumes. Byl et al. 9 developed smart shoes and pants using pressure sensors and IMUs which provide real-time visual feedback on plantar pressure, step lengths, stride widths, and joint angles. Across all patients with Parkinson's disease or stroke, there were significant gains in mobility, balance, range of motion and strength over 12 visits, with no significant differences in the gain scores between the biofeedback gait training group and conventional gait training group. Ginis et al. 10 developed a biofeedback system combining IMUs with a smartphone application which provides real-time visual feedback on cadence, stride length, symmetry, and gait speed. The biofeedback gait training group improved significantly more on balance over 18 visits and maintained quality of life at 4-week follow-up, whereas the conventional gait training group deteriorated. Schliessmann et al. 11 developed a gait training system that used shoe-fixed IMUs to provide verbal feedback on the foot-to-ground angle, stride length, stance duration, and/or swing duration depending on which parameter showed the highest deviation from the physiological norm in each participant. The deviation from a normal gait pattern significantly decreased over 3 visits in three groups of subjects in populations with or at risk for gait impairments: individuals with incomplete spinal cord injury, individuals post-stroke, and individuals aged 65 or older.
Although these studies reported positive results on improving gait quality in different populations, the biofeedback these IMU-based systems provided are mostly limited to spatial and temporal gait parameters. Only one study provided visual kinematic biofeedback on joint rotation angles (the transverse plane toe-out angle); for other planes, raw signals were interpreted by an engineer and a therapist and presented as verbal cues. 9 Since children with CP often present significant gait deficits in sagittal plane knee movements, our group reported a first prototype of a visual kinematic biofeedback system using wearable IMUs to generate feedback on the knee flexion pattern, and the feasibility of this was tested in typically developing children. 12 With this system, the user receives feedback on the deviation of the knee joint flexion pattern in the most recent step from a targeted pattern. This prototype provided a feedback signal driven by the error across the whole gait cycle as well as a peak knee flexion cue. When presented with feedback based on novel target knee flexion patterns, the children produced gait patterns with appropriate modifications to peak knee flexion; however, it was found that unintended changes in other phases of the gait cycle were sometimes produced. In order to reduce this tendency and encourage convergence to a targeted knee flexion pattern across the whole cycle, we introduce a new feedback system that provides simultaneous feedback during periods of gait in which the knee is predominantly extended (‘extension period’) and flexed (‘flexion period’). We report the principles of operation of the new feedback system, a validation study on the IMU-derived knee flexion angle on which the system is based, and a feasibility study to assess the ability of a child with CP to modify a gait deviation in response to this feedback.
Methods
Two phase visual feedback
The feedback software was developed using MATLAB (The Mathworks, Natick, MA) and the MTW Devkit (Xsens Technologies BV, Enschede, Netherlands) programming interface. Three IMUs (Xsens) that use gyroscope and accelerometer signals to measure three-dimensional orientations and accelerations are placed on the lower limb to be trained (anterior thigh, posterior shank, and the heel of the shoe) (Figure 1(a)). Orientation measures for such sensors are computed through signal integration, a process that can be susceptible to signal drift. However, IMU’s that utilize both accelerometers and gyroscopes are able to provide essentially drift free operation as long as the average acceleration during the measurement period is zero. This condition holds well for the essentially periodic and average stationary motions in treadmill walking, and signals remain stable over typical training periods of 5 to 10 minutes. The IMUs used also incorporate magnetometers to provide a measurement of heading with reference to magnetic north; however, this measurement can be susceptible to magnetic disturbance. Since the relevant kinematics of the knee during gait predominantly occur in a single plane, the heading measurement is not needed and is not used in our application. A sensor-based measurement of the knee flexion angle is calculated as the difference between the thigh and shank sensors’ rotation about the sensor’s longitudinal axis (‘roll axis’). A sampling rate of 60 Hz is used, and calibration and scaling procedures described below are used to process this raw sensor-based knee flexion angle for feedback generation.
Gait retraining IMUs setup and feedback interface. (a) Gait retraining on a treadmill with IMUs positioned on the participant’s paretic lower limb. (b) Feedback interface. Left panel: extension period; right panel: flexion period. Needle positions in the upper half of the interface indicate the performed pattern is overall more flexed than the target, while positions in the lower half indicate a more extended pattern. The red tick marks delimit the maximum range. Motivation to respond to the feedback is provided by the awarding of points added to a cumulative score, with one, two, three points respectively in the outer, middle, and inner colored sectors.
Feedback is provided shortly after initial foot contact, based on a comparison of the measured knee flexion data with the target flexion pattern. The target pattern can be prescribed as appropriate to the application: currently, a normative target knee pattern is selected from previously published optical motion data for typically developing children 13 based on walking speed normalized to height. Foot contact with the support surface is detected by the heel sensor. The time point for separating successive steps is the first sample during an event of low linear acceleration and angular velocity recorded by the heel sensor. A sample window for feedback is provisionally defined between the just detected contact time point and the immediately prior contact time point. Testing revealed that this event lags true initial contact by approximately 5% of the gait cycle period, and we adjust for this lag by shifting the feedback sample window earlier in time by a corresponding number of samples. The system provides separate feedback on extension and flexion periods of movement. The extension period is defined as terminal swing (90-100%) of the previous step, plus loading response (0-10%) and mid and terminal stance (10-50%) of the most recent step. The flexion period is the pre swing (50-60%), initial and mid swing (60-90%) of the most recent step. To achieve this, the final feedback sample window is shifted back in time by an additional 10% of the window duration.
Before calculation of the feedback metrics, dynamic time warping (DTW)14,15 is used to attenuate the effect of varying temporal alignment between the measured and target signals. The DTW algorithm returns a list of data indices for both signals that represent their maximal alignment in time. Error metrics are then calculated using these aligned signals, which are then used to drive the feedback display for the extension and flexion periods via their corresponding needles (Figure 1(b)). The signs of the mean signed errors are used to determine whether the needle is in the upper or lower half of each display. The magnitudes of needle deflections are driven by mean absolute errors via a quadratic function which allows for feedback to be provided for large errors during early training, while maintaining adequate sensitivity for fine tuning patterns with lower errors.
Validation of IMU-derived knee flexion angles
We performed a validation experiment comparing knee flexion angle data measured by our system with angles simultaneously measured by an optical motion capture system. An adolescent female with right hemiplegic CP (age 16.2 yrs, height 1.52 m, weight 44.5 kg) was the participant for this comparison. This participant was categorized to Level 1 of the Gross Motor Function Classification System (GMFCS) for children with CP. 16 The inclusion criteria of participants of this study included: 1) diagnosis of hemiplegic CP; 2) age 7 to 17 years old; 3) ability to walk on a treadmill without assistive devices; 4) ability to understand spoken English at the level needed to follow instructions. The exclusion criteria included: 1) Botulinum toxin treatment less than 16 weeks before study commencement; 2) Significant injury in lower limbs; 3) Risk factor for stroke or heart attack while exercising. For all reported study activities, written informed consent and/or assent were provided by the participant and a parent or guardian according to the regulations of the Kessler Foundation Institutional Review Board. Participant safety during treadmill walking was ensured by a pediatric physical therapist.
Reflective markers were placed on the greater trochanter, femoral lateral epicondyle, fibular head, and lateral malleolus of the participant’s right leg. Four 30 second treadmill walking trials without feedback were recorded in total, two at 0.80 m/s, and two at 1.12 m/s. Marker data were collected at 120 Hz with a compact three camera motion capture system (Optitrack Trio, NaturalPoint, Inc., Corvallis, OR) placed approximately 2 meters to the right of the subject. Knee flexion angle data for the Trio were calculated in MATLAB using the 3 D angle between a femoral vector (epicondyle to trochanter markers) and a tibial vector (malleolus to fibular head markers). Knee flexion angles calculated from IMU data as described above were resampled to 120 Hz and the datasets were aligned in time by the optimization of a variable time offset.
One session feedback experiment
A feasibility study was performed to assess whether a child with hemiplegic CP could interpret and respond appropriately to the feedback provided. An adolescent male with right hemiplegic CP (age 11.3 yrs, height 1.55 m, weight 46.3 kg, GMFCS 16 Level 1) who presented a deficit in knee flexion at mid-swing on his paretic side participated in the study.
The participant first stood on both legs with his knee extended on the paretic side. He then stood on the non-paretic leg with the shank on the paretic side swung back to 60° knee flexion relative to the extension posture. These knee flexion angles were measured using a long arm goniometer with the axis on the lateral epicondyle of the participant’s femur, the stationary arm located along the femur to the greater trochanter, and the movement arm along the fibula to the lateral malleolus. Scaling and offset for raw IMU angle data were performed based on this two point static calibration and a scaling factor of 0.75, which is consistent with the increased IMU-based knee flexion angle excursions during walking as observed in our validation study.
After the calibration procedure described above, the participant walked on the treadmill at 0.5 m/s for 1 minute while the baseline knee flexion pattern was recorded. The feedback interface (Figure 1(b)) displayed on a screen in front of a treadmill was then introduced to the participant while he was standing on the treadmill. Verbal instructions on the interface were given to the following effect: “The left dial shows your knee movement while your right foot is on the ground. The right dial shows your knee movement while your right foot is in the air. For each dial, if the needle is on the top, your knee is too bent and you should straighten your knee more, and if the needle is on the bottom, your knee is too straight and you should bend your knee more. Think about how your knee moves and follow the cues on the screen to move the needles to the colored regions to get points.” A 10-minute practice period was allotted to interactive demonstration and practice with the goal that the participant had a clear understanding of the task and how to interpret the feedback. Three training bouts followed, each one consisting of a 3 minute Feedback (FB) trial and a 3 minute Non-Feedback (NFB) trial. The NFB trials were included in accordance with motor learning principles that seek to avoid feedback dependence,3 and during these trials the participant was asked to reproduce the walking pattern that they had been learning in the preceding FB trials. Following training, the participant was asked to complete a short questionnaire to evaluate their experience and solicit input.
Results
Results of the validation experiment
Linear regressions between the datasets for each trial illustrated strong convergent validity of the IMU based knee flexion angles with the optical marker based angles, with R2 correlations varying between 0.932 and 0.966 across the four trials (Table 1). Predictive equations took the form (IMU angle) = m * (marker-based angle) + b, with the mean values across trials of the slope m and intercept b calculated as 1.27 and -33° respectively. The mean slope indicates that knee flexion angle excursions recorded by the IMU system are somewhat higher than those generated from the optical marker data. The mean intercept indicates a zero-point measurement offset between the two systems.
Trial characteristics and linear regression analysis of the validation experiment.
Results of the feedback experiment
The root-mean-square errors (RMSE) between the measured and target knee flexion patterns were calculated for the last ten strides of all baseline, FB, and NFB trials. The mean and standard deviation of the RMSE across the ten strides was then compared between trials (Figure 2(a)). The mean RMSE for the 1st FB trial was markedly lower than for the baseline trial, and a slight increase in the mean RMSE for the 2nd FB trial was followed by the lowest observed mean RMSE value in the 3rd FB trial. Performance in the 3rd FB trial showed migration toward the target pattern via increased swing phase knee flexion and maintenance of appropriate extension throughout the majority of stance phase (Figure 2(b)). Live observations and review of recorded video showed increased paretic limb step length and a more normative overall gait pattern. The mean RMSE for the NFB trials were intermediate when compared to baseline and FB values, showing evidence that the participant had some success in reproducing the target pattern without feedback. Variability between strides (error bars and shaded band in Figure 2) was larger during training than in baseline walking, as expected in the early stages of learning a new motor task. The participant was observed to be enthusiastic and engaged in the feedback interface during testing, and rated the training experience positively. On a five-level Likert scale design, the participant responded “Medium (3 of 5)” to “How easy or difficult did it seem to get points?”, “Easy (4 of 5)” to “How hard was it to tell how you had to change your walking?”, and “Very fun (5 of 5)” to “How fun or boring was it to try to change your walking to collect points?”.
Learning effect of the participant as a result of gait retraining with feedback. Data points were averaged from the last 10 strides of each trial. (a) Mean and SD (error bar) of knee flexion angle RMSE over repeated training trials. (b) Mean and SD (shaded band) of knee flexion angle curves in the baseline and last FB trial compared with the target curve.
Discussion
We report a new method for real-time pattern-based visual kinematic feedback based gait retraining. This system builds on an earlier prototype tested in typically developing children 12 by providing simultaneous feedback of knee kinematic pattern errors during the extension and flexion periods of the gait cycle, and by incorporating a more advanced scoring system to better reward participants as target patterns are approached. The feedback system works with a conventional treadmill and could be adapted for overground walking which could make it suitable for clinical, community or home setup. The participant’s gait pattern exhibited normalization of the flexion period with maintenance of the pattern in the extension period, consistent with the two phase feedback provided.
Previously reported kinematic feedback techniques for gait training in patients with CP7,8 targeted deviations at specific instants (e.g. difference in peak knee flexion angles) rather than deviations throughout the entire gait cycle (e.g. difference in knee flexion angle trajectories). Also of note, those studies used traditional optical motion capture systems to collect kinematic data and generate feedback signals. The limited portability of optical systems presents challenges for using such methods in a pediatric clinic. The hardware requirements and setup time of the IMU-based feedback platform are much lower than those for motion analysis systems 5 or virtual reality environments. 6
Our study demonstrated strong convergent validity of the IMU measurement with established optical motion capture measurement methods (the overall R2 correlations approximating 1). The study indicated a greater rate of joint excursion for the IMU system than the optical system, and we believe that this is caused by movement of the IMUs relative to the overall limb segments because they overlie muscles which contract during gait. As described in our feasibility study, the feedback system calibration procedure implements a scaling factor to compensate for this difference and produce appropriate feedback signals.
A limitation of our study is that the IMU signal validation was conducted on a single participant. Whereas our two study participants’ heights and body weights were very close to each other and yielded equal body mass indices of 19.3, validity of adoption of the scaling factor in others - particularly across a range of body types - merits further study and development. While in our testing we have observed that the use of this scaling factor does allow the system to provide qualitatively appropriate feedback cueing across a wide range of body types, our group is currently pursuing other methods to allow rapid, subject-specific sensor calibration based on dynamic walking. The optical method used in the validation study to measure kinematics used a total of four markers to measure femoral and tibial movement independently. However, this method has a limitation common to all skin marker methods: it is susceptible to error from soft tissue motion relative to the intended bone fixed landmarks. Due to the relatively low body mass index of the subject as cited above, we believe this error was small in comparison with the errors between measurement types that were our focus. An additional limitation is that this study only collected kinematic data for the knee; for clinical applicability, it is also important to study adaptation of the overall gait pattern with feedback exposure.
The potential for the use of such a system to enhance clinical pediatric gait training is supported by the demonstrated ability to deploy it in a pediatric gym, the positive reaction of the participant to the system and the appropriate response and adaptation in a single training session. Further testing is being performed to assess the ability of the system to provide therapeutic benefit in a series of participants.
Footnotes
Declaration of conflicting interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Authors NO, NE, JFC, SB, and PB are co-inventors on a provisional US patent application that describes aspects of the two phase feedback system described in this article. The authors declare no other competing or conflicting interests with respect to this article.
Funding
The authors disclose receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by Children’s Specialized Hospital Foundation (Mountainside, NJ, United States) and Kessler Foundation (East Hanover, NJ, United States).
Guarantor
PB, Kessler Foundation, United States.
Contributorship
XL, NE, PB performed data collection. XL and PB performed data analysis. XL wrote the first draft of the manuscript. NO, NE, PB developed the feedback application with design input from JFC, KB, SB, HS. All authors reviewed and edited the manuscript and approved the final version of the manuscript.
