Abstract
Gait tasks are commonly administered during motor assessments of children with neurodevelopmental disorders (NDDs). Gait analyses are often conducted in laboratory settings using costly and cumbersome experiments. In this paper, we propose a computational pipeline using computer vision techniques as an ecological and precise method to quantify gait in children with NDDs with challenging behaviors. We analyzed videos of 15 probands (PB) and 12 typically developing (TD) siblings, engaged in a preferred-pace walking task, using pose estimation software to track points of interest on their bodies over time. Analyzing the extracted information revealed that PB children had significantly less whole-body gait synchrony and poorer balance compared to their TD siblings. Our work offers a cost-effective method while preserving the validity of its results. This remote approach increases access to more diverse and distant cohorts and thus lowers barriers to research participation, further enriching our understanding of motor outcomes in NDDs.
Introduction
Neurodevelopmental disorders (NDDs) are a group of disorders characterized by abnormalities in cognitive, motor, and social domains. Among these abnormalities, motor impairments are often the first signs of an increased risk of NDDs in young children, 1 with the most salient sign being an atypical gait. Typical gait is characterized by the rhythmical movement and smooth synchronization of upper and lower limbs in a cyclical pattern that results in the displacement of the individual’s center of gravity in space. 2 A number of studies have analyzed coordination and symmetry of spatiotemporal gait characteristics in children with NDDs. For example, studies on children with cerebral palsy have examined ipsilateral 3 and contralateral 4 coordination in the upper and lower limbs, as well as stride-to-stride variability and symmetry. 5 Altogether, prior work suggests that gait synchrony is important for high functional mobility, efficient performance of daily living tasks, and improvements in quality of life.6–8 Although a precise definition of synchrony has proven elusive, prior studies support the importance of defining various measures of gait synchrony and developing methods to accurately measure such parameters for clinical studies of NDDs. These measurement methods have improved significantly with the availability of high-performance kinematic methods. For instance, researchers have utilized the Vicon motion capture system, ground-reaction force plates, 9 force-sensitive insoles, 10 and GAITRite® instrumented walkway system 11 to measure various aspects of synchrony in different populations.
Besides synchrony, dynamic balance (referred as “balance” hereafter for brevity) is another central feature of gait and defined as the ability to maintain one’s center of mass with respect to one’s base of support while moving. Most typically developing (TD) children master gait balance by the age of seven. 12 While balance is commonly evaluated in children using instruments, such as the Pediatric Balance Scale, 12 that rely on the clinician’s interpretive expertise, recent studies have utilized kinematic technologies, such as ground-reaction force plates,13–15 the GAITRite® system,16,17 and the Wii Balance Board,18,19 to propose more precise methods to measure balance.
A potential obstacle for the aforementioned kinematic methods is the requirement for the participants or patients to be physically present in the lab space. This limitation often hinders the applicability and ecological relevance of these methods, especially for individuals with cognitive and behavioral challenges. In recent years, machine learning (ML) and computer vision have shown promise in resolving the above issues and advancing gait research. 20 With high-volume data collection technologies and sophisticated data analysis tools, computational approaches have become increasingly relevant to gait analysis. Compared to conventional kinematic methods, computational methods are more cost-effective and less cumbersome. Moreover, computer vision approaches are physically non-obtrusive and less demanding for clinical populations, such as individuals with NDDs.
Accordingly, the present study followed three aims: (1) To determine the feasibility of using modern video processing technologies to extract various profiles of gait synchrony and balance from recorded videos using easy point-and-shoot cameras. Based on previous work showing the applicability of deep learning techniques in gait research, 21 we hypothesized that such techniques would enable us to quantify gait with high accuracy and specificity, without asking participants—especially children with anxiety, sensory hypersensitivity, and difficulty with imitation and attention—to be physically present in a laboratory. This aim would help evaluate whether our computational pipeline is suitable for off-site or remote video-based gait assessment. (2) To investigate the clinical usefulness of our gait analysis method using a convenience cohort of children with a 16p11.2 mutation and their TD siblings. ML-based pose estimation techniques have been previously used to study gait, 21 but not in clinical populations. Herein, we focused on quantifying and comparing gait synchrony and balance in 16p11.2 probands and a control group. Our hypothesis stated that synchrony and balance would be markedly different between these groups, with the probands having less synchronous and balanced gait. This aim would determine whether such quantifications of gait synchrony and balance could potentially be used as phenotypical descriptors in clinical studies. (3) To introduce a new measure which captures whole-body synchrony that is more robust to handedness, compared to measures in prior studies 4 (see Discussion section for more details). Our hypothesis was that using this measure would capture whole-body gait synchrony more robustly when comparing proband and control groups. This last aim would provide a new measure of gait synchrony to enhance current motor assessment methods.
Method
Participants
Children included in this study were recruited as part of a large family research meeting focusing on children with a 16p11.2 mutation (proband; PB) and their non-affected, typically developing (TD) siblings. 17 Children with 16p11.2 present mild-to-moderate neurodevelopmental impairments across language, social, and motor domains. 22 All participants’ caregivers provided written informed consent, and the experimental protocol conformed to the ethical guidelines of the Institutional Review Board of the study’s home institution. Children were administered a battery of locomotion tasks and the Pediatric Evaluation of Disability Inventory–Computer Adaptive Test (PEDI-CAT). 23
Descriptive statistics of study participants.
Materials
To study gait synchrony and balance, we analyzed data collected from the “preferred pace walk test” (PPWT), in which children were instructed to walk along a 10-m walkway at their natural pace. All videos used in the current study were originally recorded using a single point-and-shoot camera that was set up at roughly 15 feet from the beginning of a GAITRite® walkway used in the original study. The camera was pointed at ∼30° to the sagittal planes of the children’s bodies so that limb and body midline movements could be captured. Short video segments 3–9 s long (M = 5.2 s, SD = 1.4) of each child’s PPWT were manually extracted and cropped to feature only the child performing the task. This was a necessary step so as to improve the accuracy of video processing and to eliminate the effect of extraneous objects or persons in the frame.
The pre-processed videos were then fed into Detectron2,
24
a deep learning-based object detection software package. Detectron2 uses a pre-trained model to extract and track 17 points of interest (POIs; Figure 1(a)) on each child’s body (Figures 1(b) and (c)). Next, for each video frame, the POIs were used to calculate five angles Extraction of points of interest (POIs) and calculation of gait synchrony and balance measures from video recordings. (a) POIs on a body. White circles on the body silhouette denote articulated figure POIs, and blue circles denote neck and torso coordinates, calculated as the midpoint between the shoulders and hips, respectively; (b) frames of a sample input video recording; (c) a sample frame annotated with extracted POIs; (d) POI temporal signals extracted from the sample video; (e) all five angular temporal signals (ATSs) for the sample video; (f) FFT frequency-domain profiles of ATSs.
To quantify gait synchrony using the above ATSs, the Fast Fourier Transform (FFT) was applied to the first four ATSs of each participant, and the results were used to create a frequency-domain profile of each ATS (Figure 1(f)). The main swinging frequencies of each child’s arms and legs were calculated as the frequencies
Finally, the variance
Data Analysis
Data analysis was performed in two parts for each of our gait synchrony and balance parameters. First, we performed a Mann–Whitney U test on each parameter between PB and TD groups on the whole population, as well as on girls-only and boys-only subgroups. Second, we performed a correlation analysis between each parameter and age to determine associations between children’s gait profiles and age.
Results
Overall, results of the Mann–Whitney U test showed significant differences in gait synchrony between the groups (Figure 2(a)). The TD children demonstrated significantly more synchrony among their extremities (upper/lower limbs) during gait compared to the PB children. Further, a sex-based analysis revealed that when girls and boys (Figures 2(c) and (e)) were analyzed separately, no significant differences were found between PB and TD children, indicating that within their sex groups, girls and boys demonstrate similar profiles. Additionally, Pearson correlation coefficients between Gait synchrony and balance in PB and TD groups. Differences in gait synchrony (a, c, e) and balance (g, i, k) between PB and TD groups across sex. (b, d, f) Scatterplots with regression lines show correlations between Statistics of various gait synchrony and balance measures across sex. Note. Each row shows the values of a statistic across various sex groups. First level of column headers shows the sex groups as well as the breakdown of PB and TD participants within each group. Second level of column headers shows the statistics measures for each sex group. Mdn is the median value, U is the Mann–Whitney U test statistic, f is its AUC ROC effect size, and p is its p-value.
A secondary inspection for outliers identified two PB children with perfect synchrony (
Our balance analysis revealed significantly lower (i.e., better) balance scores in TD than in PB children (Figure 2(g)). This difference in balance remained significant for girls and boys when analyzed separately (Figures 2(i) and (k)), with a large effect size (Table 2).
Leave-one-out accuracy measures for the SVM classifiers.
The results of the sanity check illustrated in Figure 3 show that measures proposed in previous work, such as Sanity check results for MARP and σCRP. Significance: * p < 0.05. ** p < 0.01.
Discussion
The current study proposes a method to examine whole-body gait synchrony and balance in children using an ecologically friendly and non-intrusive approach. Indeed, synchrony is a feature of gait essential to smooth and stable locomotion,26,27 which so far has received little attention in gait research. To fill this gap and test the clinical feasibility of our synchrony parameter, we created an end-to-end pipeline of data acquisition, pose estimation (using deep learning), as well as data cleaning and analysis. Children with a 16p11.2 mutation form a genetically homogenous population with a high prevalence of neurodevelopmental delays, particularly motor impairments, and thus represent a suitable group to test the clinical utility of our parameters and ultimately contribute to the characterization of their motor phenotype. As such, using a cohort of children with a 16p11.2 mutation and their typically developing siblings, we investigated the clinical usefulness of our pipeline by measuring and comparing their whole-body gait synchrony and balance.
To put our work in a broader context, we have developed a computational pipeline applicable across various paradigms and suitable for more naturalistic and less overwhelming laboratory spaces. Overall, our findings provide the much-needed quantifiable measures of gait synchrony and balance that can help refine motor phenotypes in children with NDDs. We argue that our low-cost setup and novel, robust measures enable us to better understand of the role of synchrony in motor development, particularly with respect to motor control and smoothness of movement in the face of environmental perturbations. 28
In sum, our methods demonstrated less whole-body gait synchrony in PB children than in TD children during a locomotion task. In addition, small non-significant correlations between gait synchrony measures and age indicate that group differences were not attributed to delayed motor development in the age range under study. Only two outlying data points were identified in each group, and manual examination of their video recordings corroborated the calculations made by our computational pipeline. In particular, the two TD children with relatively lower synchrony scores completed the PPWT with their arms held stiffly by their sides. Research has shown that partly active and partly passive arm swing is necessary for optimal stability and efficiency during typical human gait. 29 Stiff arm posturing observed in the TD children might be an indicator of discomfort during testing and reaffirms the need for more ecologically friendly research approaches.
Of note, no overall sex differences were found for our gait synchrony measure. This finding suggests that regardless of their genetic status, girls and boys in the current sample demonstrate similar gait synchrony profiles as their same-sex peers. However, girls as a whole (Mdn
Similar to our results on gait synchrony, significant differences in balance were found between PB and TD children. Consistent with previous findings, 17 PB children exhibited significantly poorer balance during the locomotion task compared to their TD siblings. In contrast to the results from the gait synchrony analysis, group differences in balance remained significant across sex, suggesting that children in the PB group demonstrated poorer balance regardless of their sex. Finally, small and/or non-significant correlations between children’s balance measures and age suggest that group differences were not a result of delayed motor development.
Results of our sanity check for our synchrony parameter
While our findings highlight the promise of modern computer vision techniques in clinical gait assessment, they should be interpreted in the context of the following limitations. First, although our sample size was comparable to those of previous studies,4,6 it was relatively small and not demographically varied. For example, we conducted a post-hoc power analysis for all tests with non-significant results and found these tests underpowered, mainly due to the small sample size, which prevented us from interpreting effect sizes. We advocate for studying larger, less biased, and more heterogenous populations. Second, we did not have access to scores of standardized motor assessments or cognitive tests. Future work could explore how our gait synchrony and balance parameters are related to these developmental measures. Third, our data collection protocol and computational pipeline were not set up to study the intra-subject variability which is common in NDDs. Although high intra-subject gait variability for individuals with 16p11.2 syndrome has not been known or reported in prior work, future studies should address this issue with targeted study designs using larger populations. Fourth, we checked the precision of the pose estimation results by visually verifying the Detectron2 outputs. However, more rigorous verification and validation of the pose estimation results should be performed by comparing them to those obtained using well-established motion capture technologies. Last, our experimental setup did not allow us to test robustness to variations in recording angle, which could be addressed by using a secondary camera at a different angle.
Conclusions
Modern computer vision techniques provide low-cost and precise methods for extracting video-based information from clinical populations in naturalistic, ecologically friendly environments. In this study, we presented a data acquisition and processing pipeline to extract and analyze measures of gait synchrony and balance in children with a 16p11.2 mutation and their typically developing siblings. Hence, we used a combination of a single point-and-shoot camera and modern video processing software to extract signals that are robust to recording conditions and noise. This approach provides both researchers and clinicians with an inexpensive, ecological, and high-accuracy method to study gait and movement in individuals, including children with NDDs, who may exhibit behavioral challenges affecting test administration. The proposed approach can be implemented remotely, for example, in the context of clinical trials and geographically distant communities. We posit that our pipeline can be used to analyze other types of movement disorders, which we plan on investigating in the future.
Longstanding disparities in research sampling caused by racism and socioeconomic and geographical barriers have been pervasive in science, and have only become more apparent in the midst of the COVID-19 pandemic. 32 We contend that a low-cost and remote method of data collection, especially from continuously overlooked participants, could alleviate such disparities and advance efforts to close the knowledge and treatment gaps resulting from biased samples. Our computational approach represents one possible population-based solution 33 to increase enrollment and improve generalizability to participants of various social and genetic backgrounds. We plan on further validating our gait synchrony and balance results by studying larger cohorts of participants across different types of NDDs and more diverse demographic backgrounds.
Footnotes
Acknowledgement
The authors thank the Simons Foundation - Individual Variation project - VIP for their logistic support and the families that donate their precious time to participate in these studies to advance scientific knowledge.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
