Sage Journals: Discover world-class research

Abstract

Background

Gait accelerometer (sensor) technology has proven effective in predicting several medical outcomes, but less is known regarding its prediction of concussion symptoms relative to conventional measures of gait and balance.

Objective

To establish the reliability and validity of gait accelerometer data. We first examine test-retest reliability and the impact of footwear and walking surfaces on gait. We then examine the convergent validity between gait accelerometer data and the NIH 4-meter gait test. Finally, we compare gait accelerometer data to gait speed and balance measures for predicting concussion symptoms.

Methods

Study 1 used a crossover study design with 60 participants to evaluate retest reliability and examine the effects of footwear (shoes/no-shoes) and walking surface (tile floor/grass) on gait accelerometer data. Study 2 employed a cross-sectional design with 1008 participants to assess gait accelerometer correlations with NIH 4-meter gait and the prediction of Centers for Disease Control and Prevention (CDC) concussion symptoms relative to previously validated gait and balance measures.

Results

Retest reliability (4-day average retest interval) for the no shoes/tile surface condition ranged from .72-.91 (mean = .80). Significant effects of footwear and especially walking surface revealed by Analysis of Variances (ANOVAs) on gait accelerometer data for the power, stride, balance, and symmetry domains indicate the need to standardize these variables. Gait accelerometer data correlates significantly with NIH 4-meter gait scores. Regression analyses found that gait accelerometer data predicts CDC concussion symptom endorsement, outperforming the BESS and NIH 4-meter gait at least three-fold.

Conclusions

When standardized on footwear and walking surface, gait accelerometers achieve strong test-retest reliability, converge with established measures of gait speed, and are superior to established measures of gait speed and balance when predicting concussion symptoms. Gait accelerometers represent a rapid tool for collecting additional gait information to quantify the behavioral sequelae of concussion and potentially inform return-to-play decision-making.

Keywords

Introduction

The interpretation of human movement plays an essential role in the clinical practice of numerous medical specialties, including pediatrics, sports medicine, geriatrics, neurology, rheumatology, orthopedics, and rehabilitation. Even walking speed can predict critical outcomes, such as response to rehabilitation, functional dependence, fatality, mobility, cognitive decline, and hospitalizations.¹ To date, most work has focused on geriatric populations,² and recommendations from the Mobility Working Group emphasize gait speed as the chief predictor of mobility and various life outcomes.³

Advanced instrumentation to capture and record elements of gait functioning can add significantly to the understanding of gait.⁴ For example, gait speed assessed using accelerometers has superior sensitivity to dysfunction relative to manually collected gait speed data.⁵ Moreover, accelerometers can produce reliable outputs across a range of clinical populations.^6–12 However, the specific influence of footwear and walking surface on accelerometers has received little attention. Variability in footwear and walking surface may influence the biomechanics of movement,¹³ as well as gait outcomes in adults¹⁴ and children.¹⁵ Thus, if left unstandardized, these factors could introduce measurement error and attenuate reliability, validity, and the utility of gait assessments.

The current research uses noninvasive, continuous triaxial accelerometer data to create a visual depiction of gait (i.e., waveform), here referred to as a BioKinetoGraph (BKG). Analogous to the 12-lead electrocardiogram, BKG waveforms are related to specific gait cycle events, including cadence, foot-strike, push-off, double stance time, and swing phase. Raw data are combined to generate BKG waveforms as gravitational accelerations over time. Tracings are used to obtain the range of motion, amplitudes, and timing intervals for the various components of the waveform that relate directly to the gait cycle. The adopted method is similar to that employed in other published studies.¹⁶

Two studies are presented with the objectives of examining the reliability and validity of accelerometer data for gait. Specifically, in Study 1, we evaluate the test-retest reliability of the BKG and whether footwear and walking surface influence gait data. If they have a significant impact, these variables will need to be standardized for subsequent data collection. In Study 2, we examine the association between gait accelerometer data and an established gait measure; the NIH 4-meter gait test. Finally, we compare gait accelerometer data to standard measures of gait (NIH 4 m gait) and balance (Balance Error Scoring System; BESS) for predicting CDC concussion symptom endorsement. The goal is to determine if accelerometers can out predict traditional gait and balance measures with respect to concussion symptom endorsement, and if accelerometers can add significantly to the prediction of concussion symptoms over established measures.

Study 1

The specific aims of Study 1 were to evaluate the test-retest reliability of the gait accelerometer data and examine the influence of two extraneous variables (footwear and walking surface) to determine the effect sizes for these variables, and whether they should be standardized to optimize the accelerometer assessment.

Materials and methods

Patient and public involvement

Volunteers agreed to have their gait evaluated while systematically varying footwear and walking surfaces. Participants were informed of the goal of evaluating accelerometer capabilities and educated on the benefits of research. The university's IRB approved Study 1 (IRB #16-0191), all participants signed consent forms, and no adverse outcomes were reported. Deidentified data for studies 1 and 2 are available from the first author.

Procedure

We employed a within-subjects design, with order counterbalanced. Those enrolled in physical education classes during the fall of 2016 at a university in the southeastern United States and ambulating independently were eligible. Researchers and participants were aware of the footwear and walking surface conditions, but blind to BKG outcomes, as the data were automatically recorded and saved.

The gait protocol involves starting at one end of a 20-foot straight walking path, walking ten steps at one's usual (preferred) pace, turning, and walking back ten steps to the starting point.

Day 1. Participants wearing a sacrum gait accelerometer performed the above gait protocol covering 20 feet and back on an even, hard surface (tile floor), then on an uneven surface (grass), with and without shoes on each surface. Participants then repeated each condition (eight trials total, counterbalancing order).

Day 2. Repeated the Day 1 protocol, with retest intervals ranging from 1 to 14 days (Mean = 4.04, SD = 2.68), with 95% of the retest trials completed within nine days. Participants were instructed to wear the same pair of shoes across test intervals (but this was not verified).

Presented values reflect the mean of two walking trials under the same conditions on the same day. If a second trial was unavailable, which occurred for 10% of participants, single trial data were used. Other missing data resulted in case-wise exclusion.

The BKG and the algorithms used to extract the various gait features were initially validated using a recorded video at 240 frames per second, which is twice the sampling rate of the accelerometers. This recording was completed in a test session with four accelerometers (on both ankles, one wrist, and the sacrum) that matched the recorded data and key markers identified by the algorithm (heel-strike and toe-off) to what was observed in the video. The camera was an iPhone X configured to record at 240fps while attached to a rolling dolly system that followed the participant as they walked to keep them in the center frame. The high-speed video was used to identify the start (heel-strike) and end (toe-off) features of a gait cycle and match them to the accelerometer data. Subsequent gait events were identified on the waveform within these boundary events. It is noted that identifying gait markers from an accelerometer waveform is a widely used practice that involves the same 3-axes of data extracted from accelerometers.

Although the BKG can be evaluated from accelerometers located at each ankle, sacrum, and wrist, for this study, only the single sacrum accelerometer (near the small of the back) is presented. However, the sacrum accelerometer, located at the center of mass (CoM) can extract the same gait cycle events available from other accelerometer placements, with research suggesting that CoM data can optimally characterize synchronous information from many segmental variables and provide an analysis of one's three-dimensional trajectory.¹⁷ (Note: Additional accelerometer data from other locations were available for a subset of the sample. When compared, the gait output from the other accelerometers provided corresponding data, though with some additional variability, resulting in slightly lower test-rest reliability. In the discussion, we review some of the relevant issues with collecting data from other accelerometer placements on the body.)

The accelerometers record inertial measurement units (IMU; a general term for accelerometers, gyroscopes, and similar sensor recording devices) via transmitted signals at a rate of 200 Hz to a local Microsoft Surface Pro, which transmits the data to remote servers for BKG motion analysis following each trial. Raw data are saved to a persistent data store as a collection of timestamps and accelerometer readings of each axis. Raw inertial data are then processed through a series of algorithms (patents pending) to detect key markers of the gait cycle and extract the spatial, temporal, kinetic, and spectral features reported herein.

Prior to detecting the initial contact (heel-strike) and final contact (toe-off) events of each gait cycle, the signal is processed to remove extraneous data recorded before and after the gait trial, adjust for tilt and orientation of the accelerometer relative to gravity, and filtered to reduce noise. Correct orientation is ensured using template correlation with the y-axis pointed downward, the x-axis pointed right, and the z-axis rearward. Tilt due to the placement of the accelerometer on the body is adjusted along each axis so that the vertical axis is centered at zero. The raw signal data is then fourth-order-zero-phase-shift filtered,¹⁸ with an upper cutoff of 3hz and a lower cutoff of 0.111hz. Initial contact and final contact events were detected algorithmically by first identifying local maxima representative of the mid-swing with toe-off and heel strike events indicated by minima occurring immediately before and after, respectively. Displacement and velocity were obtained for each axis by trapezoidal integration and subtraction of the zero-phase rolling average¹⁹ equal to one gait cycle. Figure 1 provides an illustration of the type of waveform that is extracted from the sacrum accelerometer.

Figure 1.

Illustration of a BKG signature drawn from the sacrum accelerometer. The arrow shows one heel strike interval through to the next heel strike on the same leg. TO represents toe-off. HS represents heel strike. RL represents right leg. LL represents left leg. Clinical conditions can alter the size, shape, and timing of the waves.

The BKG output described here represents sixteen variables organized into four conceptual domains of balance (stability), stride (velocity/timing intervals), power (amplitudes), and symmetry (regularity).

Balance is the range of motion (ROM) at the center of mass (sacrum) as well as gait cycle variability, represented by the stability during the straight portion of the gait cycle (side balance) and those occurring while turning around (turnaround balance), the velocity of side-to-side movement (sway time), the rhythmicity of gait patterns (gait smoothness), and the force generated with lateral movement (side power), and the variability of time in double stance phase (support consistency).

Stride refers to maintaining velocity during walking and includes the average timing interval between contralateral heel strikes (stride time) and the average duration spent in the double stance phase (double stance).

Symmetry refers to the consistency of movement across the anterior-posterior and vertical dimensions. This includes a comparison of the forward and backward displacement from the center of mass (forward movement symmetry), a comparison of the distance of upward and downward displacement from the center of mass (vertical movement symmetry), and a comparison of the velocity of upward and downward movement from the center of mass (vertical sway symmetry).

Power refers to the amount and efficiency of energy generated and expended in the gait cycle. This includes the total net force generated at the center of mass (total power), the force generated with vertical movement (vertical power), the force produced with forward movement (forward power), the force generated by the foot strike (striking force), and force generated by the toe pushing off (pushing force). As a more detailed illustration of how these variables are derived from accelerometers, the striking force and pushing force variables represent the overall magnitude of force, calculated by the vector sum of amplitude values along the 3 axes, at the moment of heel-strike and toe-off, respectively. Total power, vertical power, and forward power represent the amplitude of the dominant frequency after applying a Fourier transform to the vector sum and individual axes, respectively.

Retest reliability for all variables was assessed using Pearson correlations between BKG values from Days 1 and 2. Effects of surface type (grass or tile floor) and footwear (shoes or no shoes) on the raw scores of each of the sixteen BKG variables were assessed using 2 × 2 ANOVAs. A target sample size was set at > 48 to detect a small to medium effect size (d = .35) with power = .80.

Results

Participants

In the Fall of 2016, 60 college students with no apparent health problems, aged 18 to 35 (Mean = 21.26 ± 2.79 years), 61.7% female (Height = 64.79 ± 12.87 inches; Weight = 149.80 ± 37.11 lbs.), were recruited and consented prior to initiating the procedure. All participant data are included, aside from those whose accelerometers malfunctioned.

Evaluating the effects of walking surface and footwear on gait accelerometer data

Significant and relatively large main effects were observed for surface type on the power domain variables of total power, vertical power, forward power, pushing force, and striking force, as well as stride time, forward movement symmetry, gait smoothness, sway time, side power, and turnaround balance (Supplemental Table A includes significant ANOVAs. Even after applying a Bonferroni correction for the number of comparisons, all but one of these effects remain significant for walking surface). These main effects indicate that when walking on grass (a less stable surface) as compared to tile, participants expended more energy across all axes, had longer stride times, exhibited less symmetry in forward and backward movement, displayed irregular gait patterns, showed greater side to side sway time, and manifested less balance during the turnaround.

Significant but mostly small main effects for footwear were observed for vertical power, striking force, stride time, double stance, and vertical sway symmetry. This indicates that when wearing shoes, compared to no shoes, participants displayed longer stride times, longer double stance times, greater symmetry in the speed of upward and downward movement, greater energy expenditure with vertical movement, and greater energy expenditure when the foot strikes the walking surface. (After applying a Bonferroni correction for the number of comparisons, only one significant effect, for vertical power, remains significant for footwear.) No interaction effects emerged between footwear and walking surface. Given these findings, footwear and walking surfaces were standardized for the analysis of test-retest reliability.

Reliability of accelerometer gait assessment

BKG test-retest reliability from the first and second testing days is shown in Table 1 (column 1). Retest reliability was examined by holding constant the surface and footwear conditions (tile floor with no shoes). This condition was selected because it minimizes the influence of extraneous factors (i.e., the stability of any footwear and evenness of the walking surface). Under this condition, the Pearson correlations between BKG values obtained on two different days of testing resulted in coefficients ranging from .72 to .91, with a mean of .80. This indicates good retest reliability across all four BKG domains.

Table 1.

Pearson correlations illustrating test-retest reliabilities for BKG variables by domain and correlations with NIH 4 m gait.

Domain		Variable	Reliability (r)	Correlation with NIH 4 m Gait
Power	Reliability r _average= .78
		1. Total power	.71**	−.18**
		2. Forward power	.75**	−.29**
		3. Vertical power	.80**	−.45**
		4. Pushing force	.79**	−.30**
		5. Striking force	.83**	−.30**
Stride	Reliability r _average= .86
		6. Stride time	.91**	.31**
		7. Double stance	.81**	.25**
Balance	Reliability r _average= .82
		8. Support consistency	.82**	−.17**
		9. Gait smoothness	.72**	.06*
		10. Side balance	.89**	.00
		11. Turnaround balance	.76**	−.06
		12. Side 9ower	.83**	.35
		13. Sway time	.88**	.11**
Symmetry	Reliability r _average= .75
		14. Forward Movement Symmetry	.76**	.09**
		15. Vertical Movement Symmetry	.78**	−.05
		16. Vertical Sway Symmetry	.67**	.33**

Note. * p < .05, ** p < .01. Cumulatively, BKG variables explain 25.6% variance (multiple R = .51) of NIH 4 m gait scores (F = 21.4, df = 16, 993, p < .001).

Study 2

Study 2 examines the convergent validity of the BKG with a widely used measure of gait (the NIH-4 m gait). This study also examines the ability of the BKG to predict concussion symptom endorsement as compared to established measures of gait and balance that have previously been shown to predict concussion symptoms.²⁰

The application to concussion is important, as it represents a growing public health concern,²¹ and remains the leading global contributor to all disabilities and deaths due to trauma.²²

Materials and methods

Patient and public involvement

Participants agreed to have their anonymous data aggregated for subsequent analysis to help researchers better understand gait and its relation to important outcomes.

UNCW's IRB approved Study 2 (IRB #21-0047) as an anonymous, archival data analysis.

Study 2 data were drawn from an extensive data collection involving clinical assessments in medical settings following a suspected concussion and from non-clinical settings where asymptomatic individuals were undergoing baseline testing as a condition of sports participation. The latter draws from community-based leagues and those in collegiate-level play (club athletes). Consequently, the sample represents a diverse set of individuals with respect to age and ability.

Participants

One thousand ten individuals (53.4% female) were evaluated across eight sites, with participants aged 8 to 50 years (M = 16.98, SD = 4.43). Of these individuals, 950 (54.1% female) were undergoing standardized preseason baseline assessments, while 58 (41.4% female) individuals were undergoing post-concussion evaluations in medical settings. Data were from consecutive evaluations from January 2018 to September 2020, with no exclusions. Participants were not compensated for completing the battery, as it was encouraged by their respective leagues for those completing baselines, and it was part of the medical evaluation for those receiving post-concussive care. Gait assessments were standardized and involved walking on a solid surface with no shoes.

Procedure

All participants completed the same standardized battery (known as SportGait), delivered on a Microsoft Surface Pro. The SportGait battery is administered in the following order: the CDC concussion symptom checklist, two 20-foot walks while wearing accelerometers, NIH 4 m gait test, BESS, and the Conner's Continuous Performance Test (CCPT), 3^rd edition. CCPT 3 data were not examined for this study.

The 31-item CDC concussion symptom checklist (each item endorsed as present or absent) includes 11 danger items, such as slurred speech, vomiting or nausea, persistent headache, unequal pupil, and 20 less severe symptoms, including confusion, sensitivity to light, clumsiness, and visual disturbance. Endorsed items were summed to create a single total score, with higher scores denoting a greater number of experienced symptoms.

BKG data from the two walks were available from all four accelerometers, which are similar to the commercially available accelerometers in smartphones. To match study 1 data, we here present the data from the sacrum accelerometer. Participants and researchers were blind to BKG data, as it was automatically recorded and stored in the cloud. Raw BKG data were used for these analyses.

The National Institutes of Health (NIH) 4 m Gait test involves taking the fastest of two walks down the 4 m track assessed with an automated timer. The BESS includes six poses on two surfaces, following standardized procedures, and the total BESS score was used. Raw scores were used in the reported analyses.

Results

Predicting NIH 4 m gait with the BKG

Cumulatively, the 16 BKG variables are highly correlated (multiple R = .51) with NIH 4 m gait scores (R-square = .256, F = 21.4, df = 16, 993, p < .001). This indicates that overall, BKG variables converge with a well-validated measure of gait speed.

To further explore the association between these measures, Pearson correlations were run between BKG variables and NIH 4 m gait scores. The strongest and most consistent correlations emerged for the stride and power domains (Table 1, column 2). Specifically, inverse correlations between NIH 4 m gait and all six BKG power variables indicate that less power in one's gait results in taking more time to complete the 4 meter course (i.e., slower walking speed). The positive correlations with both stride variables indicate convergence between the timing metrics derived from the BKG and NIH 4 m gait. Interestingly, three of the nine BKG variables in the symmetry and balance domains did not correlate with NIH 4 m gait scores, and of the remaining significant correlations, the r-values were generally smaller relative to those emerging for BKG power and stride variables. This indicates that several BKG variables are less (or unrelated) to gait speed and that there may be a benefit to collecting gait data in addition to speed.

To evaluate this hypothesis, we next examined whether the BKG variables can outperform a measure of gait speed when predicting concussion symptom endorsement (Note: Functionally speaking, time on the NIH 4 m gait test is equivalent to speed because the distance traversed is standardized.)

Predicting CDC concussion symptom endorsement using the BKG and NIH 4 m gait test

A hierarchical regression was performed to determine if information from the BKG can add to the prediction of CDC concussion symptom experience after accounting for the variance explained by the NIH 4 m gait test. Age and gender were entered in block one as covariates (i.e., statistically controlling for age and gender) due to there being differences between the two groups on these variables and the analyses involving raw rather than normed data. NIH 4 m gait scores were entered in block 2, and the BKG variables established in Study 1 were entered in block 3.

Combined, the covariates of age and gender account for 0.7% variance in CDC concussion symptom endorsement (R-square = .007, F = 3.518, df = 2, p = .03). The NIH 4 m gait then explains an additional 1.7% variance (R-square change = .017, F_change= 17.41, df = 1, p < .001) in concussion symptom endorsement. Finally, the BKG variables add significantly to the regression model, explaining an additional 4.9% variance in concussion symptom endorsement (R-square change = .049, F_change= 3.24, df = 16, p < .001) after accounting for the effects of age, gender, and NIH-4 m gait scores; outperforming the explained variance the NIH 4 m gait test by almost three-fold. Table 2 also presents the beta weights illustrating significant effects for BKG variables.

Table 2.

Stepwise linear regression using NIH 4 m gait and BKG variables to predict CDC concussion symptom endorsement, controlling for age and gender.

Predictor	b	b 95% CI [LL,UL]	β	sr	r	Model R² (Adj R²)	R² Change
(Intercept)	−1.019	[−2.68, 0.64]
Age	−0.031**	[−0.05, −0.01]	−0.108	−0.090	−0.072
Gender	−0.038	[−0.22, 0.15]	−0.015	−0.034	−0.047
Block 1						0.007* (0.005)	0.007*
NIH 4 m Gait	0.238**	[0.07, 0.41]	0.102	0.087	0.138
Block 2						0.024** (0.021)	0.017**
Stride time	0.004**	[0.00, 0.01]	0.129	0.081	0.093
Double stance	0.001	[−0.01, 0.01]	0.006	0.004	0.084
Gait smoothness	0.352**	[0.20, 0.51]	0.136	0.127	0.130
Forward power	0.111	[−0.68, 0.90]	0.013	0.008	−0.086
Side power	−0.655	[−1.68, 0.37]	−0.064	−0.039	0.071
Vertical power	0.078	[−0.42, 0.58]	0.017	0.009	−0.073
Total power	−0.269	[−1.14, 0.61]	−0.028	−0.018	−0.097
Side balance	−10.602*	[−20.96, −0.25]	−0.085	−0.062	−0.056
Turnaround balance	0.673	[−4.74, 6.09]	0.009	0.007	−0.055
Sway time	−0.180	[−2.07, 1.71]	−0.009	−0.006	−0.021
Support consistency	0.006	[−0.01, 0.02]	0.039	0.034	−0.036
Striking force	−0.121*	[−0.23, −0.02]	−0.098	−0.068	−0.100
Pushing force	−0.080	[−0.23, 0.07]	−0.043	−0.031	−0.093
Forward movement symmetry	−0.005	[−0.01, 0.00]	−0.048	−0.041	−0.012
Vertical movement symmetry	0.008*	[0.00, 0.01]	0.089	0.078	0.039
Vertical sway symmetry	−0.007	[−0.02, 0.00]	−0.061	−0.043	−0.006
Block 3						0.073** (0.055)	0.049**

Note. A significant b-weight indicates that the beta weight and semi-partial correlation are also significant. b represents unstandardized regression weights, β indicates the standardized regression weights, sr² represents the semi-partial correlation squared, r represents the zero-order correlation, LL and UL indicated the lower and upper limits of a confidence interval, respectively. BKG variables represent the four gait domains of stride, balance, symmetry, and power. When the BKG variables were examined independently (without entering NIH 4 m gait and BESS scores first), the BKG variables could account for significant variance even after controlling for the number of variables added (Adjusted r-square = .055, F = 8.032, p < .01). The variables stepping into this regression equation were Stride time, Gait smoothness, Striking force, Vertical sway symmetry, sway time, and Vertical movement symmetry. Age was also a significant predictor in this analysis. * indicates p < .05, ** indicates p < .01.

Predicting CDC concussion symptom endorsement using the BKG and BESS

A second hierarchical regression was employed to determine whether the BKG can add to the predictive utility of a standard measure of balance. After controlling for the effects of age and gender, the BESS adds significant incremental variance totaling 1.3% (R-square change = .013, F_change= 13.61, df = 1, p < .001) when predicting concussion symptoms. The BKG variables then explain an additional 5.7% variance after accounting for age, gender, and BESS scores (R-square change = .057, F_change= 3.78, df = 16, p < .001). This means that BKG data can add significantly to the prediction of concussion symptoms experienced by a magnitude of more than 4-fold over a well-validated measure of balance (see Table 3).

Table 3.

Stepwise linear regression using BESS total errors and BKG variables to predict CDC concussion symptom endorsement, controlling for age and gender.

Predictor	b	b 95% CI [LL,UL]	β	sr	r	Model R² (Adj R²)	R²Change
(Intercept)	−0.635	[−2.21, 0.94]
Age	−0.036**	[−0.06, −0.01]	−0.126	−0.105	−0.072
Gender	−0.020	[−0.38, −0.16]	−0.008	−0.007	−0.047
Block 1						0.007* (0.005)	0.007*
BESS Total	0.028**	[0.01, 0.04]	0.113	0.108	0.120
Block 2						0.020** (0.017)	0.013**
Stride time	0.004**	[0.00, 0.01]	0.141	0.088	0.093
Double stance	0.002	[−0.01, 0.01]	0.018	0.013	0.084
Gait smoothness	0.323**	[−0.17, 0.48]	0.145	0.124	0.130
Forward power	0.167	[−0.62, 0.96]	0.013	0.011	−0.086
Side power	−0.404	[−1.42, 0.61]	−0.040	−0.024	0.071
Vertical power	−0.064	[−0.56, 0.43]	−0.014	−0.008	−0.073
Total power	−0.381	[−1.25, 0.49]	−0.040	−0.026	−0.097
Side balance	−11.303*	[−21.62, −0.99]	−0.091	−0.066	−0.056
Turnaround balance	1.288	[−4.12, 6.69]	0.018	0.014	−0.055
Sway time	0.028	[−1.86, 1.92]	0.001	0.001	−0.021
Support consistency	0.007	[0.01, 0.02]	0.044	0.038	−0.036
Striking force	−0.117*	[−0.22, −0.01]	−0.095	−0.066	−0.100
Pushing force	−0.070	[−0.22, 0.08]	−0.038	−0.028	−0.093
Forward movement symmetry	−0.007	[−0.02, 0.00]	−0.059	−0.050	−0.012
Vertical movement symmetry	0.008**	[0.00, 0.01]	0.091	0.080	0.039
Vertical sway symmetry	−0.006	[−0.02, 0.00]	−0.054	−0.038	−0.006
Block 3						0.077** (0.059)	0.057**

* indicates p < .05, ** indicates p < .01.

For both regression analyses, higher rates of concussion symptom endorsement are associated with lower gait smoothness, more sway, less striking force, more vertical movement symmetry, and longer (slower) stride time. Importantly, BKG variables representing all four gait domains of stride, balance, symmetry, and power were significant predictors, and these findings emerged even though the values (beta weights) statistically control for other variables in the regression equation (including gait speed).

A final hierarchical regression was conducted to determine if the BKG adds to the combined predictive utility of the NIH 4 m gait and BESS. Age and gender were again entered in the first block. NIH 4 m gait and BESS scores were then entered in block two, and together, they predicted an additional 2.9% of the variance in CDC concussion symptoms (R–square change = .029, F_change = 15.29, df = 2, p < .001). The BKG variables added in block three then add 4.8% of variance to the model predicting concussion symptoms (R-square change = .048, F_change = 3.18, df = 16, p < .001). Thus, the BKG also adds significantly and out predicts the combined effects of established gait and balance measures.

Table 4 provides means and standard deviations for all variables presented separately for baseline participants and those undergoing evaluations due to a suspected concussion.

Table 4.

Means and standard deviations of baseline and post-injury samples.

	Baseline		Post-Injury		Independent Samples t-test
	M	SD	M	SD	95% CI for Mean Difference	t	df	Cohen's d
Age	17.11	4.42	14.86	4.06	1.15, 3.35	4.08**	65.48	0.51
Total Symptoms ^a	0.12	0.63	3.60	3.23	−4.34, −2.63	−8.20**	57.26	3.54
NIH 4 m gait ^a	3.32	0.52	3.61	0.67	−0.48, −0.11	−3.26**	60.15	0.55
Gait Speed (m/s)	1.25	0.38	1.14	0.20	0.01, 0.21	2.08*	1007	0.28
BESS Total Errors	11.18	5.02	13.57	5.61	−3.73, −1.04	−3.49**	1008	0.47
Forward Power	0.67	0.15	0.62	0.16	0.01, 0.09	2.34*	1008	0.32
Total Power	0.49	0.13	0.46	0.14	−0.01, 0.06	1.62	1008	0.22
Vertical Power^a	0.87	0.27	0.87	0.36	−0.10, 0.09	−0.08	61.11	0.01
Striking Force	3.85	1.04	3.58	0.99	−0.01, 0.54	1.86	1008	0.25
Pushing Force	2.85	0.67	2.66	0.86	0.01, 0.38	2.11*	1008	0.29
Stride Time	533.56	43.41	546.83	48.32	−24.86, −1.67	−2.24*	1008	0.30
Double Stance	82.95	14.52	86.86	18.20	−7.83, 0.00	−1.96	1008	0.27
Support Consistency	17.33	8.20	16.49	7.39	−1.32, 3.00	0.76	1008	0.10
Side Power	0.36	0.13	0.33	0.12	0.00, 0.06	1.67	1008	0.23
Side Balance ^a	0.025	0.01	0.03	0.01	−0.01, 0.00	−1.45	61.63	0.24
Sway Time	0.24	0.06	0.26	0.07	−0.03, 0.00	−2.23*	1008	0.30
Turnaround Balance ^a	0.06	0.02	0.06	0.02	−0.01, 0.00	−2.44*	61.82	0.39
Gait Smoothness	2.45	0.58	2.24	0.49	0.06, 0.36	2.73**	1008	0.37
Forward Movement Symmetry	−59.13	11.53	−60.43	11.97	−1.77, 4.36	0.83	1008	0.11
Vertical Movement Symmetry	−0.09	13.95	0.96	13.65	−4.74, 2.65	−0.55	1008	0.08
Vertical Sway Symmetry	21.12	11.17	20.12	11.51	−1.97, 3.97	0.66	1008	0.09

Note: * indicates p < .05, ** indicates p < .01.

Violated Levene's test for equality of variance. Welch's correction was applied to the independent samples t-test.

Discussion

Tao and colleagues reviewed gait measurements in clinical settings using a variety of technologies (including wearable accelerometers),⁴ illustrating considerable diversity in how to quantify gait. Previous research has shown that accelerometers can identify and derive specific gait cycle features^23,24 and that they relate to mild Traumatic Brain Injuries (mTBIs).²⁵ Research also suggests that accelerometers can improve the predictive utility of gait even when simply looking at gait speed assessed via instrumentation.⁵ The current studies add to this body of research, as BKG variables were shown to add significantly to the predictive validity of two frequently used and well-validated measures of gait and balance (NIH 4 m gait and BESS) when predicting the endorsement of CDC concussion symptoms. Moreover, the added predictive validity of the BKG was at least three-fold larger than these measures. This is perhaps the most crucial finding because it provides evidence that BKG data can add substantially to existing measures in predicting concussion outcomes, thereby justifying its inclusion in a clinical assessment. Moreover, the BKG also out-predicts the separate and combined effects of the NIH 4 m gait and BESS. Given the brevity of the BKG assessment (less than 2 min for the BKG as opposed to approximately ten minutes for the BESS and NIH 4 m gait), this is also an important finding, suggesting that although gait speed and balance measures are significant predictors, accelerometer-based assessments of motion provide even more predictive data, which is in keeping with the literature on mTBI.²⁵

The current findings also illustrate that reliable gait values can be extracted from the BKG for sixteen variables representing four conceptually meaningful domains of stride, balance, symmetry, and power. The value of quantifying these domains separately is that they can indicate where gait remediation may benefit most. For example, low scores on power could speak to muscle weakness, while low scores on balance could signal problems with stability, and interventions could emphasize these specific areas.

The correlations between the BKG and NIH 4 m gait are both consistent with the literature and intuitively meaningful. For example, BKG power and stride domains relate to the gait speed measure (the time it takes to traverse 4 meters), such that less power and longer time spent within individual sections of the gait cycle (stride) are associated with taking longer to cover 4 meters (i.e., slower speed) in a separate task.

The BKG and NIH 4 m gait association raises the question of whether speed is a confound for other gait metrics and whether additional information could be extracted from the BKG if gait speed were equated/standardized as part of the procedure. In the current research, we attempted to address this issue statistically by testing whether the BKG variables add to the prediction of concussion symptoms after controlling for gait speed in a separate walking test (NIH 4 m gait). Results from the hierarchical regression indicated that the BKG does, in fact, add significant incremental information over and above gait speed. We also found that of the five BKG variables that significantly predict concussion symptoms, three were either minimally or unrelated to gait speed in the NIH 4 m gait (i.e., gait smoothness, side balance, vertical movement symmetry). However, gait speed could also be methodologically controlled to gain further insight into the role of gait speed in interpreting gait metrics, and we discuss this below.

The goal of validating the BKG gait assessment is to identify unique motion signatures to establish objective biomarkers of health and illness. The long-term objective is to identify diagnostic biokinetic (gait) signatures for various vascular, neurological, and orthopedic conditions and use BKG to monitor therapeutic response or disease progression. BKG measurement can also be part of a baseline screening protocol to facilitate the detection of exercise-related injuries and to determine when it is safe to return to activity. Other studies have reported associations between accelerometer-assessed gait for geriatric syndromes such as falls and frailty,^26,27 as well as neurologic conditions such as Alzheimer's Disease,²⁸ Parkinson's Disease,²⁹ and stroke.³⁰ We here add to the literature by demonstrating the prediction of concussion symptoms over and above what is predicted by gait speed and balance. BKG data may be instrumental when patients are either unable (e.g., children or adolescents³¹) or unwilling (e.g., athletes wanting to return prematurely³²) to accurately report concussion symptoms, thereby providing a rationale for greater reliance on objective measures of functioning.

BKG variables were also shown to be sensitive to the potentially extraneous factors of footwear and especially walking surfaces. Thus, the standardization of gait assessments with respect to footwear and walking surface is critical to avoiding the effects of these confounding variables when interpreting accelerometer gait data. This is particularly relevant in clinical settings, where the interpretation of gait data has potential screening, diagnostic, monitoring, and therapeutic implications and where strict control over footwear and the walking surface can be more easily achieved.

BKG data for studies 1 and 2 were derived from a single sacrum accelerometer. Although a richer dataset could be examined from accelerometers located at the wrist (e.g., to include arm-swing data), ankles and sacrum, our rationale for adopting this simplified analysis is to mimic single accelerometer approaches using the technology available in smartphones,³³ which could result in broader adoption of the BKG. In other words, if comparable BKG data can be collected from already widely available technology, this dramatically expands the utility of this approach.

Conclusions and clinical implications

The current research documents that an analytic gait process (BKG) shows robust test-retest reliability and sensitivity to extraneous factors of footwear and especially walking surfaces (Study 1). Moreover, BKG variables can predict gait speed and self-reported concussion symptoms to a markedly more significant degree than previously validated measures of gait and balance (Study 2). These two studies provide essential initial steps in validating the BKG as a method for deconstructing gait using accelerometers and its applicability to concussion. In order to more effectively use the BKG in clinical settings, normative data would be needed for various health conditions, as well as for healthy individuals. To that end, BKG norms were subsequently collected based on a sample of 1163 ostensibly healthy individuals (47.8% female) aged 8 to 50. These data could allow for the analysis and interpretation of the BKG relative to age and gender norms rather than having to statistically control for these variables, as was the case in the current study. BKG norms using commercially available accelerometers found in smartphones were also subsequently collected based on a sample of over 4000 ostensibly healthy individuals ranging in age from 5 to 78,³⁴ and this could allow for the broader adoption of the BKG.

Limitations and future directions

Limitations include not having clinical diagnoses to validate the experience of concussion in Study 2, the absence of data from older adults, and the lack of information regarding comorbid health and orthopedic conditions that could confound the findings.

In addition, because we do not have a direct measure of speed in the BKG and because we did not standardize speed, this means that gait speed may be a confound, despite our attempts to address this statistically. Future research could not only examine the BKG under a standardized gait speed condition but could also examine if the observed findings vary when standardized speed is altered. For example, maintaining balance at slower walking speeds is known to be more difficult, and this may be exacerbated by conditions such as mTBI. Thus, standardizing to a slower speed for the BKG may elevate its sensitivity to mTBI.

The current research did not include traditional measures of gait variability (e.g., the coefficient of variance) or gait asymmetry (e.g., step length, step time asymmetry), even though we do have other measures of gait symmetry and variability (balance). Thus, future research could compare the traditional measures in the literature to those used in the current study to better understand their relative strengths and weaknesses.

Another consideration is the optimal location of the accelerometer. It is arguable that an accelerometer located at the ankle would better and more intuitively capture gait dynamics. However, ankle accelerometers are also more susceptible to extraneous factors, such as shifting position on the leg if not tightly placed, greater differences in the data due to where on the ankle the accelerometer is located, the effects of walking surface, and the presence of lower extremity injuries (assuming the goal is to assess neurological conditions); all of which would introduce noise in the data. Moreover, data from the balance and symmetry domains can be more effectively measured at the center of mass (sacrum),¹⁷ and an ankle accelerometer would provide more limited data from the corresponding gait action on the other leg. Finally, the single sacrum accelerometer makes the technology more accessible and easier to administer reliably, and other research has successfully employed sacrum-located accelerometers to assess clinical conditions using gait.^17,35 Future research could directly explore the association between simultaneously collected data from different accelerometer locations to better understand the relative tradeoffs of accelerometer placement.

Ultimately, one of the central goals of this research is to move towards a widely available and easy-to-use gait assessment that does not require extensive technology or equipment to facilitate broad adoption. To that end, recent research has explored the use of accelerometers available in smartwatches and smartphones, which are the same commercially available accelerometers used in the present study, to evaluate whether they can provide comparable data with respect to normative values, reliability, and validity. This research is already underway, and involves the use of a mobile App (SportGait Mobile), which is available in the App Store and in Google Play.³⁴ The long-term goal is to collect comprehensive normative BKG mobile data from a large cohort of healthy individuals (standardized with respect to walking on a firm surface without shoes) to define better the spectrum of normal gait functioning, as well as BKG data representing various health conditions. This would allow for the systematic investigation of the BKG signatures that could differentiate various conditions from ostensibly normal functioning. Such data could then be used to aid in early detection and diagnostic decisions, the assessment of recovery, and even to evaluate the efficacy of interventions. The present research is, therefore, an important initial step in achieving a broad adoption of the BKG to better elucidating gait functioning and its consequences.

Supplemental Material

sj-docx-1-ccn-10.1177_20597002231157947 - Supplemental material for Validation of an accelerometer-based gait assessment: Establishing test-retest reliability, convergent validity, and predictive validity for concussion symptom endorsement

Supplemental material, sj-docx-1-ccn-10.1177_20597002231157947 for Validation of an accelerometer-based gait assessment: Establishing test-retest reliability, convergent validity, and predictive validity for concussion symptom endorsement by Len Lecci, Kelly Dugan, Ken Zeiger, Julian Keith, Sasidharan Taravath, Wayland Tseh and Mark Williams in Journal of Concussion

Supplemental Material

sj-csv-2-ccn-10.1177_20597002231157947 - Supplemental material for Validation of an accelerometer-based gait assessment: Establishing test-retest reliability, convergent validity, and predictive validity for concussion symptom endorsement

Supplemental material, sj-csv-2-ccn-10.1177_20597002231157947 for Validation of an accelerometer-based gait assessment: Establishing test-retest reliability, convergent validity, and predictive validity for concussion symptom endorsement by Len Lecci, Kelly Dugan, Ken Zeiger, Julian Keith, Sasidharan Taravath, Wayland Tseh and Mark Williams in Journal of Concussion

Supplemental Material

sj-csv-3-ccn-10.1177_20597002231157947 - Supplemental material for Validation of an accelerometer-based gait assessment: Establishing test-retest reliability, convergent validity, and predictive validity for concussion symptom endorsement

Supplemental material, sj-csv-3-ccn-10.1177_20597002231157947 for Validation of an accelerometer-based gait assessment: Establishing test-retest reliability, convergent validity, and predictive validity for concussion symptom endorsement by Len Lecci, Kelly Dugan, Ken Zeiger, Julian Keith, Sasidharan Taravath, Wayland Tseh and Mark Williams in Journal of Concussion

Supplemental Material

sj-csv-4-ccn-10.1177_20597002231157947 - Supplemental material for Validation of an accelerometer-based gait assessment: Establishing test-retest reliability, convergent validity, and predictive validity for concussion symptom endorsement

Supplemental material, sj-csv-4-ccn-10.1177_20597002231157947 for Validation of an accelerometer-based gait assessment: Establishing test-retest reliability, convergent validity, and predictive validity for concussion symptom endorsement by Len Lecci, Kelly Dugan, Ken Zeiger, Julian Keith, Sasidharan Taravath, Wayland Tseh and Mark Williams in Journal of Concussion

Footnotes

Acknowledgments

We thank the many individuals at athletic clubs, universities, and medical clinics who facilitated data collection. Please contact the first author if interested in joining the multi-site research consortium collecting data on baseline and post incident assessments.

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Williams is a co-founder and shareholder of SportGait Inc. Dugan and Zeiger are employed at SportGait Inc, which is part of UNCW's Center for Innovation and Entrepreneurship. Lecci is a paid consultant and minor shareholder for SportGait Inc. Keith, Taravath, and Tseh have no conflicts to report. Tseh contributed to study 1, and Taravath to study 2. The remaining authors contributed to the two reported studies.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article

ORCID iD

Len Lecci

Supplemental material

Supplemental material for this article is available online.

References

Studenski

Perera

Patel

, et al. Gait speed and survival in older adults. JAMA 2011; 305: 50–58.

Rydwik

Bergland

Forsén

, et al. Investigation into the reliability and validity of the measurement of elderly people's clinical walking speed: a systematic review. Physiother Theory Pract 2012; 28: 238–256.

Cummings

Studenski

Ferrucci

. A diagnosis of dismobility—giving mobility clinical visibility: a mobility working group recommendation. JAMA 2012; 31: 2061–2062.

Tao

Liu

Zheng

, et al. Gait analysis using wearable accelerometers. Accelerometers (Basel) 2012; 12: 2255–2283.

Maggio

Ceda

Ticinesi

, et al. Instrumental and non-instrumental evaluation of 4 meter walking speed in older individuals. PloS one 2016; 11. https://doi.org/10.1371/journal.pone.0153583

Byun

Han

Kim

, et al. Test-retest reliability and concurrent validity of a single tri-axial accelerometer-based gait analysis in older adults with normal cognition. PLoS One 2016; 11. https://doi.org/10.1371/journal.pone.0158956

Fujiwara

Sato

Sugawara

, et al. The coefficient of variation of step time can overestimate gait abnormality: test-retest reliability of gait-related parameters obtained with a tri-axial accelerometer in healthy subjects. Accelerometers 2020; 20: 577.

Henriksen

Lund

Moe-Nilssen

, et al. Test–retest reliability of trunk accelerometric gait analysis. Gait Posture 2004; 19: 288–297.

Kluge

Gaßner

Hannink

, et al. Towards mobile gait analysis: concurrent validity and test-retest reliability of an inertial measurement system for the assessment of spatio-temporal gait parameters. Accelerometers (Basel, Switzerland) 2017; 17: 1522.

10.

Kobsar

Osis

Phinyomark

, et al. Reliability of gait analysis using wearable accelerometers in patients with knee osteoarthritis. J Biomech 2016; 49: 3977–3982.

11.

Moore

Hickey

Lord

, et al. Comprehensive measurement of stroke gait characteristics with a single accelerometer in the laboratory and community: a feasibility, validity and reliability study. J Neuroeng Rehabil n 2017; 14. https://doi.org/1-10.10.1186/s12984-017-0341-z

12.

Werner

Heldmann

Hummel

, et al. Concurrent validity, test-retest reliability, and sensitivity to change of a single body-fixed accelerometer for gait analysis during rollator-assisted walking in acute geriatric patients. Accelerometers 2020; 20: 4866.

13.

Moisan

Descarreaux

Cantin

. The influence of footwear on walking biomechanics in individuals with chronic ankle instability. PLoS ONE 2020; 15. https://doi.org/10.1371/journal.pone.0239621

14.

Arnadottir

Mercer

. Effects of footwear on measurements of balance and gait in women between the ages of 65 and 93 years. Phys Ther 2000; 80: 17–27.

15.

Wegener

Hunt

Vanwanseele

, et al. Effect of children's shoes on gait: a systematic review and meta-analysis. J Foot Ankle Res 2011; 18. https://doi.org/10.1186/1757-1146-4-3

16.

Godfrey

Del Din

Barry

, et al. Instrumenting gait with an accelerometer: a system and algorithm examination. Med Eng Phys 2015; 37: 400–407.

17.

Tesio

Rota

. The motion of body center of mass during walking: a review oriented to clinical applications. Front Neurol 2019; 10: 99.

18.

Winter

. Biomechanics and motor control of human movement. Hoboken, NJ, USA: John Wiley & Sons, 2009.

19.

Oppenheim

Schafer

. Discrete-time signal processing. Upper Saddle River, NJ, USA: Prentice-Hall, 1989.

20.

Lecci

Williams

Taravath

, et al. Validation of a concussion screening battery for use in medical settings: predicting centers for disease control concussion symptoms in children and adolescents. Arch Clin Neuropsychol 2020; 35: 265–274.

21.

Rusnak

. Traumatic brain injury: giving voice to a silent epidemic. Nat Rev Neurol 2013; 9: 186–187.

22.

Dewan

Rattani

Gupta

, et al. Estimating the global incidence of traumatic brain injury. Journal of Neurosurgery JNS 2019; 130: 1080–1097.

23.

Shimada

Ando

Matsunaga

, et al. Clinical application of acceleration accelerometer to detect the swing phase of stroke gait in functional electrical stimulation. Tohoku J. Ex. Med 2005; 207: 197–202.

24.

Preece

Goulermas

Kenney

, et al. Comparison of feature extraction methods for the classification of dynamic activities from accelerometer data. IEEE Trans Biomed Eng 2009; 56: 871–879.

25.

Dever

Powell

Graham

, et al. Gait impairment in traumatic brain injury: a systematic review. Accelerometers 2022; 22: 1480.

26.

Giladi

Shema

Brozgol

, et al. Does the evaluation of gait quality during daily life provide insight into fall risk? A novel approach using 3–day accelerometer recordings. Neurorehabil Neural Repair 2013; 27: 742–752.

27.

Moe-Nilssen

Helbostad

. Interstride trunk acceleration variability but not step width variability can differentiate between fit and frail older adults. Gait Posture 2005; 21: 164–170.. PMID: 15639395.

28.

Chung

Hsu

Wang

, et al. Gait analysis for patients with Alzheimer's disease using a triaxial accelerometer. IEEE 2012: 1323–1326. https://10.1109/ISCAS.2012.6271484.

29.

Paquet

Auvinet

Chaleil

, et al. Analysis of gait disorders in Parkinson's disease assessed with an accelerometer. Rev Neurol (Paris) 2003; 159: 786–789. PMID: 13679722

30.

Mizuike

Ohgi

Morita

. Analysis of stroke patient walking dynamics using a tri-axial accelerometer. Gait Posture 2009; 30: 60–64.

31.

Leahy

Feudtner

Basch

. Symptom monitoring in pediatric oncology using patient-reported outcomes: why, how, and where next. Patient 2018; 11: 147–153.

32.

Meier

Brummel

Singh

, et al. The underreporting of self-reported symptoms following sports-related concussion. J Sci Med Sport 2015; 18: 507–511.

33.

Silsupadol

Teja

Lugade

. Reliability and validity of a smartphone-based assessment of gait parameters across walking speed and smartphone locations: body, bag, belt, hand, and pocket. Gait Posture 2017; 58: 516–522.

34.

Lecci

Williams

Dugan

, et al. Neuromotor functioning and concussion in adolescents: Introduction and clinical analyses of a mobile gait assessment. Manuscript under review, 2022.

35.

Buganè

Benedetti

D’Angeli

, et al. Estimation of pelvis kinematics in level walking based on a single inertial accelerometer positioned close to the sacrum: validation on healthy subjects with stereophotogrammetric system. BioMed Eng OnLine 2014; 13: 146.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB

0.30 MB

0.05 MB

0.03 MB

Validation of an accelerometer-based gait assessment: Establishing test-retest reliability,convergent validity,and predictive validity for concussion symptom endorsement

Abstract

Background

Objective

Methods

Results

Conclusions

Keywords

Introduction

Study 1

Materials and methods

Patient and public involvement

Procedure

Results

Participants

Evaluating the effects of walking surface and footwear on gait accelerometer data

Reliability of accelerometer gait assessment

Study 2

Materials and methods

Patient and public involvement

Participants

Procedure

Results

Predicting NIH 4 m gait with the BKG

Predicting CDC concussion symptom endorsement using the BKG and NIH 4 m gait test

Predicting CDC concussion symptom endorsement using the BKG and BESS

Discussion

Conclusions and clinical implications

Limitations and future directions

Supplemental Material

sj-docx-1-ccn-10.1177_20597002231157947 - Supplemental material for Validation of an accelerometer-based gait assessment: Establishing test-retest reliability, convergent validity, and predictive validity for concussion symptom endorsement

Supplemental Material

sj-csv-2-ccn-10.1177_20597002231157947 - Supplemental material for Validation of an accelerometer-based gait assessment: Establishing test-retest reliability, convergent validity, and predictive validity for concussion symptom endorsement

Supplemental Material

sj-csv-3-ccn-10.1177_20597002231157947 - Supplemental material for Validation of an accelerometer-based gait assessment: Establishing test-retest reliability, convergent validity, and predictive validity for concussion symptom endorsement

Supplemental Material

sj-csv-4-ccn-10.1177_20597002231157947 - Supplemental material for Validation of an accelerometer-based gait assessment: Establishing test-retest reliability, convergent validity, and predictive validity for concussion symptom endorsement

Footnotes

Acknowledgments

Declaration of conflicting interests

Funding

ORCID iD

Supplemental material

References

Supplementary Material