Sage Journals: Discover world-class research

Abstract

Objective

Prompt, accurate, objective assessment of concussion is crucial as delays can lead to increased short and long-term consequences. The purpose of this study was to derive an objective multimodal concussion index (CI) using EEG at its core, to identify concussion, and to assess change over time throughout recovery.

Methods

Male and female concussed (N = 232) and control (N = 206) subjects 13–25 years were enrolled at 12 US colleges and high schools. Evaluations occurred within 72 h of injury, 5 days post-injury, at return-to-play (RTP), 45 days after RTP (RTP + 45); and included EEG, neurocognitive performance, and standard concussion assessments. Concussed subjects had a witnessed head impact, were removed from play for ≥ 5 days using site guidelines, and were divided into those with RTP < 14 or ≥14 days. Part 1 describes the derivation and efficacy of the machine learning derived classifier as a marker of concussion. Part 2 describes significance of differences in CI between groups at each time point and within each group across time points.

Results

Sensitivity = 84.9%, specificity = 76.0%, and AUC = 0.89 were obtained on a test Hold-Out group representing 20% of the total dataset. EEG features reflecting connectivity between brain regions contributed most to the CI. CI was stable over time in controls. Significant differences in CI between controls and concussed subjects were found at time of injury, with no significant differences at RTP and RTP + 45. Within the concussed, differences in rate of recovery were seen.

Conclusions

The CI was shown to have high accuracy as a marker of likelihood of concussion. Stability of CI in controls supports reliable interpretation of CI change in concussed subjects. Objective identification of the presence of concussion and assessment of readiness to return to normal activity can be aided by use of the CI, a rapidly obtained, point of care assessment tool.

Keywords

CONCUSSION INDEX mild Traumatic Brain Injury sport-related concussion machine learning classifier algorithm EEG return-to-play

Introduction

The Center for Disease Control (CDC) estimates that mild Traumatic Brain Injury (mTBI/concussion) accounts for 1.5 million Emergency Department (ED) visits annually in the United States, with an additional 1.6–3.8 million concussions occurring in sports and recreational activities annually¹ (among which, 1.1–1.9 million concussions occur in children under 18²). These figures are likely to be significant underestimates since it has been reported that a large percentage of patients with mild head injury/concussion may not seek medical care.^2–6 Despite the tendency to under-report, ED visits for mild head injury have significantly increased, while overall visits to the ED remained relatively unchanged,⁷ likely due to increased media attention raising awareness of the short and long-term effects of TBI and its sequelae. Children/adolescents and young adults are particularly vulnerable as significant brain development continues throughout these years. The lack of, or delay, in concussion diagnosis and/or lack of appropriate concussion management have been shown to lead to much slower recovery from concussion,^8,9 and may affect academic/cognitive and emotional functioning.^10–12 Sustaining a concussive injury before age 25 has been shown to be associated with impaired adult functioning, psychiatric disorders, low education level, welfare, and disability.¹³

Although there is currently no “gold standard” for the diagnosis and management of concussion, numerous consensus and position statements emphasize the importance of multimodal assessments of concussion, including: self-reported symptoms, motor control, mental status testing, and neurocognitive testing.^14–18 Likewise, numerous societal bodies (e.g. the American Academy of Neurology (AAN), the National Athletic Trainers’ Association (NATA)) advocate a multidimensional approach to the management of SRC consisting of computerized neurocognitive testing (CNT), assessment of postural stability (balance), vestibular and ocular assessment, and self-reported symptoms.¹⁹ In addition, the literature supporting multimodal assessments for the evaluation of mTBI/concussion has pointed to the limitations of most current, largely subjective assessment tools^14,19,20

An extensive literature demonstrates that changes in brain electrical activity occur in mTBI, as reflected in measures of connectivity (disruption in relationships between brain regions), changes in complexity of the signal (disorganization of neural networks), and changes in the frequency spectra (changes in oxygen utilization, glucose metabolism and neurochemistry)^21–24 Researchers have reported extremely high accuracy in the objective identification of traumatic structural brain injury (bleed of 1 cc or greater)²⁵ and brain function impairment (concussion)^26,27 using quantitative EEG (qEEG) as input to classifier algorithms built using machine learning (ML) methods,²⁸ in patients seen within 72 h of closed head injury (i.e. blunt or non-penetrating head trauma that does not include a break in the skull), with Glasgow Coma Scale (GCS) 13–15. Following a similar approach, Part 1 of this paper describes the derivation and performance of a quantitative, multimodal, objective Concussion Index (CI), obtained using machine learning, which includes EEG at its core as well as neurocognitive performance measures and selected symptoms. The CI is summarized into a single multivariate index. Part 2 examines clinical implications of the CI as an objective tool to aid in the assessment of concussion from time of injury through the recovery phase, and the relationship between the CI and severity as reflected in rapid or protracted recovery and total symptom burden.

Part 1: Derivation of the concussion index algorithm

This section describes in detail the Algorithm Development study for the derivation of the Concussion Index (CI). It should be emphasized that the aim of this work was to derive such an index from a set of features which can potentially lead to a rapid, accurate assessment of concussion in a hand-held device, at the point of care.

Methods

Patient population

Data was collected from August 2015 to January 2019 using BrainScope^a investigational devices (investigators were blinded to output) at 12 college and high school sites that recruited from 60 locations across the US, with approval from primary sites’ Institutional Review Boards (IRBs).^b The study included concussed subjects and controls (matched primarily for age, sex and type of sport played), as well as healthy volunteers, all between the ages of 13 and 25, who met inclusion/exclusion criteria. Injured subjects were assessed within 72 h of time of injury (Day 0), 5 days after injury (Day 5), at their clinicalsitedetermined eturn-o-lay date (RTP), and 45 days after RTP (RTP + 45). Each matched control subject was assessed at time intervals corresponding to those of their injured counterpart. In addition, in order to expand the controls, a group of non-injured healthy controls were recruited from the same population as the injured and matched controls, but were only tested at pre-season. All subjects signed written informed consent and for minors recruited to participate, parental written informed consent and adolescent assent was also obtained.

Inclusion criteria for “concussed”: Males and females between the ages of 13 and 25 having sustained a traumatic closed head injury within 72 h (3 days) from time of injury to time of BrainScope assessment, with a GCS²⁹ score ≥ 13 and no hospital admission due to either head injury or collateral injuries for more than 24 h.

Inclusion criteria for “controls”: Males and females between the ages of 13 and 25, GCS of 15 at time of assessment and no history of concussion or TBI within the past year, or Motor Vehicle Accident (MVA) requiring Emergency Department visit within the last year.

Exclusion criteria for all subjects: Subjects were excluded who had forehead, scalp or skull abnormalities or whose clinical condition, such as head trauma, would not allow placement of the electrodes; current Central Nervous System (CNS) active prescription medications taken daily (with the exception of medications being taken for Attention Deficit Disorder (ADD) or Attention Deficit Hyperactivity Disorder (ADHD); history of brain surgery or neurological disease; pregnant women; acute intoxication or evidence of illicit drug usage; previous enrollment in BrainScope studies; loss of consciousness (LOC) of 20 min or more related to the concussion injury; active fever defined as greater than 100°F or 37.78°C at time of assessment; evidence of abnormality visible on CT of the head related to the traumatic event; and patients who do not speak or read English.

Additional exclusion criteria for “controls”: Any focal neurological signs including aphasia, apraxia, diplopia, facial droop, dysarthria/slurred speech.

With regard to factors such as drugs or alcohol, fatigue, pain, and other factors which may be co-morbid with head injury, the rule followed in this investigation was to not exclude them from any subject groups (controls and head-injured patients), except as defined by exclusion criteria. By doing this, they are eliminated as differentiating factors between groups, so that features that might be sensitive to these factors do not impact the ML algorithm development task.

Definition of concussion (“Clinical Truth”) and RTP: In this study, concussed subjects were defined as those who had a witnessed head impact and who, by local site guidelines, were removed from play for 5 or more days (as subjects who return to play within 4 days of injury are not likely to have been concussed). The use of site guidelines assured broader applicability of the results to the general population of interest. RTP determination for each subject (reported as the number of days from injury date to “cleared to play” date) was made in accordance with site RTP protocols. All the sites used a gradual/graded RTP protocol at the end of which a subject was cleared to play. Furthermore, for all the colleges this protocol conformed to NCAA policy guidelines.³⁰ These guidelines included the following steps once the subject was declared symptom-free: (1) Light aerobic exercise; (2) Sport-specific activity with no head impact; (3) Non-contact sport drills and resumption of progressive resistance training; (4) Unrestricted training; (5) Return to competition. If at any point the student/athlete became symptomatic, they were returned to the previous stage.

Subject subgroups: For the purpose of further assessing performance of the CI, subjects were subdivided into the following subgroups: (1) Category 1: Controls (CNT) which included non-injured subjects who were functionally asymptomatic; (2) Category 2 (Cx2) injured subjects with RTP between 5 and 13 days (“normal/rapid recovery” group); and Category 3 (Cx3) which included injured subjects with RTP of 14 days or more (“protracted recovery” group). It is noted that McCrory et al. reported that 80% of concussed subjects recover within 14 days.¹⁴ The age, sex and race distributions of the sample were determined by the representation of each in the populations served by the participating sites involved.

Clinical assessments

All study subjects were evaluated with the following symptombased scales or assessment tools: Subsections of the Sports Concussion Assessment Tool – 3rd Edition (SCAT-3³¹) or 5th Edition (SCAT-5), including: GCS (Section 1), Symptom Evaluation (SCAT-3 Section 3 and SCAT-5 Section 2, which assesses presence and severity of 22 common concussion symptoms on a Likert scale (symptom severity self-rated from 0–6 per item, with a total score range of 0–132), Standard Assessment of Concussion (SAC, SCAT-3 Sections 4 and 8, SCAT-5 Sections 3 and 5): a brief cognitive screening tool which includes subtests of orientation, immediate memory, concentration, and delayed recall (total score in range 0–30).³² Subjects in both control and injured groups were queried regarding history of head injury and concussion.

Section 2 of the SCAT contains the Concussion Symptom Inventory³³ (CSI), the standard tool used for the quantitative assessment of presence and self-reported severity of concussion symptoms (“symptom burden”) at time of injury and for monitoring recovery from SRC. The total CSI score is constructed by adding clinical scores for the 22 symptoms (“CSI22”) queried, where the severity of each symptom is self-reported (by the patient) on a Likert (0–6) scale, with zero indicating absence of symptom. In this study, as frequently done in the literature, the total score on the 22-item CSI was used as an estimate of the total symptom burden.^33,34

Neurocognitive performance tests

Two of the neurocognitive performance tests which have been demonstrated in the literature to be affected by concussion^35,36 were included in the data reported on in this study: Simple Reaction Time (SRT; evaluates simple motor speed, information processing speed and attention), and Procedural Reaction Time (PRO; evaluates higher-order rapid responding, including visuomotor reaction time and simple decision making). While additional cognitive performance tests were included in the neurocognitive test battery for a portion of the population (Match to Sample and Go/No-Go), they were not included in the final analyses and are not reported on herein, as preliminary analyses revealed a high level of correlation between the tasks. Moreover, as inputs to the ML algorithms, only the “throughput” feature values of SRT and PRO were used, which are composite scores which jointly reflect both speed of execution and accuracy of the subject’s response. The raw neurocognitive features were age- and sex-normed and z-transformed to standard scores using a large normative database of subjects aged 13–80 collected on the device. This accounts for any effects that might be due to sex or age. These features have been shown to be both valid for use in concussion and reliable.^37–40

EEG data acquisition

BrainScope handheld investigational devices^c were used to collect 10 min of EEG data acquired under an eyes closed resting condition.^25,26 A technician observed the subjects throughout data acquisition for vigilance and alerted them if necessary, for example if they appeared drowsy. The EEG data was recorded from the limited frontal electrode montage including Fp1, Fp2, F7, F8, AFz, A1 and A2 locations of the expanded International 10–20 Electrode Placement System, and was re-referenced to linked ears. The EEG data was acquired at a sampling rate of 1 kHz and all electrode impedances were below 10 kΩ. Amplifiers had a band-pass filter from 0.3 to 250 Hz (3 dB points) and EEG data was downsampled to 100 Hz for feature extraction.

EEG data processing and quantitative EEG feature extraction

The EEG signals were processed using BrainScope’s on device real-time suite of algorithms for artifact detection,⁴¹ which identify for removal any physiologic and non-physiologic contamination (e.g. including lateral and horizontal eye movement, muscle activity), assuring quality of the EEG data. Only artifact-free data (1–2 min) was submitted to all further analyses. A large set of quantitative EEG (qEEG) features were then computed and z-transformed relative to age expected normal values, effectively removing any potential effect of age on these features. These steps are described in detail elsewhere.^27,41 The feature sets characterizing the EEG signal include measures of power (absolute and relative), mean frequency, measures of connectivity (asymmetry, coherence, phase lag, phase synchrony), measures of complexity (fractal dimension and scale-free activity), and information theory based features (entropy), across and within frequency bands. The resulting database of qEEG feature z-scores served as inputs to the derivation of the classifiers.

Multimodal classifier development

All extracted EEG features were subjected to an informed data reduction step⁴¹ as a first level of feature selection. In Machine Learning, overtraining is known to lead to poor performance on independent populations and this step reduces this risk. The resulting reduced feature set became the candidate feature set for classifier building.

The multimodality of the input features was represented by three broad types of quantitative features: electrophysiological (qEEG) feature z-scores, neurocognitive throughput feature z-scores, and selected clinical sign/symptom features. Internal and external cross-validation paradigms were used, involving multiple train & test splits of the dataset, in order to further avoid overtraining when building linear discriminant function classifiers that separate the two main classes (Control vs. Concussed), and to maximize the probability of validation in prospective independent data (previously unseen). The final classifier features and weights were derived from a training set consisting of 80% of the total dataset, while its performance was tested and reported on the remaining 20% (the Hold-Out dataset).

Classifier development involved searching the reduced dimensionality multimodal feature space using advanced ML-based techniques (e.g. Genetic Algorithms and LASSO Logistic Regression methods) in order to obtain classifier candidates that could optimally separate the two categories while avoiding overtraining. This methodology was described in detail previously.²⁷ The metrics used for evaluating classifier performance during training were the area under the curve (AUC) of the Receiver Operating Characteristic (ROC) as well as sensitivity and specificity of the classifiers at their operating point.

Statistical analysis

Demographics of the study population, including prevalence of typical concussion symptoms in the Control and Injured groups will be described. CI performance will be assessed as sensitivity and specificity of the classification algorithm at Day 0 on the independent Test and Hold-Out datasets. In addition, sensitivity will also be assessed for two distinct groups of concussed subjects. The area nder the ROC urve (AUC) will also be computed as a measure of accuracy of the classifier function.

Results

Subject demographics and symptoms prevalence at time of injury

The study population at Day 0 (time of injury) consisted of 232 Injured subjects (67.2% males, median age = 19.0 (13.8–25.0)) and 206 controls (66% males, median age = 19.0 (13.7–24.6)). Additionally, 85 non-injured healthy controls (51.8% male, median age = 17.8 (13.2–21.4)) at pre-season baseline were enrolled. The Cx group included the two subtypes described in the Patient population section above, with 93 normal recovery subjects (Cx2), and 139 protracted recovery subjects (Cx3) based on the RTP cut-off (14 days).

Table 1 shows differences in prevalence at intake for several clinical symptoms of concussion for the concussed compared with controls. As expected, concussed subjects were characterized by a significantly higher incidence of these symptoms. The median RTP for the Cx group was 15 days (range 5–70), which is consistent with the results reported by Kerr et al. in a sample of high school and college American football athletes.⁴²

Table 1.

Prevalence of common concussion signs/symptoms (Sx) at Day 0 in controls/CNT vs injured/Cx are shown in the first two rows of the Table.

	Loss of consciousness (LOC)	Retrograde amnesia (RGA)	Altered mental status (AMS)	Severe headache (SHA)	Balance problems	Dizziness	Visual disturbances (blurred or double vision)
% CNTs with symptom (Sx)	0	0	0	0	2.9%	3.4%	2.4%
% All Cx with Sx	12.5%^a	3.9%^b	51.7%^a	23.3%^a	40.9%^a	52.2%^a	29.7%^a

Notes: It is noted that “Visual Disturbances” specifically refers to blurred or double vision (diplopia); and “Severe Headache” was defined as a score ≥ 4 on the Likert scale. Significant Chi-squared differences are indicated:ap < 0.001; ^bp < 0.005 for comparisons between CNT and Cx.

Classifier performance at time of injury

Results reported in this section are based on the estimate of performance on multiple unseen datasets (Test and Hold Out), referred to as “external cross-validation.” Table 2 summarizes performance (sensitivity, specificity, and rea nder the ROC urve) of the linear classifier algorithms in an external cross-validation context (average sensitivity/specificity on the test portions of 10 train/test splits), with tandard eviation and 95% confidence intervals: sensitivity = 84.9% (95% CI: 81.6–88.2%)/specificity = 76% (95% CI: 71.7–80.3%); as well as sensitivity/specificity of the final/“All In” classifier on the unseen Hold Out group (N = 105; CNTs: N = 58; Cx: N = 47): sensitivity/specificity = 91.5%/67.2%. In addition, this table shows the average sensitivity obtained on the two injured sub-groups (“Prolonged RTP”/Cx3, “Rapid RTP”/Cx2). On the Hold Out data, the Negative Predictive Value (NPV) was 90.7% and the Positive Predictive value (PPV) was 69.4%. It is important to note that the final algorithm built using the “All In” dataset (i.e. entire dataset minus a random Hold Out group representing close to 20% of the data) was locked prior to its prospective independent validation described in Bazarian et al., 2020.⁴³

Table 2.

Summary performance table for the classifiers developed on the 10 Train/Test splits (external cross-validation) and for the “All In” classifier (performance on independent 20% Hold-Out dataset (HO)).

	Test/HO group results
	Cx	CNT	Cx3		Cx2		Cx, CNT
	Sensiitivity	Specificity	Sensitivity (Cx3)	N (Cx3)	Sensitivity (Cx2)	N (Cx2)	AUC
Mean (SD)	84.9 (5.3)	76.0 (7.0)	86.5 (5.6)	22	82.5 (8.3)	15	0.8938 (0.034)
95% CI	81.6–88.2	71.7–80.3	83–90		77.3–87.7		0.021
HO	91.5	67.2	100	28	78.9	19	0.905

Cx2: injured subjects with rapid recovery (RTP < 14); Cx3: injured subjects with prolonged recovery (RTP ≥ 14).

Notes: Area Under the ROC Curve (AUCs) are for the Cx vs. CNT classification task, that is, for discrimination between Concussed and Control at time of injury. The last row shows Sens/Spec obtained on the Hold Out dataset.

To aid in interpretability, the weighted sum output y of the linear discriminant function classifier was mapped to a continuous 0–100 range using the scaled logistic mapping (“inverted S curve”) given in equation (1) which produces a mapped classification threshold of 70 for subjects classified in the “positive class” (i.e. disease present/concussed) if CI ≤ 70.

CI = 100 - 100 / (1 + exp (- y / 2))

(1)

Four of the top-five contributors to the classification were qEEG features. It is also noted that the top 5 contributors to separation between the groups did not include neurocognitive features. The qEEG features contributing most to classification were those measuring connectivity between brain regions, including: phase, coherence, and power asymmetry/gradient features, especially in frequency bands that included alpha, beta and total power. It is of note that all but one of the qEEG features in the algorithm were from the connectivity measures, suggesting reflection of white matter tract integrity.⁴⁴

Part 2: Demonstration of CI as an objective tool to aid in assessment of concussion

This section of the paper examines clinical implications of the CI as an objective tool to aid in assessment of concussion from time of injury through the recovery phase, and demonstrates the relationship between the CI and severity as reflected in rapid or protracted recovery and total symptom burden.

Statistical analyses

Applying the CI derived in Part 1, this section of the manuscript demonstrates the clinical application of the CI to track recovery. Since the CI was derived from Time of Injury data only, the data presented in this section including Day 5 and RTP can be considered independent applications of the CI.

Statistical analyses

The change in CI values over time for controls and concussed subjects was evaluated within each group using the non-parametric Wilcoxon rank tests between time points. Results were also graphically displayed as boxplots for each group and each time point. The correlation between change in symptom burden and change in CI as reflected at the Day 5 intermediate time point, was computed for the full study population.

Results

Analysis of longitudinal change in the CI

Box plots showing the distribution of the CI at baseline/pre-season (when available), Day 0, and at subsequent evaluation time points (Day 5, RTP, RTP + 45, post-season), for the controls and the injured subjects are shown in Figure 1. Importantly, since the CI was developed using Day 0 data only, CI values for controls and injured at Day 5 and at the other follow-up time points can be considered partially independent test results. At baseline and all subsequent measurements, the controls have a median CI which is above threshold (CI ∼75) and a lower (first) quartile which is above the binary classification threshold (CI = 70). At Day 0, the injured Cx groups (Cx2 + Cx3 combined) have a median CI which is well below the threshold and which can be seen to gradually rise to a level which is on par with that of the control group at the same time point at RTP and RTP + 45. In addition, it can be seen that at Day 0 those injured subjects with “normal recovery” (rapid RTP) have a higher median CI than those with protracted recovery (prolonged RTP), suggesting that the CI is reflective of severity of concussion. The CI at Day 5 also shows very different patterns of recovery in the two subgroups of injured subjects, with the median CI of the normal recovery group at threshold, while approximately 75% of the protracted recovery group is still below threshold.

Figure 1.

Boxplot of Concussion Index (CI) for controls and injured groups for the full algorithm development dataset. Top-left: Controls, top-right: All injured subjects, bottom-left: Injured subjects with 5 ≤ RTP <14 days (Cx2), and bottom-right: Injured subjects with RTP ≥ 14 days (Cx3).

The significance of the differences between the CI scores of the controls and the injured, were assessed at each time point using the Wilcoxon rank test. The first-order statistics of the CI for Controls and Injured subjects at the various time points are reported in Table 3. The significance levels (p-values, with Bonferroni correction for multiple tests) are reported in Table 4 for the Hold-Out (Testing) Algorithm Development dataset. At Day 0 (day of injury) the CI of the injured was significantly lower than that of the controls (HO: p < 0.001) and this significance was present when considering the Cx2 and Cx3 groups separately. At Day 5, the CI of the injured was still significantly lower than that of the controls (HO: p < 0.001). At RTP, the CI of the concussed group was no longer significantly lower than that of the controls (p = 1).

Table 3.

Statistics of CI for control and injured subjects at Day 0 (time of injury), Day 5, RTP, RTP + 45 for the Hold-Out dataset.

Day	N controls	CI median (Q1, Q3)	CI mean (SD)
0	39	77.1 (60.4, 83.5)	72.1 (16)
5	36	76.9 (71, 82.4)	76.1 (7.9)
RTP	33	79.2 (69.9, 82.7)	75.8 (9.4)
RTP + 45	31	76.6 (67.9, 80.6)	74 (10)
Day	N injured (Cx2)	CI median (Q1, Q3)	CI mean (SD)
0	19	37.2 (10.9, 67.8)	39.8 (30.6)
5	9	67.4 (54.8, 84.4)	69.8 (17)
RTP	18	77.6 (68.6, 80.3)	74.8 (10.1)
RTP + 45	15	73.8 (69.2, 79.7)	74 (8.9)
Day	N injured (Cx3)	CI median (Q1, Q3)	CI mean (SD)
0	28	15.4 (9.6, 27.9)	20.5 (17)
5	26	38.4 (17, 62.8)	41.4 (27.4)
RTP	28	78.1 (71, 81.2)	76.4 (8)
RTP + 45	25	74.8 (67.3, 81.9)	73.2 (13.3)

Notes: Injured subjects are split into injured with shorter recovery (Cx2) and injured with prolonged recovery (Cx3).

SD: Standard Deviation; Q1: first quartile; Q3: third quartile.

Table 4.

Number of control and injured subjects at Day 0 (time of injury), Day 5, RTP, RTP + 45 for the Hold-Out dataset and p-values for the significance of the differences in the CI between controls and injured (all injured (Cx = Cx2 + Cx3), and separately for controls vs. injured with normal recovery (Cx2), controls vs. injured with protracted recovery (Cx3)).

Day	N controls	N injured (all Cx)	N Cx2	N Cx3	p-Value Cx vs CNT	p-Value Cx3 vs CNT	p-Value Cx2 vs CNT
0	39	47	19	28	3.2E-14	1.2E-18	8.2E-6
5	36	35	9	26	1.2E-6	4.1E-9	0.4
RTP	33	46	18	28	1.0	1.0	1.0
RTP + 45	31	40	15	25	1.0	1.0	1.0

Notes: p-values from Wilcoxon rank test were Bonferroni corrected for multiple tests and values below p = 0.05 are indicated in bold.

In order to validly interpret change over time in the concussed population, it was important to demonstrate stability of the CI in the controls over the same time period. As can be seen in Table 5, there were no significant differences between CI scores of the controls at Day 0 and at any of the subsequent time points (Day 5, RTP, or RTP + 45), which demonstrates stability of the CI over time.

Table 5.

Number of control subjects at Day 0, Day 5, RTP, RTP + 45 and p-values for the Wilcoxon rank test for a median comparing CI scores at Day 0 vs. CI scores at subsequent days (Day 5, RTP, RTP + 45).

Day	N CNT	p-Value Controls at Day 0 vs. Day X
0	206	NA
5	164	1.0
RTP	176	0.94
RTP + 45	170	0.26

Note: Test p-values were corrected for multiple tests using the Bonferroni method.

Correlation between CI and clinical “symptom burden” during recovery (Day 5)

The different patterns of recovery of the normal and prolonged recovery groups, as noted above, suggests a relationship between CI and clinical symptom severity as measured at the intermediate time point. To further explore this relationship, the correlation between quantitative measures of clinical symptom burden (measured as total CSI score) and the CI at Day 5 was evaluated at this independent intermediate time point. Although not a measure of recovery, an association between the two metrics would support a clinical manifestation of change reported by CI.

Figure 2 shows the relationship between CI and CSI at Day 5 (this time point was used to demonstrate relationship at an intermediate time point during the recovery phase) for four different subgroups of concussed subjects based on time to RTP (RTP < 14, 14 ≤ RTP < 21, 21 ≤ RTP < 28, RTP ≥ 28). It should be noted that the Pearson correlation between CI at Day 5 and CSI was highly significant (r = –0.7, p = 5.5E-31), as were the correlations between those same features when restricting to any of the four subgroups (with respective values of: r = –0.72, p < 0.001; r = –0.65, p < 0.001; r = –0.48, p < 0.01; r = –0.79, p < 0.001). Further, the figure shows that injured subjects with low CI scores tend to have high CSI scores (top left quadrant of the figure), and that most of the subjects with RTP <14 days showed this relationship. On the other hand, subjects with a relatively high CSI at Day 5 (CSI ≥ 20) had lower CI scores, most falling below CI threshold (<70) and that most of those with RTP >14 days showed this relationship.

Figure 2.

Scattergram of Concussion Index (CI) at Day 5 versus CSI (22 items), with second-order polynomial fit, for injured subjects within the full dataset. Note: circle size from smallest to largest represents increasing RTP.

Discussion

The need for objective identification of the presence of concussion and assessment of readiness to return to normal activity represents an important health care need addressed in Part 1 of this study which describes the derivation of the Concussion Index in subjects 13–25 years old. The CI is a multimodal, multivariate index with EEG at its core, integrating quantitative information about brain function impairment in concussion. Features which reflect changes in connectivity between brain regions contribute most to the index. Results of this study demonstrate high sensitivity, specificity and accuracy (AUC) of the CI in identifying the likelihood of concussion following head injury (within 72 h). Importantly, common comorbidities (e.g. ADHD, LD, headache, etc.) are not excluded from the head-injured or non-head-injured groups, such that separation between the groups are not influenced by the presence of such factors.

Although advanced neuroimaging can be used to distinguish between groups with concussive brain injury and controls,⁴⁵ such technologies are not readily available either at the sidelines, in urgent care centers, concussion clinics, or in Emergency Departments. However, studies have demonstrated significant correlation between disturbances in white matter integrity seen on DTI and the CI studied in the same subjects. These results suggest that the CI^d can act as a surrogate for other neuroimaging tools, providing many advantages in sideline testing (in locker room or nearby venue) at the time of injury.⁴⁴

Part 2 of the paper demonstrates that progressive recovery was reflected in significant changes observed on the median CI values between time of injury and Day 5. While overall the CI of the injured athletes was significantly lower than that of the controls, the CI also points to the heterogeneity within the concussed population, showing different rates of recovery at Day 5 (an independent time point not used to derive the CI algorithm). At Day 5, clear differences were seen between the median CI of those with normal and those with protracted recovery. In fact, while ∼75% of the protracted group were still below concussion threshold at Day 5, ∼50% of those in the normal recovery group were already above threshold. In addition, at clinically determined RTP (largely based on self-report of symptom persistence), there were subjects whose CI was still below threshold, indicating that some subjects may have been returned to play prior to resolution of brain function abnormalities. Similar findings have been reported in the literature attesting to the persistence of brain function abnormalities in sport-related concussions beyond the point when clinical symptoms have resolved.^46–50

Further, the demonstration that concussed subjects show no significant differences from the controls at RTP + 45 suggests that CI scores for injured subjects at RTP + 45 have returned to the point where they fall within the baseline range of a non-injured population, suggesting that brain function impairment was resolved. The reliability of such an interpretation is supported by the demonstrations of the lack of significant differences in the controls between Day 0 (treated as their baseline measurement) and RTP + 45, in conjunction with the lack of significant differences between controls and injured at RTP + 45. These findings taken together suggest that RTP + 45 in the concussed population can be considered as their “baseline,” and supports the demonstration of a “return to baseline” for the concussed subject at RTP.

It is of note that the presence and severity of concussion symptoms (“symptom burden”) as reflected in the total CSI score was significantly correlated with CI throughout the period of observation, from time of injury through RTP. While highly correlated, the CI is an objective measure which is less reliant on self-report, and less subject to under- or over-reporting or learning effects over time, adding confidence to interpretation of change over time.

Limitations of this study include the lack of other intermediate time points between 72 h of injury and RTP. Future studies should perform serial evaluations through the recovery phase to better estimate the recovery rates of individuals and the relationship to changes in CI. Another limitation is the restricted age range for which this study was performed (ages 13–25), although these ages are critical for the high school and college age student athletes Studies are currently underway to address this age limitation. In the context of the aim of constructing an objective assessment tool that could be rapidly applied on the handheld device, the inputs to the algorithm were selected with this in mind. Future studies will evaluate the contribution of other features, including oculomotor measures which are being investigated in an ongoing study.

Despite the fact that ADHD medication was not an exclusion criterion, only one injured and one control subject in our study were taking such a medication. It was therefore not possible to study the potential impact of ADHD medication on the accuracy of the CI for this subject group. Most importantly, the need for an independent validation of the CI in a prospective population is critical to validate the index. Such study has been successfully conducted⁴³ and was presented in support of BrainScope’s FDA clearance for the Concussion Index (September 2019).

Conclusions

In Gioia, 2015,⁵¹ it is stated that “ultimately, the practice of concussion assessment and management will benefit from an evidence-based medicine approach […] whereby clinicians have access to statistical bases for evaluating a patient’s scores relative to those seen in subjects with known concussions versus non-injured subjects.” The Concussion Index, a multivariate, multimodal, objective, quantitative index of brain function impairment directly addresses these needs. Once independently validated, the CI can be used as an adjunct to other clinical assessment tools in the evaluation of concussion, with the potential to allow tracking of concussion recovery. The ability to obtain such an index rapidly, in a hand held device, at the point of care, and at any point along the recovery trajectory, potentially represents significant added value as an adjunct to current clinical practice to determine concussion and track recovery in a more confident and objective way.

Footnotes

Acknowledgments

The authors acknowledge the contributions of those who made this research possible, including research staff at all the clinical sites and the athletes who volunteered for the study. In addition, the contributions to the study of Dr. R. Lopez (University of South Florida, Tampa, FL), Dr. A. Meyer (University of New Mexico, Albuquerque, NM), Dr. C. Neville (SUNY Upstate Medical University) and Dr. K. Podell (Rice University, Houston, TX) are acknowledged.

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Dr. Jacquin was employed by BrainScope Company at the time this study was conducted, but is no longer an employee. Dr. Prichep is employed by BrainScope, and is an inventor on patents licensed by BrainScope from NYU School of Medicine. All other authors were Principal Investigators at the clinical data acquisition sites participating in the Research, whose institutions received contracts to support subject recruitment, consenting and data acquisition and have no competing financial interests to disclaim.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The clinical study was funded in part by a contract to BrainScope Company Inc. from the U.S. Navy (Naval Health Research Center), contract #W911QY-14-C-0098. Disclaimer: The views, opinions and/or findings contained in this report are those of the author(s) and should not be construed as an official Department of the Navy position, policy or decision unless so designated by other documentation.

ORCID iD

Arnaud E Jacquin

Notes

References

Faul

Wald

, et al. Traumatic brain injury in the United States: emergency department visits, hospitalizations, and deaths 2002–2006. Atlanta, GA: Centers for Disease Control and Prevention, National Center for Injury Prevention and Control, 2010.

Bryan

Rowhani-Rahbar

Comstock

, et al.; on behalf of the Seattle Sports Concussion Research Collaborative. Sports-and recreation-related concussions in US youth. Pediatrics 2016; 138: e20154635.

Guskiewicz

McLeod

, et al. Knowledge, attitude, and concussion-reporting behaviors among high school athletes: a preliminary study. J Athl Train 2013; 48: 645–653.

Wallace

Covassin

Nogle

, et al. Knowledge of concussion and reporting behaviors in high school athletes with or without access to an athletic trainer. J Athl Train 2017; 52: 228–235.

Chrisman

Quitiquit

Rivara

FP.

Qualitative study of barriers to concussive symptom reporting in high school athletics. J Adolesc Health 2013; 52: 330.e3–335.e3.

Pfister

Hagel

, et al. The incidence of concussion in youth sports: a systematic review and meta-analysis. Br J Sports Med 2016; 50: 292–297.

Marin

Weaver

Yealy

, et al. Trends in visits for traumatic brain injury to emergency departments in the United States. JAMA 2014; 311: 1917–1919.

Elbin

Sufrinko

Schatz

, et al. Removal from play after concussion and recovery time. Pediatrics 2016; 138.

Asken

McCrea

Clugston

, et al. “ Playing through it”: delayed reporting and removal from athletic activity after concussion predicts prolonged recovery. J Athl Train 2016; 51: 329–335.

10.

Anderson

Godfrey

Rosenfeld

, et al. 10 Years outcome from childhood traumatic brain injury. Int J Dev Neurosci 2012; 30: 217–224.

11.

Scholten

Haagsma

Cnossen

, et al. Prevalence of and risk factors for anxiety and depressive disorders after traumatic brain injury. J Neurotrauma 2016; 33: 1969–1994.

12.

McMahon

Hricik

Yue

, et al.; TRACK-TBI Investigators. Symptomatology and functional outcome in mild traumatic brain injury: results from the prospective TRACK-TBI study. J Neurotrauma 2014; 31: 26–33.

13.

Sariaslan

Sharp

D’Onofrio

, et al. Long-Term outcomes associated with traumatic brain injury in childhood and adolescence: a nationwide Swedish cohort study of a wide range of medical and social outcomes. PLOS Med 2016; 13: e1002104.

14.

McCrory

Meeuwisse

Dvořák

, et al. Consensus statement on concussion in sport—the 5th international conference on concussion in sport held in Berlin, October 2016. Br J Sports Med 2017; 51: 838–847.

15.

Harmon

Clugston

Dec

, et al. American medical society for sports medicine position statement on concussion in sport. Br J Sports Med 2019; 53: 213–225.

16.

Broglio

Cantu

Gioia

, et al.; National Athletic Trainer's Association. National athletic trainers' association position statement: management of sport concussion. J Athl Train 2014; 49: 245–265.

17.

Albicini

McKinlay

A review of sideline assessment measures for identifying sports-related concussion. J Concussion 2018; 2: 1–11.

18.

Henry

Elbin

Collins

, et al. Examining recovery trajectories following sport-related concussion using a multi-modal clinical assessment approach. Neurosurgery 2016; 78: 232–241.

19.

Resch

Brown

Schmidt

, et al. The sensitivity and specificity of clinical measures of sport concussion: three tests are better than one. BMJ Open Sport Exerc Med 2016; 2: e000012.

20.

Patricios

Fuller

Ellenbogen

, et al. “What are the critical elements of sideline screening that can be used to establish the diagnosis of concussion” a systematic review. Br J Sports Med 2017; 51: 888–894.

21.

Sponheim

McGuire

Kang

, et al. Evidence of disrupted functional connectivity in the brain after combat-related blast. Neuroimage 2011; 54: S21–S29.

22.

Slobounov

Zhang

Pennell

, et al. Functional abnormalities in normally appearing athletes following mild traumatic brain injury: a functional MRI study. Exp Brain Res 2010; 202: 341–354.

23.

Thatcher

North

Curtin

, et al. An EEG severity index of traumatic brain injury. J Neuropsych Clin Neurosci 2001; 13.

24.

Virji-Babul

Hilderman

Makan

, et al. Changes in functional brain networks following sport-related concussion in adolescents. J Neurotrauma 2014; 31.

25.

Hanley

Prichep

Bazarian

, et al. Emergency department triage of traumatic head injury using brain electrical activity biomarkers: a multisite prospective observational validation trial. Acad Emerg Med 2017; 24: 617–627.

26.

Hanley

Prichep

Badjatia

, et al. A brain electrical activity (EEG) based biomarker of functional impairment in traumatic head injury: a multisite validation trial. J Neurotrauma 2018; 35: 41–47.

27.

Jacquin

Kanakia

Oberly

, et al. A multimodal biomarker for concussion identification, prognosis and management. Comput Biol Med 2018; 102: 95–103.

28.

Duda

Hart

Stork

DG.

Pattern classification. 2nd ed. Hoboken, NJ: Wiley-Interscience, 2001.

29.

Teasdale

Jennett

Assessment of coma and impaired consciousness. Lancet 1974; 2: 81–83.

30.

www.ncaa.org/sport-science-institute/concussion-diagnosis-and-management-best-practices (accessed 10 March 2021).

31.

SCAT3. Sport Concussion Assessment Tool, 3rd ed. Br J Sports Med 2013; 47(5): 259–262.

32.

McCrea

Kelly

Randolph

, et al. Standardized assessment of concussion (SAC): on-site mental status evaluation of the athlete. J Head Trauma Rehabil 1998; 13: 27–35.

33.

Randolph

Millis

Barr

, et al. Concussion symptom inventory: an empirically derived scale for monitoring resolution of symptoms following sport-related concussion. Arch Clin Neuropsychol 2009; 24: 219–229.

34.

Cooksley

Maguire

Lannin

, et al. Persistent symptoms and activity changes three months after mild traumatic brain injury. Aust Occup Ther J 2018; 65: 168–175.

35.

McCrory

Meeuwisse

Aubry

, et al. Concussion statement on concussion in sport: the 4th international conference on concussion in sport held in Zurich, November 2012. Br J Sports Med 2013; 47: 250–258.

36.

Johnson

Kegel

Collins

MW.

Neuropsychological assessment of sport-related concussion. Clin Sports Med 2011; 30: 73–88.

37.

LaRoche

Pfaller

Lerner

, et al. Prospective, head-to-head study of three computerized neurocognitive assessment tools (CNTs): reliability and validity for assessment of sport-related concussion. J Int Neuropsychol Soc 2016; 22: 24–37.

38.

Vincent

Bailey

Cowan

, et al. Normative data for evaluating mild traumatic brain injury with a handheld neurocognitive assessment tool. Applied Neuropsych Adult 2017; 24: 566–576.

39.

Vincent

Roebuck-Spencer

Fuenzalida

, et al. Test-retest reliability and practice effects for the ANAM general neuropsychological screening battery. Clin Neuropsychol 2018; 32: 479–494.

40.

Arrieux

Cole

Ahrens

AP.

A review of the validity of computerized neurocognitive assessment tools in mild traumatic brain injury assessment. Concussion 2017; 2: CNC31.

41.

Prichep

Jacquin

Filipenko

, et al. Classification of traumatic brain injury severity using informed data reduction in a series of binary classification algorithms. IEEE Trans Neural Syst Rehabil Eng 2012; 20: 806–822.

42.

Kerr

Zuckerman

Wasserman

, et al. Concussion symptoms and return to play time in youth, high school, and college American football athletes. JAMA Pediatr 2016; 170: 647–653.

43.

Bazarian JJ, Elbin RJ, Casa D, et al. Validation of a Machine Learning Brain Electrical Activity–Based Index to Aid in Diagnosing Concussion Among Athletes. JAMA Netw Open. 2021 Feb; 4(2): e2037349. https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2776443

44.

Wilde

Hunsacker

Scarneo

, et al. DTI indicators of white matter injury are correlated with multimodal EEG-based biomarker in slow recovering concussed college athletes. J Neurotrauma 2020. Epub ahead of print. DOI: 10.1089/neu.2018.6365.

45.

Gardner

Kay-Lambkin

Stanwell

, et al. A systematic review of diffusion tensor imaging findings in sports-related concussion. J Neurotrauma 2012; 29: 2521–2538.

46.

Barr

Prichep

Chabot

, et al. Measuring brain electrical activity to track recovery from sport related concussion. Brain Inj 2012; 26: 58–66.

47.

McCrea

Prichep

Powell

, et al. Acute effects and recovery after sport-related concussion: a neurocognitive and quantitative brain electrical activity study. J Head Trauma Rehabil 2010; 25: 283–292.

48.

Prichep

McCrea

Barr

, et al. Time course of clinical and electrophysiological recovery following sport-related concussion. J Head Trauma Rehabil 2013; 28: 266–273.

49.

Teel

Ray

Geronimo

, et al. Residual alterations of brain electrical activity in clinically asymptomatic concussed individuals: an EEG study. Neurophysiology 2014; 125: 703–707.

50.

Brooks

Bazarian

Prichep

, et al. The use of an electrophysiological brain function index in the evaluation of concussed athletes. J Head Trauma Rehabil 2018; 33: 1–6.

51.

Gioia

GA.

Multimodal evaluation and management of children with concussion: using our heads and available evidence. Brain Inj 2015; 29: 195–206.

Concussion assessment potentially aided by use of an objective multimodal concussion index

Abstract

Objective

Methods

Results

Conclusions

Keywords

Introduction

Part 1: Derivation of the concussion index algorithm

Methods

Patient population

Clinical assessments

Neurocognitive performance tests

EEG data acquisition

EEG data processing and quantitative EEG feature extraction

Multimodal classifier development

Statistical analysis

Results

Subject demographics and symptoms prevalence at time of injury

Classifier performance at time of injury

Part 2: Demonstration of CI as an objective tool to aid in assessment of concussion

Statistical analyses

Results

Analysis of longitudinal change in the CI

Correlation between CI and clinical “symptom burden” during recovery (Day 5)

Discussion

Conclusions

Footnotes

Acknowledgments

Declaration of conflicting interests

Funding

ORCID iD

Notes

References