Abstract
Introduction
A multi-domain approach to concussion assessment has been recommended that includes self-reported symptom severity in addition to neurocognitive tests and measures of postural stability. The relationship between subjective self-reported symptoms and objective measures of cognitive function in the post-injury state is not well understood. The aims of the study were to determine symptom severity throughout the post-injury continuum of care and the association between symptom severity and performance on measures of neurocognitive function.
Methods
An observational cohort study was conducted on 1257 high school and collegiate athletes (67% male and 33% female) who had sustained a concussion. Student-athletes were included in the study if they had a healthy baseline assessment and at least one follow-up injury assessment utilizing the Cleveland Clinic Concussion Application (C3 App). Symptom severity was assessed during the acute (0–7 days post-injury), subacute (8–20 days post-injury), and post-concussive (
Results
Median symptom severity during the acute post-injury phase was 10 declining to 2 during the subacute and post-concussive phases. Performance on each of the C3 App modules (SRT, CRT, PST, Trails A, and Trails B) were significantly better in athletes reporting a symptom score of ≤7 compared to those reporting a symptom score >7 at each of the post-injury phases (P < 0.05 on all comparisons).
Conclusions
Symptomatic athletes performed worse on all measures of neurocognitive function, regardless of time from injury. While symptoms alone should not be used to determine recovery, our data indicate that symptom severity may aide in deciding when to initiate post-injury neurocognitive testing to determine readiness for treatment progression.
Introduction
Self-reported symptoms, despite their limitation, are critical in the diagnosis and management of concussion in student-athletes. 1 , 2 Current summary, position, and consensus statements from multiple disciplines all agree that an athlete should be asymptomatic before returning to practice or competition;1,3,4 unfortunately, the operational definition of “asymptomatic” is unclear. 5 The stepwise return-to-play protocol recommended by the International Concussion in Sport group requires athletes to rest until asymptomatic and gradually increase activity level without reproducing concussion symptoms. 1 This guideline does not offer criteria for “asymptomatic,” leaving the team of healthcare providers to potentially interpret the athlete’s self-reported symptoms differently in a clinical context. 5 , 6 A number of symptom scales and checklists are currently in use for concussion management, and typically entail a broad range of concussion-related symptoms across a number of relevant domains: physical (e.g. headache, dizziness), cognitive (e.g. feeling in a fog), affective (e.g. feeling sad), while some include sleep (e.g. sleeping more or less than usual). 1 , 2 , 7 Previous data indicate that young athletes may experience a number of these symptoms, either at rest or with exertion, even in the absence of a concussion or history of concussion. 5 , 6 One review of 13 studies using various symptom check lists found a symptom severity range of 3.5–10 among healthy, non-concussed athletes. 5 In our own experiences using a 27-item graded symptom checklist, a mean symptom severity score of 7.5 was observed during healthy baseline testing. Despite being injury free, these individuals were not technically asymptomatic. Conversely, a number of studies have reported that some athletes intentionally under-report symptoms during the post-concussion evaluation process to expedite clearance for activity.8–10 The subjective nature of symptom scores, their presence in healthy athletes, and evidence of symptom underreporting complicate the appropriate weighting of symptom checklists in the clinical management of concussion.
Acknowledging the limitation of a symptom driven approach to concussion management, the Concussion in Sport group recommends a multifaceted, multimodal, and multidisciplinary approach to concussion management. 1 The 2016 consensus statement recognizes neurocognitive testing as a critical component of this comprehensive evaluation. 1 Modern neurocognitive testing paradigms typically involve computerized versions of traditional “paper-pencil” assessments that evaluate cognitive domains known to be impaired following concussive injury (e.g. information processing, processing speed, attention, and memory). The Cleveland Clinic Concussion application (C3 app) was developed to provide a portable, comprehensive assessment platform to incorporate documentation of the injury and recovery process, a graded symptom checklist, quantitative balance assessment, and neurocognitive evaluation.11–15 The C3 app is integral to the Cleveland Clinic Concussion Carepath, a clinical workflow that has standardized the interdisciplinary approach to concussion management and been shown to reduce cost of concussion care. 11 , 15
Cognitive recovery after concussive injury typically overlaps with the time course of symptom attenuation, although in some cases one lags behind the other. 1 , 16 , 17 The specific relationship between normalization of cognitive function and return to an asymptomatic baseline state throughout the course of recovery has not been well characterized. Importantly, recognizing disparities between an athlete’s post-concussion cognitive outcomes and symptom severity may offer an opportunity to identify individuals who will likely not follow a traditional recovery trajectory and would benefit from additional and early intervention. The primary aims of this study were to determine symptom severity throughout the post-injury continuum of care and the association between symptom severity and performance on measures of neurocognitive function. We hypothesized that symptomatic patients would demonstrate worse performance on post-injury measures of neurocognitive function.
Methods
Study design, setting, and participants
An observational cohort study was conducted in accordance with STROBE guidelines. 18 High school and collegiate student-athletes (n = 1257), ages 13–25, who met the following criteria were included in the study: 1) healthy baseline C3 app assessment; 2) concussion diagnosed by Cleveland Clinic physician or athletic trainer in accordance with published guidelines 1 and documented via C3 incident report; and 3) at least one post-injury C3 app assessment. Baseline C3 assessments were administered by athletic trainers trained in proper app administration prior to the commencement of competitive sport seasons. Follow-up assessments were administered within the school environment or in outpatient concussion clinics by an athletic trainer. All baseline and follow-up assessments were administered between June, 2013 and January, 2018. This study was approved by the Cleveland Clinic Institutional Review Board with a waiver of consent as the testing was a part of standard clinical practice outlined by the Concussion Carepath implemented in 2012. 11 , 15
Materials and data collection
Forty-six athletic trainers employed by the Cleveland Clinic in high schools and colleges across Northeast Ohio collected baseline and post-injury C3 app data as part of standard clinic practice from June, 2013 through January, 2018. The development, validation, and rationale for use of the C3 app and its components have been described in detail previously. 11 , 12 , 19 Briefly, the C3 app was developed to provide a portable platform for the collection and documentation of concussion injury-related demographics and neurologic function. The app employs a multi-domain approach to concussion assessment and includes a 27-item self-reported graded symptom checklist (GSC) using a 7-point Likert scale and assessment modules providing the following objective measures of neurologic function: 1) biomechanical measure of postural sway during performance of the Balance Error Scoring System, 2) information processing via simple (SRT) and choice reaction time (CRT) paradigms, 20 3) processing speed test (PST), 21 , 22 4) executive function and set switching using a digitized Trail Making Test A (TMT A) and B (TMT B), and 5) static and dynamic visual acuity. For the current study, self-reported GSC outcomes and the three neurocognitive modules, each described briefly below, were included in the analysis. All participants underwent baseline testing prior to the athletes’ competitive season as a standard component of Cleveland Clinic’s concussion management program. For multiple-sport athletes, baseline testing was completed once per academic calendar year. Post-injury assessments were administered at the clinical determination of the AT and/or physician and included any combination of C3 app modules.
Graded symptom checklist
Self-reported symptom severity was measured using a 27-item GSC. The GSC was adapted from the 22-item SCAT2 23 and uses a 7-point Likert scale. Questions added to the 22-item GSC adapted from the SCAT2 include three questions about sleep (sleeping more than usual, less than usual, or difficulty sleeping soundly) and two regarding neurologic symptoms (numbness/tingling and ringing in the ears). The highest possible total symptom severity score is 162. A full list of the symptoms include in the GSC is provided in Table 1, along with a proposed categorization of symptoms into four clusters: physical, cognitive, affective, and sleep.
Symptoms included in the C3 app 27-item graded symptom checklist categorized according to symptom clusters.
Simple and choice reaction time
Information processing is classically measured via reaction time paradigms, and while they are recommended in concussion assessment, there is no single test established as the “gold standard”. The SRT and CRT modules developed for the C3 app were previously validated with the Visual Choice Reaction Time Apparatus, (Model 63035 A; Lafayette Instrument Life Sciences, Lafayette, IN) and found to have excellent test-retest reliability. 20 The SRT module requires the individual to place the index finger onto a button on the iPad screen, and upon appearance of a green stimulus light, release the button and tap the stimulus as quickly as possible. 20 The interval between the appearance of the stimulus light and removal of the index finger is the measure of SRT, while the interval from removal to tapping the target is recorded as movement time. In the CRT module, participants place one index finger on each of two initiation response buttons. Two stimulus lights appear, a targeted (green) and distractor light (cyan). The participant is instructed to react only on the side of the green light to tap the stimulus as quickly as possible while inhibiting movement of the contralateral index finger. The green and cyan colors were chosen to avoid a color-blind pair between the two stimuli. For both SRT and CRT, 25 valid trials are collected and the average reaction time from trials 3–25 is recorded in milliseconds, with trials 1–2 considered practice. Reaction time is defined as the time from stimulus presentation to finger lift-off. A motor time is also computed, defined as the time from lift-off to tapping the stimulus.
Trail Making Tests A and B
The TMT-A and TMT-B are digitized versions of standard neuropsychological tests that assess set-switching and working memory, and have been shown to be sensitive in determining recovery from mTBI. 24 , 25 The TMT-A requires individuals to connect a series of numbered dots in ascending order 1–25, and TMT-B requires the connection of letters and numbers in sequential but alternating order (1-A-2-B-3-C…). In the C3 app, TMT-A and TMT-B are administered on the iPad with a stylus, and the main outcome, time to complete, is automatically computed.
Processing speed test
The PST module is an electronic adaptation of the traditional paper-pencil Symbol Digit Modalities Test (SDMT), is highly correlated with the SDMT, and has been shown to have excellent test-retest reliability. 21 , 22 This test evaluates visual memory, visual speed, and set substitution. In the C3 app, participants use a symbol-digit code presented on the iPad screen to identify the digit that matches a given symbol. The primary outcome is the number of correct symbol-digit associations completed within a 2-minute trial.
Statistical analysis
Descriptive statistics were computed to describe participant demographics. To determine symptom severity throughout the post-injury continuum of care, assessments were categorized according to time from injury: Acute (0–7 days post-injury); subacute (8–20 days post-injury) or post-concussive (>20 days post-injury) as per the concussion care pathway. 15 To determine the association between symptom severity and performance on measures of neurocognitive function, cognitive test performance for each module (SRT, CRT, PST, TMT-A, and TMT-B) was stratified according to symptom severity: Asymptomatic (0–7 on the GSC) or Symptomatic (>7) at each post-injury time period (acute, subacute, and post-concussive). The rationale for a cut-off of 7 was due to evidence that during baseline testing, healthy student-athletes (n = 592) reported the presence of symptoms such as headache or fatigue, with an average of total symptom severity score of 7.5. Next, separate linear mixed effect models compared within-phase differences for each module. Models included a random component for each athlete, as well as a random component for each time phase nested within each athlete. The main effects and interaction of the symptom score group along with the time phase group were modeled as fixed effects for each test module. Comparisons within-time phase were completed by testing contrasts of least square means. Significance was set at p < 0.05 and adjusted for multiple comparisons. Statistical analyses were performed using R software version 3.4.0 (https://cran.r-project.org).
Results
Data from 1257 student-athletes who had baseline C3 assessments, a diagnosed concussion, and at least one follow-up C3 assessment were included in the analysis. Demographic variables describing the cohort and injury-related details are provided in Table 2. The number of assessments completed in each post-injury phase is reported in Table 3 while summary data for all modules during all post-injury phases are reported in Table 4.
Athlete demographics.
Number of assessments completed in each post-injury time period (N = 1257 athletes).
Determining symptom recovery across the injury spectrum
As shown in Figure 1, student-athletes endorsed concussion-related symptoms as measured by the self-reported GSC, but scores declined sharply following the initial 0–7 days post-injury. While the GSC was not administered to all athletes every day, the median GSC total score for athletes who were assessed on post-injury day one was 20, declining to 13, 11, 8, and 5, on post-injury days two, three, four and five, respectively. Mean GSC scores were higher for each post-injury phase, indicating that the data were skewed, with a fewer percentage of student-athletes endorsing a relatively high symptom severity compared to the majority of their peers.
Total symptom severity as measured by the 27-item Graded Symptom Checklist for all tests administered through seven weeks post-injury is depicted in (a) along with the mean (red line) and median (blue line) scores for a given day. The total possible symptom severity score is 162. The mean and median symptom severity scores for each injury phase are depicted in (b).
Declines in post-injury information processing
At baseline, mean SRT performance was 286 msec. As depicted in Figure 2(a), post-injury SRT performance significantly worsened for both symptomatic and asymptomatic cohorts. However, symptomatic student-athletes continued to perform significantly worse than asymptomatic counterparts at all time periods (acute, phase, subacute phase, and post-concussive phases), (p < 0.05). In the more complex CRT paradigm, baseline reaction time was 412 msec. Post-injury performance as depicted in Figure 2(b) was significantly worse for the symptomatic (p < 0.05) but not the asymptomatic cohort, with mean CRT values of 458, 462, and 440 msec at the acute, subacute, and post-concussive time periods, respectively.
Performance on the SRT (a), CRT ( b), TMT-A (c), TMT-B (d) and PST (e) at baseline (gray) and at the acute, subacute, and post-concussive time phases. Post-injury performance was stratified into two cohorts based on symptom severity (asymptomatic (≤7 on GSC) and symptomatic (>7 on the GSC)), and was significantly different between the cohorts at all post-injury time phases.
Symptomatic athletes perform worse on trail making test at all time periods post-injury
At baseline, healthy student athletes completed the TMT-A and TMT-B in 21.7 and 45.0 seconds, respectively. As shown in Figure 2(c), during the acute and subacute post-injury time periods, the symptomatic cohort demonstrated worse performance on the TMT-A, with mean times of 23.6 and 23.0 seconds, respectively, and improved performance in the post-concussive phase to 20.5 seconds. However, the asymptomatic cohort demonstrated improved performance during all post-injury time periods. Performance was significantly better at all time periods for asymptomatic student athletes compared to symptomatic ones (p < 0.05). Post-injury performance on the TMT-B was better than baseline performance for both cohorts at all time periods, as depicted in Figure 2(d). However, asymptomatic student athletes performed significantly better than their symptomatic peers at all post-injury time periods (p < 0.05).
Symptomatic athletes perform worse on measures of processing speed at all time periods post-injury
Student athletes achieved a mean of 60.4 correct responses on the PST at baseline. Improved performance relative to baseline was observed for both the symptomatic and asymptomatic cohorts at all post-injury time periods, as depicted in Figure 2(e). However, asymptomatic athletes performed significantly better than their symptomatic peers at during all post-injury time periods (p < 0.05).
Cognitive cluster symptoms are most prevalent among symptomatic athletes
An exploratory analysis was undertaken to categorize symptoms into four clusters: physical, cognitive, affective, and sleep, as shown in Table 1. Symptoms related to the cognitive domain were most prevalent among the symptomatic cohort, followed by the physical domain and the sleep domain. Symptoms in the affective cluster were least prevalent among symptomatic athletes. Relationships between symptom cluster scores were evaluated revealing the strongest relationship between the physical and cognitive cluster (r = 0.839, p = 0.01) followed by the sleep and cognitive cluster (r = 0.765, p = 0.01). The weakest correlation (though still moderately strong) was between the affective and physical clusters (r = 0.645, p = 0.01). While this analysis was exploratory in nature, the clustering of symptoms may aide clinical decision-making by identifying deficits relative to domains of function, and implementing rehabilitation strategies and interventions such as cognitive rehabilitation, physical therapy, sleep hygiene, or counseling.
Baseline and post-injury performance (mean (SD)) during each time period stratified by symptom severity.
Discussion
A multi-domain approach to sport concussion assessment is the recommended standard of care, with objective measures of neurocognitive and motor function used to complement self-reported symptom severity and the clinical exam to determine injury status, degree of recovery, and readiness for return to school and sport. 1 , 3 , 26 While self-reported symptom severity has long been a part of the assessment protocol, its role has not been well-understood, in part due to evidence that symptoms similar to those associated with a concussion are present in healthy, non-concussed athletes and the general subjectivity of these self-reported estimates. 2 , 8 , 27 The current study aimed to determine symptom severity throughout the post-injury continuum of care and the association between symptom severity and performance on measures of neurocognitive function.
As expected, self-reported symptom severity declined steadily in the first week post-injury, with an initial median symptom severity score of 20 on day 1 among those who were tested, decreasing to a median score of 5 on day 5 post-injury. While median scores declined sharply throughout the first week, mean scores remained higher across all post-injury phases, indicating that a minority of student-athletes (∼10–20%) remained highly symptomatic compared to the majority of their peers. While not all student-athletes were tested on a daily basis, 50% of those tested remained symptomatic (GSC score > 7) until post-injury day 9.
Previous studies and our own data during healthy baseline testing indicate that high school and college student athletes often experience low levels of symptoms that are measured by the GSC including headaches, sleep disturbance, difficulty concentrating, fatigue, and irritability in the absence of injury. 5 , 6 Alla and colleagues reported that this presence of concussion-like symptoms during the healthy state makes it challenging to determine when a concussed athlete is asymptomatic, and clinically ready to be progressed through a return to play program. 5 While a specific cut-off score to define the “asymptomatic” state has not been precisely determined, our data from healthy baseline testing found that an average, student-athletes endorsed a symptom severity score of 7.5 on the 27-item GSC used in the C3 app. This score falls within the range of symptom severity scores reported by others 5 and provided rationale to use a cut-off score of ≤7 when determining symptom recovery in our cohort of concussed athletes, and in stratifying the injured athletes when analyzing neurocognitive test performance.
In recent years, a paradigm shift has occurred in clinical care, with follow-up neurocognitive testing reserved until a student-athlete has recovered sufficiently for the testing to inform clinical decisions rather than simply confirming the injury state. This change in practice has prevented unnecessary over-testing during the acute post-injury phase, which often provoked worsening of symptoms while simultaneously weakening the psychometric value of the tests. Therefore, the impact of symptom severity on cognitive performance was investigated as symptom severity scores may inform clinicians regarding when to initiate post-injury neurocognitive testing. In our cohort of 1257 injured student-athletes, the cut-off GSC score of ≤7 effectively dichotomized student-athletes with respect to their post-injury neurocognitive test performance. The symptomatic cohort of injured student-athletes performed significantly worse on all measures of neurocognitive function post-injury compared to their asymptomatic peers, as depicted in Figure 2. While previous studies have demonstrated global deficits in post-injury measures of neurocognitive function, none to our knowledge have examined the differential effects of symptom severity on test performance. Information processing as measured by a simple reaction time paradigm worsened for all participants post-injury compared to baseline performance. However, the symptomatic cohort exhibited significantly worse times compared to their asymptomatic counterparts at all post-injury time periods. In contrast, performance on the choice reaction time paradigm was worse only for the symptomatic cohort, while the asymptomatic student-athletes performed comparably to baseline levels. Test performance for SRT and CRT at baseline were similar to those reported by Bernstein and colleagues using the same testing paradigm, and post-injury times were comparable with the symptomatic group in the current study. 28
Visual scanning and cognitive flexibility were measured using the TMT-A and TMT-B demonstrating differential effects, with the symptomatic group performing significantly worse at all post-injury time phases. Interestingly, asymptomatic student athletes demonstrated improved performance relative to baseline for both the TMT-A and TMT-B during all post-injury phases. These data create some uncertainty regarding the utility of baseline testing for the TMT-A and TMT-B tests, but provide valuable information to aide in the clinical interpretation of post-injury testing to determine degree of recovery. Our results are not dissimilar to other studies with the TMT-A and TMT-B, as a high degree of baseline and post-injury variability has been reported. 24 , 25 , 29
Performance on the PST was similar to the TMT-B, with post-injury performance better overall compared to baseline values. Nonetheless, these data provide considerable value in the interpretation of post-injury data, regardless of whether baseline testing is available by which to compare post-injury performance. Our post-injury results for the symptomatic cohort were similar to those observed by Bernstein and colleagues, though higher baseline values were reported in their exploratory study. 28 Once again, a significant difference was observed in our analysis when comparing the symptomatic athletes to their asymptomatic peers at all post-injury time phases. While symptoms alone should not be used to determine recovery, our data indicate that symptom severity may aide in deciding when to initiate post-injury neurocognitive testing to determine readiness for treatment progression.
Limitations and future directions
This was a pragmatic observational study and did not include assessments at pre-determined time points. Additionally, a healthy cohort was not included by which to compare longitudinal neurocognitive test performance. Studies are planned using standardized and systematic post-injury assessment time points longitudinally for both symptomatic and asymptomatic student athletes. Furthermore, there is a need for acute prospective longitudinal studies that include a control cohort with blinded analyses of the outcome measures. Lastly, additional in-depth analyses are under way to build models that predict temporal aspect of recovery based on initial presentation of symptoms.
Footnotes
Declaration of conflicting interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Jay Alberts and Susan Linder have filed intellectual property related to the incident report mobile application used in this study.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was supported the Edward F. and Barbara A. Bell Family Endowed Chair to JLA.
