Abstract
The Sports Concussion Assessment Tool 3rd version is a sports screening tool that is often used to support return to play decisions following a head injury. The Sports Concussion Assessment Tool 3rd version is presumed to identify brain dysfunction (implying a degree of brain injury); however, the Sports Concussion Assessment Tool has never been validated with patients with definite acute brain injury. In this study, we found that all three Sports Concussion Assessment Tool 3rd version domains – symptoms, cognitive and balance assessments – were sensitive in discriminating traumatic brain injury patients (all with abnormal acute neuroimaging) from healthy controls. Through a correlation matrix (Bonferroni corrected), we found no correlation between the subjective (symptoms) and objective (examination) Sports Concussion Assessment Tool 3rd version assessments, e.g. complaints of imbalance and memory dysfunction were not correlated, respectively, with performance on testing balance and memory function. When relaxing the correction for multiple comparisons we found that of all Sports Concussion Assessment Tool 3rd version symptoms, a feeling of ‘pressure in the head’ had the largest number of co-correlations (including affective symptoms) and overwhelmingly in a pattern indicative of migraine. Taken together, that objective and subjective assessments in the Sports Concussion Assessment Tool 3rd version are poorly correlated, could suggest that symptoms in the Sports Concussion Assessment Tool 3rd version poorly reflect brain injury but rather indicate non-brain injury processes such as migraine. It follows that the current prominent orthodoxy of resting athletes following a head injury until their symptoms settle for fear of exacerbating brain injury may be unfavourable for their recovery – at least in some cases. Prospective clinical studies would be required to assess patient recovery from concussion with early active investigation and treatment versus rest – a notion supported by recent international consensus.
Keywords
Introduction
Concussion – from the Latin ‘con cutere’, meaning to shake together – can be taken in a literal sense to mean an event where there is an accelerative force to the head. Hence a post-concussion syndrome can be used as an umbrella term to indicate a clinical syndrome of symptoms and signs that result from a concussion. The latest definition of a sports concussion considers that it is a ‘traumatic brain injury induced by biomedical forces’; 1 however, this would not preclude additional diagnoses existing alongside the brain injury such as inner ear disorders. This thus means that a post-concussion syndrome could incorporate any number of specific diagnoses – from brain injury with or without non-brain injury diagnoses, including those affecting the inner ear mechanism to migraine triggered from injury to superficial structures in the head and neck.
In the sporting context, concussion is used pseudo-diagnostically to indicate the observed transient impairment of brain function following a blow to the head, which we term a ‘sports-related concussion’.2,3 In professional sport, the close monitoring of players enables the identification of most head impacts that involve a transient mental obtundation. Despite the recent recognition of sports-related concussion as a form of traumatic brain injury (TBI), 1 the full nature and degree of any associated neuropathological change is not fully understood and requires further definitive prospective studies.
International sporting consensus has developed the Sports Concussion Assessment Tool2–4 (SCAT – see Appendix 1 for weblink), which consists of cognitive tests, balance testing and symptom scores. The 3rd edition of the SCAT developed in 2012 (now replaced by the SCAT5 5 ) is used as a tool for evaluating injured athletes for concussion, both on the sideline as well as for subsequent monitoring of their recovery. Despite the poor test–retest reliability of the SCAT 6 (lessening its utility as a monitoring tool for recovery), an important concern on first assessment is the identification of brain dysfunction – alluding to some degree of brain injury 7 – which it is presumed the SCAT can detect. Our main question was to assess whether the SCAT could distinguish between healthy controls and patients with definite acute TBI.
A secondary, but related question is the relationship between symptoms and definite signs of brain injury, i.e. do persisting symptoms relate to persisting brain injury? This is particularly relevant for patients presenting to the clinic in whom the documentation of the details of the head injury is sparse – and in whom use of the term post-concussion syndrome as we define it does not prejudice the investigation towards or away from mild TBI. Certainly, the current sports consensus guidelines consider symptoms an important tool for monitoring recovery since: (i) the data for objective tools for monitoring the recovery of brain functioning are few and (ii) worsening symptoms – irrespective of their origin (peripheral or central) – impact upon sporting performance. We therefore assessed whether symptoms and signs of abnormal brain function were correlated in patients with definite brain injury. If so then in patients with definite brain injury, we should expect a correlation between related subjective complaints and objective measures, e.g. complaints of imbalance or memory disturbance should correlate, respectively, with objective measures of balance and cognition.
If the SCAT can discriminate between brain-injured patients and matched controls, then a further question that warrants investigation is whether all components of the SCAT are equally sensitive to brain injury. According to Chin et al., 6 all components of the Sports Concussion Assessment Tool 3rd version (SCAT3) show a positive correlation for test–retest reliability in non-injured controls, with the maximum value of Pearson’s r = 0.66 for the Balance Error Scoring System (BESS), reflecting testing stability between a 24 h and 8-day assessment. However, this reliability declined at a 196-day interval. 6 Nonetheless, individual components of the SCAT3 do also demonstrate good test–retest reliability in other studies. 8
Cognition is frequently used to assess TBI with studies demonstrating cognitive and behavioural impairments correlating with the trauma severity.9,10 Thus, it would be expected that the SCAT3 cognitive domain should differentiate patients from controls. There have also been reports suggesting that balance impairment is particularly common in concussion whereas cognition is less affected.11–13 It follows that finding a similar pattern in TBI – that balance is disproportionately affected – could provide additional parallels between concussion and TBI.
Another potential additional benefit of the current study would be to provide the evidence base for the use of a simple clinical test that could help in the assessment of acute TBI in a hospital setting, potentially helping with clinical decision making such as triaging admissions, which would be of great clinical utility since there is a dearth of tools sensitive to deficits in TBI.14,15
We thus prospectively assessed acute traumatic brain injury (TBI) patients using the SCAT3 as a marker of acute TBI. The patients, who all had brain imaging abnormalities, were recruited and tested on a Major Trauma Ward. To the best of our knowledge, the SCAT3 has never been employed in acute TBI patients with clinically definite brain injury confirmed with brain imaging.
Methods
Participants
All patients admitted to the Major Trauma Ward at St Mary’s Hospital, London, Paddington, are assessed by the therapy team for signs or symptoms of balance and/or dizziness, and are referred for in-patient review to the senior author (BMS). Following assessment by BMS, consecutive patients were recruited following written informed consent for all procedures as approved by the local research ethics committee. Assessments were conducted on the trauma ward, between 2 and 14 days from the trauma. Inclusion criteria were adult patients (>18 years) able to provide informed consent, no injuries to the limbs affecting balance and gait, preserved peripheral vestibular function (assessed by the head impulse test and absence of a vestibular nystagmus both with and without visual fixation) and independently living in the community without pre-existing gait or balance problems pre-admission. Exclusion criteria were patients not able to fulfil inclusion criteria. Since we were interested in the impact of brain dysfunction upon the SCAT, it was important that imbalance from peripheral vestibular disorders affecting TBI patients was either treated prior to testing (e.g. benign paroxysmal positional vertigo (BPPV)) or excluded (e.g. unilateral peripheral vestibular loss typically due to a unilateral fracture of the petrous temporal bone). Peripheral vestibular loss is easily determined by clinical examination by assessing for signs such as the head impulse test or spontaneous unidirectional nystagmus whose intensity is modulated by visual fixation.
Healthy individuals were recruited by word of mouth and written informed consent was obtained as for the patient cohort. Healthy volunteers had no previous history of head trauma or balance problems.
Sport Concussion Assessment Tool 3rd Edition
The Sport Concussion Assessment Tool 3rd edition (for official version of the tool, see Appendix 1) is a multifaceted concussion evaluation designed to assist practitioners when making return-to-play decisions for athletes aged 13 and over who may have suffered a concussion.3,4 The revised 5th edition of the SCAT
5
was developed after the time of data collection and is consequently not included in this study.
Symptom evaluation assessed a total of 22 possible symptoms that were graduated in severity, with 0 indicating no symptom, 1–2 indicating mild symptoms, 3–4 moderate symptoms and 5–6 severe symptoms. Symptom score was calculated from the total number of possible symptoms (maximum of 22), and symptom severity was calculated as the sum of the graded responses (maximum of 132).
2
The SAC provides an objective measure of immediately assessing an injured athletes’ mental status on the sport sideline.
16
The SAC is a neurocognitive test that can be conducted by an individual with no prior experience in psychometric testing
17
and contains questions designed to assess four cognitive domains: orientation to time, immediate, memory, concentration and delayed memory, resulting in a score total of 30 (total cognitive score). The responses to each assessment are dichotomous, one point is scored for each correct answer, and zero is scored for each incorrect answer.
2
The BESS is a clinical assessment of balance (Figure 1). The test within SCAT3 is a modified version of the original BESS, as the assessment does not include repeating stances on medium density foam.
18
Patients were instructed to maintain the upright posture for 20 s for each stance positions, which are performed with closed eyes on firm surfaces. These stances include feet together (double-leg), on the non-dominant foot (single-leg) and with the dominant foot directly in front of the non-dominant foot (tandem).2,19 Patients self-reported foot dominance by identifying the preferred foot to kick a football and each assessment was performed with hands on their hips.
The BESS. The BESS is tested using three ways of standing, with each stance tested for 20 s. The double leg stance (a) is carried out with both feet together. The single leg stance (b) involves balancing on the non-dominant foot with approximately 30° of hip flexion and 45° of knee flexion. The tandem stance (c) requires the non-dominant foot to be placed in front of the dominant foot so that the heel is touching the toe. All stances are performed with hands on hips and eyes closed. Errors are then recorded during the 20 s trial.
Each of the trials is scored by counting the errors or deviations from the proper stance; errors or deviations include hands lifted off iliac crest, opening eyes, stepping, stumbling or falling, moving hips into >30° abduction, lifting forefoot or heel, and a maximum score of 10 is awarded if the patient remains out of the test position for >5 s, each error recorded scores one point. If the patient commits multiple errors simultaneously, only one error is recorded. Should a patient lose their balance they were instructed to return to the original stance as quickly as possible. 2
A total BESS score is also calculated, as a sum of the three different components.
Statistics and analysis
Non-parametric statistics (Mann–Whitney U test) were used to test the differences between patients and controls. To quantify the magnitude of the effects we report, we provide η 2 values for significant U tests. Linear correlation was used to assess the within-group association between different SCAT parameters, for TBI patients. Both Mann–Whitney and linear correlational analyses were corrected for multiple comparisons. A table with the correlation matrix can be found in the supplemental materials.
Results
Patient demographics
The age range of healthy volunteers was 18–68, with an average age of 33 years ± 18.05, and a male:female ratio of 9:9 (n = 18). Table 1 shows the demographics of the 11 patients tested. Patient age range was 18–89 (average 51 ± 24 years), and a 6:5 male:female ratio of participants (n = 11). Patients were tested an average of 7.0 ± 3.33 days after injury. Following the traumatic event, patients experienced post-traumatic amnesia, loss of consciousness, and were admitted with an average Glasgow Coma Score of 14 (median 14, range 11–15). The causes of TBI were as follows: falls (50%), motor accidents (42%), assault (8%). All patients were classified as having had a moderate to severe TBI according to the Mayo criteria. 20
Patient demographics.
ASDH: acute subdural haemorrhage; GCS: Glasgow Coma Score; LOC: loss of conscious; PTA: traumatic amnesia; R/L: right/left; RTA: road traffic accident; SAH: subarachnoid haemorrhage.
Symptom evaluation
Figure 2 shows that TBI patients had both more symptoms (p < 0.05, η2 = 0.30) and a greater severity of symptoms (p < 0.001, η2 = 0.58).
Symptom scores. This shows the median number of symptoms (left panel) and median symptom severity (right panel) for controls versus patients. Error bars represent the standard error of the medians. *p < 0.05, **p < 0.01, ***p < 0.001.
We then calculated a measure of group symptom burden by obtaining the summed symptom severity scores of the group expressed as a per cent of the maximum possible symptom severity score, i.e. a symptom burden of 100% would mean that all subjects in the group had the maximum severity for that symptom (Figure 3). This group symptom burden as calculated is relevant for healthcare utilization since a very common symptom of medium severity would likely lead to overall more hours of healthcare utilization than a rare condition with severe symptoms. Figure 3 shows that the top six most burdensome symptoms for the group included fatigue (first), vestibular symptoms (imbalance – second; dizziness – sixth) and pain (neck pain – third; headache – fifth).
Symptom burden. Comparing symptom ‘burden’ between TBI patients and controls.
Objective assessment of cognition and balance performance
Standardized Assessment of Concussion (cognitive assessment)
Results of the standardized Assessment of Concussion – essentially a cognitive assessment – showed (Figure 4) that the overall test discriminated between patients and controls. All subsections were highly discriminatory between the groups (immediate recall: p < 0.01, η2 = 0.34; concentration: p < 0.001, η2 = 0.44; delayed recall: p < 0.01, η2 = 0.31; total score: p < 0.001, η2 = 0.50), except for the orientation question, which did not discriminate between patients and controls.
The Standardized Assessment of Concussion. Displayed are the median results for the four different cognitive domains tested: orientation, immediate memory, concentration and delayed recall. The total cognitive score is calculated as a sum of the four different components.
BESS
The BESS also discriminated between patients and controls (Figure 5). There were significant differences between the scores for the patient and control groups in the single leg stance (p < 0.001, η2 = 0.43), tandem stance (p < 0.05, η2 = 0.19) and the overall balance test score (p < 0.001, η2 = 0.49).
The BESS. Medians of the patients and controls for the three components and the total BESS score for all components. BESS: Balance Error Scoring System.
Linking symptoms and objective performance in brain injury
To assess the correspondence between symptoms and objective performance we cross-correlated all measures of the SCAT3 in the patient group and Bonferroni corrected for multiple comparisons.
Despite the total cognitive score being a sensitive discriminator of brain injury (Figure 4), we found no significant correlation (all p-values > 0.05) between the total cognitive score and any cognitive symptom across the patient group (and the lack of significance remained even without Bonferroni correction). The finding was the same even when individual components of the cognitive assessment (e.g. immediate and delayed recall) were analysed. Notably, cognitive complaints in the symptom evaluation component of SCAT3 – e.g. ‘difficulty remembering’ – scored relatively lowly in overall group symptom burden (Figure 3) despite the sensitivity of cognitive testing in identifying brain-injured patients. It follows that patients with acute brain injury have a low load of cognitive symptoms but a high load of objective cognitive dysfunction.
In contrast to memory symptoms, balance dysfunction was one of the most burdensome of complaints in the group of brain-injured patients (Figure 3). Surprisingly however, we found no correlation between any objective balance testing score (either balance component or total balance score) with either ‘dizziness’ or ‘balance problems’, and this held true even when we assessed these correlations without correcting for multiple comparisons. Thus, the pattern for balance in brain injury mirrors that for cognition, i.e. subjective symptoms bear no relationship to the objective assessment of brain dysfunction from acute TBI.
Discussion
To our knowledge, this is the first study to prospectively assess the SCAT3 in acute TBI inpatients with definite abnormalities on brain imaging (‘scan positive’ TBI). Our main finding was that the SCAT3 is highly sensitive in discriminating between acute TBI patients – on average seven days post TBI – and healthy matched controls with balance, cognitive assessments, number of symptom scores all showing high sensitivity in detecting TBI. The second main finding was that symptoms and signs, as elicited by the SCAT3, did not correlate in acute TBI patients.
The utility of the SCAT3 in acute ‘scan-positive’ TBI
A key question is whether there is redundancy in the SCAT3, i.e. could one of the subcomponents be particularly sensitive to brain injury and hence act as a rapid screening tool rather than assessing the entire SCAT3. The use of the SCAT to screen military personnel members for acute head trauma showed that the BESS was more affected than other SCAT components, 21 and in professional Rugby Union players, the BESS was the component that took the longest to resolve. 22 Our data found that both the balance and cognitive SCAT3 assessments were very sensitive in identifying acute TBI. It may be that in our study, subtle differences in the sensitivities of the various SCAT3 components in detecting brain dysfunction were submerged by the severity of the brain injury in our subjects as compared to typical concussed athletes.
An important finding was the lack of correlation between objective tests that could indicate brain dysfunction and symptoms of brain dysfunction. The incongruence between balance test results and patients’ symptoms has long been documented in head-injured patients23,24 and has been shown also in Professional Rugby Union players 22 but such a finding has not been previously documented in acute brain-injured patients. One explanation is that acute patients lose some awareness of the extent of their deficits. This is clinically relevant since TBI patients may be at risk of falls and further injury despite being relatively uncomplaining. We also recently showed that TBI patients with chronic symptoms have multiple balance diagnoses in the same patient, typically combining peripheral and central deficits. 25 Taken together, we recommend that when assessing balance dysfunction in head-injured patients, clinicians should place less reliance upon symptoms and should screen patients using more objective tests such as balance deficits. Similarly, cognitive screening should be considered even in the absence of cognitive symptoms. Thus, objective assessment (e.g. balance and memory testing) is likely to be a more valid marker of the mild TBI sustained in a concussion, rather than symptoms.
Surprisingly, the most burdensome symptom was fatigue which correlated strongly with sleepiness. TBI can involve brain structures important for sleep regulation 26 and provoke nocturnal insomnia and hence produce daytime somnolence. Importantly, sleep–wake dysfunction can aggravate migraine.
Implications for diagnosis and therapy
In terms of therapy, we recently showed the presence of multiple diagnoses affecting balance in individual patients with chronic vestibular symptoms post-TBI (with positive neuroimaging). 25 Indeed, we previously showed that in TBI, patients typically have multiple diagnoses often involving peripheral and central structures in the same patient. 25 Given the combination of multiple diagnoses, and the potential lack of subjective-objective correspondence, we would recommend – at least for patients with definite TBI – a comprehensive vestibular screening irrespective of vestibular symptom load. Using such an approach, we could obtain complete remission of symptoms in 80% of patients with chronic vestibular symptoms. 25 Whether a similar picture is seen in patients with concussion and normal brain imaging requires further study.
Future questions for sports concussion research
Our data pose several questions for ‘sports concussion’ and the post-concussion syndrome. First, since the BESS is both a quick and sensitive marker of brain dysfunction, should this be the first test to be used in screening athletes on the sideline? Second, how useful are symptom scores in screening for brain dysfunction? It may be more appropriate that symptoms (e.g. of migraine, anxiety, insomnia) be assessed in a specialist clinic where effective treatment for specific diagnoses can be given. Third, since symptoms and signs of brain injury are not strongly correlated, what is the evidence that imposing rest in symptomatic athletes – without an active exploration of underlying diagnoses – speeds recovery? Recent guidelines do in fact suggest active rehabilitation may be useful in some cases. 1 Conversely, athletes without symptoms could still have persisting brain dysfunction. Thus, prospective studies of specific tests that correlate with brain dysfunction and its recovery are required. In this regard, patients with definite TBI offer clarity as to which parameters reflect brain dysfunction separate from those not directly related to brain dysfunction. The exaggerated features of concussion in patients with definite brain imaging positive TBI may explain why we found clear-cut differences in the SCAT3 between our patient group and controls at an average time to testing of seven days post-TBI (Table 1) – a finding at variance with sports concussion where the SCAT3 sensitivity falls off after 3–5 days post-concussion. 26 Moreover, as the SCAT3 was designed to assess sports concussion it would be useful to prospectively assess such patients with mild TBI requiring hospitalization and in so doing establish the SCAT3’s sensitivity in differentiating between concussions and mild TBI requiring hospitalization.
Limitations and confounds
There were important clinical confounds in our study. Roughly half of the head-injured patients in our trauma unit have BPPV (unpublished audit data). Hence, all patients were screened and treated for BPPV at least a day before SCAT3 testing (and confirmation of cure confirmed on the day of SCAT3 testing). Second, most patients admitted to the major trauma ward with brain injury have a week of anti-epileptic drug prophylaxis to prevent post-traumatic seizures. 27 Of the patients we tested, 75% had been prescribed the anti-epileptic drug phenytoin, which is known to induce a cerebellar ataxia and dizziness – often in associated with a gaze-evoked nystagmus which is a cerebellar-mediated eye movement abnormality. 28 To avoid phenytoin affecting the testing, patients were only tested if off phenytoin and sufficient time allowed for any drug effects to wash out (and confirmed by serum drug assay). Additionally, at the time of testing, none of the patients displayed overt cerebellar ataxia and none had impaired tandem walking or gaze-evoked nystagmus. Indeed, it was notable in these patients that tandem standing was worse than tandem walking – a combination that is typical of a vestibular ataxia described by Brandt et al. 29 in patients with an acute peripheral vestibular loss. Importantly, all patients recruited had preserved peripheral vestibular functioning (and had any BPPV treated prior to assessment) to avoid polluting symptoms and signs of imbalance resulting from inner ear diagnoses, enabling us to relate imbalance to brain dysfunction.
Another limitation is the relatively small sample size of 11 subjects per group. Although within a single paired correlation, a sample size of 11 is adequate for a valid correlation analysis, there is however the risk of falsely accepting the null hypothesis for small effect sizes, i.e. due to the small sample size, we could have missed a significant correlation between cognitive and balance (BESS) assessments and their respective symptoms. However, given our failure to find any significant correlations between symptoms and signs – even when we did not correct for multiple comparisons (with the attendant risk of false positive findings) – suggests that any such correspondence would be weak to moderate. In addition, an a priori power analysis between two parameters that would reasonably be expected to be highly correlated, for instance the subjective symptom of ‘balance problems’ and the objective measurement of imbalance (total BESS score), showed that we would need to test circa 300 patients (H1 correlation between ‘balance problems’ and ‘total BESS score’ = 0.163, α = 0.05, power = 0.8, and correlation H0 = 0, would need a sample size of 293 participants).
Conclusion
In summary, this is the first study to evaluate the SCAT3 in acute traumatic brain-injured patients, all with documented abnormal neuroimaging. Our data show that the SCAT3 is sensitive in discriminating between acute TBI patients and healthy controls, and this sensitivity in detecting TBI was found for all domains in the SCAT3. We found no significant correlation between the cognitive and balance test scores and their complementary symptoms, implying at best, a modest correspondence between patients’ symptomatic complaints and objective assessment of their complaints. One explanation could be that objective testing may reflect brain injury whereas symptoms may be a combination of brain injury and other factors that are not directly related to brain injury (e.g. in predisposed individuals, a migraine process – uncorrelated with the severity of brain injury – may cause headache and dizziness). Equally, the loss of insight due to the brain injury could result in a relative diminution of symptoms. Hence, future prospective studies should assess whether concussion patients with normal brain imaging show a similar lack of correspondence between symptoms and signs for other measurement scales (i.e. other than the SCAT).
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: BMS is supported by grants from the Medical Research Council (UK), National Institute for Health Research (UK), The Racing Foundation, The Imperial NIHR Imperial Biomedical Research Centre, EPSRC, Imperial Health Charity, US Department of Defense – Congressionally Directed Medical Research Program.