Effect of subconcussive impacts on functional outcomes over a single collegiate football season

Abstract

Context

In collision sports, particularly American football, athletes can accumulate thousands of subconcussive impacts, or head acceleration events (HAEs), across a single season; however, the short-term consequences of these impacts are not well understood.

Objective

To investigate the effects of the accumulation of impacts during practices on cognitive functions over a single football season.

Design

Prospective observational study.

Setting

Athletic training room and University laboratory.

Participants

Twenty-three NCAA Football Bowl Subdivision players.

Main outcome measures

Helmet accelerometers during practices and virtual reality testing (VR; balance, reaction time, spatial memory) before and after the season.

Results

Preseason had the majority of ≥80 G impacts while during the season had the majority of ≥25 G to <80 G impacts and positional differences showed that linemen had the majority of both types. Virtual reality analysis revealed that scores significantly decreased after the season for spatial navigation (p < 0.05) but not for balance or reaction time. Significant correlations (p < 0.05) were found between cognitive measures and player demographic variables.

Conclusions

Even in the absence of clinical symptoms and concussion diagnosis, repetitive impacts may cause cognitive alterations. Documenting the distribution of impact quantity and intensity as a function of time and position may be considered by coaches and clinicians to reduce the accumulation of impacts in athletes exposed in contact sports.

Keywords

Subconcussive impacts head acceleration events accelerometer virtual reality football

Introduction

Repetitive subconcussive impacts, or head acceleration events (HAEs), are impacts that involve the transfer of mechanical energy to the brain with sufficient force to injure axonal or neuronal integrity but do not result in clinical symptoms.^1,2 These impacts are often unmanaged or undiagnosed, as they are less severe than a full-blown concussive injury, and result in no immediate neurological deficits, making them challenging to operationally define.³ A recent review by Mainwaring and colleagues⁴ found no consensus definition in use for these impacts. Despite this, there has been growing concern surrounding these impacts as they can accrue to large numbers over the course of a season or career^5,6 and have a possible link to neurological impairments.⁶ Post-mortem studies, done in both humans and animals, support the notion that repeated HAEs may have an accumulative effect⁷ and affect cognitive processes.⁸ The link between repetitive HAEs and cognitive function,^9–12 white matter changes on diffusion tensor imaging (DTI^13,14), alterations of cerebral blood flow,¹⁵ functional brain alterations,^9,12,15,16 and cerebral spinal fluid changes¹⁷ have been previously demonstrated. However, the intensity and frequency of the impacts necessary to cause damage is still unknown.

Players of collision sports, such as football, are at a high risk for the accumulation of repetitive head impacts. Previous studies have shown that high school football players can average 600 impacts over a season and collegiate players average 1000 impacts.¹ However, these exposures can also vary based on player position and previous concussive history. Research has begun to link these exposure metrics to cognitive and neurobehavioral disturbances, including increased risk for Alzheimer’s disease, chronic traumatic encephalopathy (CTE), or amyotrophic lateral sclerosis (ALS).⁶ However, there is still much debate regarding the progression of these disturbances and their link to repetitive impacts.¹⁸

Accelerometers are now commonly used in research to track the number of HAEs in American football. College football players (compared to high school football and other college sports) experience the most impacts across a season, the highest average peak linear and rotational acceleration per impact, and the highest cumulative linear and rotational acceleration per impact.¹⁹ The type of practice is also a contributing factor to the number of occurring impacts. Full-pad practices have been shown to have the highest number of impacts per session and these numbers increase in game settings.^19–21

Position played is a major confounding factor and current literature suggests that offensive and defensive linemen sustain the greatest number of impacts, at a lower magnitude however, than other positions, as they are normally on the line of scrimmage and are involved in short distance, low magnitude impacts on almost all plays.^20–22 Athletes in the skill or speed positions while in the open field experience fewer, but higher magnitude impacts.²¹ These positions include quarterback, running back, fullback, tight end, corner back, safety, and linebacker; players that often build up momentum prior to tackling or being tackled.²³ Playing position also was a significant factor in the athletes’ symptom reporting of more non-clinical post-impact symptoms (headache, concentration difficulties, dizziness, etc.), with offensive linemen in particular reporting more-frequent symptoms after repetitive impacts, suggesting that these linemen experience more subconcussive injury.²⁴

Commonly, the processes affected after concussion are memory, attention, concentration, processing speed, and reaction time.^25–27 Additionally, cognitive changes have also been associated with repetitive HAEs.^11,28 Visual working memory impairments, with altered activation in the dorsolateral prefrontal cortex,¹² and abnormal regional cortical activation patterns¹³ have been demonstrated in football players after a single season, even in the absence of clinical symptoms of concussion. However, many studies using traditional clinical concussion tools report no findings over the course of a single season^29–31 or report improvements in score after the season.^32–35

Therefore, this study aimed to examine the effect of multiple high intensity collisions, or HAEs, on brain functionality after participation in a single season of football using a novel virtual reality tool. Furthermore, by monitoring cognitive changes, we can examine how exposure to repetitive HAEs affects these processes in a way that is both time and cost effective and could help inform clinical protocols. We hypothesized that the number of HAEs accumulated throughout the season (as an additive effect) would influence functional properties critical in athletes (balance, reaction time, spatial memory). Additionally, the number of impacts would be a confounding factor along with player position, how long they have played football, and/or previous history of concussive injury.

Methods

Design: Before the season started, all participants underwent a baseline test on virtual reality and the accelerometer BodiTrak system was installed in their helmets. Once preseason began, participants wore their accelerometers during 53 practice sessions. After the season ended, all participants were tested again on the virtual reality system (see Procedures below for details on the specific modalities).

Participants: 24 participants were recruited and enrolled in the study. 1 participant dropped out during baseline testing, so their data was not used and the sample size moving forward was n = 23. All participants (n = 23) were Penn State University NCAA Football Bowl Subdivision football players. Background information is contained in Table 1. These participants were part of a larger study that also included imaging data which has been previously published.¹⁵ All participants provided informed written consent, as approved by the Penn State University Institutional Review Board (IRB#2907) and were compensated for their involvement in the study. It is important to note that none of the participants included in this study were diagnosed with a concussion by medical personnel during the course of the season.

Table 1.

Participant demographic information.

Variables	(n = 23)
Demographics, range (M ± SD)
Age	19–23 (20.78 ± 1.28)
Number of Years Playing	6–19 (10.89 ± 3.34)
Number of Previous Concussions, n (%)
0	14 (60.9)
1	7 (30.4)
2	2 (8.7)
ADHD Diagnosis, n (%)	2 (8.7)
Psychiatric Diagnoses, n (%)	0 (0)
Position Breakdown, n (%)
Offensive Line	8 (34.8)
Defensive Line	8 (34.8)
Tight End	2 (8.7)
Linebacker	2 (8.7)
Safety	2 (8.7)
Running Back	1 (4.3)

Procedures

BodiTrak System: Impacts to the head for all participants were monitored using helmet sensors from the Head Health Network BodiTrak system. These sensors, created by Vista Medical, were adapted to be individually placed inside each individual participants’ helmet using a 3M VHB adhesive and are comprised of elastic fabric with pressure monitors and impact sensors. With sensor installation, the sensors are placed directly on the inner surface of the helmet between the shell and the padding and the helmet is not altered in any way. All sensors were placed by a certified athletic trainer (author M.S.) who was trained on proper installation techniques and the sensors were monitored throughout the season for integrity and functionality. The sensors provide estimates on linear acceleration (in units of G) and location of impact and were in place throughout the season. Impacts were only recorded during practices (not games) for a total of 53 possible sessions.

Virtual Reality System: A 3D TV system (HeadRehab.com) with a head mounted accelerometer was used. The accelerometer is attached to an adjustable headband which each participant wore over their left ear. Data were collected before season (baseline) and after the season. Three different modules were used: balance, reaction time, and spatial memory (see^36–38 for reliability and validity of all modules for use in detecting sport-related concussion. Briefly: spatial memory (sensitivity 95.8%/specificity 91.4%); reaction time (sensitivity 95.2%/specificity 89.1%); balance (sensitivity 85.7%/specificity 87.8%)). In the balance task, participants are instructed to hold tandem Romberg position for all trials. In the first trial, the virtual room (Figure 1) is completely still (for a baseline measure) and in the subsequent 6 trials, the virtual room moves in various directions. The system measures the position and orientation in yaw, pitch, and roll directions. In the reaction time module, participants are instructed to stand feet shoulder width apart, hands on their hips. The virtual room moves, and they are instructed to move their body in the same direction as the virtual room. The system measures both reaction time (in ms) and errors of anticipation (wrong direction of response). In the spatial memory module, participants are shown a 3 D randomized representation of a virtual corridor. They are shown a pathway with multiple turns to a door and then the return trip. Afterwards, they are instructed to repeat the exact pathway using a joystick. The system measures how many errors it takes to get to the door and the total time to complete the task. Raw data were analyzed using SideLine v10.1 test reporting module which uses mathematical algorithms to produce a score on a scale of 0 (fail) to 10 (perfect) and this score was used for analysis.

Figure 1.

Representation of the virtual room.

Statistical Analysis: All statistical analyses were performed using SPSS V25. For all analyses, statistical significance was considered reached at p < 0.05. Normality of the VR data was assessed using histogram plots and the Shapiro-Wilk test. Due to non-normality, a Wilcoxon Signed Rank Test was used to assess differences in virtual reality scores before the season versus after the season. Additionally, participants were grouped into categories of concussion history (i.e., yes or no) or position categories (i.e, speed (quarterback, running back, halfback, fullback, wide receiver, tight end, defensive back, safety, and linebacker) or non-speed (offensive and defensive linemen) based off of Lehman’s classifications²³) and then differences in virtual reality scores before the season verses after the season were also examined. Point-Biserial correlations were used, given the ability to measure the association between continuous and dichotomous variables, to assess the relationship between accelerometer data, virtual reality data, and participant demographic information. Impacts were categorized into categories of ≥80 G (to reflect higher intensity impacts) and ≥25 G to <80 G (to reflect lower intensity impacts) based on work done by Mihalik²⁰ and on pilot work done at Penn State. Correction for multiple comparisons was done on correlation analyses for the individual virtual reality components as each modality (balance, reaction time, or spatial memory) was a unique hypothesis (p = 0.05/6 = 0.0083).

Results

Accelerometer Data: A total of 129 impacts at ≥80 G were recorded across the season (53 sessions) with 78.3% (101 impacts) occurring during the preseason (13 sessions) and 21.7% (28 impacts) occurring during the regular season (40 sessions). A total of 3557 impacts at ≥25 G to <80 G were recorded across the season with 32.6% (1158 impacts) occurring during the preseason and 67.4% (2399 impacts) occurring during the regular season. The raw number of individual impacts at ≥80 G and ≥25 G to <80 G obtained over the season are presented in Table 2. The average incidence rate for impacts at ≥80 G across the season was 0.0047 ± 0.0038 while the average incidence rate for impacts ≥25 G to <80 G across the season was 0.1228 ± 0.0878.

Table 2.

Accelerometer data for all participants (n = 23) including position information.

Participant number	Position	Monitored sessions (53 total)	Total ≥80 G	Incidence rate of ≥80 G^a	Total ≥25 G and ≤80 G	Incidence rate of ≥25 G and ≤80 G^a
1*	Tight End	27	0	0.0000	17	0.0139
2	Offensive Line	42	3	0.0025	95	0.0779
4	Defensive Line	37	5	0.0041	116	0.0952
5	Defensive Line	38	7	0.0057	180	0.1477
6	Safety	45	5	0.0041	101	0.0829
7	Defensive Line	47	11	0.0090	148	0.1214
8	Offensive Line	47	10	0.0082	455	0.3733
9*	Offensive Line	26	0	0.0000	28	0.0230
10	Offensive Line	49	15	0.0123	344	0.2822
11	Offensive Line	30	5	0.0041	221	0.1813
12	Running Back	43	13	0.0107	110	0.0902
13	Safety	51	2	0.0016	124	0.1017
14	Defensive End	37	4	0.0033	155	0.1272
15	Defensive End	32	7	0.0057	95	0.0779
16**	Defensive Line	2	2	0.0016	9	0.0074
17	Defensive Line	36	2	0.0016	99	0.0812
18	Defensive Line	47	2	0.0016	74	0.0607
19	Offensive Line	38	1	0.0008	183	0.1501
20	Linebacker	45	1	0.0008	151	0.1239
21	Offensive Line	48	7	0.0057	301	0.2469
22	Tight End	37	13	0.0107	205	0.1682
23**	Linebacker	17	12	0.0098	83	0.0681

Players who missed part of the season due to a non-neurological injury are indicated by an asterisk (*). Players whose season ended early due to non-neurological injury are indicated by a double asterisk (**).

^aIncidence rate calculated as: (number of impacts)/(population size*time of study). For example, for participant 1: 17/(23*53).

By practice type, full pads had the highest raw cumulative number of ≥80 G (78) impacts followed by scrimmages (30), upper pads only (19) and then helmets only (2). For the ≥25 G to <80 G impacts by practice type, full pads have the highest raw cumulative number (1934) followed by upper pads only (1225), scrimmages (300), and then helmets only (98). However, practice type rate showed that the scrimmage practices had the highest average number of impacts per practice followed by full pads, uppers, and lastly, helmets.

Virtual Reality Data: Virtual reality results were produced on a scale of 0 (fail) to 10 (perfect) for each module: spatial memory, balance, and reaction time. Average scores for the before season and after season were calculated for all 23 participants and Wilcoxon Signed Rank Test were run (significance, p < 0.05). In the spatial memory module, a significant change (Z = −2.160, p = 0.03, effect size r = 0.45) in score from before season (M = 8.79, SD = 1.90) to after season (M = 7.74, SD = 2.85) was observed. In the balance module, there was a non-significant change (Z = −1.02, p = 0.31, r = 0.21) in scores from before season (M = 7.89, SD = 2.49) to after season (M = 8.97, SD = 0.74). The reaction time module also showed a non-significant change (Z = −1.40, p = 0.16, r = 0.29) in score from before season (M = 4.78, SD = 3.30) to after season (M = 5.98, SD = 2.18).

Additionally, when data was examined based on previous concussion history (yes or no) there were significant differences found. Players with a previous history of concussion (n = 9) had a significant decrease (Z = −2.016, p = 0.04) in the spatial memory module from before the season (M = 9.03, SD = 1.12) to after the season (M = 6.44, SD = 3.19). No other significant differences were found for either group in the other VR modules.

Correlation Data: Point-Biserial Correlations were run between predictors of interest (position category, years played football, and how many previous concussions) and main outcomes (before and after VR scores and impacts) and are presented in Table 3. After correction for multiple comparisons, only non-speed versus speed position category was significantly negatively correlated with before the season spatial memory VR scores.

Table 3.

Point-biserial correlations.

	Before spatial	Before balance	Before Rxn time	Post spatial	Post balance	Post Rxn time
Non-Speed vs Speed	–.537**	0.123	0.111	–0.367	0.122	0.107
Years Played	–0.019	–0.046	0.283	–0.283	–0.211	–0.107
How Many Cx	0.008	0.072	0.083	–0.282	–0.175	0.043
Total 80 G	–0.095	0.068	–0.096	0.061	–0.143	–0.159
Total 25 G	–0.123	0.089	0.230	0.197	–0.013	–0.311

cx = concussion; Total 80 G = impacts ≥80 G; Total 25 G = impacts ≥25 G to <80 G; Before = before season; rxn time = reaction time; Post = after season.

**Correlation is significant after correction for multiple comparisons.

Discussion

To our knowledge, this study is the first to explore neurocognitive changes, via VR measures, after a single season of football. By implementing testing throughout the season, we were able to begin to track players’ neurocognitive abilities and how it relates to the accumulation of impacts. This study revealed several findings of interest that will be discussed in the following text.

In terms of raw accelerometer data, first, preseason practices contained the vast majority of the high impacts (≥80 G). Second, practice type influenced the cumulative number of impacts received. Full pads had the highest raw number of impacts both at ≥80 G and at ≥25 G to <80 G, which has been previously demonstrated.¹⁵ Third, speed players (tight end, linebacker, defensive back, running back) experienced more cumulative impacts at ≥80 G and non-speed players (linemen) had more cumulative impacts at ≥25 G to <80 G, which has been previously demonstrated.^21,22 An athletic trainer, physician, or sport coach using this technology can monitor impacts in real time, referring back to specific time points, drills, or injury instances. They can assess whether or not a certain drill is causing too many head impacts, or if an instructed movement technique is exposing the head to increased risk of trauma. Furthermore, coaches may be able to draw conclusions about how to structure practices, whether to include certain drills, or whether certain players need revisions to their techniques.

Virtual reality data revealed a significant decrease in spatial memory scores from before the season to after the season (p < 0.05) as well as no significant change in balance and reaction time scores. Furthermore, when players were separated into groups based on previous concussion diagnosis (yes or no), only players with previous history of concussion had significant decreases in spatial memory scores after the season (p < 0.05). Cognitive impairments have been previously demonstrated, mostly in MRI studies, showing visual working memory impairments being associated with altered activation in the dorsolateral prefrontal cortex (DLPFC)¹² and decreases in cortical thickness.³⁹ This study shows impairments in memory without the use of advanced MRI suggesting potential utility as a measure of memory when MRI may not be feasible. It also highlights the importance of spatial memory as a beneficial and low cost (both in a sense of time and money compared to other common research modalities) test for detecting neurocognitive changes as a function of cumulative impacts that players experience over a single football season in the absence of diagnosed concussion.

The correlation analyses also revealed findings of interest. Non-speed versus speed position category was significantly negatively correlated with before-season spatial memory VR score, with non-speed players having higher before the season spatial memory score. These correlations suggest that in the future, spatial memory assessment could be a critical clinical tool in determining which players may be at risk for cognitive deficits throughout the season and that monitoring players’ neurocognitive function during the entire season, even in the absence of any clinical symptoms, may be an important injury prevention approach.

The information from this study could potentially help identify players at risk for high number of repetitive head impacts and cognitive deficits, as previous literature has shown links between concussion history and cognitive functioning.⁴⁰ Retired professional football players, diagnosed with three or more concussions, were found to report more cognitive symptoms, specifically memory impairment and cognitive impairment, than retired players with no history of concussion.⁴¹ Our finding of a cognitive decline from before the season to after the season are in line with previous reports that a history of concussions in college football players may result in lower cognitive functions including memory changes.⁴² However, simply examining players before and after the season for deficits could potentially be misleading. These pre to postseason changes seem to provide support for the notion that clinical symptoms are not a necessary observation for the presence of deficits in brain functioning. However, it is still unclear if these changes could be a compensatory adaptation to cumulative impacts or a more lasting alteration of brain functional integrity.

This study is not without limitations. The present study was based on a fairly small cohort of football players so future work should try to replicate these findings in larger sample sizes, as well as examine a greater variety of players (including females). Additional during-season cognitive testing would also prove beneficial as testing on a variety of cognitive and functional tasks could also further increase understanding of changes within a season. Furthermore, only practice accelerometer data, and not game data, was used collected which could confound the results. Future studies should examine these sessions as well as there is potential for higher overall number of impacts, especially the impacts at higher intensities. Lastly, there is concern over the use of accelerometers in contact sports due to the accuracy of the systems used. Forces measured by the helmet systems may not accurately represent the forces experienced by the brain itself. The system used in this study likely has inconsistencies in forces measured and margins of error like many other systems used in research. Additionally, incorporating new technologies along with virtual reality could lead to the development of more beneficial assessment tools than currently used in clinical practice.

Conclusion and clinical implications

Presently, the medical community does not have proven quantitative measures to diagnose concussions, nor any proven measures to predict an individual’s susceptibility to concussive and subconcussive injury. The quantitative data regarding impacts obtained in practices via helmet sensors throughout the football season allowed us to assign an objective value to individual events that had previously only been assessed subjectively. This information, combined with the results from the virtual reality testing, may allow coaches and policymakers to more confidently embrace the idea that minimizing the number and intensity of head impacts will benefit the participant and prevent decreased cognitive testing scores after participating in one football season.

No exact diagnostic testing for concussion exists as of yet and the accelerometer technology should not be used as a diagnostic measure. However, medical professionals may be able to use technologies such as virtual reality and helmet sensors to assist in making decisions about the health and well-being of patients. Until the medical community has found definitive diagnostic measures for concussive and/or subconcussive injuries, these technologies could be used as another tool to assist in making more informed decisions about patient safety.

Footnotes

Acknowledgements

We would like to thank Katie Finelli for her assistance with data collection and the Penn State Football players for their time.

Data availability statement

Data is available upon reasonable request to the authors.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article:: Final preparation of this manuscript was partially supported by a NCAA grant “Effects of Cumulative Head Acceleration Events (HAE) Over A Singe Athletic Season on Brain Functional and Structural Integrity” (author SMS). The remaining authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Alexa E Walter

Supplemental Material

Supplementary material for this article is available online.

References

Bailes

Petraglia

Omalu

, et al. Role of subconcussion in repetitive mild traumatic brain injury. J Neurosurg 2013; 119: 1235–1245.

Manley

Gardner

Schneider

, et al. A systematic review of potential long-term effects of sport-related concussion. Br J Sports Med 2017; 51: 969–977.

Belanger

Vanderploeg

McAllister

Subconcussive blows to the head: a formative review of short-term clinical outcomes. J Head Trauma Rehabil 2016; 31: 159–166.

Mainwaring

Pennock

KMF

Mylabathula

, et al. Subconcussive head impacts in sport: a systematic review of the evidence. Int J Psychophysiol 2018; 132: 39–54.

Baugh

Stamm

Riley

, et al. Chronic traumatic encephalopathy: neurodegeneration following repetitive concussive and subconcussive brain trauma. Brain Imaging Behav 2012; 6: 244–254.

McKee

Cantu

Nowinski

, et al. Chronic traumatic encephalopathy in athletes: progressive tauopathy after repetitive head injury. J Neuropathol Exp Neurol 2009; 68: 709–735.

Shultz

MacFabe

Foley

, et al. Sub-concussive brain injury in the Long-Evans rat induces acute neuroinflammation in the absence of behavioral impairments. Behav Brain Res 2012; 229: 145–152.

Broglio

Eckner

Paulson

, et al. Cognitive decline and aging: the role of concussive and subconcussive impacts. Exerc Sport Sci Rev 2012; 40: 138–144.

Breedlove

Robinson

Talavage

, et al. Biomechanical correlates of symptomatic and asymptomatic neurophysiological impairment in high school football. J Biomech 2012; 45: 1265–1272.

10.

Koerte

Lin

Muehlmann

, et al. Altered neurochemistry in former professional soccer players without a history of concussion. J Neurotrauma 2015; 32: 1287–1293.

11.

McAllister

Flashman

Maerlender

, et al. Cognitive effects of one season of head impacts in a cohort of collegiate contact sport athletes. Neurology 2012; 78: 1777–1784.

12.

Talavage

Nauman

Breedlove

, et al. Functionally-detected cognitive impairment in high school football players without clinically-diagnosed concussion. J Neurotrauma 2014; 31: 327–338.

13.

Bazarian

Zhu

Zhong

, et al. Persistent, long-term cerebral white matter changes after sports-related repetitive head impacts. PLoS One 2014; 9: e94734.

14.

Davenport

Whitlow

Urban

, et al. Abnormal white matter integrity related to head impact exposure in a season of high school varsity football. J Neurotrauma 2014; 31: 1617–1624.

15.

Slobounov

Walter

Breiter

, et al. The effect of repetitive subconcussive collisions on brain integrity in collegiate football players over a single football season: a multi-modal neuroimaging study. Neuroimage Clin 2017; 14: 708–718.

16.

Abbas

Shenk

Poole

, et al. Alteration of default mode network in high school football athletes due to repetitive subconcussive mild traumatic brain injury: a resting-state functional magnetic resonance imaging study. Brain Connect 2015; 5: 91–101.

17.

Neselius

Brisby

Theodorsson

, et al. CSF-biomarkers in olympic boxing: diagnosis and effects of repetitive head trauma. PLoS One 2012; 7: e33606.

18.

Iverson

Gardner

Shultz

, et al. Chronic traumatic encephalopathy neuropathology might not be inexorably progressive or unique to repetitive neurotrauma. Brain 2019; 142: 3672–3693.

19.

Reynolds

Patrie

Henry

, et al. Comparative analysis of head impact in contact and collision sports. J Neurotrauma 2017; 34: 38–49.

20.

Mihalik

Bell

Marshall

, et al. Measurement of head impacts in collegiate football players: an investigation of positional and event-type differences. Neurosurgery 2007; 61: 1229–1235; discussion 1235.

21.

Broglio

Sosnoff

Shin

, et al. Head impacts during high school football: a biomechanical assessment. J Athl Train 2009; 44: 342–349.

22.

Schnebel

Gwin

Anderson

, et al. In vivo study of head impacts in football: a comparison of national collegiate athletic association division I versus high school impacts. Neurosurgery 2007; 60: 490–495; discussion 495–496.

23.

Lehman

EJ.

Epidemiology of neurodegeneration in american-style professional football players. Alzheimers Res Ther 2013; 5: 34.

24.

Baugh

Kiernan

Kroshus

, et al. Frequency of head-impact-related outcomes by position in NCAA division I collegiate football players. J Neurotrauma 2015; 32: 314–326.

25.

Iverson

Gaetz

Lovell

, et al. Cumulative effects of concussion in amateur athletes. Brain Inj 2004; 18: 433–443.

26.

Jordan

Matser

Zimmerman

, et al. Sparring and cognitive function in professional boxers. Phys Sportsmed 1996; 24: 87–98.

27.

Collins

Grindel

Lovell

, et al. Relationship between concussion and neuropsychological performance in college football players. JAMA 1999; 282: 964–970.

28.

Vogelpohl

Lindsey

Stickley

, et al. A prospective evaluation on the effects of one high school football season on neurocognitive test scores in high school football athletes. Int J Athl Ther Train 2017; 22: 49–55.

29.

Caccese

Best

Lamond

, et al. Effects of repetitive head impacts on a concussion assessment battery. Med Sci Sports Exerc. Epub ahead of print 14 January 2019. DOI: 10.1249/MSS.0000000000001905.

30.

Gysland

Mihalik

, et al. The relationship between subconcussive impacts and concussion history on clinical measures of neurologic function in collegiate football players. Ann Biomed Eng 2012; 40: 14–22.

31.

Kaminski

Wikstrom

Gutierrez

, et al. Purposeful heading during a season does not influence cognitive function or balance in female soccer players. J Clin Exp Neuropsychol 2007; 29: 742–751.

32.

Brett

Solomon

GS.

Comparison of neurocognitive performance in contact and noncontact nonconcussed high school athletes across a two-year interval. Dev Neuropsychol 2017; 42: 70–82.

33.

Miller

Adamson

Pink

, et al. Comparison of preseason, midseason, and postseason neurocognitive scores in uninjured collegiate football players. Am J Sports Med 2007; 35: 1284–1288.

34.

Volberding

Melvin

Changes in ImPACT and graded symptom checklist scores during a division I football season. Athl Train Sports Health Care 2014; 6: 155–160.

35.

Munce

Dorman

Odney

, et al. Effects of youth football on selected clinical measures of neurologic function: a pilot study. J Child Neurol 2014; 29: 1601–1607.

36.

Teel

Slobounov

SM.

Validation of a virtual reality balance module for use in clinical concussion assessment and management. Clin J Sport Med 2015; 25: 144–148.

37.

Teel

Gay

Arnett

, et al. Differential sensitivity between a virtual reality balance module and clinically used concussion balance modalities. Clin J Sport Med 2016; 26: 162–166.

38.

Teel

Gay

Johnson

, et al. Determining sensitivity/specificity of virtual reality-based neuropsychological tool for detecting residual abnormalities following sport-related concussion. Neuropsychology 2016; 30: 474–483.

39.

Tremblay

De Beaumont

Henry

, et al. Sports concussions and aging: a neuroimaging investigation. Cereb Cortex 2013; 23: 1159–1166.

40.

Montenigro

Alosco

Martin

, et al. Cumulative head impact exposure predicts later-life depression, apathy, executive dysfunction, and cognitive impairment in former high school and college football players. J Neurotrauma 2017; 34: 328–340.

41.

Guskiewicz

Marshall

Bailes

, et al. Association between recurrent concussion and late-life cognitive impairment in retired professional football players. Neurosurgery 2005; 57: 719–726; discussion 719–726.

42.

Rabadi

Jordan

BD.

The cumulative effect of repetitive concussion in sports. Clin J Sport Med 2001; 11: 194–198.