Abstract
Background:
In response to concerns over concussions and repeated head impacts that occur during sports, the National Collegiate Athletic Association (NCAA) mandated that all member institutions enact a concussion management plan (CMP). Although institutional and health care provider self-reports have been investigated, compliance with NCAA protocol recommendations has not been examined.
Purpose:
To examine the CMPs from the 65 institutions within the NCAA Power 5 conferences for compliance with the NCAA 2015 concussion guidelines.
Study Design:
Descriptive epidemiology study.
Methods:
Each institution’s publicly available CMP was obtained in 2015, reviewed, and coded for compliance with each of the required 47 components. Overall compliance rate, item-level, category-level, and institution-level compliance was assessed. Independent predictors of compliance, including each institution’s athletic training staff size, academic performance, and athletic performance, were examined with quasi-binomial regression.
Results:
CMPs varied substantially in length and level of detail. The overall compliance rate for all components across all institutions was 94.3% (2880/3055). Twelve components achieved 100% (65/65) compliance, and the lowest levels of compliance were clustered in “return to learn.” There were 22 institutions that achieved a 100% compliance rate; the lowest institutional compliance was 59.6%. There were no significant associations between the independent predictors and institutional compliance.
Conclusion:
Overall compliance with NCAA concussion management requirements was high, but there remains room for improvement. The lowest level of compliance was clustered in the return-to-learn section. There were limited details provided in the reducing head trauma component. Items with lower compliance (reducing head trauma, return to learn) tended to be outside the core competencies of the medical staff, indicating an area for improvement. Encouragingly, many institutions and specific components demonstrated full compliance.
In recent years, there has been growing concern regarding the health-related consequences of concussions and repeated head impacts sustained during sports. In response, all 50 states, the District of Columbia, and multiple sports organizations have enacted policies aimed at improving athlete safety. 41 At the collegiate level, the National Collegiate Athletic Association (NCAA), created in 1910 as the governing body of intercollegiate athletics, is charged, in part, with making the game of football safer. 45 A century later, sport safety remains an ongoing challenge; however, the focus has now shifted from football fatalities to concussion and head trauma concerns across all sports. Indeed, concussions occur in all NCAA-sponsored sports, and later-life neuropathologies have been identified in former football, soccer, ice hockey, and baseball players. 29,49 The topic of sports-related concussion first appeared in the NCAA Medical Handbook in 1933, in which it was stated that the seriousness of concussions was often overlooked. The first official protocols were recommended in 1994. 8,40 Although changes occurred gradually over the next 16 years, NCAA concussion regulation has rapidly evolved since 2010. 34
On April 29, 2010, the NCAA enacted its Concussion Policy and Legislation, which became effective in August 2010 (NCAA Bylaw: 3.2.4.17) and stated that all member institutions must have a concussion management plan (CMP). This plan was to include the following components: (A) an annual process that ensures student-athletes are educated about concussion signs and symptoms, acknowledge receipt of information, and have a responsibility to report symptoms to a medical staff member; (B) student-athletes with suspected concussions must be removed from the current practice or game and evaluated by a medical staff member with experience in the evaluation and management of concussions; (C) student-athletes diagnosed with a concussion are precluded from same day return to play (RTP); and (D) student-athletes must be cleared by a physician or physician’s designee prior to RTP after a concussion. 34 Subsequent years saw minor modifications to the concussion guidelines as well as the publication of “best practices” recommendations. Bylaw 3.2.4.17 was expanded (Bylaw: 3.2.4.17.1) in January 2016 to require a member institution’s CMP to include, by August 2016, procedures for (A) compliance with 3.2.4.17; (B) preparticipation baseline testing for each student-athlete; (C) reducing exposure to head injuries; (D) education about concussion, including a policy that addresses return to learn (RTL); (E) proper and appropriate concussion management, consistent with known best practices, made available to any student-athlete who has suffered a concussion; (F) annual review of the process of identifying, removing from participation, and assessing a student-athlete for a possible concussion; and (G) certification of compliance signed by the institution’s athletics director. 33 A set of guidelines, referred to as the NCAA Concussion Safety Protocol Checklist (NCAA-CSPC), was distributed to all 65 member institutions of the autonomous 5 conferences collectively referred to as the “Power 5”: the Atlantic Coast Conference, Big Ten Conference, Big-12 Conference, Pac-12 Conference, and Southeastern Conference, to ensure their CMPs were complaint with Bylaw 3.2.4.17.1. 33,36 Each institution’s CMP was reviewed for compliance by the Concussion Safety Protocol Committee in 2015. 33,37
Since the 2010 Concussion Policy was enacted, there has been limited investigation of the evolving NCAA concussion management policies, with Baugh et al 5 reporting high, but not total, self-reported compliance with the concussion policy among NCAA athletic department personnel. Therefore, the purpose of this study was to examine the compliance of CMPs of schools within the NCAA Power 5 conferences with the NCAA concussion guidelines as revised in 2015. Given that these CMPs had been previously reviewed for compliance by a committee within the NCAA, we hypothesized that compliance would be high. However, we expected that areas shown in previous studies to have been less well implemented would likely have lower overall compliance. Additionally, we examined the association between athletic training staff size, academic performance, athletic performance, and conference with compliance.
Methods
The Power 5 CMPs and NCAA-CSPC were obtained through the publicly available NCAA concussion safety website. 35 The NCAA-CSPC is divided into 8 sections with required components in each. 36 The administrative section, section 8, contains only submitting the plan to the NCAA which, by default, all 65 institutions performed; therefore, the administrative section was not scored in this analysis. The remaining 7 sections had a total of 47 components that were evaluated (Table 1).
Mandatory Components of the NCAA Concussion Safety Protocol Checklist a
a For the purposes of this review, only sections 1 to 7 (N = 47) were analyzed. NCAA, National Collegiate Athletic Association.
Each component was scored as addressed (yes) or not addressed (no) in each institution’s CMP. In instances where a required component was not directly addressed (eg, “preferred to provide written care instructions to another responsible adult”), the component was scored as not addressed, since “preferred” does not meet the “required” standard. Conversely, multiple concussion management plans indicated that they were using comprehensive assessments (eg, the Sport Concussion Assessment Tool–3rd edition [SCAT-3]), and they were credited with performing each of the individual assessments (eg, balance testing) within the comprehensive assessment. Length of CMPs was determined by converting PDFs to Microsoft Word documents and taking the page and word counts. Each CMP was examined by a member of the research team to determine whether it included each of the evaluated components; a random 10% of the plans were coded by a second member of the research team and evaluated for interrater reliability, which was deemed sufficiently high (kappa: 0.976).
The athletic training staff size was obtained by reviewing the intercollegiate athletic department websites for each institution; only staff athletic trainers (ATs) who provided clinical athletic training services as of January 2016 were counted. Graduate assistants, interns, and research staff were not included in the count. The institution’s academic performance was categorized by the NCAA’s Graduation Success Rate (GSR) for the incoming 2008 recruitment class—the most recently available data. 38 The GSR reflects the percentage of first-time freshmen enrolled as student-athletes who graduate within 6 years regardless of transfer status. 38 The institution’s athletic performance was categorized by the Learfield Sports Directors’ Cup (Directors’ Cup), which awards points according to performance in NCAA championship events in 20 (10 male and 10 female) sports, with higher numbers reflecting better performance. 24
Data Analysis
The overall compliance rate of the 47 components across all 65 institutions was calculated based on the number of components directly addressed in the CMP divided by the total number of components. The mean institutional compliance rate was calculated as the number of components directly addressed by the institution’s CMP divided by the 47 possible components. Second, a compliance rate per NCAA-CSPC section (N = 7) and components (N = 47) was calculated by summing the number of items that the institution complied with and dividing it by the total possible items, either within the section or overall, respectively. Descriptive statistics are provided to illustrate the levels of compliance overall and within the NCAA-CSPC sections. Simple quasi-binomial regressions (log-link) were performed to investigate the institution’s overall compliance rate (0%-100%) with each of the 4 independent variables: athletic training staff size, academic performance, athletic performance, and conference. Post hoc it was decided to specifically examine compliance with the NCAA-CSPC “Return to Learn” section across these 4 independent variables, using the same methodology as described for overall compliance.
Results
The concussion management plans showed considerable diversity in length (mean ± SD, 14.2 ± 13.8 pages; range, 4-67 pages) and word count (mean ± SD, 4701.9 ± 5635.7 words; range, 1186-32,270 words). Among plans that identified their most recent update (n = 51), all but one had been updated in 2015.
Sections
Seven sections of the NCAA-CSPC were evaluated for compliance. The section with the highest compliance was “Pre-Participation Assessment” at 99.1% (322/325), while the lowest overall was “Return to Learn” at 86.4% (618/715) (Table 1).
Components
The overall component compliance was 94.3% (2880/3055). The majority of components had very high compliance (Tables 2-5).
Section 3: Recognition and Diagnosis of Concussion a
a ATC, athletic trainer certified.
Section 4: Post-Concussion Management a
a SA, student-athlete.
Section 5: Return to Play a
a RTP, return to play; SA, student-athlete.
Section 6: Return to Learn
a ADAAA, Americans with Disabilities Act and Amendments; SA, student-athlete.
There were 12 components that achieved 100% (65/65) compliance, primarily in the sections “Recognition and Diagnosis of Concussion” and “Return to Play.” Member institutions were frequently compliant within the “Pre-Season Education” section (98.2%; 383/390), including concussion education being provided to student-athletes (100%; 65/65), coaches (100%), team physicians (98.4%), ATs (98.5%), and the director of athletes (95.3%). Similarly, there was high compliance within the “Pre-Participation Assessment” section (99.1%; 322/325); almost all CMPs required a preparticipation assessment, including concussion history (100%; 65/65), symptom evaluation (100%), cognitive assessment (98.5%), balance assessment (100%), and the team physician determining either clearance to RTP or the need for additional testing (96.9%).
The lowest compliance items were clustered in “Return to Learn” and included “modification of schedule/academic accommodations for up to two weeks, as indicated, with help from the identified point-person” (73.8%; 48/65), “re-evaluation by team physician if concussion symptoms worsen with academic challenges” (76.9%; 50/65), “remaining at home/dorm if student-athlete cannot tolerate light cognitive activity” (81.5%, 53/65), and “re-evaluation by team physician and members of the multi-disciplinary team, as appropriate, for student-athletes with symptoms lasting longer than two weeks” (81.5%, 53/65). An additional area of low compliance was “documentation of oral and/or written care to both the student-athlete and another responsible adult” (76.9%; 50/65). All other components exceeded 83% compliance.
Institutions
The mean institutional compliance rate was 44.3 out of 47 (SD, 3.3; range, 28-47) with 22 institutions (33.8%) achieving a 100% (47/47) compliance rate. The lowest institutional compliance rate was 59.6% (28/47).
Predictors
The mean athletic training staff size was 12.1 ± 3.1 ATs (range, 7-20 ATs). The overall GSR for the 65 institutions was 85.2% ± 5.5% (range, 71%-98%). The overall Directors’ Cup score was 675.2 ± 299.6 (range, 91-1448). The overall percentage compliance by institution was not predicted by the independent variables athletic training staff size (t = 0.117, P = .907), conference (P value range, .225-.852), GSR (t = −1.41, P = .163), or Directors’ Cup score (t = −1.05, P = .300). The compliance within the “Return to Learn” category, at the institutional level, was not predicted by the independent variables athletic training staff size (t = 0.16, P = .870), conference (P value range, 0.068-0.437), GSR (t = −1.08, P = .284), or Directors’ Cup score (t = −1.696, P = .095).
Discussion
Our findings reveal high compliance with the required components of CMPs among Power 5 institutions. It is noteworthy, however, that despite the availability of a checklist and internal review of the policies, 5.7% of components were not adequately incorporated into the published CMPs. Similar to previous studies of self-reported practice patterns across a more diverse setting, these findings suggest that, while highly compliant overall, there is continued room for improvement within the Power 5 CMPs. 5 Beyond the specific data findings, 2 themes emerged: (1) medical management of concussions demonstrated high compliance whereas administrative components had lower compliance, and (2) there were 2 general approaches presented in the CMP “Return to Play” section—procedural (eg, checklist or cutoff score driven) and clinically based (eg, clinician driven) progressions.
In the past 2 decades, there has been a substantial increase in the utilization of objective concussion assessment tools across diverse sports medicine populations. All CMPs analyzed in this study included a multifaceted preparticipation assessment with more than 98% of CMPs including concussion history, symptom evaluation, cognitive assessment, and balance testing. This represents an improvement on the 71.2% of Division I ATs who reported using multiple neurological examinations in 2011, albeit the specific examinations differed between studies. 18 Across divisions in 2013, less than half of responding institutions self-reported utilizing a multifaceted battery including both balance and cognitive testing; however, Division I respondents reported using the most such tools (3.6) during concussion assessment. 6,10 In 2015, Kerr et al 19 reported that only 55.2% of NCAA Division I member institutions’ preparticipation assessments were compliant with NCAA guidelines, although multiple respondents indicated they planned on incorporating preparticipation assessments in 2014; our results might reflect those adaptations. It is important to note these prior studies included many institutions outside the Power 5 conferences, which likely have fewer resources, thus limiting the direct comparability of our study to those conducted previously. After a suspected concussion, almost all CMPs in this study (≥96.9%) incorporated objective physical tests in their evaluation, as opposed to only 17% in 1999 and 79.2% in 2011. 17,18 Similarly, CMPs were highly compliant with medical clearance requirements and stepwise progressive RTP (98.9% overall). Thus, it appears that the medical management of sports-related concussion is highly compliant with NCAA best practices and consistent with consensus statement recommendations. 28
The sports medicine staff, typically composed of team physicians and ATs, is the cornerstone of concussion recognition, diagnosis, and management within intercollegiate athletics, and they typically possess considerable expertise and experience with concussions. 4,11,18 For ATs, the concussion-specific educational competency is related to identifying the signs, symptoms, interventions, and RTP criteria. There are, however, no educational competencies related to academic accommodations that would likely be associated with RTL protocols. 32 In medical education, while the medical management of brain injury is extensively discussed, there are no medical education milestones directly or indirectly related to academic accommodations and/or RTL postconcussion. 1,2 A potential mismatch between the requirements of a CMP and current consensus might explain the lower compliance in the realm of RTL. Specifically, RTL protocols including “modification of schedule/academic accommodations” (73.8%), “gradual return to classroom/studying as tolerated” (83.1%), and “re-evaluation by team physician if concussion symptoms worsen with academic challenges” (76.9%) had the lowest compliance. Kerr et al 19 reported that many (67.2%) NCAA Division I institutions had RTL programs in 2013, but only 4.8% included academic support personnel. Herein, most CMPs provided a multi-disciplinary team (95.4%), but there was limited discussion of the utilization of academic support services, neuropsychologists, or disability services for academic accommodations after a concussion. These results suggest there may be a need for increased attention to appropriate academic accommodations, within institutional guidelines, to assist with successful return to the classroom and academic environment. Indeed, Kerr et al 19 have explicitly recommended that the NCAA work with member institutions to develop RTL programs.
The NCAA CMP requirements are broadly stated, thereby allowing each intercollegiate athletic department and medical staff flexibility in concussion management, given the individualistic presentation and resolution of each concussion. Two common approaches were noted within the CMPs, which we generally classified as (1) procedural or checklist driven or (2) clinically driven. Indeed, one protocol specified the exact times and tests that were to be performed after a suspected concussion as well as specific step-by-step protocols for RTP progression. In a similar manner, another institution presented sport-specific activities (eg, number of jogging laps during the initial phase of exercise) for their progressive RTP protocol. Conversely, clinician judgment–driven protocols were also presented wherein specific testing timelines were based on when the treating physician deemed they were warranted. Both approaches are consistent with the overall NCAA CMP requirements and, given the lack of evidence supporting or refuting determinants of recovery, 12,44 clinical judgment is certainly warranted so long as assessments are consistent with the test’s psychometric properties. 19 Conversely, utilizing a strict objective guideline may provide sports medicine clinicians with support, offsetting pressure from coaches and student-athletes pushing for RTP. 21,31,43
Another area of high but incomplete CMP compliance was the section “Reducing Exposure to Head Trauma” (92.3% [60/65]). The CMP checklist acknowledged that reducing head trauma may be hard to quantify, and the majority of institutions stated adherence to the 2014 Inter-Association Consensus: Year Round Football Practice Contact Guidelines and Independent Medical Care Guidelines. Many CMPs made general comments about student-athlete centered or safety-first approaches; however, specific plans to reduce head trauma were not presented frequently. This is a particularly important consideration, as later life neuropathologies (eg, chronic traumatic encephalopathy) may be related to repetitive head impacts independent of concussion diagnosis; in addition, a number of studies suggest measurable changes in outcome (eg, neuroimaging or cognitive changes) after a season of sports participation regardless of whether a concussion is diagnosed. 7,9,14,26,30,42,46,47 The quantification of head impacts could provide, at a minimum, general information about the frequency and magnitude of impacts in college football. 15 Several potential strategies have been discussed that could reduce head trauma, including reduced number of contact practices, helmetless tackling, 48 and educational interventions. 20 Furthermore, although football is the primary focus in the head trauma reduction section, repeated head impacts frequently occur in many other NCAA-sponsored sports, including soccer, ice hockey, lacrosse, and wrestling, but few CMPs identified plans beyond football. 13,25,27 Thus, future iterations of NCAA CMPs should encourage reducing head impacts in all sports.
Although sample populations do not directly align, this study found higher rates of concussion education than that found in previous studies. For example, in 2015 Kerr et al 19 indicated that 92% of their sample provided concussion education to student-athletes and 85.6% provided concussion education to coaches. In 2014, Baugh et al 5 found that 71.5% of Division I athletes received annual concussion education. Although the vast majority of schools in this study indicated within their CMP that they provide concussion education to the relevant stakeholders, the nature of this education and the education’s effectiveness in improving concussion knowledge, attitudes, and norms is unknown. 22 Previous research has indicated that even when concussion education is provided in accordance with NCAA policy, there is significant variation in both content and modality of delivery as well as questions regarding effectiveness. 22 Additional research is needed to understand how the concussion education provision is operationalized at the institutional level.
Examination of variation in overall rates of compliance as well as rates of compliance within the “Return to Learning” section, in which there was the most variation, yielded no significant predictors among those evaluated. That is, the number of full-time staff ATs, the conference of competition, a measure of academic prowess, and a measure of athletic competition were not associated with the level of CMP compliance at these schools. In Division II and III institutions, a smaller staff was indicated as a limitation for incorporating multifaceted CMPs, thus it was somewhat surprising there was no relationship between staff size and CMP compliance in this study. However, this may be due to the fact that the staffs at NCAA Power 5 schools are larger than those at smaller Division II and III programs. This may additionally reinforce previous findings of heterogeneity in concussion management practices across NCAA divisions of competition. 6 Alternatively, the small variation in compliance rates within the present sample may reduce our ability to detect statistically significant trends in CMP compliance related to staffing size.
In addition to the highly compliant medical management, there were multiple areas of improvement noted in the CMP review compared with previous self-reported responses. Specifically, herein 100% of CMPs indicated that the final determination for RTP was from the team physician or medically qualified physician designee, whereas in 2013 some respondents reported that either coaches (6.8%) or athletes (6.6%) could make the RTP decision. 5 Current NCAA guidelines call for athletic health care providers to have “unchallengeable autonomous authority” for the medical management of student-athlete injuries, including concussions. 39 Furthermore, there was 100% compliance with the recommendation that student-athletes suspected of a concussion be removed from participation, evaluated by the team physician or AT, and, if a concussion diagnosis is confirmed, removed for at least the remainder of the calendar day.
It is important to note that this review focused solely on the written CMP, and the results should not be extrapolated to the actual clinical concussion management practices; the actual practice could be better or worse than is currently stated. Indeed, several lawsuits have challenged the clinical management of concussions among NCAA member institutions, 3,23 and previously high-profile apparent violations of concussion management guidelines have been reported. 16 The review of CMPs can be seen as a step toward an overall enforcement by the NCAA to help protect the health and well-being of student-athletes. It is encouraging to see relatively high compliance across this subset of institutions; however, it is unclear the extent to which this compliance translates to the rest of the NCAA Division I institutions or to Division II or III, where this CMP review has not yet been implemented. To date, only this subset of the most competitive and well-resourced schools have to make their CMPs publicly available and have an NCAA-mandated review process for compliance. Moving toward a more uniform and equitable process across all levels of competition is an essential next step to ensuring compliance in concussion management for all schools and the athletes attending these schools.
Footnotes
Acknowledgment
The authors would like to acknowledge the assistance of Christine Flora, who assisted the research team with the data collection for this study.
One or more of the authors has declared the following potential conflict of interest or source of funding: T.A.B. has received research-related funding from the NCAA as part of the NCAA/DoD CARE consortium (award number W81XWH-14-2-0151). W.P.M. receives royalties from ABC-Clio Publishing, Springer, and Wolters Kluwer, and his research is funded in part by a grant from the Football Players’ Health Study, funded by the NFL Players’ Association, and philanthropic support from the National Hockey League Alumni Association through the Corey C. Griffin Pro-Am Tournament. C.M.B. receives research support from the Football Players’ Health Study, funded by the NFL Players’ Association; has received funding from the NCAA; and is supported by the National Institute of Mental Health (award number T32MH019733).
Ethical approval for this study was waived by the University of Delaware Institutional Review Board.
