Brain smart – Let's play safely: Evaluation of a concussion education program in schools

Abstract

Objectives

There are two objectives for this paper. First, to determine effects of a concussion education program in a local school board in terms of concussion-specific knowledge in children and confidence in identifying and managing concussion in adults. Second, to identify differences and similarities in concussion knowledge between children who participate in sports and those who do not and between children with a history of one or more concussions and those without.

Design

A cross sectional survey regarding concussion knowledge was distributed randomly to students and adults at both pre- and posteducation timepoints. A concussion education program was disseminated across the school board for students between the distribution survey timepoints. Following the education program, adults and students completed their respective post-test surveys. Chi-squared tests in SPSS determined the significance of between-group differences.

Results

All 17 adults (100.0%) who had received concussion education recently reported confidence in their knowledge of concussion management, compared to 35.7% adults who had not received education for over a year (p = 0.020). For students, all of whom completed the concussion education training between the pre- and post-tests, there were no significant differences in concussion knowledge scores between athletes and non-athletes (either in or outside of school) or between those with a history of concussion and those without. There were no significant changes in concussion knowledge between the pre- and post-tests, except for one question.

Conclusion

Concussion education programs increase confidence of concussion management protocols in adults involved in sport, but they require improvements to better support knowledge amelioration, particularly for target groups that are at high risk of sustaining another injury.

Introduction

A concussion is a mild traumatic brain injury induced by a biomechanical force, which can be a blow to the head, neck, or body causing the brain to move or rotate within the skull.^1,2 In recent years, more attention has been placed on the risks of concussion, specifically as they pertain to sports, which are the leading cause of concussion in young adults and youth. ³ Contact sports, particularly ice hockey, rugby, and American football, are among the sports being targeted for the long-term outcomes associated with sustaining multiple concussions, including neurodegenerative and mental disorders. ⁴ Resources now exist such that children and youth are provided with evidence-based information following injury,^5–7 which may increase concussion-specific knowledge for young people who have sustained one or more concussions. As individuals who have sustained at least one concussion are at high risk for sustaining multiple further injuries, ⁸ this education is important and timely.

An increased focus on the outcomes of concussions has spurred the development and implementation of concussion protocols and education programs by sports organizations, school boards, and governments. ⁵ Rowan's Law, a concussion safety act, passed in the Ontario

Government in 2018, is the first of its kind in Canada. The Act promises to ensure education on sports-related concussion (SRC) for athletes, coaches, and parents, and to monitor return-to-learn (RTL) and return-to-activity (RTA) strategies for patients with SRCs. ⁹ As noted, similar programs have been established in schools across the province. ⁵ Therefore, children who participate in organized sports activity, both in and outside of school, now have mandated exposure to concussion education resources, which aims to address the high risk of injury in this population.^5,9

Because concussion education programs are relatively new, there are many questions that must be answered regarding their effectiveness. Firstly, little is known about their ability to translate knowledge to participants and to incentivize behavioural change (e.g., reporting behaviours, RTL/RTA decisions), for students and school employees, who are responsible for overseeing management and maintaining accommodations in the classroom.^5,10 Greater concussion knowledge has been correlated with improved reporting behaviours in youth athletes, ¹¹ yet there is a lack of evidence supporting the fact that young athletes are sufficiently educated. Coaches similarly require concussion education because higher coach education has been hypothesized to improve athlete reporting behaviours. ¹² Higher levels of education have been found to correlate with increased adult confidence in RTA management and concussion identification, and again there is a paucity of evidence regarding the effectiveness for such programs in understanding of concussion from a coaching perspective. ¹²

Multiple studies have been conducted to investigate the effectiveness of educational interventions among teachers, coaches, and athletes, with varied findings and focuses.^10,12–14 This study it is the first of its kind in this jurisdiction. It was designed as a pilot test to determine the benefits and challenges associated with implementing a mandatory concussion education program. Secondary goals of the study include aiming to add to the growing body of literature by examining the effectiveness of educational interventions and knowledge translation activities for teachers, coaches, parents, and students in a local school board. Further analyses investigate the impacts of athlete versus non-athlete status and concussion history on concussion knowledge in students in this school board.

Methods

Two surveys, one for students and one for adults, were randomly distributed among seven eligible schools, both elementary and secondary that contained similar demographics (e.g., socioeconomic status, grade levels, geographic location), at the beginning of the study. Students in grades 3, 6, and 9 were eligible to participate, if they provided written informed consent, and all adults involved in school sports who provided written informed consent were similarly included. The study protocol was approved by the Hamilton Integrated Research Ethics Board. There were separate surveys for students and adults, both of which investigated the participant's involvement in sport and experience with concussion. The adult survey also asked questions about the organization’s concussion protocols, education programs, assessment strategies, and adult confidence. All adults who were asked to participate in the study were either employees of the school board or volunteer coaches. Confidence is defined in the survey as the self-reported comfort of each adult with respect to concussion identification and knowledge of management. Adults were asked to answer either ‘yes’ or ‘no’ as to whether they were confident in their abilities.

All students in the school board were required to complete a four-month concussion education program designed by the Ontario Physical and Health Education Association (OPHEA), implemented by the adults within the board. Completion of the course was mandated, and all schools in the board were successful in completing the program for their students. Not all adults were required to complete training (i.e., those who were not involved in the implementation of the program did not go through any concussion training). The education program was implemented in the time between the pre-training and post-training surveys and was mandated as part of all students’ required curriculum during that academic year.

The OPHEA presents several concussion-related documents, both for elementary and secondary schools, that address issues such as signs and symptoms of concussion, RTL accommodations, and RTA recommendations. ¹⁵ The goals of concussion education for school staff, as defined by the OPHEA, include gaining knowledge and understanding of concussion definition, signs and symptoms; identification and proper response to a possible concussion; mechanisms for supporting students in RTL and RTA; and ways to promote concussion prevention and education. ¹⁵ All of these points were included in the adult survey, while the student education program and survey focused on the first point (i.e., concussion definition, signs and symptoms). Both the surveys and the education program were designed with respect to the OPHEA guidelines.

Identical surveys were distributed randomly four months after the pre-education surveys among the school board following the longitudinal intervention to obtain a post-test measurement. Different schools and grade levels were included in the pre-education and post-education surveys, due to the convenience sampling model employed in the study. A school board representative distributed surveys in the order that schools completed the OPHEA program, until the number of participants was equal between the pre- and post-test. Therefore, the participants who completed the pre-test were not the same as the participants who completed the post-test. This was the first time that this education program was given in this school board, and all schools and grade levels completed the program, therefore the pre-education students had not been exposed to the program, while the entirety of the post-education sample had taken the program.

Analysis

Survey data were collected and managed using REDCap, the research electronic data capture tool hosted at McMaster University, ¹⁶ then exported to IBM SPSS Statistics, version 25, for analysis. Descriptive statistics were obtained for the demographic-type questions for the surveys. Chi-squared tests were completed to obtain p-values for hypothesized associations among the data. These hypotheses included:

Post-education concussion knowledge scores for both adults and students would be higher than pre-test concussion knowledge scores;

A higher proportion of adults with recent exposure (within one year) to concussion education programs would report more confidence in concussion identification and management, compared to adults with more distant exposure or no exposure to concussion education programs;

Students who participated in sports, either in or outside of school, would have higher concussion knowledge scores than students who did not participate in sports;

Students with a history of one or more concussions would have higher concussion knowledge scores than students without any concussion history.

Analysis was performed for both the pre-education surveys and post-education data separately, and then in comparison. Separate analyses examined demographic trends and overall scores on the knowledge portions of the survey. Data were compared to identify changes in knowledge scores between the pre- and post-education surveys.

Results

The student survey was completed by 475 participants, ranging from ages 11 to 20 years, mean age 13.1 years. This sample included 213 males (43.3%), 258 females (52.1%), and 4 others (0.8%). The majority of students played sports in school (n = 306, 64.4%) and outside school (n = 312, 65.7%) (Table 1). Athletes ranged from recreational to competitive level. This demographic information, which combines the pre- and post-education samples, plus statistics regarding the number of concussions sustained by each individual can be found in Table 1.

Table 1.

Child participant demographic information.*

	Student respondents, n	Student respondents, %
Mean age n (SD)	13.1 (SD = 1.07)	N/A
Male	213	43.3
Female	258	54.3
Other gender	4	0.8
Sports in school	306	64.4
No sports in school	164	34.5
Sports outside school	312	65.7
No sports outside school	161	33.8
No past concussions	371	78.1
1–2 past concussions, n (%)	83	17.5
3–5 past concussions, n (%)	15	3.2
6–10 past concussions, n (%)	5	1.1
Total respondents	475	100

*Percentages are not mutually exclusive.

A total of 61 participants completed the adult survey, many of whom fit into various and multiple roles within their schools; however, all of these individuals were involved with sports within their school. All adults completed the same survey, regardless of their role. These roles included coach (n = 54, 88.5%), teacher (n = 39, 63.9%), parent (n = 12, 19.7%), and referee (n = 2, 3.3%) (Table 2). Coaching experience ranged from one year to 40 years. This demographic information, plus statistics regarding the number of concussions observed by each coach in the past sport season can be found in Table 2.

Table 2.

Adult participant demographic information.*

	n	%
Coach	54	88.5
Teacher	39	63.9
Parent	12	19.7
Athletic Trainer	0	0
Referee	2	3.3
0 observed concussions	28	45.9
1–2 observed concussions	27	44.3
3–5 observed concussions	6	9.8
Total participants	61	100

*Percentages are not mutually exclusive.

Between the pre- and post-education samples, there were no statistically significant changes in student knowledge in all but one question (Table 3). This question, which required students to acknowledge that concussions can lead to serious memory loss, and long-term effects including difficulties with learning and thinking, was answered correctly by 261 students (93.9%) on the post-education survey, compared with 174 students (87.9%) in the pre-education survey, p = 0.008. The questions with the lowest number of correct answers were the same for both the pre- and post-education surveys. These questions required students to identify the characteristics of a concussion, define the signs and symptoms associated with concussion, and recognize how a concussion could be sustained (Table 3).

Table 3.

Correctly answered student knowledge questions.*

Question	Correct n, pre-test	Correct %, pre-test	Correct n, post-test	Correct %, post-test	P-value
What is a concussion?	160	80.8	214	77.0	0.663
Which of the following signs or symptoms may indicate a concussion?	149	75.3	202	72.7	0.273
A concussion can occur from any direct hit to the head, neck of jaw, a hit to the body or a sideways hit to the head?	156	78.8	228	82.0	0.431
It is important to watch a person after a suspected concussion, as the signs and symptoms may appear later?	188	94.9	258	92.8	0.613
It is OK to return to the game or physical activity on the same day after a concussion as long as the person feels OK?	191	96.4	257	92.4	0.181
Concussions can lead to serious memory loss, and long-term effects including difficulties with learning and thinking?	174	87.9	261	93.4	0.008
Everyone recovers from a concussion at different times?	187	94.4	257	92.4	0.571
Total population	198	N/A	278	N/A	N/A

*Percentages are not mutually exclusive.

Other analyses run based on student questionnaires identified no significant difference in concussion knowledge, as measured by survey scores, between students who played sports, either in or outside of school, and those who do not (Table 4). Similarly, there were no significant differences in concussion knowledge between students with no concussions and students with one or more concussions (Table 4). In all of these categories, the most common score was seven out of seven (49.1–56.1% of each group achieved this score) followed by six out of seven (17.6–22.2% of each group achieved this score), and five out of seven (11.4–22.4% of each group achieved this score).

Table 4.

Student concussion knowledge and demographics.*

Knowledge score (out of 7)	Total Population	0 (n, %)	1 (n, %)	2 (n, %)	3 (n, %)	4 (n, %)	5 (n, %)	6 (n, %)	7 (n, %)	P-value
Sport in school	306 (64.4)	1 (0.3)	2 (0.7)	3 (1.0)	8 (2.6)	22 (7.2)	35 (11.4)	64 (20.9)	171 (55.9)	0.169
No sport in school	164 (34.5)	3 (1.8)	1 (0.6)	1 (0.6)	5 (3.0)	7 (4.2)	37 (22.4)	29 (17.6)	82 (49.7)
Sport outside school	312 (65.7)	3 (1.0)	3 (1.0)	1 (0.3)	9 (2.9)	19 (6.1)	41 (13.1)	61 (19.6)	175 (56.1)	0.143
No sport outside school	161 (33.8)	1 (0.6)	0 (0.0)	3 (1.9)	4 (2.5)	9 (5.6)	31 (19.3)	34 (21.1)	79 (49.1)
Concussion history (one or more)	103 (21.7)	1 (1.0)	1 (1.0)	2 (1.9)	1 (1.0)	8 (7.8)	14 (13.0)	24 (22.2)	52 (50.5)	0.956
No concussion history	371 (78.1)	3 (0.8)	2 (0.5)	2 (0.5)	12 (3.2)	21 (6.7)	60 (16.1)	71 (19.1)	201 (54.0)

*Percentages are not mutually exclusive.

A higher proportion of adults who had attended a concussion education program reported confidence in their ability to identify and manage concussions, compared with those who had not. All 17 adults (100.0%) who had received concussion education recently reported confidence in their knowledge of concussion management, compared to 35.7% adults who had not received education for over a year (p = 0.020). Thirty (49.2%) adults reported not having attended a concussion education program within a year, while 29 (47.5%) reported that they had received concussion education recently (i.e., within a year). When asked whether they regularly used standardized assessment tools for identifying concussions, more adults who had received recent concussion education (12, 41.4%) responded “yes”, compared to adults who had not recently attended concussion education (6, 20.0%) (p = 0.346). Of the adults who reported never having attended a concussion education program, only one (7.1%) reported using standardized assessment tools (Table 5).

Table 5.

Adult concussion confidence and training, pre-test.*

		No. confident in their abilities of concussion management	% confident in their abilities of concussion management	No. not confident in their abilities of concussion management	% not confident in their abilities of concussion management	No. who regularly use standardized assessments	% who regularly use standardized assessments
When did you last have training in concussion education?	Within 1 month	1	100.0	0	0	1	33.3
	6 mos. ago	8	100.0	0	0	5	35.7
	1 year ago	4	57.1	3	42.8	6	50.0
	Over a year	5	71.4	2	28.5	5	29.4
	Never	4	40	6	60	1	7.1
P-value	N/A	0.020	0.02	0.06	0.06	0.346	0.346
Total participants	61	23	100	11	100	18	100

*Percentages are not mutually exclusive.

Discussion

The findings from this study demonstrate the effectiveness of current concussion education programs in influencing knowledge and in modifying adult confidence. In particular, students demonstrated significantly increased knowledge on only one question, a true or false question that inquired as to whether concussions can lead to serious memory loss and long-term difficulties with thinking and learning. This lack of change could be attributed to the fact that scores were relatively high on the pre-education survey and that different samples were studied for the pre- and post-education surveys. However, given that the adults within the school board were responsible for disseminating the education program, and more than half of those adults who took part in the study reported that they had not recently taken part in any concussion education, there is room for concern that the messages were not sufficiently understood or adequately presented by the program facilitators. This is especially apparent because the questions on the surveys were basic measures of understanding of concussions and suggests that the objectives of the program may not have been met in this iteration.

There were also no differences in the knowledge of athletes and non-athletes. This finding suggests that in spite of the additional resources and education that are mandated for students who participate in organized sports, there is no increase in concussion knowledge, compared to those available to students who are not involved in sport. This finding is worrisome considering the concussion protocols in place for most amateur sports organizations across the province of Ontario, many of which have been guided by the most up-to-date recommendations for concussion management and education, the 2016 Berlin Consensus on Concussion in Sport.^17–19 Given the increased risk of injury that comes with playing sports, this trend demonstrates the need for improvement of concussion education programs, which should be able to increase knowledge among individuals who are likely to sustain an injury. ³

Similarly, there were no significant differences in concussion knowledge between students who had sustained one or more concussions, compared with those who had not. This finding is also cause for concern because patients should be given access to concussion resources following an injury, if they seek medical care, and because they are at an increased risk of sustaining future concussions. ³ It is possible that the education intervention in this study covered similar topics to the content explored in education for athletes and students with concussion, thus accounting for the lack of differences that was expected between the groups.

Although the education program did not affect the knowledge of survey participants, adults who had participated in an education program within the last year reported greater confidence in their knowledge of concussion management and improved familiarity with concussion protocols. While not statistically significant, there was a greater proportion of adults who had completed concussion education reported regular use of standardized assessments than those who had not. Both pieces of evidence suggest that education for coaches may be integral in creating an understanding of concussion protocols within an organization and the role of the coach in carrying out the organization’s guidelines (i.e., going beyond a simple awareness that such protocols exist). These gains, however, do not carry over to a coach’s general knowledge of concussion, as evidenced by a lack of change in scores between adults who had attended a concussion education program recently, and those who had not. This finding is interesting, as it suggests that coaches attend education sessions with the goal of growing their confidence in management, instead of strictly increasing knowledge of signs and symptoms.

Findings from other studies regarding concussion knowledge following the implementation of legislation are varied, which may come, in part, from the variations between concussion education programs. For example, two reviews by Tomei et al. and Mrazik et al., respectively, demonstrate that most concussion education is targeted at individuals who are involved in sports.^20,21 The current study investigated the effects of concussion education on all students, including both athletes and non-athletes, and the literature is limited in this regard. Still, there is some evidence that can be used to analyze the results of the current study. In particular, while there are reports of increased knowledge immediately following a concussion education program, studies have shown mixed results in the long term. ²¹ As the current study employed a longitudinal education program, the post-test survey may not have assessed the short-term gains of study participants. Furthermore, discussions on the effects of American legislation and education programs have shown significant variability – from no changes in knowledge to significant differences – which often has to do with logistical differences in various settings (e.g. access to educator training resources, interest from the staff carrying out the education), ²² and these factors may have influenced this study’s effectiveness.

There are limitations to this research that must be addressed. While the post-education surveys were distributed in the same school board as the pre-education surveys, and all students took part in the concussion education intervention, distribution was random. These decisions were made given the nature of the pilot study design of the project, and the anonymized survey sampling which prohibited researchers from performing the post-test surveys with the individuals who completed the pre-test surveys. However, these choices did mean that different schools and different grades were surveyed for the pre- and post-education surveys, thus any changes (or lack of changes) between the two surveys cannot be conclusively attributed to the education program. Ultimately, these conclusions are weak, although the study was successful in completing the pilot project and establishing this program in the local school board. The authors recommend that future studies implement a pre-test and post-test study design among the same general samples (i.e., the same schools and grade levels) to control for more variables.

Furthermore, there was a small sample size for adults, potentially limiting the study’s external validity and signifying an underpowered study. Due to the small sample size, it was difficult to appropriately determine differences between groups, and thus analysis could not examine trends between different types of adult participants. The research team did not determine a priori what minimum sample sizes would be required to reach statistical significance in either student or adult groups, which may explain why there were so few statistically significant results.

Lastly, employing the cross-sectional study design required making several assumptions. In particular, the research team had to assume that all students who participated in the survey had received equal exposure to the intervention by the post-education survey, which may not have been true if some students had had prolonged absences from school or if different schools dedicated different amounts of time to this education. Similarly, while adults who imparted the OPHEA program were required to complete training for the role, it is possible that the training was inadequate, or the OPHEA training was deficient for another reason. With random survey distribution, the goal was to account for individual differences; however, systematic differences in the amounts of time or energy dedicated to this education program between schools were not accounted for in the study design.

The current study may be applicable to other schools in Ontario, as they also begin to implement concussion education programs. In particular, the educators of this concussion program were employees of the schools, and their training for carrying out the program consisted of high quality and informative modules on a website that is freely available to all Ontarians. This applicability raises concern that other programs may similarly fail to impart a change in knowledge score. As such, designers of concussion education programs should examine the strengths and weaknesses of this, along with other, preexisting programs, and more research should be conducted to determine what works well in such programs and what does not facilitate learning.

There is a need for future research on the topics covered by the current study, as several trends are identified regarding the effectiveness of concussion education programs. Research should be conducted to identify the optimal method by which to impart knowledge, as the program examined in the current study was not successful in this respect. With Rowan’s Law being introduced to the province and emphasizing concussion education for athletes, ⁹ future research should examine how the Act, and others similar to it, contributes to athlete and coach knowledge and management of concussions, as it may induce changes to the current landscape of concussion education programs and research. More attention should be paid to ensuring that high-risk groups (athletes, individuals with concussion histories) are exposed to concussion-related education that may aid in the prevention of future injuries. Therefore, it would be of interest to investigate differences in concussion knowledge between athletes in contact and non-contact sport, as well as across different sporting levels, who may have different risks of concussion in their sports. It would also be worthwhile to investigate whether gender differences exist in concussion knowledge. Lastly, the finding that education programs increase confidence in adults should also be examined in more detail to gain an understanding of what elements of an education program increase confidence and how those components could be modified to enhance knowledge translation.

Conclusion

Concussion education programs are relatively new, and there is still much to be learned about their effectiveness in improving knowledge and confidence and incentivizing behavioural changes. The current study demonstrated that education programs can effectively improve confidence in identifying and managing concussions, for adults in particular. However, the study also found no significant changes in the knowledge of their participants following a concussion education program, and no differences between the concussion knowledge of athletes versus non-athletes or that of students with no past concussions versus those with one or more. These findings are worrisome in that the individuals who are at highest risk for concussion do not demonstrate a better understanding of the injury that their lower risk counterparts. It is also possible that the disappointing results were due to a combination of errors in the study design and a poor distribution of materials to students. The current study suggests that more research should be conducted to determine the characteristics of an optimal education program.