Abstract
Purpose
The Glasgow Coma Scale (GCS) is the gold standard in assessing severity for traumatic brain injury (TBI). This study aimed to compare GCS-rated severity to caregiver-perceived severity. Researchers assessed levels of agreement and explored which factors explained any differences and predicted whether a GCS score was provided.
Methods
Caregiver-perceived severity, demographic, and medical information was collected via electronic survey from 107 youth with TBI. GCS-rated severity was collected via medical-chart review.
Results
The strength of agreement fluctuated by severity (mild/concussion k = 0.60, moderate k = 0.1, severe k = 0.64). Caregiver perceptions were more severe in 62% of cases. No medical or demographic variables explained discrepancies. GCS was documented in 38% of medical records. Binomial regression modeling found that participants seen in the emergency department were most likely to receive a GCS rating.
Conclusion
Results suggest that caregivers may be reliable evaluators of their child's TBI severity, particularly for those classified as mild or severe injuries with GCS ratings. Allowing caregiver-perceived severity ratings could increase inclusivity in research, so participation is not limited to children who go to the emergency department. More research is needed to determine if caregiver-perceived severity tracks with long-term outcomes or if caregivers base severity ratings on factors outside of the initial injury event.
Introduction
A traumatic brain injury (TBI) is caused by a bump, blow, or jolt to the head that disrupts the brain's normal functioning. 1 TBIs may disrupt a young person's development, with cascading, long term effects on the child's health, cognition and social functioning.2–4 Pediatric TBI represents a significant public health concern, with estimates as high as 640,000 emergency department visits annually in the US. 5
The Glasgow Coma Scale (GCS; Teasdale and Jennett 6 ) is the primary tool for categorizing adult and pediatric TBI severity. Before standardized severity ratings, providers were unable to reliably monitor their patient's changing status and unable to predict the degree and duration of the altered consciousness.6,7 The GCS evaluates patients in three areas: eye response (from closed to spontaneous), verbal response (from no audible responses to orientated responses), and motor response (from no movement to obeying motor commands). Higher values in each area are indicative of higher function, and the points are summed with the total score indicating the severity of injury. Total GCS scores between 3 and 8 indicate a severe TBI, 9 through 12 indicate a moderate TBI, and 13 to 15 indicate a mild TBI. With advanced understanding of mild TBI, a GCS score between 13 and 15 that includes positive imaging findings is considered a mild-complicated injury. 8 While other severity measures (e.g., Injury Severity Scale; Baker et al. 9 ) exist and research indicates additional variables also contribute to severity of injury (e.g., duration of posttraumatic amnesia, positive findings on brain imaging), GCS remains the most used severity measure in pediatric and adult research studies. 10
The usefulness of GCS in predicting patient outcomes appears stronger when considering outcomes closer to the time of injury. 11 This is especially true in acute settings. For example, a study across pediatric emergency departments found that GCS was inversely related to the need for medical or neurosurgical intervention, such that a lower GCS score (poorer functioning) was related to an increased need for medical intervention. 12 Similarly, when building a mortality prediction tool for pediatric TBI, researchers found GCS to be a significant predictor of mortality in the pediatric intensive care unit. 13
On the other hand, the usefulness of GCS is less consistent when predicting long-term outcomes. Just one year post-injury, a study of young people and adults found that survival with moderate/severe disabilities was equally likely across all GCS-rated severities (47% in mild TBI, 45% in moderate TBI and 48% in severe TBI). 14 In a study examining long-term outcomes (10 + years) for youth with severe TBI (GCS = 3 or 4), 15% of participants experienced only minor deficits that did not impede their resumption of normal life. 15 Alternatively, children with mild GCS scores have been found to have a substantial risk for long-term disability. 16
Ultimately, the pursuit of understanding long-term outcomes of pediatric TBI is undermined by the short-term reliability of GCS alone. A reliance on GCS as the primary measure of TBI severity artificially reduces the heterogeneity of the TBI population included in research, because studies often require a GCS rating amongst their inclusion criteria. Patients with mild TBIs or those injured at home are less likely to seek medical care. 17 A study within a large children's hospital revealed that only 11.7% of children with a concussion who sought medical care presented to the emergency department, 18 and it is unclear if GCS is reported reliably in other medical settings like urgent care or outpatient clinic visits. If these persons are not included in research studies because they lack a GCS rating of severity, research literature may not reflect diverse outcomes across all TBI experiences. These gaps indicate that additional means of classifying pediatric TBI severity are needed to better assess long-term outcomes and allow for more inclusive representation in research.
It is not yet known if caregivers are reliable reporters of the severity of their child's TBI. However, caregivers (especially those of young children) frequently serve as proxies to provide details related to their child's injury and its impact. Caregiver report is currently used to establish a history of a TBI 19 and determine longer term outcomes including quality of life, 20 executive functioning, 21 and psychosocial functioning. 22 However, one recent Canadian study found poor agreement between medical records and caregiver-reported TBI (kappa = 0.161). 23
To expand the understanding of severity ratings for pediatric TBI, the current study examined how caregiver-perceived severity of injury compared to GCS-rated severity of injury in the electronic medical record of children with TBI. Researchers posed the following questions: 1) Does caregiver-perceived severity of injury match the GCS-rated severity of injury? 2) When differences exist, are there factors that help to explain these differences? 3) What factors predict when a GCS score is assigned?
Methods
This study was approved by institutional review boards prior to the initiation of activities, and all parents provided informed consent.
Eligibility
The participants in the current study were recruited as part of a larger, longitudinal study examining return-to-school supports for children with TBI. Eligibility for this umbrella study included English-speaking students who were currently enrolled in K-12 schooling (elementary, middle, high school in the US), and a history of at least one TBI. Students who experienced multiple TBIs or who had pre-morbid diagnoses (e.g., autism, attention-deficit/hyperactivity disorder) remained eligible. Participants were recruited through relationships with children's hospitals, school programs, children's medical centers and social media. For the current study, only participants recruited from a specific children's hospital were included, as the informed consent at this site allowed researchers to access hospital data within the electronic medical record, including GCS scores.
Recruitment procedures
Recruitment for the umbrella study at this children's hospital was conducted under a partial HIPAA waiver. Research staff identified potential participants using the hospital's daily trauma report, neurosurgery patient list, and relevant clinics (e.g., physical medicine, sports medicine, and complex concussion clinics). Research staff contacted the caregivers of eligible patients. Caregivers who wanted to participate would provide an initial verbal consent. Study data were collected and managed using REDCap electronic data capture tools, a secure, web based application designed to support data capture for research studies. 24 The provision of verbal consent triggered the release of the first set of electronic surveys through a personalized REDCap link. Through this link, caregivers completed a formal consent, intake surveys, questionnaires, and assessments. The current study only utilized data from the intake survey, in which caregivers provided demographic information as well as information related to the nature of their child's TBI.
Measures
Caregiver-perceived severity
On the intake survey, caregivers were asked a multiple-choice question about their child's “severity of injury” with the following potential responses: 1) mild/concussion, 2) moderate, 3) severe, and 4) was not told/informed. Option 4 was not selected by any of the participants in the current study.
Mechanism of traumatic brain injury
Similarly, caregivers were asked to categorize the mechanism of their child's injury. They were first asked if the injury occurred in a sports-related context (yes or no). Caregivers then categorized their child's injury into one of six categories: 1) assault, 2) car accident, 3) fall, 4) struck by or with object, 5) other, 6) unknown. There were no instances of caregivers reporting that the mechanism of injury was “unknown.” In cases where caregivers selected “other,” they were asked to describe how the injury occurred. Based on the data dictionary created for this study (Supplementary Material), researchers reviewed comments and categorized the injury into one of the four categories listed above.
Highest acuity of care
Caregivers were asked if they sought medical care after their child's injury (yes or no). For those who did pursue medical care, caregivers could select any/all locations: 1) pediatrician, 2) urgent care, 3) emergency department, 4) specialized clinic (e.g., concussion clinic). Higher acuities of care represent a level of care equipped for more severe and immediate needs and with more specialized staff and resources. By this standard, an emergency department represented the highest acuity of care, followed by urgent care, and finally a combined category of pediatrician/specialized clinic.
GCS-Rated severity
Three trained research coordinators completed medical records screening, recruitment, and intake interviews for all participants enrolled from this study site. Given a partial HIPAA waiver, part of the recruitment process was to collect eligibility information prior to contacting a family. When research coordinators noted a GCS score in the chart, they recorded it in the REDCap intake survey. To confirm reliability of the reported GCS scores and explore for those that were not included in the initial REDCap survey, a researcher (last author) completed a medical record review for all participants with completed intake surveys. Examples of records reviewed include notes from the emergency department, clinic notes, and consults from specialty physicians (e.g., trauma, neurosurgery, physical medicine) or other clinicians (e.g., speech-language pathologists, neuropsychologists). For purposes of this study, hospital-reported GCS rating was only recorded for instances when a specific score was included within the medical record. Since one objective of this study was to examine the use of GCS reporting following TBI, severity descriptions made only in a narrative description (e.g., “concussion” or “moderate TBI”) were not included.
Data analysis
Fleiss’ Kappa statistic examines the level of agreement between two raters.25,26 In the current study, the Fleiss kappa statistic assessed the inter-rater agreement between GCS-rated severity (determined by medical providers and listed in the electronic medical record) and the caregiver-perceived severity collected at study intake.
To identify factors that might explain discrepancies between caregiver-perceived severity of injury and GCS-rated severity, researchers used the Fischer's Exact test. The independence of each of the selected dependent variables (i.e., sex, mechanism of injury) with the independent variable (agreement between caregiver-perceived severity and GCS-rated severity) was examined. The same variables were used to find factors that could explain the nature of the discrepancy in cases of a mismatch between GCS-rated severity and caregiver-perceived severity.
A binary logistic regression was used to determine which variables predicted whether a GCS score would be included in a child's medical record. To select variables for inclusion in the regression, researchers first evaluated their associations with receiving a GCS score using either a chi-square test or Fisher's Exact test. The chi-square was used for all variables that had categories large enough to meet the necessary assumptions. For variables with less than the necessary counts per category, a Fisher's Exact test was used. Based on these analyses, the variables considered included sex, ethnicity, academic grade at injury, mechanism of injury, caregiver perceptions of severity, and highest acuity of care.
Results
Participants
A total of 107 participants were included in this analysis. The demographic characteristics of the participants are presented in Table 1. The average age of injury for participants was 10.89 years (standard deviation [SD] = 4.29, range = < 1–17 years). Sixty-two participants were male (57.9%) and 45 (42.1%) female. The majority (69.2%) of participants were white, 18.7% African American, 8.4% multiracial, 1.9% Hispanic/Latinx, and 1.9% Asian. Of the 107 participants, 38% (n = 41) had a GCS-rated severity. Of the 41 participants with GCS-rated severity, caregiver-perceived severity matched the GCS-rated severity in 28 cases (68%), leaving 13 cases where severities did not match. Table 2 shows the specific discrepancies for these 13 cases.
Demographics for each group.
Mean value.
Standard Deviation.
Sample size.
Cases with a discrepancy between caregiver-perceived and GCS-rated severity.
GCS scores: 3–8 = Severe, 9–12 = Moderate, 13–15 = Mild.
Tables 3 and 4 present injury mechanism, severity, and medical care information by group. Based on caregiver-perceived severity, 55 (51.4%) participants experienced a mild TBI, 22 (20.6%) sustained moderate TBIs, and 30 (28%) were severe. The average time-since-injury (the time between when the child sustained their TBI and when the caregivers were completing the intake survey) was 0.94 years (SD = 2.15, range = 0.02–11.6 years, 25th percentile = 0.14, median = 0.20, 75th percentile = 0.45). This large standard deviation can be explained by the inclusion of several outliers with times-since-injury as high as 11 years. An emergency department was the highest acuity of medical care sought in the majority of cases (76.6%). Pediatrician/specialized clinic was the highest acuity of care in 15% of cases and urgent care was the highest level of care sought in 8.4% of cases.
Injury information and level of care for participants based on severity-reporting groups (caregiver only and GCS + caregiver). Data is n (%) unless otherwise indicated.
Severity depicted in this table is exclusively the caregiver-perceived severity.
Sample size.
Mean value.
Standard Deviation.
Mechanism of injury for each group. All data is na (%).
Sample size.
How does caregiver-perceived severity of injury compare to GCS-rated severity?
Researchers used Fleiss’ Kappa to assess the agreement between GCS- and caregiver-perceived severity. Results demonstrated a moderate agreement, k = 0.50, 95% confidence interval (CI) [0.27, 0.72], p < 0.001. The kappa results for the individual severities were as follows: mild/concussion k = 0.60, p < 0.001, moderate k = 0.10, p = 0.51, severe k = 0.64, p < 0.001.
What factors explain discrepancies between caregiver-perceived severity of injury and GCS-rated severity?
Researchers used the Fisher's exact test to examine the associations between child/injury variables and whether GCS-rated severity matched caregiver-perceived severity for the 41 children who had both ratings. There was no statistically significant association found with “matching severity ratings” for school level at injury (preschool, elementary, middle/high school) (p = 0.14), highest acuity of care (p = 0.32), sex (p = 1), race (p = 1), mechanism of injury (p = 0.15), or GCS-rated severity (p = 0.16). Caregiver-perceived severity was the only variable that was associated with “matching severity ratings” at levels of statistical significance (p = 0.003). This reinforces earlier findings that the likelihood of discrepancies varies at different severity ratings.
With only 13 cases in which caregiver perceptions and GCS scores differed, the sample size was too small for statistical analysis to identify factors associated with these discrepancies. Instead, descriptive analysis was utilized and found that in 62% (n = 8) of cases, GCS-rated severity was less severe than the caregiver-perceived severity.
What factors predict the assignment of a GCS score?
Researchers assessed potential variables to include in the model by chi-square or Fisher's exact tests. Results revealed that only highest acuity of care was significantly associated with whether a GCS was provided. Children who sought care in an emergency department were significantly more likely to receive a GCS-rated severity (p = 0.017) than children who sought care elsewhere. A binomial logistic regression analyzed the effects of acuity of care on the likelihood that participants were provided a GCS score. The model correctly categorized 61% of cases and described 24.6% (Nagelkerke R2) of the variance in the provision of a GCS score.
Discussion
The overarching goal of this study was to determine if caregivers can reliably report the severity of their child's TBI, compared to GCS-rating. GCS has been the standard for assessing TBI severity since its creation in 1974. 6 If GCS is considered the “gold standard,” then the moderate agreement of interrater reliability between caregiver-perceived and GCS-rated severity suggests that caregivers are reasonable assessors of pediatric TBI severity. For mild and severe injuries specifically, the agreement between caregiver perceived- and GCS-rated severity is considered “substantial,” further suggesting that in the absence of a GCS score, caregiver-perceived severity may be a reasonable substitute. However, this agreement invites further study into the nuances of different severity ratings and how these different severity ratings track with post-injury outcomes.
Caregiver-perceived severity was the only variable to be associated with discordance between ratings. It was less likely to agree with GCS-rated severity for injuries that the caregivers perceived as moderate. Caregivers were also more willing to assign a moderate severity than GCS ratings indicated. This could suggest that caregivers perceive “mild” or “severe” as being too extreme to represent their child's injury experience. A “moderate” rating may be seen as more reasonable, allowing for a wider representation of experiences. However, these same findings could also suggest that caregivers can adequately recognize severity extremes but are less adept at recognizing the nuances of a moderate injury. More research is needed to determine caregiver perceptions of each severity level, and the factors caregivers consider when evaluating the severity of their child's injury.
In cases where a discrepancy between caregiver-perceived and GCS-rated severity exists, caregivers were more likely to assign a higher severity rating. There is no research available to determine the factors caregivers are considering in their assessments of TBI severity, but research related to other medical diagnoses provides some preliminary guidance. Research comparing caregiver-perceived severity of eczema to physician-rated severity suggests that parents look to different factors when assessing severity and that a parent's unique access to their child's daily life may provide a better understanding of the severity of their child's condition. 27 A study comparing parent-rated severity of respiratory tract infections to physician-rated severity also found that parents were more likely to provide higher severity ratings. 28 Parent and physician severity ratings were influenced by different factors. Parents were influenced by their child's symptoms (e.g., cough) while clinicians were more influenced by the results of their examination (e.g., lung sounds on auscultation). Similar work in the autism community suggests caregiver severity is influenced more by the degree to which the child's autism impacts the family than the actual autism symptoms themselves, as rated by clinicians or professionals. 29 A qualitative study in febrile illness found that parent-rated severity was based on the level of deviation from the child's typical behavior and the presence of physical features. 30 In the case of pediatric TBI, more research is needed to determine the factors that influence caregivers’ perception of severity, whether it be the acute injury or its later effects on the child or family. This is an important questions for researchers and clinicians to consider as the severity rating at the time of injury may not consistently predict severity of impairments years after the injury occurs (e.g., Horn et al.). 31
This study also sought to determine what variables predicted whether a child would receive a GCS score. Highest acuity of care was the only variable to be significantly associated with GCS assignment, where children who received care in the emergency department represented the largest category and reference group of the model. While 98% of participants with a GCS score went to the emergency department, 51% of participants without a GCS score also went to the emergency department. The results suggest that while GCS is most likely to be administered in the emergency department, it is not consistently provided there. The inconsistency of GCS reporting aligned with previous research that also reported the presence of a GCS-severity rating to be associated with care referral. 32 A study of a large national data set revealed pediatric TBIs were less likely to be reported in states with less access to insurance/medical care. 33 By way of GCS, disparate access to medical resources could mean disparate access to research participation as well. The results of this study reveal a profound gap in care and a limiting factor in research participation.
Limitations
The results of this study should be considered in light of its limitations. Because patients were recruited from both outpatient and inpatient settings, there is some variability between when the child's injury occurred and when the family began participating in the study. In a study of patients with eczema, discordance in parent and physician ratings grew as time since the diagnosis increased. 27 A caregiver's assessment of the severity of their child's TBI made shortly after the injury is likely based primarily on the injury event itself or its immediate effects. As more time passes, caregivers may consider additional factors. 34 Caregiver perception of the severity of their child's TBI may change with time and further research should examine the factors that shape a caregiver's perception of their child's injury severity. Additionally, willingness of families to participate in the umbrella study may be a product of increased concerns around their child's TBI. This exposes the study sample to a potential selection bias. This study cannot account for the context in which caregivers considered the severity of their child's TBI. Some caregivers may have felt it more accurate to relay a severity if they were given one by a medical provider than to provide their own, though this also requires recall from the time of the injury. A small sample size limits the generalizability of these results, but also reinforces the need to expand research inclusion to include persons who are less likely or unable to seek medical care following a TBI. While beginning with a sample of 107, less than half had a GCS rating, limiting the ability to compare ratings. This exemplifies the difficulties of depending on GCS ratings for study inclusion.
Conclusion
The current reliance on GCS for research and outcome measurement may be limiting the understanding of the heterogeneous impacts of TBI on the developing brain. Most youth do not seek care within the emergency department 18 and the results of this study suggest that even those who do may not be given a GCS rating. Caregiver-perceived severity may be a reliable replacement in the absence of GCS and should be considered as a valuable means to increase inclusion in research studies. Further work should also consider the factors on which caregivers base their impressions of a child's severity of injury. To better understand outcomes of pediatric TBI, research protocols must be expanded to incorporate the heterogeneity that exists in both acute and chronic outcomes.
Supplemental Material
sj-docx-1-prm-10.1177_18758894251385506 - Supplemental material for Assessing the validity of caregiver-perceived severity in pediatric traumatic brain injury: A comparative analysis with the Glasgow Coma Scale
Supplemental material, sj-docx-1-prm-10.1177_18758894251385506 for Assessing the validity of caregiver-perceived severity in pediatric traumatic brain injury: A comparative analysis with the Glasgow Coma Scale by Sophia Nichols, Angela H Ciccia and Jennifer P Lundine in Journal of Pediatric Rehabilitation Medicine
Footnotes
Acknowledgements
The authors gratefully acknowledge STATBI research assistants and the families who participated in this work.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study is supported by the Centers for Disease Control & Prevention (CDC), grant number RFA-CE-19-003, Evaluation of Return to School Programs for Traumatic Brain Injury. The authors have no other financial or non-financial conflicts to report.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
