Abstract
Objective
Citation analyses identify the most-cited publications in a given field, which aids in understanding areas of the literature that are well-developed and those where additional research is required. Our objective was to perform a citation analysis in concussion to understand the state of the science from a bibliometric perspective.
Design
We performed a keyword search for articles related to concussion in Harzing's Publish or Perish, which scrapes Google Scholar for citation metrics. This approach was used to identify the 50 articles with the most lifetime citations as well as the 50 articles with the highest citation rate.
Main outcome measures
Citations and citation rates.
Results
Per our citation analysis, we found that concussion guidelines are among the most cited publications (comprising ≥20% of each citation cohort), yet there is a dearth of widely cited clinical trials to inform them; only one randomized trial (studying the effects of rest following concussion) was included in our citation analysis. The majority of study designs (≥40% of each citation cohort) were cross-sectional. Concussion recovery and secondary complications of concussion were common study topics, with ≥20% of publications in each citation cohort focused on these issues. The publications included in our analysis were authored by 596 authors from only 12 countries, suggesting a lack of global representation in concussion research.
Conclusions
Existing reviews and consensus statements have called for additional, high-quality research in concussion; our citation analysis quantifies this need. Further, although concussion is a global problem with its incidence and burden increasing in the developing world, our citation analysis demonstrates that the most-cited and discussed articles in concussion are published by authors from only 12 countries. Going forward, to address the global problem that is concussion, a more global research perspective is called for.
Introduction
Globally, every year, millions of people experience a concussion. 1 The scale of this injury and its potential for long-term sequelae2–4 prompted the formation of the many international and national research taskforces aimed at advancing our understanding concussion and how to manage it.5–9 Multidisciplinary collaboration has also worked towards this goal, bringing together perspectives ranging from neurology to sports medicine to rehabilitation sciences to further concussion research. As suggested by these initiatives, and in general, research interest in concussion has heightened. This is perhaps best demonstrated by its peaking research output: a PubMed search of “concussion” shows an exponential growth in the number of publications on the topic from 2000 to 2020.
Despite this intensified research activity, there remain many unanswered questions about how concussion can best be prevented, diagnosed, managed, and treated. A review of widely-cited concussion consensus statements demonstrates that best practice in concussion management is still largely informed by moderate-level evidence, and in many sub-areas, calls for further high-quality evidence have been made in order to form definitive clinical guidelines.10–14 As this relatively young field has grown, new and more nuanced questions about concussion have inevitably surfaced that are only in the early stages of investigation. Therefore, while research activity in concussion has increased, there remain important knowledge gaps to be addressed by future research.
A more complete understanding of the state of the science can be achieved through a citation analysis, a bibliometric examination of all peer-reviewed publications on concussion. 15 Citation analyses can identify publications with the largest impact on the field, in terms of total number of citations, as well as which publications have the highest annual citation rate (a proxy for the current relevance and influence of a given publication).16,17 These analyses are also helpful for identifying which sub-fields are well-studied and have widely discussed research findings, and which ones are emerging. 18 To date, citation analyses have been performed in many neurological disciplines to better understand their respective literatures.19–25 Framing concussion literature by means of a citation analysis can complement existing guidelines and reviews by showcasing the papers with the most influence on this growing field.
To this end, we performed a citation analysis to identify the most cited studies, study designs, and study topics in concussion. While a citation analysis on sport-related concussion has been published, 26 we focused the present citation analysis on concussion broadly given the impact of this injury on the civilian and non-athletic populations, and to better understand the basic and clinical science of concussion at large. This study, therefore, provides a cross-sectional summary of the most cited articles in concussion, which may help inform future research questions and efforts.
Methods
We performed a search for articles on concussion with the most lifetime citations and the highest annual citation rate using Harzing's Publish or Perish, 27 which generates citation metrics based on data collected from Google Scholar. More specifically, we performed a title and keyword search in Publish or Perish using the following search terms: “concussion*” OR “concuss*” OR “mild brain injur*” OR “mild traumatic brain injur*” OR “mTBI*” OR “postconcuss*” OR “post-concuss*”. We did not place a restriction on publication date or language of publication. The search was initially run on June 1st, 2020, and was re-run prior to publication (on March 1st, 2022) to ensure recency.
Results from Publish or Perish were organized in order of most lifetime citations and were then exported into Microsoft Excel. A similar process was repeated for articles with the highest annual citation rate. For each of these citation cohorts, the top 100 articles were imported in order to reach a final list of 50 articles with the most lifetime citations and highest annual citation rate, as it was expected that some articles would have to be excluded for not meeting eligibility criteria (as defined below).
The two authors reviewed articles in each citation cohort against the exclusion criteria, which were: (1) non peer-reviewed publication, (2) irrelevant to concussion, and (3) focused on more severe forms of brain injury. Three articles (across both citation cohorts) were excluded based on this initial screen. After this initial screen, each cohort of articles was categorized, and information on the following was extracted into the data collection sheet independently by each author: journal, journal type (speciality vs. general), country of publication, international collaboration (whether this occurred or not), category/study topic, study design, outcomes studied, and special population (sports, pediatrics, etc.) studied. The authors then met to resolve discrepancies in coding. For the present review, there were five discrepancies that were resolved by consulting the original article and re-appraising the article and categorization in question.
Results
Altogether, 67 articles were included in our citation analysis; all but 17 articles in the total lifetime citation cohort overlapped with the 50 included in the highest annual citation rate cohort. These articles are listed in Tables 1 and 2, along with their citation metrics.
Publications on concussion (n=50) with the highest annual citation rate.
Publications on concussion (n=50) with the most lifetime citations.
Study citation metrics and demographics
The articles in the total lifetime citation and highest annual citation rate cohorts had been cited, on average, 1032.5 ± 653.9 and 935.4 ± 718.5 times, respectively. The average annual citation rates for articles in these two cohorts were, respectively, 77.0 ± 89.7 and 85.7 ± 85.6. These differences were not significant (p > 0.05), likely because of the overlap of 33 articles between citation cohorts.
In both citation cohorts, 12/50 (24.0%) of articles involved international collaboration, with most of the remaining studies being conducted within the United States. At the author level, we found that the 67 unique articles that were included in our study had 596 authors from 12 countries. (N.B. there were not 596 individual authors, as some authors were included on multiple publications.) Combined, nearly 90% of all articles were authored by individuals from the United States and Canada. A heat map of these 596 authors (generated using Heatmapper 28 ) is provided in Figure 1.
Distribution of authors (n = 596) of the publications included in our citation analysis: Legend: Black = 0 authors; Yellow = 1–9 authors; Orange = 10–19 authors; Pink = 20–99 authors; Red = 100 + authors.
When looking at each citation cohort individually, there were an average of 9.04 and 6.5 authors on each publication in the highest annual citation rate and total lifetime citation cohorts (a significant difference, p < 0.05). A breakdown of institutional affiliations of each author listed on each unique publication is provided in Table 3.
Institutional affiliations of authors.
Study topic and design
The studies included in this citation analysis were belonged to one of 9 distinct study topics, as summarized in Table 4. Consensus statements and studies on recovery as well as secondary complications of concussion were common in both citation cohorts.
Study topics of publications included in our citation analysis, by citation cohort.
Similar to study topics, the distribution of study designs was comparable across citation cohorts. Studies could be categorized into one of 7 study designs, as detailed in Table 5. Of note, both citation cohorts had more cross-sectional clinical studies (≥40%) than studies of any other design. There were more reviews (systematic, scoping, and narrative) in the highest annual citation rate cohort than the total lifetime citation cohort. The distribution of consensus statements was similar across citation groups, with 10 in the highest annual citation rate cohort and 11 in the most lifetime citations cohort. The number of randomized trials in the citation cohorts was also similar (with 2 and 1 per group, respectively), and surprisingly low.
Study designs of publications included in our citation analysis, by citation cohort.
Discussion
Our citation analysis of the 50 articles with the highest annual citation rate and the 50 articles with the most lifetime citation in concussion yielded 67 unique publications. Alongside systematic reviews and meta-analyses, this citation analysis provides another lens through which to review concussion literature. It creates an understanding of which topics in concussion are most studied, and which study designs form the evidence base for the most-cited–and thus most discussed and influential –literature in this field. This citation analysis also offered important insight into where research on concussion is being conducted.
International collaboration was not uncommon among the top-cited articles in concussion, with approximately a quarter of all articles (in either citation cohort) being a product of such collaboration. However, most of these international collaborations involved only developed nations. Further, all the authors listed on the publications included in our review had institutional affiliations in only 12 countries (Table 3). This lack of a global perspective in concussion has been recognized by others, 1 and is particularly important to address given that the burden of brain injury is highest outside of North America and Europe. 29 Further, in many developing nations (that were not reflected in the publications included in our citation analysis), the most common causes of injury differ (e.g. there is a higher incidence of documented motor vehicle collision-related injury than sport-related injury in developing nations), 29 which means that the presentation and management needs of concussion differ regionally. Even though the injury is the same, regional differences in terms of how these injuries are caused and may present warrant concerted research. Promisingly, however, a review of clinical trials on concussion (as per www.clinicaltrials.gov) shows a more worldwide distribution of concussion research, despite the persistence of a North American and European research hub. This global problem requires a global solution.
In regards to study topic, both citation cohorts were similar (Table 4). Each cohort had ≥10 publications on each of the following topics: consensus statement, recovery of concussion, and secondary complications of concussion (e.g. developing chronic cognitive impairment or mood-related sequalae). With respect to recovery of concussion, it is still mainly studied with symptoms as the primary end-point. Symptom management of course remains a leading concern in concussion management, though it is important to recognize that recovery can–and often should–be defined based on other outcomes, whether they be neurophysiological recovery (which may outlast symptom recovery) or on a more functional level, return to activity/vocation. 30 Though it may have been expected that the highest citation rate cohort (i.e. the most currently relevant studies) would employ alternate measures of recovery (e.g. functional neuroimaging), the lack of studies in this cohort that focused on such outcomes may reflect either a continued emphasis on symptom resolution or that such studies are, relatively, still in their early stages. Regardless, future research should consider a more multi-disciplinary set of outcome measures. Further, most studies were clinical, with a minority of animal studies that focused on the neurophysiology or biomechanics of concussion, suggesting a shift away from studying the basic science of the injury and towards a more patient-centric model.
In line with this, an examination of study design (Table 5) showed that only 1 and 2 animal studies were included in the highest annual citation rate and total lifetime citation cohorts, respectively. Most studies (20 and 25 for the aforementioned groups, respectively), were cross-sectional, examining patients from the acute to the chronic stages of injury. Longitudinal studies that track patients throughout their recoveries remain limited, which may be attributable to the greater investment and difficulty of conducting such studies. Further, in combination, review articles and consensus statements accounted for more than ≥40% of articles in either citation cohort. While this suggests that there is now a greater amount of evidence available for consolidation (corroborated by the overall increased research output in concussion), it remains somewhat concerning that only 2 articles in the highest annual citation rate group (and 1 in the total lifetime citation rate cohort) were randomized trials. Together, this suggests that while more evidence is being consolidated (through review or consensus statement), and these sources are being widely-cited, the evidence itself is not of the highest quality. Though there are other trials that are included in said reviews and consensus statements that are not among the most-cited publications on concussion, they are cited less often. Further, these reviews and trials themselves call for additional, high quality evidence to inform clinical decision making. Our citation analysis supports that notion.
Conclusion
Our citation analysis adds to the concussion literature by identifying influential articles in our field, namely the 50 with the current highest annual citation rate and the 50 with the most lifetime citations. Overall, there is a need for more high-quality evidence to inform clinical management in concussion, as well as additional outcome measures (in addition to symptom reports) to better understand concussion recovery. International collaboration in the field is not uncommon, but needs to include more representation from research centres outside of North American and Europe. Concussion is a global problem, and can best be solved with a global perspective.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
