A Systematic Review of Reported Outcomes in Randomized Controlled Trials Targeting Early Interventions in Moderate-to-Severe Traumatic Brain Injury

Abstract

Traumatic brain injury (TBI) is a leading cause of death and disability worldwide. Randomized controlled trials (RCTs) are the cornerstone to evaluate the efficacy of an intervention. To assess the methodology of clinical research, we performed a systematic review that evaluated the different outcomes used in RCTs targeting the early phase of moderate-to-severe adult TBI from 1983 to October 31, 2023. We extracted each outcome and organized them according to the COMET and OMERACT framework (core area, broad domains, target domains, and finally outcomes). A total of 190 RCTs were included, including 52,010 participants. A total of 557 outcomes were reported and classified between the following core areas: pathophysiological manifestations [169 RCTs (88.9%)], life impact [117 RCTs (61.6%)], death [94 RCTs (49.5%)], resource use [72 RCTs (37.9%)], and adverse events [41 RCTs (21.6%)]. We identified 29 broad domains and 89 target domains. Among target domains, physical functioning [111 (58.4%)], mortality [94 (49.5%)], intracranial pressure target domain [68 (35.8%)], and hemodynamics [53 (27.9%)] were the most frequent. Outcomes were mostly clinician-reported [177 (93.2%)], while patient-reported outcomes were rarely reported [11 (5.8%)]. In our review, there was significant heterogeneity in the choice of end-points in TBI clinical research. There is an urgent need for consensus and homogeneity to improve the quality of clinical research in this area.

Introduction

Every year, approximately 69 million individuals sustain a traumatic brain injury (TBI) worldwide. ¹ In Europe, the annual incidence ranges from 262 to more than 1,000 cases per 100,000 people, ^{1 –4} and leads to 82,000 deaths. ⁵ TBI survivors experience significant disability: neurocognitive dysfunction such as memory or attention loss ⁶ and neuropsychological difficulties with mood or sleep disturbances. ^{7 –9} Certain chronic health issues such as disinhibition, apathy, or anosognosia ^{10 –12} are not necessarily visible in the patient’s environment but have major consequences on the patient’s quality of life and social functioning. ^{13 –15} TBI induces major public health costs ^16,17 and has a significant impact on the life of the patient’s family. ^18,19 To decrease disability and improve patient outcome, multiple randomized controlled trials (RCTs) assessing different interventions have been conducted in TBI. However, numerous RCTs have led to negative results. ^20,21 Despite efforts in transparency and reporting in clinical research, it has been suggested that the lack of consistency between studies in the choice, relevance, and reporting of outcomes can bias results and lead to negative results. ^22,23 Moreover, it has been recently proposed by several initiatives that outcomes in RCTs should also focus on patient priorities so that clinical research can be meaningful to both patients and clinicians. ^24,25

To the best of our knowledge, there is no international consensus on what outcomes should be used and how they should be reported in RCTs focusing on moderate-to-severe TBI. The aim of this study was to perform a systematic review of the literature, to describe the outcomes used in RCTs, and assess interventions in the early phase of moderate-to-severe TBI.

Methods

We performed a systematic review of the literature to identify, classify, and compare outcomes reported in RCTs performed in moderate-to-severe TBI patients. Details of the protocol for this systematic review were registered on the International Prospective Register of Systematic Reviews PROSPERO (number 2020 CRD42020194273) and can be accessed at https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42020194273.

Search strategy

We searched the PubMed database to identify RCTs performed in the early phase of moderate-to-severe TBI. Articles from June 1, 1983 (first identified RCT), to October 31, 2023, were reviewed. The search terms included “Traumatic Brain Injury,” “randomized controlled trial” [MeSH Terms] and excluded “Pediatrics” [MeSH Terms] OR “Adolescent” [MeSH Terms] patients. The complete algorithm is available in the Supplementary Data S1 section.

Types of participants and studies

Adult patients ≥18 years were included. The definition of moderate-to-severe TBI was as follows: a baseline Glasgow Coma Score ≤12 ^21,26 and/or hospitalization in an ICU ≥24 h and/or invasive mechanical ventilation ≥24 h. Studies including both mild and moderate-to-severe TBI were included in the review only if moderate-to-severe TBI represented at least 50% of the included patients. Only RCTs describing interventions targeting the early phase of TBI were selected. An early intervention was defined as an intervention performed in the first 2 weeks after TBI at the latest. The location (e.g., pre-hospital, emergency ward, critical care unit, neurosurgical ward) and the time to the intervention had to be clearly assessable by the reviewers. All types of interventions were eligible (e.g., surgery, extracorporeal devices, pharmacology, endovascular radiology). We retrieved full texts in English.

Eligibility of studies and classification or outcomes

Three reviewers (T.D., Y.D., and R.C.) independently assessed all titles. Abstracts and then full texts of potentially relevant RCTs were assessed independently by two reviewers (T.D. and Y.D.) and any disagreement on the eligibility was resolved through consultation of a third reviewer (R.C.) to reach a consensus. After abstract selection, the full texts of the articles were retrieved and the three reviewers checked for eligibility. Two reviewers (T.D. and Y.D.) reviewed all the outcomes and classified them into domains according to a pre-validated framework. ^25,27,28 In case of discrepancy, a third reviewer (R.C.) was consulted to reach a consensus. Furthermore, R.C. reviewed the final organization and classification of domains and outcomes (see framework below). The data extracted were as follows: first author, year of publication, participating countries, sample size, mean age of participants, study duration, type of intervention, and outcomes used. Data were recorded in a Microsoft Excel^® file.

Framework and classification of outcomes

We collected all outcomes that were used in both the primary and secondary end-points. We organized these outcomes according to a previously published framework developed by the OMERACT group (Outcome Measures in Rheumatology) and recommended by the COMET (Core Outcome Measures in Effectiveness Trials) initiative. ^25,27,29 This framework describes five core areas, which are the first layer of classification: death, physiopathological manifestation, life impact, resource use, and adverse events. The second layer of classification proposes to organize outcomes in core areas into broad domains. According to the COMET initiative, ^25,27 there is a consensus on 38 broad domains in clinical research. For instance, in the physiopathological core area, common broad domains are blood and lymphatic system outcomes, cardiac outcomes, endocrine outcomes, etc. In the life impact core, common broad domains are social or physical functioning. The adverse event core area includes outcomes broadly labeled as some form of unintended consequence such as adverse reactions, safety, harm, negative effects, or toxicity of an intervention. ²⁹

Because these broad domains may not provide a high degree of granularity, the third layer of classification proposes to organize outcomes as target domains. Finally, the original outcomes as reported in the RCTs are set in the last layer. Figure 1 provides an overall description and an example of this framework adapted to the review. Supplementary Figure S1a, S1b, and S1c provide additional examples for other core areas. This framework has also been previously used in a literature review involving patients with severe TBI or stroke with cranioplasty. ³⁰ Moreover, outcome assessments were organized as patient-reported outcome measures, observer-reported outcome measures, clinician-reported outcome measures, performance outcome measures, and surrogate measures (https://www.fda.gov/about-fda/division-patient-centered-development/clinical-outcome-assessments-coas-medical-device-decision-making). Patient-reported outcomes provide information on the patient’s health condition and are directly reported by the patient (no outside interpretation). Clinician-reported outcome are reports coming from a trained health care professional to interpret signs, behaviors related to a patient’s condition. Observer-reported outcomes are signs or behaviors related to a patient’s condition, assessed by individuals who observe the patient in daily life (caregiver, family). Performance reported-outcomes are measurements collected when a patient is asked to complete a standardized and well-defined task. Surrogate outcomes are defined as other outcomes not applicable to the previous definitions, usually derived from physiological data (i.e., biomarker, values issued from a monitoring device).

FIG. 1.

Illustration of the concept of core area, broad domain, target domain, and outcome according to the COMET initiative and OMERACT Handbook.

Data analysis

We described the number and types of core areas (mortality, pathophysiological manifestations, resource use, life impact, adverse events) followed by broad domains, target domains, and finally outcomes, as well as their proportions among all included RCTs. Given the population of patients (moderate-to-severe TBI), we also specifically described the neurological outcomes in the acute and late stages of the disease used in RCTs. For this review, the acute stage of TBI corresponds to the acute-care setting (in-hospital after primary admission) and the late stage corresponds to the post-acute care, after hospital discharge. The acute neurological outcomes are in the death and pathophysiological manifestation core area and the nervous system broad domain. The late neurological outcomes are in the life impact core area across different broad domains. We also reported the number of countries involved, included participants, intervention types, and the duration of RCTs. Finally, we described the median number of outcomes per RCT as well as their assessment as observer-reported, clinician-reported, patient-reported, performance, or surrogate outcome measure. Continuous variables are presented as median and interquartile ranges. Categorical data are presented as frequencies and percentages. Descriptive analyses were performed with the software R studio (version 4.2.2).

Results

Trial characteristics

From June 1, 1983, to October 31, 2023, 190 RCTs including 52,010 participants were selected (Supplementary Fig. S2). The median sample size was 81 [interquartile: 39–204] patients. Studies took place in 59 countries, including the United States [48 (25.3%)], China [23 (12.1%)], France [15 (7.9%)], Iran [12 (6.3%)], and the United Kingdom [7 (3.7%)]. Thirty-one (16.3%) RCTs were international, involving two countries or more. The median duration of RCTs was 28 [16–48] months. Duration was not reported in 30 (15.8%) RCTs. The characteristics of the included RCTs are provided in Supplementary Table S1.

Core areas, domains, and outcomes

Among the 190 RCTs, we retrieved 557 different outcomes [median number of outcomes per RCT 10 (6–16) ]. Among the RCTs, the use of core areas was as follows: 169 RCTs (88.9%) for pathophysiological manifestations, 117 RCTs (61.6%) for life impact, 94 RCTs (49.5%) for death, 72 RCTs (37.9%) for resource use, and 41 RCTs (21.6%) for adverse events. In the review, we identified 29 broad domains and 89 target domains. Among all broad domains, nervous system outcomes [131 RCTs (68.9%)], physical functioning [111 RCTs (58.4%)], and mortality [94 RCTs (49.5%)] were the most common. Among target domains, mortality [94 (49.5%)], physical functioning [111 (58.4%)], intracranial pressure target domain [68 (35.8%)], and hemodynamics [53 (27.9%)] were the most frequent. Figure 2 displays the organization of outcomes according to core areas, broad domains, and target domains.

Outcome assessment was as follows: clinician-reported outcome [177 (93.2%)], surrogate outcome [139 (73.2%)], performance outcome [19 (10%)], and patient-reported outcome [11 (5.8%)]. The most frequent clinician-reported outcomes were in the physical functioning [111 (58.4%)] and mortality [94 (49.5%)] broad domains. The broad domain with the most patient-reported outcomes was quality of life [8 (4.2%)] (Figure 3).

FIG. 2.

Core areas, broad domains, target domains in randomized controlled trials assessing interventions in the early phase of moderate-to-severe traumatic brain injury.

FIG. 3.

Classification of outcome assessments reported in randomized controlled trials related to the early phase of moderate-to-severe traumatic brain injury.

Pathophysiological manifestation core area

In the pathophysiological manifestation core area, we found 16 broad domains and 58 target domains. In this core area, the nervous system broad domain was predominant [131 (68.9%) RCTs]. In the nervous system broad domain, the most frequent target domains were intracranial pressure [68 (35.8%], consciousness [32 (16.8%)], and medical management of intracranial hypertension [29 (15.2%)]. Supplementary Table S2 displays all outcomes and their organization in target and broad domains in this core area.

Life impact core area

In this core area, we found 7 broad domains and 16 target domains. The most frequent broad domains were physical functioning [111 (58.4%) RCTs] and cognitive functioning [25 (13.5%) RCTs]. Supplementary Table S3 displays all outcomes and their organization in target and broad domains in this core area.

Resource use core area

In this core area, we found 4 broad domains and 12 target domains. The most common broad domains were hospital resource use [53 (27.9%)] and the need for further intervention [48 (25.2%)]. Supplementary Table S4 displays all outcomes and their organization in target and broad domains in this core area.

Adverse events core area

Supplementary Table S5 displays all outcomes and their organization in target and broad domains in this core area.

Acute neurological outcomes

The neurological outcomes used in RCTs in the early stage of TBI management are in the death or pathophysiological manifestation core area and in the nervous system broad domain. We retrieved 128 different outcomes classified in 16 target domains. The most common acute neurological outcomes were mortality [88 (46.3%) RCTs], intra-cranial pressure [57 (30%)], and cerebral perfusion pressure [38 (20%)]. Figure 4 displays the organization of acute neurological outcomes and their organization in their target domains.

FIG. 4.

Acute neurological outcomes.

Late neurological outcomes

The late neurological outcomes used in RCTs were in the life impact core area (Fig. 5), across the seven different broad domains. We retrieved 77 outcomes classified among 16 target domains. The most common late neurological outcomes were the Glasgow Outcome Scale (GOS) [63 (33.2%) RCTs], the GOS-Extended (GOS-E) [45 (23.7%)], and the Disability Rating Scale [22 (11.6%)]. Figure 5 displays late neurological outcomes and their organization in target and broad domains.

FIG. 5.

Late neurological outcomes.

Discussion

This systematic review provides a scope of the outcomes, target domains, broad domains, and core areas used in RCTs evaluating the efficacy of early interventions in adult patients presenting moderate-to-severe TBI. The multiplicity of domains, outcomes, and outcome measures reflects a lack of consensus on the outcomes that should be used in RCTs in this field.

The lack of consensus is a major issue in clinical research. ³¹ It may lead to potential outcome reporting bias, the selection of outcomes that may not be relevant to stakeholders (e.g., clinicians, patients), thus reducing the usefulness of RCTs and limiting their comparability. ³² When reported, heterogeneity in operationalization was also noted. For instance, the GOS, which is commonly used as a gold standard in TBI clinical research, could be evaluated in many different ways: postal mail, ^26,33,34 phone calls, ^21,33 and face-to-face interviews, ^34,35 but the most frequent situation was lack of a description. ³⁶ The lack of consistency in reporting could lead to major differences in the classification of patients across studies and potentially across arms of randomization especially when blinding is not possible. Other issues include variability in the measure and reporting. ³⁷ Patient-reported outcomes are of increasing importance in clinical research. They are emphasized in other medical fields such as rheumatology or nephrology ^24,25 and also in TBI ³⁸ that patient and caregiver perspectives are mandatory for meaningful interventions for all stakeholders in daily practice. Unfortunately, our systematic review emphasizes that the use of self-reported outcomes currently remains confidential in TBI clinical research. The elaboration of a standardized core outcome set in clinical research associating patient-centered outcomes, health care professional outcomes, or even policy-maker outcomes could dramatically improve the quality of reporting and consistency of results in RCTs. ^24,39 Such core outcome sets enhance the relevance of clinical research for all stakeholders. Without a core outcome set routinely reported in trials, selective reporting of outcomes favoring interventions may occur, possibly leading to an overestimation of the true effect. In addition, failure to report adverse events can expose patients to unrecognized risks of harm. ⁴⁰ These methodological issues contribute to the considerable inefficacy and waste in biomedical research: every year, this waste is estimated to be over 85% of the 240 billion USD invested around the globe. ³¹

A core outcome set is an agreed minimum set of outcomes achieved by consensus and clinically relevant, to be measured and reported in all clinical trials of a specific disease. ²⁴ Core outcome sets are developed through rigorous methodology, ⁴¹ can minimize reporting bias, promote consistency across clinical trials, and enable direct comparisons of the effect of the different interventions. Further studies are needed to obtain a better understanding of what matters the most for TBI patients and develop reliable and validated patient self-reported outcomes. Several initiatives of core outcome set development have been undertaken in rheumatology (OMERACT) ²⁵ and nephrology (Standardized Outcomes in Nephrology—SONG). ²⁴ These initiatives have led to the identifications of key outcomes for clinical research through a consensus process that involved health care providers, policy makers, patients, and their caregivers. In TBI clinical research, other authors have also stressed the importance of collecting meaningful end-points to adequately categorize the patient. The 8-level GOS-E is usually stated in two categories to classify the patient as having a “good” or “poor” outcome. In addition, this oversimplification is mainly the consequence of an investigator-based assessment and the patient’s perspective could provide more insight regarding the way patients should be classified. ³⁸ To this end, other researchers have proposed to perform cognitive tests in addition to functional outcome evaluations. ⁴² The COMET initiative was developed in 2010 to facilitate the development and implementation of core outcome sets. ²⁸ In 2010, a group proposed a battery of a minimal set of nine outcomes for clinical research. ⁴³ The authors chose a certain number of outcomes among the most commonly used. These authors proposed 3 different layers of outcomes in TBI research. In the first layer, 8 different outcomes ranging from standard outcomes such as functional recovery to health-related quality of life were proposed to be systematically evaluated. However, there was no consensus or ranking between these outcomes. The second and third layers proposed specific scales in specific clinical contexts. However, the high number of scales proposed raised major feasibility problems. Consequently, these initiatives have not been transformed into major modifications in the methodology of clinical research in TBI patients. Other initiatives such as the National Institute of Neurological Disorders and Stroke (NINDS) are highly meaningful. This initiative proposes a classification of neurological outcomes in TBI and consciousness disorders. However, the NINDS classification exclusively handles neurological outcomes (e.g., neuropsychological testing, functional outcomes). However, many of the outcomes we found in our systematic review that may be relevant to certain stakeholders failed to fall within this classification (e.g., pneumonia, cerebral biomarkers, nutrition). Moreover, since our review is endorsed by the COMET initiative, we were in line with their recommendations and methodology. ^25,28,44 The elaboration of a core outcome set relevant to all stakeholders, patients, their caregivers, health care professionals, and policy makers, is therefore urgently needed in this area to tackle the litany of 30 years of negative RCTs. ^{21,26,33 –35}

Limitations

Similar to most studies, this study has some limitations. First, we only included studies available on PubMed—thereby introducing a potential selection bias. However, the vast heterogeneity we found renders unlikely that studies published elsewhere could alter our results. We also excluded non-English studies. In seven studies, we were unable to obtain the full text so we could not include them. We only reported the “outcome measures” published in the articles without reviewing supplementaries, appendixes, and protocols when there were available, which could induce a potential reporting bias. For all of these reasons, our results may overestimate the scope, inconsistency, and variability of outcome reporting. However, the broad domains were independently identified by two reviewers and then discussed with another reviewer following the OMERACT recommendations. ²⁵ Moreover, the framework that we used should provide consistency and relevance for the scientific community. Finally, there is no consensus on the name and organization of target domains considering the specificity of each medical field and the level of granularity being reached in clinical research outcomes. Considering the lack of similar work in the literature, our choices of target domain organization were not made upon other previously published data. However, three different authors checked for the consistency and relevance of these target domains.

Conclusions

To the best of our knowledge, this is the first systematic review that provides the scope of outcomes used in moderate-to-severe TBI RCTs. This study shows the diversity of domains and outcomes reported in RCTs performed in the early stage of moderate-to-severe adult TBI patients. Our findings stress the need for a standardized set of outcomes for TBI in clinical research in a core outcome set. Our group has started the elaboration of a core outcome set in clinical research for TBI and is currently reaching out for potential collaborations on this topic.

Transparency, rigor, and reproducibility summary

The study design and analytic plan of the review were registered in PROSPERO (number 2020 CRD42020194273) on July 27, 2020. We included adult patients with moderate-to-severe TBI who were included in an RCT testing an intervention in the acute phase of pathology. Data analysis is descriptive (median number and interquartile range, number and frequency). There were 190 RCTs included out of 52,010 participants from 1983 to the October 31, 2023. We extracted each outcome and organized them according to the COMET and OMERACT framework (core area, then broad domains, then target domains, and finally outcomes). A total of 557 original outcomes were identified.

Among core areas, 169 RCTs (88.9%) used pathophysiological manifestations, 117 RCTs (61.6%) used life impact, 94 RCTs (49.5%) used death, 72 RCTs (37.9%) used resource use, and 41 RCTs (21.6%) used adverse events. In the review, we identified 29 broad domains and 89 target domains. Among broad domains, nervous system outcomes [131 RCTs (68.9%)], physical functioning [111 RCTs (58.4%)], and mortality [94 RCTs (89.4%]) were the most common. Among target domains, physical functioning [111 (58.4%], mortality [94 (49.5%)], intra-cranial pressure target domain [68 (35.8%)], and hemodynamics [53 (27.9%)] were the most frequent. Patient-reported outcomes were seldom used [11 (5.8%)]. The data generated to perform this review have not been uploaded on a public repository but are available on request to the corresponding author.