Should We Use the IMPACT-Model for the Outcome Prognostication of TBI Patients? A Qualitative Study Assessing Physicians’ Perceptions

Abstract

Introduction. Shared Decision-Making may facilitate information exchange, deliberation, and effective decision-making, but no decision aids currently exist for difficult decisions in neurocritical care patients. The International Patient Decision Aid Standards, a framework for the creation of high-quality decision aids (DA), recommends the presentation of numeric outcome and risk estimates. Efforts are underway to create a goals-of-care DA in critically-ill traumatic brain injury (ciTBI) patients. To inform its content, we examined physicians’ perceptions, and use of the IMPACT-model, the most widely validated ciTBI outcome model, and explored physicians’ preferences for communicating prognostic information towards families. Methods. We conducted a qualitative study using semi-structured interviews in 20 attending physicians (neurosurgery,neurocritical care,trauma,palliative care) at 7 U.S. academic medical centers. We used performed qualitative content analysis of transcribed interviews to identify major themes. Results. Only 12 physicians (60%) expressed awareness of the IMPACT-model; two stated that they “barely” knew the model. Seven physicians indicated using the model at least some of the time in clinical practice, although none used it exclusively to derive a patient’s prognosis. Four major themes emerged: the IMPACT-model is intended for research but should not be applied to individual patients; mistrust in the IMPACT-model derivation data; the IMPACT-model is helpful in reducing prognostic variability among physicians; concern that statistical models may mislead families about a patient’s prognosis. Discussion: Our study identified significant variability of the awareness, perception, and use of the IMPACT-model among physicians. While many physicians prefer to avoid conveying numeric prognostic estimates with families using the IMPACT-model, several physicians thought that they “ground” them and reduce prognostic variability among physicians. These findings may factor into the creation and implementation of future ciTBI-related DAs.

Keywords

shared decision making goals-of-care decisions prognosis traumatic brain injury critical care outcomes IMPACT-model qualitative research

Critically ill traumatic brain injury (ciTBI) patients are incapacitated. They depend on family members or health care proxies as surrogate decision makers (“surrogates”) for any medical decisions, including the life-or-death (“goals-of-care”) decision about continuation of care or withdrawal-from-life-sustaining interventions. Surrogates make the goals-of-care decision based in part on the patient’s long-term prognosis, which is typically communicated to the surrogates by physicians. Physicians commonly rely on their own experience to derive an estimate of prognosis, but may also apply predictions obtained from validated prognostic models, such as the intracerebral hemorrhage (ICH) score^1,2 and FUNC scores³ for patients with an ICH or the International Mission for Prognosis and Analysis of Clinical Trials in TBI (IMPACT) model^4,5 for patients with TBI to obtain an estimate for a patient’s severity of disease and long-term prognosis.^6,7 While the authors of these models recommend that these models should not be used for endorsing limitations of care or making treatment decisions without also considering individual patient factors not included in such scores,^6,8,9 in clinical practice unfortunately many physicians use them to inform clinical decision making.⁸

However, in ciTBI physicians vary considerably in their perceptions of neurologic prognosis and withdrawal-from-life-sustaining therapies.^10–13 Goals-of-care decisions lack standardization, and physicians are prone to adding their personal opinions and biases based on their impression of the patient when offering (or not offering) certain treatments at their discretion.^10,11,13 In many other diseases, a more standardized and collaborative approach to the communication of projected outcomes, delineation of treatment options with their risk-benefit ratios, as well as the systematic elicitation and inclusion of patient values and preferences has been achieved by the use of shared decision-making tools (decision aids).^14–17 Recently, two large professional medical societies highlighted the need for shared decision making in the intensive care unit (ICU) to facilitate information exchange, deliberation, and effective decision making.¹⁸ This approach has also been suggested for difficult decisions in TBI.¹⁹ Currently, no decision aid exists for goals-of-care decisions in ciTBI patients or any other diseases in the neuro-ICU.¹⁷ The International Patient Decision Aid Standards (IPDAS) serves as a framework for the development, rating, and certification of high-quality decision aids and recommends a systematic approach, beginning with the exploration of stakeholders’ (i.e., physicians’ and surrogates’) communication practices and preferences for decision making.^20,21 Furthermore, this framework advises the presentation of probabilities, such as those derived from disease-specific validated outcome models.²¹

In ciTBI, the IMPACT model remains the most widely validated outcome model but has significant weaknesses. It is based on admission criteria only and does not include the clinical course of a patient after admission, or response to supportive care in the ICU for the first crucial 3 to 5 days. Therefore, the IMPACT model may be poorly suited to be included in a future goals-of-care decision aid for ciTBI, because potential care-limiting decisions should not be based on prognostic probabilities derived from admission criteria only. However, to our knowledge, to date there is a paucity of better ciTBI outcome prediction models, which include the patient’s hospital course and response to treatment, leaving the IMPACT model as a leading candidate for use in a goals-of-care decision aid in ciTBI.

The objective of the current study was to examine physicians’ awareness, perceptions, and use of the IMPACT model, as well as their preferences for communicating prognostic estimates during family meetings with ciTBI surrogates. The information gained from this study would crucially inform the content and building of a future goals-of-care decision aid in ciTBI.

Methods

Study Design and Participants

This current study is one component of a larger qualitative study exploring key stakeholders’ (surrogate decision makers and physicians) attitudes and preferences for outcome prognostication for ciTBI patients, and its communication to families and surrogate decision makers²² that was conducted between October 2015 and August 2016. Participating physicians were recruited using purposive and snowball sampling from seven academic medical centers from five geographic regions of the United States (Northeast, Mid-Atlantic, South, West, and Midwest; Figure 1) representing four different specialties (neurosurgery, neurocritical care, trauma surgery, and palliative care). Inclusion criteria included English-speaking with willingness to undergo an audio-recorded interview, caring for ciTBI patients, practicing as an attending for ≥2 years after fellowship training, and routine engagement in goals-of-care discussions for ciTBI patients.

Figure 1

Geographic distribution of participating physicians. This map shows the number and location of participating physicians in the United States.

The University of Massachusetts Medical School (UMMS) Institutional Review Board approved the study with written consent for in-person interviews (n = 10), or verbal consent with an approved script for telephone interviews (n = 10).

Measurements and Interviews

Baseline demographics were obtained using a written survey. We developed a semistructured interview guide with open-ended questions. To address the objectives of our current study, one part of the interview guide was aimed at exploring physicians’ communication practices during family meetings of ciTBI patients discussing the patient’s prognosis, as well as physicians’ awareness, knowledge, use, and perception of the IMPACT model to guide estimates of the patient’s prognosis. The interview guide was iteratively refined with input from content and physician-patient communication experts, and tested in two mock interviews. The institutional review board approved the final interview guide before its use. One trained interviewer (SM) conducted all interviews in person or over the phone. All interviews were digitally audio-recorded and transcribed verbatim by a professional transcription service. Participants were offered a $25 gift card for their participation.

Coding Procedures and Analysis

We developed a coding scheme using parallel deductive and inductive methods. The initial coding scheme was developed based on the interview guide content by a neurointensivist (SM) and an experienced qualitative researcher (KM). Two coders (TQ and JM) cooperatively coded five transcripts while making inductive changes to the initial coding scheme when necessary, with review of changes by SM and KM. Conceptually similar themes were combined and emerging themes added; conflicts were resolved using a third reviewer. Using this revised coding scheme, coders then independently coded the same three additional transcripts, resulting in >80% congruency. All previous and subsequent transcripts were then coded separately using the revised coding scheme, with no further inductive changes required, indicating theme saturation (no new themes emerged). All codes were reviewed by three authors to ensure correct classification. Recruitment was continued to diversify the sample and to confirm that indeed no novel information would be obtained through the analysis of several additional transcripts. Theme saturation and feasibility resulted in a final sample size of 20 physicians. Qualitative analysis was performed using NVIVO (QSR International Pty Ltd., Melbourne, Australia). Baseline characteristics of the study groups were summarized using descriptive statistics.

Results

Baseline Characteristics

The average interview duration was 37 minutes (range = 19–65 minutes). Baseline characteristics of the study cohort are shown in Table 1. The mean number of years in attending practice was 14. Half of the participants were specialized in neurocritical care, with the other half distributed among neurosurgery, trauma, and palliative care. The geographical location of participants is shown in Figure 1. Of the approached physicians, one declined participation.

Table 1

Participating US Physicians’ Baseline Characteristics (N = 20)

Baseline Characteristics	Physicians
Age (years), mean ±SD	47 ± 8
Female	7 (35)
Race
Caucasian	13 (65)
African American	1 (5)
Asian	6 (30)
Ethnicity
Non-Latino	19 (95)
Latino	1 (5)
No. of years in attending practice, mean ±SD (range)	14 ± 10 (2–40)
Specialty
Neurocritical care	10 (50)
Neurosurgery	7 (35)
Trauma surgery	2 (10)
Palliative care	1 (5)

Note: Data are presented as n (%) unless otherwise specified.

IMPACT Model Awareness and Use in Clinical Practice

Overall, only 12 (60%) of the physicians expressed awareness of the IMPACT model, of whom 8 were nonsurgical neurocritical care specialists. Of the 12 physicians with knowledge of the IMPACT model, 7 indicated using the model at least some of the time to form a patient’s prognosis in clinical practice, although none used it exclusively to derive a prognosis. One physician mentioned he/she might show the output from the IMPACT model to select families “once in a while if there is a family that is very number-oriented and they want percentages.” Of the seven neurosurgeons interviewed, three expressed awareness of the IMPACT model, of whom one was “barely aware,” and one acknowledged its use for discussions among physicians, but not with families. Of the two trauma surgeons, only one stated that he/she “had heard about” the IMPACT model, but was not using it for prognostication due to lack of familiarity with this model.

Major Themes

We identified four major themes representing a variety of views on the communication practices of numeric estimates of the patient’s prognosis during family meetings and the utility of the IMPACT model for ciTBI prognostication and clinical decision making. When physicians were unaware of the IMPACT model specifically, we included general statements about the use of health outcome models, as these statements included important insights into the physicians’ perceptions of the utility of outcome scores for neurological prognostication. Below we list each theme with a representative quote. Additional quotes for each theme are shown in Table 2.

Table 2

Themes and Representative Physician Quotations

Theme	Representative Quotations
IMPACT model is for research, not individuals	“Those calculators are about populations. They’re not about individuals, and frankly, the use of those calculators for his purpose is a perversion of the original construct of them.”
	“We played around with using IMPACT but I think it’s a problem. I think it’s good for research, but I’m not convinced that IMPACT is great for an individual patient because it gives you probabilities and I think it’s difficult to express those probabilities to families.”
	“I am always very scared when I hear of colleagues trying to use the IMPACT calculator or the CRASH calculator. These things were never designed for clinical decision making, they were designed to inform clinical trial design.”
	“I think there is benefit to us and residents in teaching ourselves what to know. Also for research. When you’re trying to look at severity of injury between groups those scales are useful but I have concerns about trying to take information that’s gathered from a population and to apply it to a human.”
	“It’s one thing if the models are really well validated, you have large subgroups, and then you can tell. If you have subgroups that don’t really represent who the patient is, then the result the model provides could be dangerous. Quirky things where patients have an unusual background, these models may not hold.”
Mistrust in the IMPACT model data	“I don’t think the data is really good. I think for me TBI is probably the most difficult disease in which to prognosticate and I do not think that any of the decision tools that we have there personally are very good. . . . I do not think the disease has been studied very well.”
	“The problem with a traumatic brain injury IMPACT score is that it has lumped together all of the heterogeneous TBI disease processes based on the Glasgow Coma Scale. It doesn’t help up shift out the different types of traumatic brain injury DAI versus subdural versus contusion.”
	“Everybody knows that all of these scales are tinted by the databases that they come from and the fact that there was withdrawal of care from all of them. There hasn’t really been a good prognostic scale that was prospectively gathered.”
	“I very much avoid giving percentages because percentages I think, when you have come up with a percentage you better have damn good data to do that with and most often we do not have good data for that.”
	“I hate the fact that, that even in an IMPACT calculator we don’t have confidence [intervals] anymore, and I think that is just crazy.”
	“I do not put any stake in the calculators that are out there right now.”
	“There’s not a lot of science available to really. . . . Prognosis in TBI is really non–clear cut.”
The IMPACT model is helpful in reducing physician variability	“At times, we will also calculate an IMPACT score to add to the clinical information we have about that patient. It is not done in isolation but rather in conjunction with clinical examination. . . . Putting that all together, we start to get a good idea of how we feel the patient is doing.”
	“I think overall from a global stand point . . . it reduces the variability of what people are hearing.”
	“We use it to give us some guidance, some expectations. But we don’t use it, for instance, if the IMPACT mortality rate is 86% or something, they don’t then say, ‘We’re going to stop because the outcome is going to be very poor.’ So that’s sort of how I use it, to provide a baseline of early expectations for the family.”
	“I have used the IMPACT score at times, and put those number in to have some information.”
	“We usually use the IMPACT prediction model so that includes everything from CT findings, the presence or absence of asymmetric pupils, hypertension or hypoxia and some of the hemoglobin values. We actually, for a severe TBI patient will float all that information into the calculator to generate a likelihood of mortality and morbidity.”
	“I think we have as physicians, we actually have a naïve feeling that provision of quantitative information is somehow reassuring to families. And so, so we at our practice use those models, we do an IMPACT score on most every patient and we look at them and we talk about it at the bedside.”
	“For the cases that seem really uncertain I’ll try to use the IMPACT database.”
	“It’s also if I have an uncertainty after looking at general imaging and seeing the patients, I use the database to guide me. I think it’s been helpful in getting an idea of where your confidence should be.”
	“I think it is helpful to ground the physicians a little bit. In short, I think it reduces the variability of prognosis that a large group of physicians may give to this very heterogeneous group of patients.”
Presenting numbers derived from statistical models may mislead patient families	“I try to . . . hold away from those models because the percentages will be interpreted by families in whatever way they want. Even if you give them the 80% chance of mortality, if they’re optimistic, they will latch on to the 20%.”
	“I would never give the result of an IMPACT score and tell them that there is a 60% chance of mortality. I don’t think that’s really appropriate.”
	“I really shy away from giving numbers because families will often hook onto those numbers and make an absolute judgment.”
	“Those models are the models that say that there is zero chance of improving are helpful. And the model that says there is one half of one percent a chance of improving is not so helpful, because that is the ray of light that enters into the room that is anything to grasp a hold of. . . . When you say there is a one percent chance that you can recover or particularly have a good recovery. You know in a society in which you buy a lottery ticket and have a one in a hundred and fifty million chance of winning and you will still buy the ticket. I mean how can you not grab hold of the one percent and hold it to be true.”
	“The things that we use, we say, ‘The percentages x, plus or minus y.’ Family members, unless they are very mathematically sophisticated, really don’t interpret these numbers the way that physicians and scientist interpret these numbers. They become simplified and used against you later.”
	“The confidence intervals are there, you can calculate them, you actually need to dig deeper to do it, its not there in the decision, that we use clinically a lot. That is when I stop using quantitative members with patients and families because if I am going to talk about a raise, more than 80% of independent function defined as being angulatory at 3 months whether 95% confidence interval compared to 91%. If I were really going to provide accurate data to families. I would probably need to describe it like that; otherwise, I think I am being potentially incomplete, dishonest, or obfuscating. So, once I started to think like that, I should talk more in qualitative ranges.”
	“I have had residents who’ve met families in the ER and looked up an ICH score or Glasgow Scale and in the ER they’ve told families that their loved ones have a 78% chance of dying. I think that’s really unhelpful. I think one of the problems with some of these tools is that they’re going to be disseminated to the masses and you’re going to have people with little information who have access to a percentage or a likely outcome.”
	“In my practice now I almost refuse to give them a number, because if you say there is a 70% bad outcome there is still a 30% who have a good outcome and I can’t tell you which one you’ll be in.”
	“I think those sorts of scales are not particularly helpful, because one in a hundred, I wouldn’t mind those odds. I’d buy a lottery ticket if I had a one in a hundred chance.”
	“The thing that I don’t do on a regular basis is use outcome-prediction models from the medical literature with quantitative predictions to provide some sort of level of presumed clarity. I usually ask and determine in the context of the discussion whether that is information that frankly matters to the family or they process information in a different way.”
	“You cannot walk up to a grieving mother in the ER and tell them that their son has a this much percent chance of dying. That’s just not what they need to hear.”

TBI, traumatic brain injury; DAI, diffuse axonal injury; CT, computed tomography; ICH, intracerebral hemorrhage; ER, emergency room.

The IMPACT Model (Outcome Scores in General) Are Research Tools and Should Not Be Used in Individual Patients

Many physicians expressed significant apprehension about applying the IMPACT model to individual patients. Specifically mentioned was the opinion that the IMPACT model and other outcome scores are research tools and were designed to inform clinical trial design, but should not be applied at the bedside to an individual. As one physician stated,

I am always very scared when I hear of colleagues trying to use the IMPACT calculator or the CRASH calculator. These things were never designed for clinical decision making, they were designed to inform clinical trial design.

Mistrust in the IMPACT Model Data

Several physicians raised concern about the quality of the data underlying the IMPACT model, as well as the quality of the data generated by TBI outcome research in general due to heterogeneity among the different injury patterns and the stratification into different TBI severities by the Glasgow Coma Scale. This mistrust in TBI outcome data also represented the main reason for physicians not applying the IMPACT-model in clinical practice:

I don’t think the data is really good. I think for me TBI is probably the most difficult disease to prognosticate and I do not think that any of the decision tools that we have personally are very good. . . . I do not think the disease has been studied very well.

Even physicians who were not aware of the IMPACT model expressed a genuine concern and mistrust in using any prognostic models and calculators for the determination of a patient’s prognosis:

I do not put any stake in the calculators that are out there right now.

There’s not a lot of science available really—Prognosis in TBI is really not clear cut.

The IMPACT Model Is Helpful in Reducing Physician Variability in a Heterogeneous Disease

Despite the inherent mistrust in using the IMPACT model (or any outcome model in general), all physicians acknowledged the challenges in forming a prognosis reliably for many of these critically ill patients. Therefore, several physicians viewed the IMPACT model as helpful in reducing the variability among physician opinions when many different physicians and services are caring for a patient:

I think it is helpful to ground the physicians a little bit. In short, I think it reduces the variability of prognosis that a large group of physicians may give to this very heterogeneous group of patients.

Several physicians warned that the IMPACT model should not be used in isolation but in conjunction with all available clinical data:

At times, we will also calculate an IMPACT score to add to the clinical information we have about that patient. It is not done in isolation but rather in conjunction with clinical examination. . . . Putting that all together, we start to get a good idea of how we feel the patient is doing.

Presenting Numbers Derived From Statistical Models May Mislead Patient Families

A frequently mentioned concern regarding the clinical use of the IMPACT model for families was the reluctance by physicians to provide numeric prognostic estimates to families. Participants stated several reasons for this concern. The first reason was the assumption that families will misinterpret numeric estimates due to stress and despair, thereby creating false hope and unfounded optimism:

Percentages will be interpreted by families in whatever way they want. Even if you give them the 80% chance of mortality, if they’re optimistic, they will latch on to the 20%.

In addition, physicians expressed apprehension about potentially low numeracy among surrogate decision makers, resulting in oversimplification of numeric estimates:

Family members, unless they are very mathematically sophisticated, really don’t interpret these numbers the way that physicians and scientists interpret these numbers. They become simplified and used against you later.

Although a subgroup analysis of attitudes and awareness of the IMPACT model by physician specialty was considered, it was not performed due to the small sample size.

Discussion

Our study found substantial variability in expert physicians’ awareness, utilization, and perception of the IMPACT model, but consistency in their reluctance to use numeric prognostic estimates during family meetings. These findings present both insights and challenges to promoting informed decision making among surrogates while illustrating physicians’ approaches to prognostication for ciTBI patients.^11,13

With the creation of a ciTBI goals-of-care decision aid in mind, which meets as many IPDAS criteria as possible, including the criteria to “present probabilities” for the outcome prognostication,²¹ we purposefully examined physician responses to the IMPACT model as the most widely validated prediction model in TBI.^4,5,23,24 Parsimonious outcome models, in general, are not designed to guide treatment decisions or limitations of care, but due to their ease of use, in the reality of clinical practice, physicians often turn to these tools for guidance to inform clinical decision making and may misuse them by applying them to individual patients.⁸ We found in our study that most of the participating TBI expert physicians do not endorse the communication of probabilities derived from the IMPACT model in a ciTBI decision aid, while some find the model helpful in grounding physicians in their prognostication.

Our findings are in line with a recent qualitative study assessing attending physician attitudes and use of clinical prediction models when discussing end-of-life care. Physicians in that study reported that they use clinical prediction models primarily to “validate” the prognosis the physician had already derived to enhance their predictive confidence. However, similar to the results of our study, physicians were hesitant to provide actual numerical data to patients and had concerns of promoting prognostic overconfidence in end-of-life care.²⁵

Other published literature has also raised caution about the utility and appropriate use of prognostic scores to guide treatment decisions and provide outcome prognostication in neurological emergencies.^8,26 Because these scores are parsimonious by design, they omit unique patient characteristics that might affect patient’s prognosis such as preexisting comorbidities, including cognitive decline, and major in-hospital complications. Decisions based on prognostic scales that result in limitations of care during the early phases of catastrophic brain injuries, including TBI, may lead to clinical nihilism and self-fulfilling prophecies.^8,13,26 For example, in patients with an ICH, a multicenter study of 742 neurologists and neurosurgeons showed that knowledge of the ICH score considerably influenced physician decision making and prognostication.⁷ The physicians’ prediction of 30-day mortality and initial treatment intensity recommendations changed significantly before and after the ICH score was provided to them in two hypothetical ICH cases. Recommendations for treatment limitations were 60% higher when the ICH score suggested no chance of functional independence, but were significantly lower with an ICH score suggesting independence at 30 days. Similarly, another multicenter study in ICH patients revealed that physician and nurse clinical judgments correlated more closely with actual 3-month functional patient outcomes than with two of the most commonly applied ICH prognostic scores.⁹ In the setting of ciTBI, prospective comparisons of patients’ outcomes as predicted by the IMPACT score versus independent clinician judgment have not been published. However, in a large Italian TBI quality performance database study, the actual ciTBI patient outcomes were consistently better over the 11 years of collected data than the outcomes predicted by the IMPACT model.²⁷ This research suggests that skepticism about the IMPACT model expressed by the physicians in our study is indeed warranted.

A second important concern raised by several participants in our study was mistrust in the data from which the IMPACT model was derived. This mistrust may be based on awareness that the IMPACT model explains only 35% of the variability in outcomes after TBI.²⁴ While the IMPACT model has been externally validated,^5,28 its source data were combined from many observational studies and clinical trials of different patient populations, years under study, and data quality.^4,29 Therefore, in order to achieve adequate power in the predictive model, the selection of variables affecting prognosis was limited to few common variables collected in all source studies.³⁰ This recognition of the limitation of standardized data collection across research studies eventually led to the creation of the TBI Common Data Elements.^31–33 There are several ongoing single- and multicenter prospective TBI studies that are currently working on the development of more precise TBI prediction models (TRACK-TBI, CENTER-TBI, and OPTIMISM study) to include information about the clinical inpatient course of a patient after admission, and response to early supportive care.^34–36

There are fundamental uncertainties involved in communicating individual risk estimates.³⁷ Models that are well calibrated to predict the outcomes of a large population do not predict well (discriminate) for individuals.³⁸ Predictive models are typically validated based on calibration, not discrimination, although only discrimination is the relevant standard if the model is intended for individual prediction.³⁹ Our finding that physicians were reluctant to use a model that was validated at the population level to predict at the individual level may indicate their appreciation of the limitations of applying population-based risk models to individuals.

The reluctance of physicians to present numeric prognostic estimates derived from the IMPACT model is discordant to the communication preferences of surrogate decision makers of ciTBI patients when discussing prognosis. A recent study showed that surrogates felt that prognostic probabilities help limit uncertainty and ambiguity, and hearing numeric estimates conveys more confident decision making.²² A high numeracy and health literacy level of the participating surrogate decision makers may have biased the results of this study, however. This discordance between physician practice and surrogate preference regarding delivery of information about prognosis in the setting of ciTBI requires further study.

Physicians in our study worried that families will not adequately interpret the meaning of the numeric estimates due to low numeracy and health literacy, high stress levels, and false hope. Several studies have confirmed that surrogates may interpret prognostic estimates with an optimistic bias.^40,41 Reasons other than simple misunderstandings of numerical risk information are thought to underlie this optimistic bias; psychological biases such as “the need to remain hopeful,” skepticism in the physicians’ ability to accurately predict death, and positive patient attributes as perceived by the surrogate, but not low numeracy and health literacy, have been suggested as some of the cognitive causes for optimistic bias.⁴⁰ Furthermore, only a very small proportion of surrogates rely exclusively on prognostic information provided by physicians.⁴¹ Patient attributes greatly influence surrogates’ own impression of the patient’s prognosis, including strength of character, life history, appearance, and faith.⁴¹ Furthermore, other research has shown that neither surrogates’ personal estimates of the patient’s outcome nor their understanding of the physician’s prognostication differed between conveying the patient’s long-term prognosis numerically versus qualitatively.⁴² For some patients or their surrogate decision makers, conveying the “gist,” which is the qualitative, vague, bottom-line meaning of the information, rather than a numeric prognostication, may be preferred by physicians, patients, and by their surrogates themselves.^40,43,44 The perspectives captured from some of the physicians in our study support this notion.

In contrast, however, presenting numeric prognostic estimates to surrogate decision makers using the best available clinical prediction models, albeit imperfect and with a large degree of uncertainty, has important advantages. First, as recent research has shown, many surrogate decision makers in ciTBI patients want to know these numbers.²² Second, as physicians stated in our study, clinical prediction models may help limit the prognostic variability among physicians with different experience levels and biases; they help “ground” them. Third, avoiding the communication of probabilities could suggest to families a lack of confidence in the physician’s overall competence in caring for ciTBI patients. The clear communication of probabilities and the associated uncertainty appropriately tailored to the family’s numeracy level is, however, a difficult skill to learn and teach. Having a tool available that uses icon arrays and images to convey this numeric information may help.

As we develop a decision aid for the goals-of-care decision in ciTBI patients, we have carefully weighed the options of including or omitting the IMPACT model from the decision aid. We are well aware of the limitations of the IMPACT model and that the goals-of-care decision, a possible treatment limiting decision, must not be based on the admission variables only without considering the patient’s clinical course. However, complete omission of the IMPACT model from the decision aid would equal omission of all outcome probabilities from the decision aid, given the current absence of a better ciTBI outcome prediction model that includes elements of a patient’s clinical course or response to supportive ICU care over the first crucial 3 to 5 days after TBI. Such omission would be against the IPDAS criteria, and clearly against what families and surrogate decision makers need.²² Therefore, we are considering including the IMPACT model in the decision aid, well knowing that we must also explain the uncertainty associated with it, as it only includes admission criteria. One possible consideration is to use the IMPACT model to calculate the predicted mortality and poor outcome at 6 months (IMPACT predicts only these two dichotomized outcomes), provide this information to the physician who is going to discuss goals-of-care in order to “ground them,” and encourage the physician to add additional elements derived from the clinical course not included in the IMPACT model, such as response to treatment in the first few days, certain severe in-hospital complications (e.g., cardiac arrest, need for renal replacement therapy, refractory increased intracranial pressure, ischemic stroke from herniation),³⁶ preexisting comorbidities, or cognitive status before the TBI. Physicians may then “tweak” the prognosis to the better or worse from the “base” prognosis derived from the IMPACT model before it is included and presented in the decision aid to the patient’s family. As the ongoing TRACK-TBI, CENTER-TBI, and OPTIMISM studies^34–36 yield improved prediction models, we will update the decision aid with this new information. This approach is one of many possible approaches to the described outcome prediction dilemma in patients with TBI and has been modeled by the inventors of a decision aid for families of chronically ventilated medical ICU patients.⁴⁵

Study Strengths and Limitations

Our study has several strengths and limitations. We recruited physicians from across the United States representing five geographical areas, with overrepresentation from the East Coast, and underrepresentation of African American and Latino physicians. Physicians from these racial and ethnic backgrounds, and those from other areas of the United States and the world, may have different experiences, views, and approaches to communicating prognosis, which we may not have fully captured. We reached theme saturation after interviews with 20 physicians, which may seem small, but is a sample size that is typical for qualitative interview studies. It was designed to uncover some of the key issues in this area but not to produce generalizable results. Half of our cohort included nonsurgical neurointensivists, while the surgical participants were recruited from neurosurgery and trauma. It is possible that there may have been differences between these surgeons and nonsurgical physicians in the awareness, use, and perception of the IMPACT model. Half of our participants were recruited from a single center (UMMS). Therefore, it is possible that the institutional culture and center-specific practices may have biased the results. However, given the similarities of the physician comments from this and other study sites, our results likely represent findings that are not center-specific. Finally, the reservations expressed by physicians regarding the use of the IMPACT model may not be generalizable to all models in neurocritical care or models outside of this field.

In summary, our study identified significant variability of the awareness, perception, and use of the IMPACT model among physicians. While many physicians prefer to avoid conveying numeric prognostic estimates with families using the IMPACT model, several physicians thought that they “ground” them and reduce prognostic variability among physicians. These findings may factor into the creation and implementation of future ciTBI-related decision aids. Furthermore, research efforts should continue to focus on accumulating reliable ciTBI outcome data.

Footnotes

This work was done in the Department of Neurology at the University of Massachusetts Medical School, Worcester, MA. Sources of Funding: Financial support for this study was provided entirely by NIH/NICHD grant 5K23HD080971 (PI Muehlschlegel) and the NIH Clinical and Translational Science Award to the University of Massachusetts Medical School (UL1TR000161). The funding agreement ensured the authors’ independence in designing the study, interpreting the data, writing and publishing the report.

None of the authors have any conflicts of interest.

Prior presentations: Portions of this work were presented in poster form at the Annual Neurocritical Care Society Meeting in National Harbor, MD in September 2016. The final work was presented in poster form at the Society for Medical Decision Making 39^th Annual North American Meeting in Pittsburgh, PA in October 2017.

References

Hemphill

3rd Bonovich

Besmertis

Manley

Johnston

. The ICH score: a simple, reliable grading scale for intracerebral hemorrhage. Stroke. 2001;32(4):891–7.

Clarke

Johnston

Farrant

Bernstein

Tong

Hemphill

3rd . External validation of the ICH score. Neurocrit Care. 2004;1(1):53–60. doi:10.1385/NCC:1:1:53.

Rost

Smith

Chang

et al . Prediction of functional outcome in patients with primary intracerebral hemorrhage: the FUNC score. Stroke. 2008;39(8):2304–9. doi:10.1161/STROKEAHA.107.512202.

Steyerberg

Mushkudiani

Perel

et al . Predicting outcome after traumatic brain injury: development and international validation of prognostic scores based on admission characteristics. PLoS Med. 2008;5(8):e165. doi:10.1371/journal.pmed.0050165.

Roozenbeek

Lingsma

Lecky

et al . Prediction of outcome after moderate and severe traumatic brain injury: external validation of the International Mission on Prognosis and Analysis of Clinical Trials (IMPACT) and Corticoid Randomisation After Significant Head injury (CRASH) prognostic models. Crit Care Med. 2012;40(5):1609–17. doi:10.1097/CCM.0b013e31824519ce00003246-201205000-00028.

Rabinstein

Hemphill

3rd . Prognosticating after severe acute brain disease: science, art, and biases. Neurology. 2010;74(14):1086–7. doi:10.1212/WNL.0b013e3181d7d928.

Zahuranec

Fagerlin

Sanchez

et al . Variability in physician prognosis and recommendations after intracerebral hemorrhage. Neurology. 2016;86(20):1864–71. doi:10.1212/WNL.0000000000002676.

Hemphill

3rd White

. Clinical nihilism in neuroemergencies. Emerg Med Clin North Am. 2009;27(1):27–37. doi:10.1016/j.emc.2008.08.009.

Hwang

Dell

Sparks

et al . Clinician judgment vs formal scales for predicting intracerebral hemorrhage outcomes. Neurology. 2016;86(2):126–33. doi:10.1212/WNL.0000000000002266.

10.

Turgeon

Lauzier

Burns

et al . Determination of neurologic prognosis and clinical decision making in adult patients with severe traumatic brain injury: a survey of Canadian intensivists, neurosurgeons, and neurologists. Crit Care Med. 2013;41(4):1086–93. doi:10.1097/CCM.0b013e318275d046.

11.

Turgeon

Lauzier

Simard

et al . Mortality associated with withdrawal of life-sustaining therapy for patients with severe traumatic brain injury: a Canadian multicentre cohort study. CMAJ. 2011;183(14):1581–8. doi:10.1503/cmaj.101786.

12.

Cote

Turgeon

Lauzier

et al . Factors associated with the withdrawal of life-sustaining therapies in patients with severe traumatic brain injury: a multicenter cohort study. Neurocrit Care. 2013;18(1):154–60. doi:10.1007/s12028-012-9787-9.

13.

Izzy

Compton

Carandang

Hall

Muehlschlegel

. Self-fulfilling prophecies through withdrawal of care: do they exist in traumatic brain injury, too? Neurocrit Care. 2013;19(3):347–63. doi:10.1007/s12028-013-9925-z.

14.

Stacey

Legare

Col

et al . Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev. 2014;(1):CD001431. doi:10.1002/14651858.CD001431.pub4.

15.

Cox

Lewis

Hanson

et al . Development and pilot testing of a decision aid for surrogates of patients with prolonged mechanical ventilation. Crit Care Med. 2012;40(8):2327–34. doi:10.1097/CCM.0b013e3182536a63.

16.

Cox

White

Abernethy

. A universal decision support system. Addressing the decision-making needs of patients, families, and clinicians in the setting of critical illness. Am J Respir Crit Care Med. 2014;190(4):366–73. doi:10.1164/rccm.201404-0728CP.

17.

Muehlschlegel

Shutter

Col

Goldberg

. Decision aids and shared decision-making in neurocritical care: an unmet need in our neuro-ICUs. Neurocrit Care. 2015;23(1):127–30. doi:10.1007/s12028-014-0097-2.

18.

Kon

Davidson

Morrison

et al . Shared decision making in ICUs: an American College of Critical Care Medicine and American Thoracic Society policy statement. Crit Care Med. 2016;44(1):188–201. doi:10.1097/CCM.0000000000001396.

19.

Shutter

Timmons

. Intracranial pressure rescued by decompressive surgery after traumatic brain injury. N Engl J Med. 2016;375(12):1183–4. doi:10.1056/NEJMe1609722.

20.

Ottawa Hospital Research Institute (OHRI). Decision aid development toolkit [cited 28 November 2016]. Available from: https://decisionaid.ohri.ca/resources.html

21.

Elwyn

O’Connor

Stacey

et al . Developing a quality criteria framework for patient decision aids: online international Delphi consensus process. BMJ. 2006;333(7565):417. doi:10.1136/bmj.38926.629329.AE.

22.

Quinn

Moskowitz

Khan

et al . What families need and physicians deliver: contrasting communication preferences between surrogate decision-makers and physicians during outcome prognostication in critically ill TBI patients. Neurocrit Care. 2017;27(2):154–62. doi:10.1007/s12028-017-0427-2.

23.

Murray

Butcher

McHugh

et al . Multivariable prognostic analysis in traumatic brain injury: results from the IMPACT study. J Neurotrauma. 2007;24(2):329–37. doi:10.1089/neu.2006.0035.

24.

Lingsma

Roozenbeek

Steyerberg

Murray

Maas

. Early prognosis in traumatic brain injury: from prophecies to predictions. Lancet Neurol. 2010;9(5):543–54. doi:10.1016/S1474-4422(10)70065-X.

25.

Hallen

Hootsmans

Blaisdell

Gutheil

Han

. Physicians’ perceptions of the value of prognostic models: the benefits and risks of prognostic confidence. Health Expect. 2015;18(6):2266–77. doi:10.1111/hex.12196.

26.

Skrobik

Kavanagh

. Scoring systems for the critically ill: use, misuse and abuse. Can J Anaesth. 2006;53(5):432–6. doi:10.1007/BF03022613.

27.

Di Deo

Lingsma

Nieboer

et al . Clinical results and outcome improvement over time in traumatic brain injury. J Neurotrauma. 2016;33(22):2019–25. doi:10.1089/neu.2015.4026.

28.

Han

King

Neilson

Gandhi

. External validation of the CRASH and IMPACT prognostic models in severe traumatic brain injury. J Neurotrauma. 2014;31(13):1146–52. doi:10.1089/neu.2013.3003.

29.

Steyerberg

Murray

Maas

. Datasets included in IMPACT [cited 2 February 2017]. Available from: http://www.tbi-impact.org/?p=impact/datasets

30.

Maas

. Standardisation of data collection in traumatic brain injury: key to the future? Crit Care. 2009;13(6):1016. doi:10.1186/cc8163.

31.

Maas

Harrison-Felix

Menon

et al . Common data elements for traumatic brain injury: recommendations from the interagency working group on demographics and clinical assessment. Arch Phys Med Rehabil. 2010;91(11):1641–9. doi:10.1016/j.apmr.2010.07.232.

32.

Wilde

Whiteneck

Bogner

et al . Recommendations for the use of common outcome measures in traumatic brain injury research. Arch Phys Med Rehabil. 2010;91(11):1650–60.e17. doi:10.1016/j.apmr.2010.06.033.

33.

NIH/NINDS. NINDS Common Data Elements—Traumatic brain injury 2012; version 2.0 [updated June 2012; cited 22 January 2017]. Available from: https://www.commondataelements.ninds.nih.gov/TBI.aspx#tab=Data_Standards

34.

Yue

Vassar

Lingsma

et al . Transforming research and clinical knowledge in traumatic brain injury pilot: multicenter implementation of the common data elements for traumatic brain injury. J Neurotrauma. 2013;30(22):1831–44. doi:10.1089/neu.2013.2970.

35.

Maas

Menon

Steyerberg

et al . Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI): a prospective longitudinal observational study. Neurosurgery. 2015;76(1):67–80. doi:10.1227/NEU.0000000000000575.

36.

Muehlschlegel

Carandang

Ouillette

Hall

Anderson

Goldberg

. Frequency and impact of intensive care unit complications on moderate-severe traumatic brain injury: early results of the Outcome Prognostication in Traumatic Brain Injury (OPTIMISM) Study. Neurocrit Care. 2013;18(3):318–31. doi:10.1007/s12028-013-9817-2.

37.

Politi

Han

Col

. Communicating the uncertainty of harms and benefits of medical interventions. Med Decis Making. 2007;27(5):681–95. doi:10.1177/0272989X07307270.

38.

Diamond

. What price perfection? Calibration and discrimination of clinical prediction models. J Clin Epidemiol. 1992;45(1):85–9.

39.

Rockhill

. Theorizing about causes at the individual level while estimating effects at the population level: implications for prevention. Epidemiology. 2005;16(1):124–9.

40.

Zier

Sottile

Hong

Weissfield

White

. Surrogate decision makers’ interpretation of prognostic information: a mixed-methods study. Ann Intern Med. 2012;156(5):360–6. doi:10.7326/0003-4819-156-5-201203060-00008.

41.

Boyd

Evans

et al . “It’s not just what the doctor tells me:” factors that influence surrogate decision-makers’ perceptions of prognosis. Crit Care Med. 2010;38(5):1270–5. doi:10.1097/CCM.0b013e3181d8a217.

42.

Lee Char

Evans

Malvar

White

. A randomized trial of two methods to disclose prognosis to surrogate decision makers in intensive care units. Am J Respir Crit Care Med. 2010;182(7):905–9. doi:10.1164/rccm.201002-0262OC.

43.

Reyna

. Theories of medical decision making and health: an evidence-based approach. Med Decis Making. 2008;28(6):829–33. doi:10.1177/0272989X08327069.

44.

Reyna

. A theory of medical decision making and health: fuzzy trace theory. Med Decis Making. 2008;28(6):850–65. doi:10.1177/0272989X08327066.

45.

Cox

Wysham

Walton

et al . Development and usability testing of a Web-based decision aid for families of patients receiving prolonged mechanical ventilation. Ann Intensive Care. 2015;5:6. doi:10.1186/s13613-015-0045-0.