Sage Journals: Discover world-class research

Abstract

Background

Physician's estimates of a patient's prognosis are an important component in shared decision-making. However, the variables influencing physician's judgments are not well understood. We aimed to determine which physician and patient factors are associated with physicians' predictions of critically ill patients' six-month mortality and the accuracy and confidence of these predictions.

Methods

Prospective cohort study evaluating physicians' predictions of six-month mortality. Using univariate and multivariable generalized estimating equations, we assessed the association between baseline physician and patient characteristics with predictions of six-month death, as well as accuracy and confidence of these predictions.

Results

Our cohort was comprised 300 patients and 47 physicians. Physicians were asked to predict if patients would be alive or dead at six months and to report their confidence in these predictions. Physicians predicted that 99 (33%) patients would die. The key factors associated with both the direction and accuracy of prediction were older age of the patient, the presence of malignancy, being in a medical ICU, and higher APACHE III scores. The factors associated with lower confidence included older physician age, being in a medical ICU and higher APACHE III score.

Conclusions

Patient level factors are associated with predictions of mortality at six months. The accuracy and confidence of the predictions are associated with both physician and patients' factors. The influence of these factors should be considered when physicians reflect on how they make predictions for critically ill patients.

Keywords

Prognosis critical care critical care outcomes clinical decision-making life support care forecasting

Introduction

Shared decision-making is the process in which clinicians and patients (or their families) collaborate in the process of making health care decisions. For critically ill patients, shared decision-making requires an understanding of a patient's values and preferences and an estimate of the patient's expected course (i.e. prognosis).¹ The prognosis includes a patient's chances of survival and if they do survive, what their future level of function, and ultimately quality of life will be.^2,3 Several tools have been developed to help predict patient's outcomes during or after an admission to the intensive care unit (ICU).^4–6 These tools, however, are not routinely used in clinical practice because of their cumbersome nature and the underlying baseline differences in patients included in the development of these models (i.e. the calibration).^7–9 Studies have looked at physician's prognostic accuracy in predicting mortality^10–13 and other patient-centered outcomes.¹³ However, there is a paucity of data evaluating how individual patient and physician factors influence physicians' accuracy and confidence in these predictions. It is also unclear what factors influence whether physicians predict patients will survive or die. Understanding how these factors influence prognosis is important, as the process is generally determined by physician intuition.^14,15 This can help improve our understanding of an important element of shared decision-making.

The aim of this study was to determine which individual patient and physician factors are associated with physicians' predictions that a patient will live or die and which of these factors were associated with actual mortality at six months. Additionally, we explored which physician and patient factors were associated with both discriminative accuracy and confidence of these predictions.

Materials and methods

Study design and setting

We conducted a secondary analysis of a previously published study that evaluated the discriminative accuracy of physicians in predicting a variety of patient-centered outcomes, including six-month mortality as well as physical and cognitive function.¹³ This was a prospective cohort study in which patients and physicians were recruited from five ICUs (three medical, two surgical) in three hospitals affiliated with the Penn Health System. Recruitment occurred from October 2013 to May 2014, with six-month follow up being completed in December 2014. Written or oral informed consents were obtained from physicians and patients or surrogates when patients were not capable. The study was approved by the University of Pennsylvania Institutional Review Board (Project Approval Number 81886).

Study population

Patients were included if they were admitted to the ICU for at least three calendar days and required mechanical ventilation for a minimum of 48 h, vasopressors for more than a minimum of 24 h, or both. The primary attending physician for at least two calendar days was asked to predict different outcomes, including patient mortality at six months. Additionally, they were asked to rate their confidence in their predictions using a Likert scale from 1 (no confidence at all) to 5 (very confident).¹³

Measurements and variables

Baseline physician and patient variables were collected prospectively. Physician variables included age, gender, and years since graduation. Patient-related information included demographics, baseline function (i.e. cognition, independent toileting, ability to climb 10 stairs before ICU admission), medical comorbidities, and Acute Physiology and Chronic Health Evaluation III (APACHE III) score.¹³

For the purpose of our analysis, the primary outcome was defined as the direction of physicians' prediction of patients' mortality at six months. This was a dichotomous outcome, either predicted alive or dead. Additionally, we explored which factors were associated with patients' actual six-month mortality. As a secondary analysis, we also explored which individual patient and physician factors were associated with accuracy and confidence in these predictions. These analyses were done separately for predictions of six-month survival and mortality.

Statistical analysis

Descriptive statistics were used to summarize baseline characteristics of the physicians and patients. Data were described using proportions for categorical data and mean and standard deviation (SD) if data were normally distributed or median and interquartile range (IQR) if data had non-normal distribution for continuous data.

To assess the association between individual patient and physician factors with the mortality prediction, we used univariate logistic regression models. For those continuous predictors, in order to assess for the linear relationship between the variable and the study outcome, we categorized the predictor in deciles and assessed the proportion of outcome in each decile. We also used a graphical approach by plotting the predictors against the logit of the probability of the outcome using the locally weighted scatterplot smoothing (LOWESS) plots. Important a priori clinical predictors and variables that were significantly associated (using a threshold value of p = 0.1) with the primary outcome in the univariate analysis were included into a multivariable logistic regression model. Subsequently, we removed the variables that were not significant (at a p = 0.05 level) in the Wald test. The final model retained only the variables that were significantly associated with the outcome. We performed a correlation coefficient matrix with all the proposed independent variables to assess for multicollinearity. Additionally, we tested an a priori interaction term for the combination of malignancy and severity of illness using APACHE III score, assuming that doctors would be more likely to predict mortality in patients with malignancy if they were also presenting with more severity of disease. Since most physicians performed more than one prediction and the unit of analysis was the prediction itself, we conducted a generalized estimating equations (GEE) analysis approach to account for the effect of clustering at the physician level.¹⁶ The discrimination of the final models was assessed by plotting the area under the receiver operating characteristic (ROC) curve. The Hosmer–Lemeshow goodness of fit test was used to assess the calibration of the final models.

For the secondary analyses, we explored physician and patient factors associated with the accuracy and confidence in the predictions. We defined a confident prediction as predictions that were rated as either a 4 (confident) or 5 (very confident) result. Unadjusted and adjusted GEE models were performed so as to explore the association between baseline physician and patient factors with accuracy and confidence. For each of the outcomes, we built two separate models depending on the direction of the prediction (i.e. alive vs. dead at six months). Finally, since a nonlinear relationship between many potential continuous predictors (i.e. patient's APACHE III score) and the log(odds) of either physician's accuracy and confidence was presumed, we explored graphically the linear association of the continuous predictor and the probability of the outcome (Figure 1). We deemed this to be relevant, for example, if physicians appeared to be more confident or accurate at the extremes of severity of illness. All analyses were performed using Stata version 13.

Figure 1.

Linear association between physicians' and patients' variables and confidence of predictions. Association between physicians' age, patients'age, and APACHE III score with the logit of the probability of an accurate prediction for both survical and mortality. The plots have been created using locally weighted scatterplot smoothing (LOWESS) functions.

Results

During the study period 340 patients or surrogates were approached of which 303 consented and were enrolled. Of these, we were able to get physician predictions for 300 of the patients. Forty-seven physicians contributed at least one prediction per patient. The baseline characteristics of physicians and patients were described elsewhere. Physicians' median age was 41 (IQR 38–51) and 80% (N = 36) were male. Patient's median age was 62 years (IQR 53–71) and 57% (N = 172) were male (Table 1S, supplemental digital content). For the 300 patients included in this study, mortality at six months was predicted for 99 patients (33%), with 130 (43%) dying by six months.

Table 1.

Univariate analysis of the association between baseline characteristics and outcomes.

	Mortality prediction at six months	Mortality occurred at six months
	Odds ratio (95% CI)	Odds ratio (95% CI)
Physician variables
Age, years	1.10 (1.02–1.18)	0.96 (0.90–1.03)
Male gender	1.27 (0.27–5.93)	1.47 (0.32–6.80)
Years since graduation	1.10 (1.02–1.18)	0.96 (0.90–1.03)
Patient variables
Demographics
Age, median (IQR)	1.02 (1.00–1.04)	1.03 (1.02–1.05)
Male, N (%)	0.95 (0.59–1.55)	0.77 (0.48–1.22)
Race
White (reference)	1	1
African American	1.21 (0.73–2.03)	1.29 (0.79–2.11)
Other	1.21 (0.39–3.78)	1.94 (0.65–5.83)
Level of education, college or more	0.91 (0.55–1.49)	0.68 (0.43–1.09)
Patient's residence, home vs. chronic facility	0.83 (0.31–2.16)	0.42 (0.16–1.10)
Employed	1.09 (0.88–1.35)	1.21 (0.98–1.49)
Married or living with partner	1.07 (0.83–1.39)	0.99 (0.78–1.27)
Patient's insurance
Private (reference)	1	1
Medicare	1.0 (0.60–1.69)	1.45 (0.88–2.34)
Other	0.76 (0.33–1.75)	0.88 (0.40–1.93)
Hospitalized in previous year	1.43 (0.81–2.55)	2.28 (1.31–3.98)
Able to walk up 10 consecutive stairs in the month before	1.00 (0.54–1.84)	0.85 (0.48–1.53)
Toileting independently before hospitalization	0.93 (0.44–1.95)	0.84 (0.41–1.72)
Cognitive abnormal	1.57 (0.84–2.91)	1.98 (1.07–3.63)
Previous morbidity
Functional comorbidity index	1.05 (0.93–1.18)	1.04 (0.93–1.18)
Neurological condition (stroke, etc.)	1.43 (0.79–2.59)	1.11 (0.62–1.99)
Congestive heart failure	0.93 (0.57–1.55)	1.17 (0.73–1.88)
Coronary artery disease	0.63 (0.37–1.06)	0.74 (0.46–1.22)
Chronic obstructive pulmonary disease	1.43 (0.81–2.52)	1.24 (0.71–2.16)
Renal failure requiring dialysis	1.63 (0.76–3.52)	1.81 (0.84–3.87)
Malignancy (treated for cure or metastatic)	2.96 (1.77–4.94	2.77 (1.68–4.58)
Malignancy (treated for cure)	1.81 (1.03–3.19)	1.34 (0.78–2.33)
Malignancy (known metastatic or palliative approach)	3.97 (1.97–7.98)	9.62 (3.90–23.75)
Liver disease	1.62 (0.79–3.31)	3.79 (1.75–8.20)
Cognitive impairment	1.87 (0.70–5.02)	3.35 (1.14–9.72)
Psychiatric comorbidity	1.00 (0.60–8.24)	0.71 (0.44–1.16)
Variables related to ICU admission
Primary ICU diagnosis medical	4.22 (2.17–8.24)	3.61 (2.05–6.37)
APACHE 3 score (per 10-point increase)	1.19 (1.10–1.29)	1.28 (1.18–1.40)

Mortality prediction

The univariate association between baseline patient and physician factors and with the prediction and actual mortality at six months are shown in Table 1. Patients were predicted to die if they were older, had a history of malignancy, were in a medical ICU (compared to a surgical ICU), and had a higher APACHE III score. The factors associated with both predictions of mortality and actual mortality, included patient's age, presence of malignancy, a medical ICU diagnosis, and the APACHE III score. Other variables that were associated with the actual mortality but not predicted mortality included being hospitalized in the previous year, liver disease, and the presence of cognitive impairment. Physician factors associated with predictions of mortality at six months in the univariate analysis included older age and more years since graduation.

The multivariable logistic regression model for predicted mortality included patient's age (OR 1.03; 95% CI 1.01–1.05, for each year of increase), the presence of malignancy (OR 2.63; 1.51–4.59), being in a medical ICU (OR 5.10; 95% CI 2.48–10.68), and patient's APACHE III score (OR 1.16; 95% CI 1.06–1.27, for every 10-point increase) (Table 2). This model showed a moderate discrimination (area under the ROC curve 0.77; 95% CI 0.72–0.83) and good calibration (p = 0.89). When performing our analysis using a GEE approach, similar point estimates and confidence intervals were obtained (Table 2). These same factors were all associated with actual six-month mortality when combined in a multivariable model (Table 2). The pre-defined interaction term of malignancy and APACHE III was not significant in any of these models.

Table 2.

Multivariable analysis^a of the association between baseline characteristics, predictions of six-month mortality, and actual six-month mortality.

	Mortality prediction at six months		Mortality occurred at six months
	Logistic regression	GEE	Mortality occurred at six months
	Odds ratio (95% CI)	Odds ratio (95% CI)	Odds ratio (95% CI)
Patient's age, years	1.03 (1.01–1.05)	1.03 (1.01–1.05)	1.05 (1.03–1.07)
Malignancy	2.63 (1.51–4.59)	2.48 (1.43–4.30)	2.61 (1.47–4.66)
ICU diagnosis medical vs. surgical	5.10 (2.48–10.68)	4.49 (2.04–9.92)	5.25 (2.70–10.22)
APACHE score (per 10 unit of increase)	1.16 (1.06–1.27)	1.15 (1.05–1.25)	1.27 (1.16–1.39)

All analyses are multivariable and are adjusted by the variables exhibited in the table.

Accuracy of predictions

Among predictions of mortality at six months, patients of older age, who were in a medical ICU and had higher APACHE III scores were associated with higher accuracy (Table 3). These same factors were associated with inaccurate predictions of six-month survival (Table 3). The LOWESS plots of baseline factors compared to the accuracy of predictions are shown in Figure 1.

Table 3.

Association between baseline characteristics and accuracy of predictions.

	Accurate prediction of survival at six months (76.41% were accurate predictions)		Accurate prediction of death at six months (81.82% were accurate predictions)
	Unadjusted OR (95% CI)	Adjusted OR^a (95% CI)	Unadjusted OR (95% CI)	Adjusted OR^a (95% CI)
Physician's age	0.95 (0.92–0.98)	0.97 (0.93–1.01)	0.98 (0.92–1.04)	0.93 (0.85–1.02)
Physician's gender, male	0.89 (0.36–2.24)	1.16 (0.53–2.56)	1.00 (0.33–2.99)	0.84 (1.49–4.81)
Patient's age	0.97 (0.95–0.99)	0.96 (0.93–0.98)	1.03 (0.99–1.07)	1.05 (1.01–1.98)
Patient's gender, male	1.30 (0.70–2.43)	1.13 (0.53–2.40)	0.58 (0.19–1.71)	0.53 (0.14–1.99)
ICU diagnosis medical vs. surgical	0.42 (0.20–0.89)	0.34 (0.15–0.80)	2.67 (0.70–10.24)	17.49 (2.18–140.21)
APACHE score (per 10 unit of increase)	0.79 (0.70–0.90)	0.78 (0.69–0.90)	1.17 (0.99–1.39)	1.23 (1.02–1.51)

All adjusted analyses include physicians' age and gender, patients' age and gender, medical ICU and APACHE score.

Confidence of predictions

Predictions of six-month survival were more often confident (43.3%) compared to six-month predictions of mortality (35.7%), although this finding was not statistically significant (p = 0.21). Among those predictions of death at six months, higher physicians' age was associated with nonconfident predictions while being in a medical ICU was associated with confident predictions (Table 4). Among those predictions of survival at six months, the adjusted analysis showed that patient's older age, being in a medical ICU and higher APACHE III score were associated with nonconfident predictions. Figure 1S shows the LOWESS plots of baseline variables with the confidence of predictions (supplemental digital content).

Table 4.

Association between baseline characteristics and confident predictions.

	Confident prediction of survival at six months (43.3% were confident predictions)		Confident prediction of death at six months (35.7% were confident predictions)
	Unadjusted OR (95% CI)	Adjusted OR^a (95% CI)	Unadjusted OR (95% CI)	Adjusted OR^a (95% CI)
Physician's age	0.94 (0.90–0.98)	0.97 (0.93–1.01)	0.90 (0.84–0.98)	0.88 (0.81–0.97)
Physician's gender, male	0.46 (0.19–1.13)	1.01 (0.53–1.92)	0.47 (0.17–1.29)	0.90 (0.26–3.01)
Patient's age	0.98 (0.96–0.99)	0.97 (0.94–0.99)	0.99 (0.96–1.02)	1.00 (0.97–1.03)
Patient's gender, male	1.30 (0.74–2.29)	1.22 (0.65–2.30)	0.68 (0.30–1.55)	0.62 (0.24–1.64)
ICU diagnosis medical vs. surgical	0.28 (0.16–0.52)	0.24 (0.12–0.48)	3.80 (0.64–22.5)	6.11 (1.02–36.31)
APACHE score (per 10 unit of increase)	0.86 (0.78–0.95)	0.87 (0.78–0.98)	1.06 (0.94–1.20)	1.01 (0.93–1.24)

All adjusted analyses include physicians' age and gender, patients' age and gender, medical ICU, and APACHE score.

Discussion

Among a cohort of critically ill patients and the ICU attending physicians that were responsible for their care, we found a variety of patient factors were associated with physicians' predictions of six-month mortality vs. survival. We also found both physician and patient factors that were associated with the accuracy and confidence in these predictions. Patient factors associated with the direction (towards mortality) and the accuracy of the prediction included older age, a higher severity of illness, a history of malignancy and being in a medical ICU. When death was predicted, older physicians were less confident in their predictions while being in a medical ICU was associated with higher confidence.

Prognostication is an important element in providing care for critically ill patients. Regardless of the underlying mechanisms and pathways that lead to a prognosis, it has an important role in shared decision-making.^3,17–19 This study is novel as it used a quantitative approach to measure which factors are associated with physicians' predictions of critically ill patients' mortality. The patient-level factors associated with such predictions of mortality have face validity, since older age, the presence of malignancy, and the severity of disease are known factors associated with increased mortality.^20,21 Furthermore, the same set of variables associated with physician's predictions were also associated with the observed mortality at six months. Previous literature suggests that physicians' predictions of critically ill patients' mortality perform well in comparison with objective predictive tools.^{10,11,22–25} Our results thus suggest that the process driving physician's judgements are potentially driven by these universal poor prognostic factors, such as age, higher severity of illness, and the presence of life limiting diseases, like malignancy.^4,5

An interesting aspect of our study is the potential clustering effect of the predictions based on the individual physicians. We expected that since 47 physicians performed predictions for 300 patients, some of the factors associated with the outcome would be dependent on the individual physicians. GEE represents a flexible method to analyze for correlated observations.¹⁶ The results of GEE compared to other methods that ignore independence would not change the point estimates (i.e. odds ratio, coefficients) but would rather affect the standard error and thus the confidence intervals.¹⁶ However, in this study the confidence intervals of the conventional logistic regression approach were similar to those of the GEE model (Table 3). This suggests the predictors associated with the outcome are independent of the subjectivity of individual physicians.

One of the strengths of our study is that it reports associations of patient and physician factors with accuracy and confidence in predictions. The relationship of these factors with accuracy was different depending on the direction of the prediction. Interestingly, the same variables associated with incorrect predictions of survival were the same variables associated with correct predictions of mortality. When physicians think a patient will die, factors such as age, being in a medical (vs. surgical) ICU, and severity of disease are considered and increase the likelihood of a correct prediction. However, when physicians think patients will survive, they may neglect these factors and make incorrect predictions. Another interesting finding results from the association of physicians' age and the accuracy and confidence of the predictions. Our findings showed that a higher physicians age was associated with nonconfident mortality predictions. This finding is interesting in face of previous literature that suggested that the most experienced doctors performed more accurate predictions.²⁴ Perhaps, this reflects that physicians become less confident being older because they have experienced being humbled in their predictions more frequently.

This study has several limitations that merit discussion. First, although we aimed to explore which individual patient and physician variables influence physicians' judgement on patient's prognosis, we do not know how physicians used these variables in their predictions and how they incorporated the evidence to their judgments. Furthermore, it is not clear if physicians incorporated information on available predictive tools (i.e. APACHE III score) to make their predictions. Although it is not clear whether these judgments were made solely on intuition or expert intuition (or both), the fact that all the final predictors retained in our model are universally recognized as poor prognostic factors suggest that they might have consciously or subconsciously contributed to their predictions.^4,5 Second, the variables associated with predictions of mortality were similar to the variables associated with the actual six-month mortality, suggesting a possible self-fulfilling prophecy. However, since this study focused on six-month mortality rather than in hospital mortality, it might be less likely that it influences outcomes distant in time. A self-fulfilling prophecy also seems unlikely given that in this dataset there were only 29 patients who were predicted to die in hospital that actually did die in hospital.¹³ Third, this study was conducted at three hospitals associated with an academic medical center, and therefore our results might not be generalizable to hospitals in different regions or in nonacademic environments.

Conclusions

In conclusion, this study demonstrates that several patient factors are associated with predictions of mortality at six months, while patient and physician factors are associated with the accuracy and the confidence of these predictions. Interestingly, the same factors associated with increased accuracy of predictions of mortality were associated with decreased accuracy when predictions of survival were made. This could have important implications for clinical practice, suggesting that the presence of increased age, being in a medical ICU and higher severity of disease should be highlighted as important predictors of death, even when physicians think a patient will survive. This should be considered when physicians reflect on how they make predictions for critically ill patients and the way this information is used in shared clinical decision-making with family members and patients.

Supplemental Material

Supplemental material for Factors associated with physicians’ predictions of six-month mortality in critically ill patients

Supplemental Material for Factors associated with physicians’ predictions of six-month mortality in critically ill patients by Bruno L Ferreyro, Michael O Harhay and Michael E Detsky in Journal of the Intensive Care Society

Footnotes

Acknowledgements

Dr Ferreyro would like to thank Dr Federico Angriman and Dr Andre Amaral for their thoughtful feedback on these study findings. Dr Ferreyro and Dr Detsky would also like to thank Toronto Clinical Research in Progress (CRIP) group for their input while this study was conceived.

Authors' contribution

Drs Detsky and Ferreyro had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Ferreyro, Detsky.

Acquisition, analysis, or interpretation of data: Ferreyro, Harhay, Detsky.

Drafting of the article: Ferreyro, Detsky.

Critical revision of the article for important intellectual content: Ferreyro, Harhay, Detsky.

Statistical analysis: Ferreyro, Harhay, Detsky.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: MOH was supported by the National Institute of Health (NIH)/National Heart, Lung and Blood Institute (NHLBI) [grant K99 HL141678]; MED was supported by the National Heart, Lung, and Blood Institute [T32-HL098054].

References

Légaré

Witteman

. Shared decision making: examining key elements and barriers to adoption into routine clinical practice. Health Aff (Millwood) 2013; 32: 276–284.

Truog

Campbell

Curtis

, et al. Recommendations for end-of-life care in the intensive care unit: a consensus statement by the American College [corrected] of Critical Care Medicine. Crit Care Med 2008; 36: 953–963.

Kon

Davidson

Morrison

, et al. Shared decision making in ICUs: An American College of Critical Care Medicine and American Thoracic Society Policy Statement. Crit Care Med 2016; 44: 188–201.

Herridge

. Prognostication and intensive care unit outcome: the evolving role of scoring systems. Clin Chest Med 2003; 24: 751–762.

Vincent

J-L

Moreno

. Clinical review: scoring systems in the critically ill. Crit Care 2010; 14: 207.

Breslow

Badawi

. Severity scoring in the critically ill: part 1–interpretation and accuracy of outcome prediction scoring systems. Chest 2012; 141: 245–252.

Vincent

J-L

Opal

Marshall

. Ten reasons why we should NOT use severity scores as entry criteria for clinical trials or in our treatment decisions. Crit Care Med 2010; 38: 283–287.

Lemeshow

Klar

Teres

. Outcome prediction for individual intensive care patients: useful, misused, or abused?. Intensive Care Med 1995; 21: 770–776.

Hallen

SAM

Hootsmans

NAM

Blaisdell

, et al. Physicians' perceptions of the value of prognostic models: the benefits and risks of prognostic confidence. Health Expect 2014; 18: 2266–2277.

10.

Sinuff

Adhikari

NKJ

Cook

, et al. Mortality predictions in the intensive care unit: comparing physicians with scoring systems. Crit Care Med 2006; 34: 878–885.

11.

McClish

Powell

. How well can physicians estimate mortality in a medical intensive care unit?. Med Decis Making 1989; 9: 125–132.

12.

Afessa

Keegan

. Predicting mortality in intensive care unit survivors using a subjective scoring system. Crit Care 2007; 11: 109.

13.

Detsky

Harhay

Bayard

, et al. Discriminative accuracy of physician and nurse predictions for survival and functional outcomes 6 months after an ICU admission. JAMA 2017; 317: 2187–2189.

14.

Christakis

. Prognostication and bioethics. Daedalus 1999; 128: 197–214.

15.

Kellett

. Prognostication–the lost skill of medicine. Eur J Intern Med 2008; 19: 155–164.

16.

Hanley

. Statistical analysis of correlated data using generalized estimating equations: an orientation. Am J Epidemiol 2003; 157: 364–375.

17.

Teno

Fisher

Hamel

, et al. Decision-making and outcomes of prolonged ICU stays in seriously ill patients. J Am Geriatr Soc 2000; 48: S70–S74.

18.

Charlson

Sax

MacKenzie

, et al. Resuscitation: how do we decide? A prospective study of physicians' preferences and the clinical course of hospitalized patients. JAMA 1986; 255: 1316–1322.

19.

Christakis

Lamont

. Extent and determinants of error in doctors' prognoses in terminally ill patients: prospective cohort study. BMJ 2000; 320: 469–472.

20.

White

Reid

Harris

, et al. A systematic review of predictions of survival in palliative care: how accurate are clinicians and who are the experts?. PLoS ONE 2016; 11: e0161407–e0161420.

21.

Lambden

Zhang

Friedlander

, et al. Accuracy of oncologists' life-expectancy estimates recalled by their advanced cancer patients: correlates and outcomes. J Palliative Med 2016; 19: 1296–1303.

22.

Rocker

Cook

Sjokvist

, et al. Clinician predictions of intensive care unit mortality. Crit Care Med 2004; 32: 1149–1154.

23.

Marcin

Pollack

Patel

, et al. Combining physician's subjective and physiology-based objective mortality risk predictions. Crit Care Med 2000; 28: 2984–2990.

24.

Christensen

Cottrell

Murakami

, et al. Forecasting survival in the medical intensive care unit: a comparison of clinical prognoses with formal estimates. Methods Inf Med 1993; 32: 302–308.

25.

Radtke

Pfister

Kuhr

, et al. Is “gut feeling” by medical staff better than validated scores in estimation of mortality in a medical intensive care unit? – The prospective FEELING-ON-ICU study. J Crit Care 2017; 41: 204–208.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.20 MB