Abstract
Objective
Intensive care unit (ICU) discharge is a decision process that is usually performed subjectively. We evaluated whether a subjective score (Sabadell score) is associated with hospital outcomes.
Methods
We conducted a prospective cohort study from August 2014 to May 2015 at a tertiary-care private hospital in Brazil. We analyzed 425 patients who were discharged alive from the ICU to the wards. We used univariate and multivariate analysis to identify risk factors associated with a composite endpoint of worse outcomes (later ICU readmission or ward death) during the same hospitalization.
Results
Forty-three patients (10.1%) were readmitted after ICU discharge, and 19 died in the ward. Compared with patients with successful outcomes, those with the composite endpoint were older and more severely ill, had a nonsurgical reason for hospitalization, more frequently came from the ward, were less frequently independent during daily activities, had sepsis, had higher C-reactive protein concentrations at ICU admission, and had higher Sabadell scores at discharge. The multivariate analysis showed that sepsis and the Sabadell score were independently and significantly associated with worse outcomes.
Conclusion
Sepsis at admission and the Sabadell score were predictors of worse hospital outcomes. The Sabadell score might be a promising predictive tool.
Background
Survivors of critical illness are frequently vulnerable to new complications after intensive care unit (ICU) discharge, including readmission and short-term mortality. 1 Risk factors for such adverse events following patient discharge from the ICU should be identified to avoid increases in morbidity and/or mortality. Several risk factors associated with worse outcomes have been studied, such as age, nighttime discharge, admission source, and transfer to a high-dependency unit.2,3 In a recent retrospective cohort of patients with oncohematological diseases, we found that ICU readmissions increased mortality by 10-fold and were associated with male sex, emergency surgery, a longer hospital stay before ICU transfer, and invasive mechanical ventilation. 4 Most of these risk factors are associated with a higher severity of illness at ICU admission or previous chronic health problems. However, the means by which to combine their relevancy and weight into a straightforward decision is still unknown. Additionally, some previous studies focused on data solely at ICU admission. Assessment at ICU discharge would likely allow for better prediction of later outcomes. 4
Physicians usually rely on their clinical judgment to decide whether the patient is ready to be discharged from the ICU and to choose the proper discharge facility (e.g., the ward or high-dependency unit). 5 Some previous data suggest that death might be more accurately predicted by an ICU physician’s subjective impression than by usual scoring systems. 6 In fact, the Sabadell score is a subjective tool for prediction of the post-ICU prognosis that exhibits good discriminative ability.7,8 Nevertheless, its validation is restricted to developed countries, and it may not fully apply to low- and middle-income countries.
The primary objective of this study was to evaluate the ability of the Sabadell score to predict worse outcomes after ICU discharge in a tertiary-care hospital located in a middle-income country.
Methods
This was a prospective cohort analysis of patients admitted to an “open format” mixed medical-surgical ICU. We characterized our unit and hospital in a previous report. 4 Briefly, it is a private tertiary-care hospital in São Paulo, Brazil; cardiac surgical patients are managed in another unit within our hospital. Admission and discharge decisions are made after discussion between the patient’s attending physician and the intensive care physician. This study was approved by the local institutional ethics committee of Hospital Sirio-Libanes, which waived the requirement for informed consent because of the observational design of the study (CAAE: 21503913.5.0000.5461).
The study population comprised consecutive adult patients (>18 years of age) who were admitted from August 2014 to May 2015 and discharged alive. The exclusion criteria were an ICU length of stay of <12 h, pregnancy, and patient unsuitability for ICU readmission (death on the unit or transfer to another hospital or to palliative care). Data were retrieved from the ICU administrative database (Sistema Epimed™; www.epimedmonitor.com)9,10 and included age, sex, Simplified Acute Physiology III Score,11,12 admission source, diagnosis, comorbidities, resource use during ICU stay, frequency of nighttime and weekend discharges, Sequential Organ Failure Assessment (SOFA) score at ICU admission and discharge,
13
C-reactive protein and lactate levels at ICU admission and discharge, and hospital mortality. Sepsis was defined according to a previous consensus definition at the time of data collection.
14
Readmission was defined as ICU admission of a patient who had been previously admitted to the ICU during the same hospitalization. If multiple readmission episodes occurred, only the first was considered for the present analysis. Based on the time of ICU discharge, the patients were categorized into daytime (7:00
At ICU discharge, the attending intensivist scored the patient’s subjective prognosis as reported in the Sabadell score.7,8 Mutually exclusive groups of patients were defined as follows: score of 0, good prognosis in the long term (>6 months); score of 1, poor prognosis in the long term (>6 months) and suitable for ICU readmission without restrictions; score of 2, poor prognosis in the short term (<6 months) and with debatable suitability for ICU readmission; and score of 3, not expected to survive the hospital stay. The Sabadell score is entirely subjective and integrates the physician’s knowledge and impression about the patient’s condition, previous performance before ICU admission, and medical history during the healthcare facility stay. The ward team was blind to the Sabadell score; likewise, the ICU physicians who evaluated requests for ICU readmission were blind to the Sabadell score (the decision regarding ICU readmission was at the discretion of the ward physician only).
Statistical analysis
Data were analyzed with IBM SPSS Statistics for Windows, Version 20.0 (IBM Corp., Armonk, NY, USA). Normally distributed data were tested with the Kolmogorov–Smirnov test. Data are presented as the mean (standard deviation) or median [25th–75th percentile] accordingly. Categorical variables are presented as absolute number (percentage). Parametric variables were compared between groups with an unpaired Student’s t-test or analysis of variance, and nonparametric variables were compared between groups using the Mann–Whitney test or Kruskal–Wallis test. Categorical data were compared with the chi-squared test. All statistics were two-tailed, and a p-value of <0.05 was considered statistically significant.
A composite endpoint that included ICU readmission or unexpected ward death was assessed. A readmission event is strongly associated with an increased risk of hospital mortality3,4 and is usually applied as a metric of quality of care. 15 Thus, combining readmission with ward death into a composite endpoint sounds plausible, is clinically meaningful, avoids competing risk between these variables, and increases the statistical power of the study to evaluate risk factors for these events. 16 A multivariate logistic regression analysis was performed, with worse outcomes as the dependent factor. Variables with a p-value of <0.1 in the univariate analysis were included in the logistic model. Multicollinearity was excluded with the variance inflation factor before modeling. 17 The model was refined using the backward stepwise method, excluding the least significant variable at each step if the significance level was >0.05. As a sensitivity analysis, the same procedure was performed for ICU readmission. Calibration and discrimination of the prediction model were evaluated with the Hosmer–Lemeshow goodness-of-fit test and the area under the curve (AUC), respectively. Survival curves were constructed with the Kaplan–Meier method censored at 90 days and compared with the log-rank test.
Results
Of 425 patients admitted to the ICU and discharged alive during the study period, readmission occurred in 43 patients (10.1% of discharged patients) and death occurred in 19 patients (4.5%). The characteristics of the study group are presented in Table 1. At ICU admission, patients who developed worse outcomes after ICU discharge were older, were sicker, were more frequently male, had a nonelective surgical reason for admission, were usually from the ward, more frequently had sepsis, were less frequently independent during daily activities, more frequently required invasive procedures, had higher SOFA scores both at ICU admission and discharge, had higher C-reactive protein values at admission, and higher Sabadell scores at discharge.
Patient characteristics at intensive care unit admission and discharge.
Data are presented as n (%), mean ± standard deviation, or median [25th–75th percentile].
SAPS III, Simplified Acute Physiology III Score; SOFA, Sequential Organ Failure Assessment; CRP, C-reactive protein; ICU, intensive care unit.
*p-value for comparison between patients with and without worse outcomes.
At ICU discharge, the attending physicians classified the patients as follows: score of 0, 219 (51.5%); score of 1, 134 (31.5%); score of 2, 65 (15.3%); and score of 3, 7 (1.6%). The baseline characteristics and ICU procedures were different between the two groups (Table 2). Ward mortality was 4.5% among all patients and differed for each Sabadell score group: score of 0, 0.5%; score of 1, 2.2%; score of 2, 16.9%; and score of 3, 57.1% (p < 0.001) (Figure 1). The readmission rates were also different for each group: score of 0, 4.6%; score of 1, 14.2%; score of 2, 20.0%; and score of 3, 14.3% (p = 0.001). When considering any unwanted outcome, the observed incidences were as follows: score of 0, 5.0%; score of 1, 14.9%; score of 2, 26.2%; and score of 3, 57.1% (p < 0.001).
Survival curves in the ward for patients in each group according to the Sabadell score.
Clinical characteristics according to the Sabadell score at intensive care unit discharge.
Data are presented as n (%), mean ± standard deviation, or median [25th–75th percentile].
SAPS, Simplified Acute Physiology III Score; SOFA, Sequential Organ Failure Assessment; CRP, C-reactive protein; ICU, intensive care unit.
In the multivariate analysis (Table 3), independent risk factors for worse outcomes were sepsis at ICU admission and Sabadell scores at ICU discharge. The Hosmer–Lemeshow test was not statistically significant for the final model. The AUC was 0.73 [95% confidence interval (CI), 0.66–0.81; p < 0.001]. The sensitivity analysis with ICU readmission as the outcome identified the same independent predictors, but a Sabadell score of 3 was no longer significant (Table 4). The Hosmer–Lemeshow test was not statistically significant for the readmission model. The AUC was 0.71 (95% CI, 0.63–0.79; p < 0.001).
Factors associated with worse outcomes after intensive care unit discharge in the multivariate analysis.
OR, odds ratio; CI, confidence interval
Factors associated with intensive care unit readmission in the multivariate analysis.
OR, odds ratio; CI, confidence interval.
Discussion
The main finding of our single-center prospective observational cohort study is the applicability of a subjective score (Sabadell score) to prediction of unfavorable events after ICU discharge. Because these events suggest poor discharge decision-making and are associated with a longer hospital stay, increased resource consumption, and greater morbidity, tools with which to correctly identify patients who are ready to be discharged are required.
Our ward mortality rate is similar to that in a previous multicenter study that validated the Sabadell score in Spain 8 and is in line with a recent meta-analysis that aimed to estimate hospital mortality among patients discharged alive from the ICU. 18 However, we observed a readmission rate of 10.1%, which is higher than that in most previously published studies19–21 as well as in the above-mentioned meta-analysis, which suggested that readmission rates generally range from 4% to 6% in critically ill patients. 18 Our high readmission rate might have been due to case-mix differences (e.g., the presence of a step-down unit in our hospital, a high proportion of patients with oncohematological diseases in our cohort, different end-of-life practices in middle-income countries wherein some patients who should receive palliative care in the wards are readmitted). In fact, Kramer et al. 20 recently acknowledged the problem with case-mix and readmission rates. In-hospital mortality adjustment nullified the differences in standardized mortality between ICUs with high and low rates of readmission. Thus, quality indicators and predictive tools might work differently across facilities. This is one of the reasons why we decided to study the validity of the Sabadell score in our institution.
Notably, the Sabadell score was originally described to predict ward mortality.7,8 However, ICU readmission is associated with post-ICU death.3,4,8 A recent European Society of Intensive Care Medicine report suggests that this metric can serve as a quality indicator 15 because some deaths associated with readmission are thought to be preventable. 22 Therefore, the relationship between the Sabadell score and readmission as a possible endpoint was plausible. Actually, our observed mortality associated with Sabadell score strata is similar to that in a previous report, 8 and the observed AUC is similar to that described for the Stability and Workload Index for Transfer (SWIFT) score, a predictive tool for ICU readmission. 23 A recent systematic review of published tools to predict adverse events (i.e., ICU readmission, death after ICU discharge, or medical emergency team activation) revealed a single tool (the Minimizing ICU Readmission score) that could predict both post-ICU discharge mortality and readmission. 24 The Minimizing ICU Readmission score includes data related to the severity of illness (Simplified Acute Physiology II score at ICU admission, central venous catheter use during the hospital stay, SOFA score at ICU discharge, and systemic inflammatory response syndrome in the last 2 days before ICU discharge) and the discharge policy (discharged from the ICU at night). Interestingly, this score has a discrimination ability similar to our findings (AUC, 0.74; 95% CI, 0.68–0.79). 25 Finally, our sensitivity analysis for ICU readmission, which should be viewed only as a hypothesis-generating analysis, identified the same predictors; however, a Sabadell score of 3 was no longer significant, probably because ICU readmission was not considered adequate for these patients (e.g. end-of-life planning). It should be stressed that in our study, the ward physician was solely responsible for the decision regarding ICU readmission, and he/she was unaware of the Sabadell score and the objective of the present study.
Clinical decision-making is frequently required of critical care practitioners and should be performed in a timely fashion. 26 In many instances, there is a lack of solid evidence-based support for such decisions, 27 especially related to ICU discharge policies. Some previous publications suggest that subjective physician impressions are more accurate in the prediction of outcomes after critical illness than are contemporary prognostic scores.28–30 A systematic review suggested that physicians’ predictions were more accurate than scoring systems’ predictions (area under the summary receiver operating characteristic curve: 0.85 ± 0.03 for physicians and 0.63 ± 0.06 for scoring system predictions, p = 0.002). 6 Although scoring systems might reduce undesirable variability among health practitioners, an experienced physician usually subconsciously integrates all relevant information (e.g., history, clinical parameters at ICU admission, current state at ICU discharge, and presence of vulnerability). Therefore, their subjective impression about a patient’s prognosis could be as accurate or even better in the prediction of outcomes. Our results corroborate this hypothesis as previously demonstrated in a multicenter Spanish cohort. 8
Our study has several limitations. First, it was a single-center prospective analysis of a private tertiary-care oncology center, which might limit the generalizability of our findings and our sample size. Thus, our results are hypothesis-generating, and caution is required when interpreting them. However, this is the first analysis involving a cohort derived from a middle-income country, and the concordance with previous publications is reassuring. Second, we only studied admission and discharge factors associated with outcomes. A patient’s history in the ward probably influences his or her hospital outcome, but this was not evaluated in our study. Third, the composite endpoint of ICU readmission and ward death gives the same importance for both outcomes. Some may argue that the relative importance of each of the components might be different for patients and clinicians. Although we acknowledge that this might be true, readmission is a relevant outcome. Furthermore, the composite endpoint increases the power of our study to detect potentially relevant risk factors and avoids the competing risks of readmission and death. 16 Fourth, a reliability evaluation of the Sabadell score for repeatability or agreement between physicians was not carried out. A subjective score might have great variability according to the doctor who applies it. Finally, we only asked intensive care physicians to predict survival and readmission. Recent data suggest that nurses’ perceptions are also relevant. 31
Conclusion
In our cohort of critically ill patients discharged alive from the ICU, sepsis and the Sabadell score were identified as independent risk factors for worse outcomes. Future studies should validate the clinical utility of this score as a clinical decision-making tool.
List of abbreviations
ICU, intensive care unit; SOFA, Sequential Organ Failure Assessment; AUC, area under the curve.
Footnotes
Declaration of conflicting interest
The authors declare that there is no conflict of interest.
Funding
This study was supported by institutional funding from the Research and Education Institute, Hospital Sirio-Libanes, São Paulo, Brazil. Financial support for this study was provided entirely by a grant from the Research and Education Institute of the Hospital Sirio-Libanes. The funding agreement ensured the authors’ independence in designing the study, interpreting the data, and writing and publishing the report. We thank the Research and Education Institute of the Hospital Sirio-Libanes for providing this financial support. No external funding was obtained for this study.
Previous presentation
Some results of this study were presented at the 29th European Society of Intensive Care Medicine Annual Congress, 1–5 October 2016, in Milan, Italy.