Using a CriSTAL scoring system to identify pre-morbid conditions associated with a poor outcome after admission to intensive care in people 70 years or older

Abstract

Older people admitted to intensive care are considered to have lower physiological reserves, an increased susceptibility to infection and longer recovery times, resulting in generally poorer outcomes after intensive care treatment. However, biological heterogeneity makes identification of those with the best chances of survival within their group difficult and risks subjecting those at the end of their lives to unsuccessful treatments. There is no fit-for-purpose outcome prediction tool capable of identifying patients most at risk of these poor outcomes at the point of admission to intensive care. This retrospective study sought to identify factors associated with mortality in older patients (≥70 years) admitted to a teaching hospital critical care unit using objective variables readily available at the point of admission. A total of 15 variables were tested for a significant association with mortality. Of these, eight were identified as significant variables (myocardial infarction within 6 months, an abnormal ECG, congestive cardiac failure (NYHA ≥2), chronic pulmonary disease, chronic liver disease, metastatic cancer, a stay in hospital ≥5 days preceding ICU admission, and frailty (Clinical Frailty Score ≥4)). These variables were used from the basis of a novel outcome prediction model. The aim of such a model would be that it could be used at the point of referral to intensive care to inform considerations regarding admission, and to facilitate conversations with the patient and family regarding realistic treatment expectations.

Keywords

Mortality elderly intensive care admission

Introduction

The UK population aged over 65 years has increased by 21%, and the population aged over 85 years has increased by 31% since 2005.¹ The combination of an older, frailer population who are undergoing major surgery and availability of novel treatment options is resulting in an increased demand for all healthcare services.²

Older patients have less physiological reserve, an increased susceptibility to infection, and consequently much longer recovery times, meaning admission to intensive care (ICU) results in generally poorer outcomes.^3–6 Of patients who survive beyond hospital discharge, many encounter physical disability, depressive symptoms, cognitive impairment, and/or post-traumatic stress disorder in addition to the burden of new or worsened physical illness.^7,8 As a result, ICU survivors, irrespective of age, report lower quality-of-life (QOL) outcomes for all domains at baseline and at 6 months to 14 years after discharge.⁹

Admission to intensive care (IC) can be a life-changing event for any individual: loss of financial stability, ongoing poor health, or inability to live independently are all significant outcomes for those who may have apparently achieved a ‘good’ recovery and outcome. This is even more significant for those who may be nearing the end of their lives and in whom admission to IC and ‘survival’ may be accompanied by a significant change in circumstances after discharge and for the last few months of their life. Previously, ICU outcome prediction models have focused on different numerical age brackets to calculate likelihood of poor response to interventions, however, recently the degree of frailty in an individual has shown to be a more accurate predictor of short- and long-term mortality, the decline in functional independence and a lower quality of life after a stay in ICU.^10–12

Understanding the complex relationship between pre-morbid factors and eventual outcomes in the IC population carries huge significance for giving patients and families useful, relevant information on the burdens and potential benefits of treatment, particularly as they get older. Currently, outcome prediction tools focus only on population-level outcomes by using a combination of pre-morbid factors and acute physiology values to produce a standardised numerical value, which represents the predicted outcome from that set of circumstances.^13–15 However, to date, there is no fit-for-purpose stratification tool for use at the point of admission to IC, which might be used to inform such conversations.

Chronic disease variables, markers of chronic ill health and degree of frailty, have been previously identified as significant predictors of short-term (30-day) mortality for elderly ward patients by the Criteria for Screening and Triaging to Appropriate aLternative care (CrISTAL) study. CriSTAL criteria were identified from a large literature review with the aim to develop a screening tool to identify elderly patients in hospital wards at the end of life and quantify the risk of death in hospital or soon after discharge.¹⁶ The variables in the study were selected due to their simplicity, objectivity, and independence from each patient’s immediate resuscitation status.

The aim of this study was to review the CriSTAL criteria in an IC population to determine whether they are also able to identify patients who are less likely to survive despite ICU admission. It is intended that the final model would be able to offer a starting point to begin discussions with the patient and their relatives about likelihood of survival from an ICU admission and the possibility of alternative treatment options, e.g. end-of-life care.

Methods

The project was registered (CEU project number 7710) as an extension of a previous service review registered and conducted at Sheffield Teaching Hospitals (CEU project number 6825) in September 2016.

A predetermined sample size of 1000 elderly patients (70 years and older) admitted as an emergency to critical care units at the Northern General Hospital (NGH) or the Royal Hallamshire Hospital (RHH) in Sheffield between March 2015 and December 2017 were included in the review. Variables used were adopted from the original CriSTAL scoring system with the aim of creating a new, unique IC scoring system for our unit and included data that are currently routinely collected or noted as part of the patient’s admission and stay in ICU. Fifteen variables from the original model were identified for further investigation in an ICU population. These consisted of ten chronic disease variables (chronic kidney disease (CKD) ≥ Stage 3, myocardial infarction (MI), abnormal ECG prior to admission (any ECG not showing sinus rhythm and/or brady/tachycardia), congestive cardiac failure (CCF), chronic obstructive pulmonary disease, chronic liver disease, cancer, dementia, psychiatric history) and four markers of health (any amount of documented weight loss, stay in hospital ≥5 days preceding ICU admission, ICU readmission during the same hospital stay) and frailty. The Clinical Frailty Score (CFS) was used to grade the varying degrees of frailty. The CFS is an internationally-validated nine-point assessment tool designed to quantify frailty in the critically ill.^10–12 The CFS ranges from 1 (very fit) to 9 (terminally ill). Groups used in this review were CFS grade: ≤3 (No significant frailty), 4 (Vulnerable), 5 (Mildly frail), 6 (Moderately frail), 7 (Severely frail), and 8 (Very severely frail). Each patient admitted to critical care in Sheffield is routinely assigned a CFS based on their overall health status prior to hospital admission and during their hospital stay, and scores are documented in the patient’s record. These scores were used for analysis in this study. Of these 15 independent variables, six were sub-divided into severity groups to attempt to identify the point when certain chronic diseases become significantly associated with mortality in an ICU population. These were CKD (Stages 3, 4, and 5), MI (time of occurrence within 6 months and ≥6 months prior to admission to ICU), CCF (NYHA grades 1, 2, 3, and 4), cerebrovascular disease (transient ischaemic attack and stroke), cancer (isolated solid tumour, local metastasis, and metastatic disease), and frailty (Rockwood CFS grades 4, 5, 6, and 7). Each parameter was assigned a numerical value. In Cr1 all variables were scored 1 point whilst in Cr2 parameters were weighted with more significant variables assigned higher scores (see Online Appendix 1).

All patient level of care was also documented as either level 2 (patients receiving basic respiratory support, e.g. mask/hood CPAP or single-organ support) or level 3 (patients receiving advanced respiratory support, e.g. invasive mechanical ventilation or requiring a minimum of two organs supported). This enabled the two levels of care to be analysed separately, reducing the influence of selection bias.

A retrospective epidemiological database analysis of admission and outcome data was collected from the electronic patient data management system (Metavision). Only ICU-based outcomes were studied whereby survival was indicated by discharge from the unit, whilst mortality was death in the unit. The sample was divided randomly into two equally sized groups (Training and Test) for two different retrospective analyses using SPSS version 23.0 for Windows (IBM Corp. 2014. Armonk, NY: IBM Corp). The purpose of this was to examine the prognostic ability of two modified CriSTAL models (Cr1 and Cr2). The first half of the dataset (Training) would identify significant variables and test the Cr1 model which contained all 15 variables mentioned previously, whereby the absence of a predictor scored 0 points whilst the presence of a predictor scored 1 point, totaling a maximum score of 15. Binary data were analysed with a logistic regression, which was used to determine the mortality odds ratios, along with the degree of significance of each CriSTAL variable on outcome. This analysis was then used to create the new modified CriSTAL model (Cr2), which would then be applied and tested on the second half of the sample (Test). Using a two-sample method meant no patients would be tested twice in an attempt to reduce bias in the model and best replicate its clinical use in different patient cohorts. Due to the small sample used in this study, a p value less than 0.1 was used to determine variables included in Cr2 to formulate a model that included clinically meaningful as well as statistically significant variables causatively included in the Cr2 model. Performing a “Backwards LR” in binary logistic regression function to the training sample, insignificant criteria were eliminated in a stepwise manner and the associated impact on the model’s predictive power calculated. Using mortality odds ratios as a guide, the remaining criteria were weighted appropriately and formed the new modified CriSTAL model (Cr2) (Online Table 1). Final analysis focused on comparing the predictive power of each model and its clinical appropriateness using receiver operator characteristic (ROC) associated AUC values.

Results

Data were obtained from the period 1 March 2015 to 31 December 2016 for a total of 1000 consecutive elderly (70 years and older) patients admitted to critical care, with a mean age of 77.5 years ( ± 5.83 SD; range: 70–101 years). The distribution in levels of care was approximately 3:1 for level 2 and level 3 organ supports, respectively (755 level 2 patients and 245 level 3 patients). In all, 728 (72.8%) patients were alive at discharge and the remaining 272 (27.2%) died in the unit. The total sample was split randomly into two comparable size groups termed Training (mean age: 77.3 ± 5.83 SD; total mortality: 29.2%) and Test (mean age: 77.7 ± 5.83 SD; total mortality: 25.2%) for different analyses. In total, frailty (CFS ≥ 4) was observed in 699 (69.9%) (median = CFS4 (IQR ≤ 3–4), range 1–7) patients, whilst the remaining 301 (30.1%) were deemed to have no significant degree of frailty.

Analysis of the Training sample with the Cr1 model identified eight variables to be clinically important in predicting mortality (p ≤ 0.1) including: MI within 6 months of admission (p = 0.009; OR 2.67; 95%CI 1.28–5.58), abnormal ECG (p = 0.001; OR 2.28; 95%CI 1.42–3.67), congestive cardiac failure (NYHA ≥ 2) (NYHA = 2; p = 0.76; OR 2.43; 95%CI 0.91–6.50), chronic obstructive pulmonary disease (p = 0.026; OR 1.83; 95%CI 1.08–3.12), chronic liver disease (p < 0.001; OR 12.73; 95%CI 3.14–51.62), metastatic cancer (p = 0.21; OR 3.46; 95%CI 1.21–9.93), stay in hospital ≥ 5 days preceding critical care admission (p < 0.001; OR 2.83; 95%CI 1.62–4.97), and frailty (CFS ≥ 4) (CFS 4; p = 0.054; OR 1.86; 95%CI 0.99–3.48). Logistic regression results showed that both models were able to predict population-based outcome to a reasonable degree (Cr1 – 76.6% vs Cr2 – 79.6%). Of these results, specificity was high, whilst sensitivity was comparatively low (Cr1 – 98.3% and 38.4%, respectively, vs Cr2 – 96.2% and 28.2%, respectively).

Both models were able to differentiate patients depending on outcome according to the independent T tests. The survival group tested using Cr1 (mean 2.79; SD 1.52; N = 354) had statistically lower CriSTAL scores than patients who died in the unit (mean 3.65; SD 1.50 N = 156) p < 0.001 (mean difference −0.96; 95%CI −1.25 to −0.67). Similarly using the Cr2 model, survivors (Cr2 – mean 3.72; SD 2.53; N = 374) scored significantly lower than deceased patients who died in the unit (Cr2 – mean 6.20; SD 3.32; N = 126) p < 0.001 (mean difference −2.48; 95%CI −3.29 to −1.84).

Figure 1(a) and (b) represents the two models’ predictive power in the form of ROC curves. Associated AUC results for Cr1 (AUC 0.67; 95%CI 0.62–0.72; p < 0.001) and Cr2 (AUC 0.72; 95%CI 0.66-0.77; p < 0.001) indicate that whilst both are predictive of mortality, Cr1 would be classified as a poor predictive model (AUC 0.6–0.7) whereas Cr2 would be classified a fair prediction model (AUC = 0.7–0.8). Further analysis of the Cr2 model revealed a score of ≥ 4 would correctly identify 73% of dying patients; however, the associated sensitivity trade off suggests that 36.1% of dying patients would be incorrectly classified.

Figure 1.

Receiver operator curves for the Cr1 (a) and Cr2 (b) models. The Cr1 dataset was based on the original CrISTAL dataset. The Cr2 model consisted of fewer variables than the Cr1 dataset.

Discussion

Initial analysis of 500 elderly patients identified eight variables significantly predictive of mortality including moderate-to-severe degrees of frailty, indicating that incorporating frailty into an ICU outcome model was appropriate – a result in agreement with much of the similar literature.^4,12,17–19 Six of these variables already have a large evidence base indicating these would be expected to be significant predictors of mortality in our population. However, two parameters (the presence of any ECG abnormality and a hospital stay ≥5 days preceding admission) were novel, statistically significant indicators in predicting mortality in patients over 70 years of age in our ICU patient population.

Both CriSTAL models (Cr1 and Cr2) expressed statistically significant p values in the independent T tests indicating that CriSTAL scores for patients who survived and died were proven to have absolute differences: both models were therefore able to statistically differentiate patients by outcome. However, the significant mean difference in the Cr1 cohort was only equivalent to 0.96 of a point, a value which clinically cannot be used to effectively separate both groups of patients. The weighting of variables did improve the mean difference between mortality and survival in the Cr2 model, increasing the difference on the groups to 2.48. Whilst both models are statistically appropriate, neither can be suggested at present as clinically useful in any predictive sense simply due to the minimal separation in score for each outcome. However, the benefit of being able to identify poor predictive factors in an ICU population of older patients is helpful when providing information to patients and relatives regarding their views on the likely benefits and burdens of treatment. This was the original aim of our review and adds weight to the concept of conducting conversations on likelihood of survival and therefore potential benefit from ICU admission based on outcome data for the critical care unit.

Logistic regression outputs (Online Appendix 2) were able to calculate the ability to predict population outcome based on the variables in the scoring system. Both models seemed to lack the capability to correctly predict patients likely to die (percentage correct – Cr1 38.4%; Cr2 30.2%) with the later Cr2 model performing notably worse. This indicates the elimination of eight variables from the Cr1 model, whilst classed as insignificant, did contribute to better mortality prediction. Conversely, both models’ ability to predict survival was extremely high (percentage correct Cr1 – 93.8%; Cr2 – 96.3), with Cr2 possessing the higher correct percentage. This analysis found that both models could be clinically useful in helping discussions of likely survival and although eliminating statistically insignificant variables improves the model’s usability and reduces completion time, it does sacrifice accuracy of mortality prediction. However, the high percentage of correct results for survival indicate that patients with no or very few variables of morbidity are extremely likely to survive suggesting that both models might serve better as a survival predictor.

Overall, the calculated predictive capacities of Cr1 and Cr 2 were deemed to be poor (AUC = 0.67) and fair (AUC = 0.72), respectively. Well-validated current ICU mortality predictors such as APACHE IV (AUC = 0.86–0.89), ICNARC (AUC = 0.87), and SAPS III (AUC = 0.80) all possess notably better predictive capabilities than Cr2.^20,21 However, there is evidence that previous versions of APACHE and SAPS underperform when applied to an elderly cohort.²² When compared against models designed and tested for use in the elderly, the predictive power of Cr2 was similar to other larger, more complex, and disease-specific models (AUC = 0.66–0.75).^23–25 The study hypothesis that our final model (Cr2) would accurately predict ICU mortality in a retrospective cohort was not proven to be true, results from the ROC curve indicate that scoring above 4 points would correctly identify almost three-quarters of patients likely to die and when compared to the MPM₀III model (AUC = 0.721-0.80),^20,21 Cr2 exhibited competitive AUC results whilst only containing half the number of variables. Another factor in its favour is usability, as unlike the MPM₀III, the information required is available for use at the point of consideration of referral assuming a medical record is available. This implies Cr2 does have a reasonable level of survival prediction, which could usefully inform conversations with referring teams, patients, and relatives. Furthermore, of the 15 variables identified as significant predictors of elderly ward patient mortality, eight variables, including the finding of two novel parameters (ECG abnormality and hospital stay ≥ 5 days preceding admission), were found to be significant predictors of mortality once the individual was admitted to ICU. Despite identifying numerous predictors of mortality, we failed to achieve strong prognostic sensitivity and specificity when incorporated into a model, indicating some patients would be liable to false-positive and false-negative results. Falsely identifying a patient as likely to survive will not affect any treatment decisions, however, it can provide additional burden and harm to the patient and relatives. Alternatively, the implications associated with incorrectly identifying a patient likely to die can have huge significance on treatment decisions and decisions regarding end-of-life care.

In conclusion, our findings indicate that the Cristal criteria alone could not be used as a clinical decision aid. However, the ability to be able to use them to explore patient and family expectations of survival during conversations regarding admission to intensive care, can be a helpful way of framing those conversations without direct reference to age.

This study demonstrates that using only a few simple chronic disease markers at the point of admission has the potential to predict short-term ICU mortality. Chronic morbidities and frailty are better considerations when making clinical decisions for critically ill elderly patients rather than chronological age alone. These variables should therefore play integral part in any discussions with the patient or relatives regarding realistic treatment expectations and preferences for end-of-life care.

Supplemental Material

Supplemental material for Using a CriSTAL scoring system to identify pre-morbid conditions associated with a poor outcome after admission to intensive care in people 70 years or older

Supplemental material for Using a CriSTAL scoring system to identify pre-morbid conditions associated with a poor outcome after admission to intensive care in people 70 years or older by K Jankowski and DC Bryden in Journal of the Intensive Care Society

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Levine

Sachs

Jin

, et al. A prognostic model for 1-year mortality in older adults after hospital discharge. Am J Med 2007; 120: 455–460.

NHS England. Critical Care bed capacity and urgent operations cancelled 2016/2017 data. Available at: https://www.england.nhs.uk/statistics/statistical-work-areas/critical-care-capacity/critical-care-bed-capacity-and-urgent-operations-cancelled-2016-17-data/ (2016, accessed 27 September 2018).

Boumendil

Aegerter

Guidet

, et al. Treatment intensity and outcome of patients aged 80 and older in intensive care units: A multicenter matched-cohort study. J Am Geriatr Soc 2005; 53: 88–93.

Fuchs

Chronaki

Park

, et al. ICU admission characteristics and mortality rates among elderly and very elderly patients. Intensive Care Med 2012; 38: 1654–1661.

Bagshaw

Webb

SAR

Delaney

, et al. Very old patients admitted to intensive care in Australia and New Zealand: a multi- centre cohort analysis. Crit Care 2009; 13: R45.

Alonso

Ebert

Kern

, et al. Outcome predictors of acute stroke patients in need of intensive care treatment. Cerebrovasc Dis 2015; 40: 10–17.

Capuzzo

Bianconi

Contu

, et al. Survival and quality of life after intensive care. Intensive Care Med 1996; 22: 947–953.

Wolters

Peelen

Welling

, et al. Long-term mental health problems after delirium in the ICU. Crit Care Med 2016; 44: 1808–1813.

Dowdy

Eid

Sedrakyan

, et al. Quality of life in adult survivors of critical illness: a systematic review of the literature. Intensive Care Med 2005; 31: 611–620.

10.

Rockwood

Song

MacKnight

, et al. A global clinical measure of fitness and frailty in elderly people. Can Med Assoc J 2005; 173: 489–495.

11.

Bagshaw

Stelfox

McDermid

, et al. Association between frailty and short- and long-term outcomes among critically ill patients: a multicentre prospective cohort study. Can Med Assoc J 2014; 186: E95–E102.

12.

Le Maguet

Roquilly

Lasocki

, et al. Prevalence and impact of frailty on mortality in elderly ICU patients: a prospective, multicenter, observational study. Intensive Care Med 2014; 40: 674–682.

13.

Moreno

Metnitz

PGH

Almeida

, et al. SAPS 3 - From evaluation of the patient to evaluation of the intensive care unit. Part 2: Development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med 2005; 31: 1345–1355.

14.

Zimmerman

Kramer

McNair

, et al. Acute physiology and chronic health evaluation (APACHE) IV: Hospital mortality assessment for today's critically ill patients. Crit Care Med 2006; 34: 1297–1310.

15.

Vincent

Moreno

Takala

, et al. The SOFA (sepsis-related organ failure assessment) score to describe organ dysfunction/failure. Intensive Care Med 1996; 22: 707–710.

16.

Cardona-Morrell

Hillman

. Development of a tool for defining and identifying the dying patient in hospital: Criteria for Screening and Triaging to Appropriate aLternative care (CriSTAL). BMJ Support Palliative Care 2015; 5: 78–90.

17.

Ferrante

Pisani

Murphy

, et al. The Association of Frailty With Post-ICU Disability, Nursing Home Admission, and Mortality: A Longitudinal Study. Chest 2018; 153(6): 1378–86.

18.

Somme

Maillet

Gisselbrecht

, et al. Critically ill old and the oldest-old patients in intensive care: short- and long-term outcomes. Intensive Care Med 2003; 29: 2137–2143.

19.

Hope

Adeoye

Chuang

, et al. Pre-hospital frailty and hospital outcomes in adults with acute respiratory failure requiring mechanical ventilation. J Crit Care 2018; 44: 212–216.

20.

Keegan

Gajic

Afessa

. Comparison of APACHE III, APACHE IV, SAPS 3, and MPM0III and influence of resuscitation status on model performance. Chest 2012; 142: 851–858.

21.

Kuzniewicz

Vasilevskis

Lane

, et al. Variation in ICU risk-adjusted mortality – impact of methods of assessment and potential confounders. Chest 2008; 133: 1319–1327.

22.

Minne

Ludikhuize

de Jonge

, et al. Prognostic models for predicting mortality in elderly ICU patients: a systematic review. Intensive Care Med 2011; 37: 1258–1268.

23.

de Rooij

Abu-Hanna

Levi

, et al. Identification of high-risk subgroups in very elderly intensive care unit patients. Crit Care 2007; 11: R33.

24.

Langlais

Nesseler

Le Pabic

, et al. Does the clinical frailty score improve the accuracy of the SOFA score in predicting hospital mortality in elderly critically ill patients? A prospective observational study. J Crit Care 2018; 46: 67–72.

25.

Zilberberg

Shorr

Micek

, et al. Clostridium difficile-associated disease and mortality among the elderly critically ill. Critical Care Med 2009; 37: 2583–2589.