Abstract
Introduction
Patients diagnosed with end-stage liver disease (ESLD) often struggle with a heavy symptom burden that compromises their quality of life. Introduction of specialty palliative care (PC) may help address these issues but is underutilized in ESLD. This study aimed to assess the performance of the surprise question (SQ) in this population as a potential screen to identify patients with a life expectancy of less than 12 months.
Methods
A retrospective cohort analysis was performed. In a previous study (COMPASS Trial, NCT03022630), attending hepatologists were asked the SQ: “Would you be surprised if this patient were to die in the next 12 months?” as a prompt to consider consultation to specialty PC. Here, we consider the full collection of SQ answers and reconcile against record of death to examine the accuracy of the SQ as a predictor in this patient population.
Results
The SQ had high sensitivity (93%) but low specificity (38%) for predicting 12-month mortality in ESLD patients. The SQ also had a positive predictive value of 42% and a negative predictive value of 92%. Patients who were SQ+ (physician responded: No, I would not be surprised) were more likely to die within the year than those who were SQ−.
Conclusion
In ESLD, the SQ may serve as a useful screening test to identify patients at higher risk of mortality which may prompt PC engagement.
Introduction
End-stage liver disease (ESLD) is defined as the culmination of progression of chronic liver disease to cirrhosis, decompensation, and chronic liver failure, featuring portal hypertension or hepatocellular failure-related complications. 1 The prevalence of cirrhosis in North America is estimated to be ranging from 300 to 1000 per 100,000 and is increasing over time. 2 Age-adjusted mortality from chronic liver disease and cirrhosis also continues to increase over time. 2 Although liver disease is the 12th leading cause of death in the US, 3 strikingly, only 6.4% of those deaths include the provision of hospice care. 4 Patients with ESLD suffer from significant symptom burden including fatigue, pruritis, pain, bleeding, altered mental status, edema, and dyspnea. 5 The palliative care (PC) needs of this population are considerable, particularly in the last year of life, given the high morbidity and mortality.6-8
PC can and should be delivered concurrently with disease-specific care to address physical and non-physical symptoms, mitigate suffering, and improve quality of life for patients with serious illnesses. Improved outcomes have been demonstrated in both cancer and non-cancer serious illnesses, such as chronic obstructive pulmonary disease (COPD), interstitial lung disease (ILD), heart failure, renal disease, and progressive neurologic disorders.9-11 However, there is a lack of consensus on the optimal timing of PC consultations in chronic disease states, specifically in ESLD. Historically, PC involvement typically has occurred late in the disease course with prior studies identifying a need for improved end of life care in ESLD due to the high symptom burden 12 and high rates of health care utilization in patients’ last year of life. 13 Patients with cirrhosis and ESLD typically have inconsistent and limited access to PC services for assistance with goals of care conversations and advanced care planning.14,15 As the clinical trajectory of ESLD tends to be a gradual decline with fluctuations in deterioration, the most appropriate time to consult PC remains elusive. 16
The surprise question (SQ) is a simple, experience-based, intuitive screening tool that helps clinicians identify patients with a limited life expectancy and who therefore may benefit from PC engagement. 17 The clinician would ask “would I be surprised if this person were to die in the next 12 months?” In previously published literature, the SQ has been used in patients with end stage renal disease (ESRD) to predict need of hemodialysis, 18 to trigger goals of care conversations in patients with cancer regardless of prognosis, 19 to integrate PC into neurocritical care 20 and to trigger PC consultations in gynecology/oncology, emergency department patients, and patients with COPD.21-23 Prior studies have determined that the SQ is effective at finding patients, especially with non-cancer illnesses, who would benefit from PC integration and advanced care planning.24,25 Additionally, the SQ has been used alongside clinical prediction tools, showing improved prognostic accuracy in the last 6 months of life in a population of patients on hemodialysis. 26
Although the SQ has been utilized as a trigger to consult PC for patients with ESLD, 27 it remains unclear if there is concordance with clinician responses and patients’ 1 year mortality. This study aims to examine the concordance of the SQ with the observed mortality of patients with ESLD whose hepatologists were prompted with the SQ.
Methods
Study Design, Setting, Population
This work is a retrospective analysis (IRB #192012) of the overarching set of participant eligibility determining questions that were asked and answered during a previous trial: Creation of Models for Palliative Assessments to Support Severe Illness Trial (COMPASS Trial, NCT03022630). 27 Patients were admitted at an urban, academic referral center in Nashville, TN. Potential patients were identified within the electronic medical record using hepatology service codes and/or attending physician name. Patients’ electronic medical records were manually reviewed by study personnel to confirm preliminary eligibility. The hepatologist of record was presented daily with the SQ for each patient on their inpatient service with indications of ESLD from February 2017 to January 2018. The SQ queries the provider: “Would you be surprised if this patient were to die in the next 12 months?” The SQ was used as an eligibility screening question in COMPASS to assess appropriateness for consideration of a PC intervention for patients requiring inpatient care for ESLD. We examine the concordance of all screening questions asked and answered relative to the actual survival at 1-year post-presentation.
Patient Characteristics and Data Sources
Demographics and clinical data were abstracted from the electronic health record (EHR). EHR-derived mortality data were supplemented with vital statistics supplied by the Tennessee Department of Health. (TDOH IRB# TDHIRB-2020-0221). ESLD etiology was ascertained via examination of ICD-9 and ICD-10 codes preceding the SQ. Generally, 1 of the following underlying causes emerged: alcoholic cirrhosis, nonalcoholic steatohepatitis (NASH), chronic Hepatitis C, or chronic Hepatitis B. Where 2 causes were consistently documented, chart review was conducted to confirm. Likewise, when 1 of the more routine causes was not found, chart review was conducted to ascertain “other status,” which often included: primary sclerosing cholangitis, primary biliary cirrhosis, hepatocellular carcinoma, cholangiocarcinoma, autoimmune hepatitis, metabolic syndrome, Budd-Chiari syndrome, congestive hepatopathy, cryptogenic cirrhosis, cholestatic liver disease, and biliary atresia. MELD scores were calculated by extracting creatinine, bilirubin, INR, and sodium levels for patients, all drawn and reported within 48 hours of each other and preceding the SQ ask. As the parent COMPASS study and SQ work was conducted from 2017-2018, the prevailing MELD models at the time did not include dialysis status. Therefore, this was not used to adjust scores. Calculation when appropriate variables could be ascertained was performed using: https://www.mdcalc.com/calc/78/meld-score-model-end-stage-liver-disease-12-older. All authors had access to the study data and reviewed and approved the final manuscript.
Outcomes
The primary outcome was the accuracy of the SQ in predicting 12-month mortality in patients with ESLD.
Statistical Analyses
We present summary statistics on demographic and baseline characteristics (ie, age, sex, race, ethnicity, ESLD etiology, MELD). Data is shown for the overall cohort, then dichotomized into 2 groups using responses to the SQ (“No, I would not be surprised.” [SQ+] or “Yes, I would be surprised.” [SQ−]). Categorical variables are described using frequencies and proportions and continuous variables are described using means and standard deviations, as well as medians and interquartile ranges. Kaplan-Meier curves are presented by SQ response with point estimates and 95% confidence intervals (CIs). The log rank test is used to compare the time to death after SQ response between groups. We assess the accuracy of the SQ in predicting 1-year mortality by calculating prevalence, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). For a number of patients, the SQ was posed more than once to the hepatologist on service. Response to the subsequent SQ may not have been answered by the original provider. As such, we qualitatively detail the performance of the SQ ask over time. Analyses were conducted using: https://vassarstats.net/; https://eurekastatistics.com/kaplan-meier-survival-curve-grapher/; https://www.statskingdom.com/kaplan-meier.html; and BlueSky Statistics (v7.3 Commercial Edition, Chicago, IL).
Results
Of 409 potentially eligible participants for the SQ during the COMPASS trial, 378 patients were included. We excluded 31 patients for specific reasons: duplicate patient, research ineligibility, or lack of index test (ie, no response to the SQ provided). Of those included, response results indicated that the clinician would be surprised (‘YES’) if the patient died in the next year in 104/378 patients (27.5%) but would not be surprised (‘NO’) among 274/378 patients (72.5%) (Figure 1). Study Flow Diagram. Cohort identification originated from the collection of patients initially screened in the COMPASS study (NCT03022630). Abbreviations: SQ, surprise question.
Patient Demographics.
Data presented as number of patients (%) for categorical data. Continuous data presented as either mean (SD) or median (IQR). SQ = NO indicated that the provider would not be surprised if the patient were to die in the next 12 months. SQ = YES indicated that the provider would be surprised if the patient were to die in the next 12 months. Abbreviations: SQ, surprise question; SD, standard deviation; IQR, interquartile range; ESLD, end-stage liver disease; EtOH, alcoholic cirrhosis; NASH, nonalcoholic steatohepatitis; Hep B, hepatitis B; Hep C, hepatitis C.
MELD Score Ranges and Associated Surprise Question Answers.
MELD score ranges delineated with associated SQ answers. Data presented as number of patients and percentage of total with MELD scores calculable (n = 323). SQ + (NO) indicated that the provider would not be surprised if the patient were to die in the next 12 months. SQ - (YES) indicated that the provider would be surprised if the patient were to die in the next 12 months. Abbreviations: SQ, surprise question.
Patients with a ‘NO’ response to the SQ (SQ+) had lower survival rates at 365 days compared to those with a ‘YES’ response (SQ−) (58.4% vs 92.3%; log-rank test: P < 0.001) (Figure 2). In reviewing the accuracy of the SQ predicting 1 year mortality, our data show that 114/274 patients (41.6%) who were predicted by the SQ to die had recorded deaths within 1 year. In contrast, 160/274 patients (58.3%) who were predicted by the SQ to die did not have observed or recorded deaths within 1 year (ie, 160 false positives). Lastly, of the 104 patients in which the SQ did not predict death within 1 year, 8 died (ie, false negatives) (Table 3). As such, this test has a 93.4% sensitivity (95% CI .87, .96), 37.5% specificity (95% CI .31, 0.43), 41% positive predictive value (PPV) (95% CI .35, .47), and a 92% negative predictive value (NPV) (95% CI .85, .96) (Table 4). Kaplan-Meier Curve. Kaplan-Meier Curve illustrating the survival probability of patients whose providers answered YES (blue line, SQ −) or NO (red line, SQ +) to the surprise question (SQ). Log-rank test: P < 0.001. Surprise Question Response and Patient Status at 365 Days. Presentation of patient status at 365 days and concordance with surprise question answers as 2 × 2 table. Data served as basis for calculating the sensitivity, specificity, and positive and negative predictive values of the SQ in ESLD patients. Data presented as number of patients (%), where indicated. Accuracy of the Surprise Question in Predicting Mortality at 365 Days. Data summarizing attributes of the surprise question’s performance in the ESLD population considered. Estimated values displayed with appropriate 95% confidence intervals.
In addition, the SQ was repeated for 80 patients who were readmitted. These patients were readmitted on average at 50 days from the initial SQ assessment (range 3 to 203 days). Interestingly, the clinician assessment responses remained consistent for 64 (80%) of these patients, with 55 remaining ‘NO’ and 9 remaining ‘YES’. Notably, 25 of the 55 (45%) whose response remained ‘NO’ died within a year.
Discussion
Our study evaluated the performance of the SQ as a screening test in the unique context of patients with ESLD. Our findings illustrate that patients with ESLD whose hepatologist answered ‘NO’ to the SQ during a hospital admission had a lower cumulative 1-year survival as compared to those patients for whom the hepatologist answered ‘YES’ to the SQ. The sensitivity of the SQ for patients with ESLD in our study was 93.4% (higher than what has been published in the literature for non-ESLD populations), while the specificity of the SQ in this population was 37.5%. Gonzalez et al published a sensitivity of 85% for the SQ in a recent prospective cohort of ambulatory patients with heart failure. 29 Additionally, in a meta-analysis of studies examining the accuracy of the SQ for predicting death, the pooled sensitivity was 71.4% (95% CI [66.3-76.4]) and the pooled specificity was 74.0% (95% CI [69.3-78.6]). 30 The sensitivity in our sample of patients with ESLD may have been higher due to our more advanced ESLD population (median MELD score of 20) from a single quaternary referral center with a large volume transplant program, reflecting a sicker population.
In our study the NPV was found to be 92.3% and the PPV 41.6%. Our elevated NPV aligns with prior studies as summarized by Davis et al, wherein a ‘YES’ answer to the SQ is generally more likely to predict survival as opposed to a ‘NO’ response predicting mortality. 31 Considering that an answer of ‘YES’ accurately predicts survival in 92.3% of cases in this patient population, the SQ could be used as a tool to prioritize engagement of PC consultations for those patients with ESLD whose hepatologist answered ‘NO’ to the SQ. The SQ was originally developed as a screening tool for PC consultation and advance care planning rather than a prognostic tool. Asking if someone “would be surprised if a patient were to die” does not fully equate to “will this person die” or even “what is the probability of death.” The SQ is not an algorithm ingesting only objective data. There is clinical intuition and judgment involved. Thus, a human psycho-cognitive element is introduced when using the SQ to evaluate patient status. However, its prognostic accuracy for mortality has been examined in various conditions and settings outside of ESLD. 30
The MELD score is a widely utilized, validated prognostic tool used to estimate 3-month survival in patients with ESLD. 28 For those participants in our study for whom a MELD score could be calculated, we examined the percentage of ‘NO’ or ‘YES’ answers to the SQ, across a range of MELD scores (Table 2). Given that increasing MELD score is known to be associated with increased risk of 3-month mortality, 28 it is not surprising that zero ‘YES’ responses to the SQ were received for those with a MELD of 40 or greater. Similarly, only 1 ‘YES’ SQ answer was recorded among those with a MELD of 30-39. At the other extreme, there were nearly equal numbers of ‘NO’ and ‘YES’ SQ answers for those with a MELD of 9 or less. For the remaining MELD score categories, there is variability in the SQ answers. We hypothesize that this finding supports the incorporation of the SQ into clinical assessment, as this could potentially facilitate earlier PC engagement for those patients in whom a clinician is concerned about the clinical trajectory, regardless of MELD score.
In addition to increased risk of mortality, ESLD causes significant morbidity. The complications of ESLD are associated with high symptom burden, which has been reported to exceed that of other chronic diseases and cancer. 32 A single center study of patients with ESLD awaiting liver transplant demonstrated that early PC improved symptom burden and mood. 33 A separate single center study of patients with ESLD awaiting liver transplant showed that those who received palliative care services had higher quality end of life outcomes and fewer ICU admissions, compared to those not receiving palliative care. 34 Similarly, a recent study of US inpatients with ESLD demonstrated that PC utilization is associated with decreased invasive procedures, shorter length of stay, and lower hospitalization charges. 35 Despite the benefits of PC in the ESLD population, it remains underutilized and is delivered late in life, diminishing the potential benefit to patients and families. 36
In examining the use of the SQ for patients with ESLD as an opportunity to consider PC referral, we specifically note a subset of patients in our study (n = 80) who were admitted more than once and therefore had repeated assessment with the SQ. With 80% of clinician responses remaining consistent upon repeat SQ ask, there appears to be durability of clinical assessment in this population. We hypothesize that this represents a missed opportunity for earlier integration of PC for this subset of patients whose hepatologists answered ‘NO’ to the SQ had they been referred to PC at the first ask. Earlier integration of PC for patients and their families could have enhanced the understanding of the disease course and anticipated progression. This may have resulted in fewer readmissions, more hospital-free days, and improved quality of life. While earlier integration of PC would not be expected to decrease mortality in patients with ESLD, it has potential to improve symptom management, illness understanding and psychosocial support for these patients nearing the end-of-life. This is especially poignant given that nearly 50% of patients whose treating clinicians repeatedly answered ‘NO’ to the SQ in fact died within 365 days.
We note the following strengths to our study. First, to our knowledge, this is the only study examining the prognostic accuracy of the SQ for predicting mortality in patients with ESLD. This offers a “lower tech” tool that is grounded on a provider’s clinical and intuitive knowledge base, and can be readily implemented to improve patient care by identifying those who would benefit from earlier engagement with PC. Second, because responses in the COMPASS trial were solicited from hepatologists, prognostic assessment may be more clinically suggestive as compared to non-hepatologists. Third, we obtained complete data regarding the primary outcome of survival by extracting data from the EHR and supplemented by the Tennessee Department of Health.
We also note the following limitations to our study. Our parent study was conducted in 2017-2018, at which time the MELD calculator did not include a variable for hemodialysis. Thus, the MELD score calculations herein were calculated based on how MELD scores were performed at the time of the parent study. The population we examined was predominantly white, non-Hispanic, and male so may not be representative of all ESLD patients. Second, the SQ was utilized in the inpatient setting. This likely inflated the number of ‘NO’ answers, as inpatients are sicker compared to outpatients. Third, the SQ was answered only by physicians. These responses reflect individual clinical assessments and may not have been representative of all hepatology providers engaged (eg, advanced practice providers, nursing professionals). Finally, this is a single-center study of patients admitted to an academic quaternary care hospital, potentially limiting generalizability to other care settings.
ESLD is associated with significant morbidity and mortality. Early PC engagement has been proposed as a solution to improve the physical and psychological burden associated with ESLD. 37 In this study, we sought to evaluate hepatologist’s clinical intuition in an inpatient setting, by examining the accuracy of the SQ in predicting mortality within 12 months for patients with advanced liver disease. While the SQ should not be solely used as a prognostic indicator of death, our study proves that the SQ can be utilized by hepatologists as a screening tool with good sensitivity for identifying patients with ESLD who may be at higher risk of death and therefore may benefit from PC co-management.
Footnotes
Acknowledgments
The authors would like to thank Mary Lynn Dear, PhD and William Hiser for their support through various stages of coordinating and conducting the research. The authors would like to particularly acknowledge Christopher Lindsell, PhD for guidance, mentorship, and encouragement throughout this project. The authors would also like to express their gratitude to the patients and caregivers without whom this study would not have been possible.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The project described was supported in part by the Vanderbilt Institute for Clinical and Translational Research (VICTR) Learning Healthcare System Platform under CTSA award No. UL1 TR002243 from the National Center for Advancing Translational Sciences. Its contents are solely the responsibility of the authors and do not necessarily represent official views of the National Center for Advancing Translational Sciences or the National Institutes of Health.
