Abstract
Introduction
Several prediction scores for triaging patients with upper gastrointestinal (GI) bleeding have been developed, yet these scores have never been compared to the current gold standard, which is the clinical evaluation by a gastroenterologist. The aim of this study was to assess the added value of prediction scores to gastroenterologists’ Gut Feeling in patients with a suspected upper GI bleeding.
Methods
We prospectively evaluated Gut Feeling of senior gastroenterologists and asked them to estimate: (1) the risk that a clinical intervention is needed; (2) the risk of rebleeding; and (3) the risk of mortality in patients presenting with suspected upper GI bleeding, subdivided into low, medium, or high risk. The predictive value of the gastroenterologists’ Gut Feeling was compared to the Blatchford and Rockall scores for various outcomes.
Results
We included 974 patients, of which 667 patients (68.8%) underwent a clinical intervention. During the 30-day follow up, 140 patients (14.4%) developed recurrent bleeding and 44 patients (4.5%) died. Gut Feeling was independently associated with all studied outcomes, except for the predicted mortality after endoscopy. Predictive power, based on the AUC of the Blatchford and Rockall prediction scores, was higher than the Gut Feeling of the gastroenterologists. However, combining both the Blatchford and Rockall scores and the Gut Feeling yielded the highest predictive power for the need of an intervention (AUC 0.88), rebleeding (AUC 0.73), and mortality (AUC 0.71 predicted before and 0.77 predicted after endoscopy, respectively).
Conclusions
Gut Feeling is an independent predictor for the need of a clinical intervention, rebleeding, and mortality in patients presenting with upper GI bleeding; however, the Blatchford and Rockall scores are stronger predictors for these outcomes. Combining Gut Feeling with the Blatchford and Rockall scores resulted in the most optimal prediction.
Introduction
There is an increasing role for evidence-based medicine in clinical practice. This is accompanied by the development of prediction scores and their use is increasingly being recommended and adopted in clinical guidelines. 1 A prediction score (or risk score/decision rule) is a tool for physicians based on several predictors – such as patients’ history, physical examination, test results, and other disease characteristics – which give an estimation on the probability of a likely diagnosis, prognosis, or response to treatment. 2 Such tools can be of added value for the physician in daily clinical practice.
Upper gastrointestinal (GI) bleeding is a common clinical problem and accounts for 25–35 hospitalizations per 100,000 person-years.3,4 The severity of the disease may vary from no active bleeding to rapid exsanguinations, and yet the course remains difficult to predict. Almost all patients suspected for upper GI bleeding are therefore admitted to the hospital and endoscopy is being performed within 24 hours after hospitalization. 5 This results in a high pressure on hospital capacity, possibly unnecessary discomfort for the patient, and high healthcare costs. Accurate predicting of the course and outcome of upper GI bleeding should ideally facilitate triage into low- and high-risk groups and would thus help clinical management.
Several prediction scores for upper GI bleeding have been developed. 6 The most commonly used scores are the Blatchford and Rockall scores.7,8 The Blatchford score is a validated score using pre-endoscopic variables, such as clinical and laboratory data, and has the primary goal to predict the need for an intervention, such as an upper endoscopy with a haemostatic procedure. The Rockall score is a validated score based on clinical, laboratory, and endoscopic variables and primarily predicts mortality. Although these scores are validated and recommended by international guidelines, 5 it seems that gastroenterologists confronted with upper GI bleeding do not often incorporate these scores into clinical practice.
From previous studies we have learned that scores are more likely to be implemented if they are easy to use, if recommendations are being made based on the score (instead of just assessment), if they can be incorporated in the normal daily usual workflow, and if they are computerized. 9 However, the willingness of a physician to use scores is also important. The reasons for a physician not using scores may be: they are difficult to calculate, they take time, and, most importantly, they do not add to their own clinical knowledge or ‘Gut Feeling’. Moreover, it has been reported that clinical decision making may be even better than prediction scores in predicting whether patients with upper GI bleeding should be admitted to the intensive care unit. 10
In the current study, we assessed the added value of prediction scores to the Gut Feeling of gastroenterologists in patients with suspected upper GI bleeding presenting to the accident and emergency department (A&E).
Methods
Patients and outcomes
All patients of 18 years or older that were admitted to A&E for suspected upper GI bleeding (i.e. presentation with self-reported melaena or haematemesis) between October 2009 and April 2012 were included in eight participating hospitals in the Netherlands. Patients were treated according the treatment protocols of the participating centres and no interference was made with regard to patient management. Patients were followed for 30 days after presentation by the study coordinator of the participating hospital.
Diagnostic criteria for upper gastrointestinal bleeding
The primary outcomes were: (1) need for clinical intervention; (2) 30-day mortality (in- and out-hospital); and (3) 30-day rebleeding rate. A clinical intervention was defined as a blood transfusion or any operative, radiological, or endoscopic intervention to control the haemorrhage and/or the occurrence of rebleeding or mortality. 7 Mortality was defined as all-cause mortality in- and out-hospital. Rebleeding was defined according to the criteria set by the Peptic Ulcer Bleed study as recurrent haematemesis of fresh blood (>200 ml), active bleeding or fresh blood found during endoscopy, or two of the following: (1) Hb drop >20 g/l within 24 h; (2) Hb increase <10 g/l after adequate blood transfusion; (3) systolic RR <90 mmHg (after being higher initially); or pulse rate >110/min (after being lower initially) within 30 days after initial stabilization. 11
Gut Feeling and prediction scores
Questions regarding the Gut Feeling of the gastroenterologist
Variables tested according to use of Rockall and Blatchford scores
Statistical analysis
Patient characteristics, comorbidities, and endoscopic findings were analysed using standard descriptive statistics. Sensitivity and specificity rates and negative and positive predictive values were calculated for both Gut Feeling (medium/high risk compared to low risk and high risk compared to low/medium risk) and the prediction scores at a cut-off level of <1 for predicting/excluding the need of an intervention and >2 for predicting rebleeding and mortality. Logistic regression analyses were performed to assess the association between Gut Feeling and each individual outcome (need for intervention, rebleeding, and mortality predicted before and after endoscopy), as well for the prediction scores and the various outcomes. The area under receiver operating curve (AUC) was calculated to compare the discriminative power of the Gut Feeling and the prediction scores, with an AUC of 0.5 indicating no and a value of 1.0 indicating perfect discrimination between high and low risk. Ordinal logistic regression analyses were performed to identify risk factors associated with Gut Feeling. Statistical analysis was performed with SPSS 14.0 (SPSS, Chicago, IL, USA).
Ethical considerations
This was a prospective observational study, in which patient data were entered anonymously in a central database. The protocol did not include any (additional) interventions and no additional testing was performed. Therefore, the Dutch Law on Medical Research on Humans did not apply here and approval by a medical ethical committee was not required, as agreed by the Medical Ethics Committee of the Sint Antonius Hospital, Nieuwegein, the Netherlands (20 July 2009).
Results
In total, 1001 patients were included, with Gut Feeling being completed for 974 patients (Figure 1). The mean age of the studied population was 65 (range 18–99) years and 37% of the patients were female. Other baseline characteristics are shown in Table 4.
Flowchart of study population. Baseline characteristics for different outcomes Values are n (%) or mean ± SD unless otherwise stated. BP, Blood pressure; GI, gastrointestinal; NSAID, nonsteroidal anti-inflammatory drug; PPI, proton pump inhibitor; SSRI, selective serotonin re-uptake inhibitor.
Prediction of need for clinical intervention
In total, 667 patients (69%) underwent a clinical intervention (e.g. blood transfusion, operative or endoscopic procedure to control the haemorrhage). The number of interventions increased significantly with higher estimated risks by the Gut Feeling, as well as with increasing Blatchford scores (Figure 2). The Blatchford score showed high sensitivity rates while the Gut Feeling revealed higher specificity rates (Table 5). After correcting for the Blatchford score, the Gut Feeling was still independently associated with the need for an intervention (odds ratio, OR, 1.5, 95% CI 1.0–2.3 for medium risk; OR 4.9, 95% CI 3.0–8.2 for high risk). However, the Blatchford score had a better predictive power than the Gut Feeling of the gastroenterologists (AUC 0.86 vs. 0.78, respectively). Combining the Gut Feeling with the Blatchford score improved the predictive power to 0.88 (Figure 3). Regression analyses showed that haemoglobin and urea levels, haematemesis, and a history of collapse had the highest effect on the Gut Feeling of gastroenterologists.
Association Gut Feeling and prediction scores with outcome. (a) Prediction of clinical intervention. (b) Prediction of rebleeding. (c) Prediction of mortality before endoscopy. (d) Prediction of mortality after endoscopy. Left Y-axes illustrate the percentage of patients with outcome predicted by the Gut Feeling; right Y-axes illustrate the mean prediction score for every group of the Gut Feeling. ROC curves for various outcomes. The sensitivity, specificity and predictive values of Gut Feeling and the prediction scores for various outcomes Values are n/total (%). NPV, negative predictive value; PPV, positive predictive value.
Prediction of rebleeding
In total, 140 patients (14.4%) developed a rebleeding. The rebleeding rate and the mean Rockall score increased significantly with higher estimated risks by the Gut Feeling (Figure 2). Sensitivity rates were highest for the Rockall score and specificity rates for the Gut Feeling (Table 5). The Gut Feeling was independently of the Rockall score associated with rebleeding (OR 1.7, 95% CI 1.1–2.7 for medium risk; OR 3.4, 95% CI 2.1–5.7 for high risk). Rebleeding was slightly better predicted by the Rockall score compared to the Gut Feeling, but both predict rebleeding only to a moderate degree (AUC 0.69 vs. 0.68, respectively). Combining the Rockall score and the Gut Feeling improved the predictive power to an AUC of 0.73 (Figure 3). An intervention and, to a lesser extent, haematemesis and a history of a collapse had the highest effect on the Gut Feeling of gastroenterologists.
Prediction of mortality
Forty-four patients (4.5%) died. The intra-observer agreement between the Gut Feeling before and after endoscopy was low (kappa 0.41), suggesting that the results of endoscopy significantly changed the gastroenterologists’ views on the prognosis. Mortality rates increased with higher Gut Feeling risk estimations before and after endoscopy (Figure 2). The sensitivity of the Gut Feeling was highest before endoscopy, while the specificity was highest after endoscopy. Both Blatchford and Rockall scores showed high sensitivity rates (95.6% and 95.7%, respectively) (Table 5). A high Gut Feeling risk estimation before endoscopy was a significant predictor for mortality, also after adjustment for the risk scores (OR 1.8, 95% CI 0.8–4.1 for medium risk; OR 3.3, 95% CI 1.4–8.0 for high risk); however, the Gut Feeling after endoscopy was no predictor after correcting for the Rockall score (OR 1.4, 95% CI 0.6–3.0 for medium risk; OR 2.0, 95% CI 0.8–4.7 for high risk). The predictive power of the Gut Feeling and the risk scores are shown in Figure 3. The Rockall score combined with the Gut Feeling provided the most optimal prediction for mortality (AUC 0.77). The Gut Feeling before endoscopy was mostly affected by haemoglobin level, liver disease, haematemesis, and a do not resuscitate status of the patient. The Gut Feeling after endoscopy was mostly affected by a do not resuscitate status, liver disease, and whether an intervention was performed.
Discussion
We found that the Gut Feeling of gastroenterologists was a good predictor for the need of a clinical intervention, rebleeding, and mortality in patients presenting with upper GI bleeding; however, the Blatchford and Rockall prediction scores had overall higher predictive values. Prediction scores had a higher sensitivity and thus performed better in excluding an unfavourable outcome, while the Gut Feeling showed a higher specificity and thus had a better performance in predicting an unfavourable outcome.
We observed an overall tendency in gastroenterologists to overestimate the risk of an adverse outcome (positive predictive value), especially for the risk of rebleeding and mortality. An explanation could be that the estimation of gastroenterologists of the risk of rebleeding and mortality is based on older literature citing high rebleeding and mortality rates while a considerable reduction in rebleeding and mortality has been observed over the last decade as a result of better acid suppression, advanced endoscopy haemostatic techniques, and improvement of radiological haemostatic interventions.14,15
With regard to the outcomes in this study, both the Gut Feeling and the prediction scores performed well in excluding mortality and rebleeding as high negative predictive values were observed for those endpoints. For predicting the need of a clinical intervention, high positive predictive values were observed, especially for the Gut Feeling, meaning that gastroenterologists were able to predict the need for an intervention. The high sensitivity rates of the Blatchford and Rockall scores can be used to select patients who need closer monitoring. A high specificity – such as we found for the Gut Feeling – is however also of clinical importance because hospital admission and endoscopy are both a burden to patients, but to some extent also to the hospital, and a decision to do so should be based on well established risk factors.
These risk factors used in the prediction scores and the risk factors gastroenterologists based their Gut Feeling on are different for the various outcomes. For predicting the need of a clinical intervention, gastroenterologists mainly based their Gut Feeling on low haemoglobin levels and presentation with haematemesis and collapse. In the Blatchford score, a collapse and haemoglobin levels also play an important role, but other parameters used in this score, such as blood urea levels, tachycardia, and blood pressure were not used to a large extent by gastroenterologists. 7 A recent systematic review by Srygley et al. 16 identified tachycardia and haemoglobin level as important predictors for a clinical intervention, while they also identified a history of cirrhosis, malignancy, and nasogastric lavage with red blood as risk factors. For rebleeding, the most important predictors identified in the literature are age, comorbidities, active bleeding, and location of the bleeding.17–19 The Gut Feeling of gastroenterologists was, however, mainly based on whether an intervention was performed (which might be a reflection of active bleeding) and to a smaller extent on haematemesis and collapse. Lastly, important predictors of mortality found in literature are age, comorbidities, and haemodynamic instability.8,20–23 Gastroenterologists found haemoglobin level, liver disease, haematemesis, and a do not resuscitate status the most important indicators for mortality, which partly overlap with the predictors known from the literature.
This study has several strengths and limitations. This is the first study that shows that prediction scores are better than clinical estimation of experienced gastroenterologists in risk classification of patients with upper GI bleeding. Secondly, not only patients with an established upper GI bleeding but also patients with a suspected upper GI bleeding were included, as this is clearly a better reflection of the population actually presenting to the A&E and requiring risk classification. Moreover, the predictive power of the Blatchford and Rockall scores may increase somewhat after excluding variceal bleeding as these scores were originally developed for patients with nonvariceal bleeding, resulting in a larger difference than Gut Feeling. A limitation of this study was that due to the prospective design, we were unable to compare Gut Feeling to other more recently developed risk scores (e.g. PNED, AIM65)21,23 as we did not include variables such as time from onset of symptoms to admission and albumin. Retrospective collection of these parameters would likely have resulted in missing values and bias.
The results of our study may have important implications for clinical practice. We have shown that the use of prediction scores increases the predictive power for all clinical relevant outcomes of upper GI bleeding over Gut Feeling. The use of these scores will thus lead to a better prediction of the need for a clinical intervention. The risk scores were superior to the clinical estimation of experienced gastroenterologists, and given that A&Es are usually manned by less experienced registrars and junior house officers, there is definitely a reason for incorporation of these scores in clinical practice. However, we have also shown that combining prediction scores with the Gut Feeling of a gastroenterologist predicts even more accurately. This emphasizes the need for gastroenterologists always to be included in clinical decision making and triaging of patients presenting with upper GI bleeding. Based on our results, we propose to combine Gut Feeling and the established risk scores, balancing both sensitivity and specificity; however, this needs be confirmed in a prospective follow-up study.
In conclusion, this is the first study in which the added value of prediction scores was compared to Gut Feeling of gastroenterologists in predicting the outcome in patients with upper GI bleeding. We found that Gut Feeling is an independent predictor for adverse outcome; however, prediction scores have higher sensitivity and better predictive power compared to Gut Feeling. We therefore suggest using prediction scores in combination with Gut Feeling of gastroenterologists in making clinical decisions regarding treatment and monitoring of patients with upper GI bleeding.
Footnotes
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
Conflict of interest
The authors declare that there is no conflict of interest.