Abstract
Background:
Early and effective treatment of patients with sepsis requires early recognition in emergency department and understanding the severity of the disease. Many studies have been conducted for this purpose, and many of scoring systems have been developed that provide early recognition of these patients and show their severity.
Objectives:
The aim of this study is to evaluate the efficacy of the scoring systems used to determine the mortality of patients with infections admitted in emergency department.
Methods:
In all, 400 patients who admitted to Uludağ University Hospital Emergency Department were prospectively included in this study. In addition to Systemic Inflammatory Response Syndrome score, Quick SOFA score, Mortality in Emergency Department Sepsis score, Modified Early Warning Score, and Charlson Comorbidity Index score in all patients, CURB-65 score was calculated in the patients diagnosed with pneumonia. It has been aimed to determine the power of these scores’ predictive mortality rates and their superiority to each other.
Results:
It was found that Mortality in Emergency Department Sepsis score and Quick SOFA score could be used with similar efficacy (respectively p = 0.761 and p = 0.073) in determining early mortality in emergency department (5th and 14th days) and that MEDS score was more effective (p < 0.001) in predicting the 28th-day mortality. While these recommendations were valid in patients diagnosed with pneumonia, it was determined that CURB-65 score could also be used to estimate 5th-, 14th-, and 28th-day mortalities (respectively, for the 5th day, p = 0.894 and p = 0.256; for the 14th day, p = 0.425 and p = 0.098; and for the 28th day, p = 0.095 and p = 0.158). The power of Systemic Inflammatory Response Syndrome score, previously used to identify sepsis, in predicting mortality was detected to be lower.
Conclusion:
Mortality in Emergency Department Sepsis score and Quick SOFA score could be used with similar efficacy in determining early mortality in emergency department. However, if you want to predict 28th-day mortality rate, it can be better to use Mortality in Emergency Department Sepsis score or CURB-65 (in patients diagnosed with pneumonia).
Background
Sepsis is a syndrome that can be caused by community or hospital-acquired infections, which has serious morbidity and mortality reasons and increasing frequency. It results in 15% to 40% death in affected patients despite improvements in treatment and effective antibiotics.1–3 Some studies have shown that patients admitted to intensive care units constitute 10% of the patients with sepsis, and this admission has been shown to occur most frequently in emergency departments.4–6 In addition, 20% of these patients are hospitalized in the emergency room for more than 6 h. 7
Early and effective treatment of patients with sepsis requires early recognition in emergency department and understanding the severity of the disease. Many studies have been conducted for this purpose, and many of scoring systems have been developed that provide early recognition of these patients and show their severity. The use of these systems is recommended to make the required distinction fast, high quality, and efficient.8,9
Although these systems are considered to diagnose sepsis, in fact they have been developed to ensure the prediction of patients at high risk among the ones with suspected infection. Apart from these, there are systems that can predict mortality of patients. These early warning scores have been developed for early detection of patients at risk of mortality and can be simply performed by bedside and primarily with physiologic parameters.10,11
The aim of this study is to evaluate the efficacy of the scoring systems used to determine the mortality of patients with infections admitted in emergency department. In these patients, the following were calculated:
Systemic Inflammatory Response Syndrome score (SIRS);
Quick SOFA score (qSOFA);
Mortality in Emergency Department Sepsis score (MEDS);
Modified Early Warning Score (MEWS);
Charlson Comorbidity Index (CCI);
CURB-65 score of patients with pneumonia.
They were prospectively compared with the mortality of the patients, and it was aimed to determine high-risk group more correctly and to make comparison.
Methods
For the planned prospective observational study, approval of ethics committee with the assessment date of 24 May 2016 and decision number 2016-10 / 2 was obtained from Clinical Research Ethics Committee, Faculty of Medicine, Uludağ University.
Patients with community-acquired infections above 18 years of age who were admitted to the Emergency Department of Uludağ University Applied Research Center for Health between 1 June and 1 September 2016 and who were evaluated in yellow and red triage were included in the study. Exclusion criteria were determined as patients under 18 years of age, trauma patients, pregnant patients, and neutropenic patients. Patients who left the hospital for any reason without treatment, patients unwilling to participate, and patients who died within 1 h after being admitted in emergency department were also excluded from the study. Patients with missing data were not included in the study.
Infections were diagnosed according to clinical, radiological, and laboratory findings:
- Community-acquired pneumonia (CAP) was defined as the presence of a new infiltrate on chest radiography together with clinical symptoms suggestive of lower respiratory tract infection. 12
- Acute pyelonephritis was defined as the presence of two of the following: (a) axillary temperature ⩾ 38.3°C or chills, (b) flank pain or costovertebral angle tenderness or pain on bimanual palpation of the kidney, and (c) mictional syndrome (including two or more of the following: dysuria, frequency, suprapubic pain, or urgency), together with the presence of pyuria (a positive leukocyte esterase dipstick test result, subsequently confirmed by urinalysis with more than 10 leukocytes/mL in urine without centrifuging or more than 5 leukocytes per high-power field in centrifuged sediment) or a positive urine culture. 13
- Intra-abdominal infection was defined based on clinical, laboratory, and radiological findings. 14
- Skin and soft tissue infections were diagnosed according to clinical findings and included acute bacterial skin and skin structure infections (ABSSSI) and others such as diabetic foot infection and chronic wound infection. 15
- Surgical site infection was diagnosed according to the CDC (Centers for Disease Control and Prevention) definitions. A surgical site infection is defined as an infection that occurs after surgery in the part of the body where the surgery took place. 16
For each patient, name, surname, patient number, age, gender, Turkish Republic (TR) identification number, phone number, phone number of a relative to be reached, admission date, admission time, body temperature, heart rate, blood pressure, respiratory rate, saturation, Glasgow Coma Scale (GCS), AVPU (Alert–Verbal Stimuli–Painful Stimuli–Unresponsive) score, presence of altered mental status, date and time of first applied antibiotic, first administered antibiotic, infectious foci, CCI, whether there was story of antibiotherapy or chemotherapy within the last 3 months, whether there was change or addition of antibiotics after hospitalization, laboratory findings, SIRS score, 17 qSOFA score, 18 CURB-65 score, 19 MEDS score, 20 MEWS 21 and hospitalization, referral, discharge, or death status were recorded. Following that, on the 5th, 14th, and 28th days of admission, information was obtained by calling the patient or his or her relative who could be reached. To verification and avoiding bias (for the patients who could not be reached), status of death was questioned with the help of Central Civil Registration System (MERNIS).
Receiver operating characteristic (ROC) analysis was conducted to investigate the efficacy of SIRS, qSOFA, CURB-65, MEDS, MEWS, and CCI scoring systems in distinguishing between dead and living patients; cut-off point, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), area under the curve (AUC), and related p values of scoring systems were reported. In the analyses, MedCalc Statistical Software version 16.4.3 (MedCalc Software bvba, Ostend, Belgium; 2016) was used; statistical significance level was determined as p < 0.05.
Results
A total of 476 patients who met the criteria among 33,746 patients were admitted to the Emergency Department Hospital of Uludağ University Applied Research Center for Health between June 1 and September 1 2016. Among them, 400 were included in the study.
Of these 400 patients, 55.25% (n = 221) were male and 44.75% (n = 179) were female. Median age of the patients was found as 60.39 years (18–94). In all, 6% (n = 24) patients died on the 5th day, 11% (n = 44) on the 14th day, and 17% (n = 68) on the 28th day. ROC analyses applied to scoring systems showed cut-off values of MEDS, MEWS, and CURB-65 scoring systems and CCI. These values are given in Table 1.
The cut-off values found according to ROC analyses.
ROC: receiver operating characteristic; SIRS: Systemic Inflammatory Response Syndrome score; qSOFA: quick SOFA score; MEDS: Mortality in Emergency Department Sepsis score; MEWS: Modified Early Warning Score; CCI: Charlson Comorbidity Index; CURB-65: Confusion, Urea (BUN > 19 mg/dL), Respiratory rate ⩾30/minute, Systolic blood pressure <90 mmHg or Diastolic blood pressure ⩽60 mmHg, Age ⩾65 years).
Standard values in literature have been used.
Only those patients with the diagnosis of pneumonia are included in the calculation.
Along with the values detected, sensitivity, specificity, PPV, and NPV found according to 5th-, 14th-, and 28th-day mortalities of all scoring systems and p values determined with the AUCs are shown in Tables 2, 3, and 4. Graphical comparison of the AUCs according to the 28th-day mortality is shown in Figure 1. Analyses of comparison made between scoring systems and p values found are given in Table 5. The p values of the tests which are statistically not different from each other and which can be used interchangeably are indicated as bold and underlined.
Analysis of scoring systems according to the 5th-day mortality.
SIRS: Systemic Inflammatory Response Syndrome score; qSOFA: quick SOFA score; MEDS: Mortality in Emergency Department Sepsis score; MEWS: Modified Early Warning Score; PPV: positive predictive value; NPV: negative predictive value; AUC: area under the curve.
Analysis of scoring systems according to the 14th-day mortality.
SIRS: Systemic Inflammatory Response Syndrome score; qSOFA: quick SOFA score; MEDS: Mortality in Emergency Department Sepsis score; MEWS: Modified Early Warning Score; PPV: positive predictive value; NPV: negative predictive value; AUC: area under the curve.
Analysis of scoring systems according to the 28th-day mortality.
SIRS: Systemic Inflammatory Response Syndrome score; qSOFA: quick SOFA score; MEDS: Mortality in Emergency Department Sepsis score; MEWS: Modified Early Warning Score; PPV: positive predictive value; NPV: negative predictive value; AUC: area under the curve.

Comparison of AUCs according to the 28th-day mortality.
For all patients; p values found according to comparative analysis made between scoring systems.
SIRS: Systemic Inflammatory Response Syndrome score; qSOFA: quick SOFA score; MEDS: Mortality in Emergency Department Sepsis score; MEWS: Modified Early Warning Score.
We compared the baseline characteristics of dead and living patients according to the 28th-day mortality (Table 6).
Comparison of baseline characteristics of dead and living patients according to the 28th-day mortality.
AVPU: Alert–Verbal Stimuli–Painful Stimuli–Unresponsive.
Median values are given (minimum–maximum).
When we consider the 5th-day mortality, the scoring system with the largest AUC, that is, showing the best performance in distinguishing dead and living patients, has been found as MEDS (>8). qSOFA (⩾2) comes second while MEWS (>5) comes third. According to the comparative analysis made between scoring systems, there is no statistical difference between MEDS, qSOFA, and MEWS (p = 0.761 and p = 0.079).
When we consider the 14th-day mortality, the scoring system with the largest AUC, that is, showing the best performance in distinguishing dead and living patients, has been found as MEDS (>8). qSOFA (⩾2) comes second while MEWS (>3) comes third. According to the comparative analysis made between scoring systems, no difference was statistically found between MEDS and qSOFA (p = 0.073). There is difference between MEDS and MEWS (p = 0.016).
When we consider the 28th-day mortality, the scoring system with the largest AUC, that is, showing the best performance in distinguishing dead and living patients, has been found as MEDS (>6). qSOFA (⩾2) comes second while MEWS (>5) comes third. According to the comparative analysis made between scoring systems, there is statistically significant difference between MEDS, qSOFA, and MEWS (p < 0.001 and p < 0.001).
When the analyses were repeated for 347 patients who were followed up in hospital, it was detected that the results obtained in the analysis for all the patient groups were valid here.
Categorization of patients according their diagnoses and mortality rates are shown in Table 7. Pneumonia was diagnosed in 41.75% of patients, skin and soft tissue infection in 19%, urinary system infection in 15.75%, and abdominal infection in 14.75%.
Infectious foci in patients and the 28th-day mortality rates.
The most common infectious disease in the patients we studied was detected as pneumonia with a rate of 41.75%. When the scoring systems applied to these patients are analyzed, sensitivity, specificity, PPV, and NPV found according to 5th-, 14th-, and 28th-day mortalities and p values determined with AUCs are specified in Tables 8, 9, and 10.
Analysis of scoring systems according to the 5th-day mortality in patients with pneumonia.
SIRS: Systemic Inflammatory Response Syndrome score; qSOFA: quick SOFA score; MEDS: Mortality in Emergency Department Sepsis score; MEWS: Modified Early Warning Score; PPV: positive predictive value; NPV: negative predictive value; AUC: area under the curve; CURB-65: Confusion, Urea (BUN > 19 mg/dL), Respiratory rate ⩾30/minute, Systolic blood pressure <90 mmHg or Diastolic blood pressure ⩽60 mmHg, Age ⩾65 years).
Analysis of scoring systems according to the 14th-day mortality in patients with pneumonia.
SIRS: Systemic Inflammatory Response Syndrome score; qSOFA: quick SOFA score; MEDS: Mortality in Emergency Department Sepsis score; MEWS: Modified Early Warning Score; PPV: positive predictive value; NPV: negative predictive value; AUC: area under the curve; CURB-65: Confusion, Urea (BUN >19 mg/dL), Respiratory rate ⩾30/minute, Systolic blood pressure <90 mmHg or Diastolic blood pressure ⩽60 mmHg, Age ⩾65 years).
Analysis of scoring systems according to the 28th-day mortality in patients with pneumonia.
SIRS: Systemic Inflammatory Response Syndrome score; qSOFA: quick SOFA score; MEDS: Mortality in Emergency Department Sepsis score; MEWS: Modified Early Warning Score; PPV: positive predictive value; NPV: negative predictive value; AUC: area under the curve; CURB-65: Confusion, Urea (BUN >19 mg/dL), Respiratory rate ⩾30/minute, Systolic blood pressure <90 mmHg or Diastolic blood pressure ⩽60 mmHg, Age ⩾65 years).
When we consider the 5th-day mortality in the patients with pneumonia, the scoring system with the largest AUC, that is, showing the best performance in distinguishing dead and living patients, has been found as CURB-65 (>2). MEDS (>8) comes second while qSOFA (⩾2) comes third. According to the comparative analysis made between scoring systems, there is no statistical difference between CURB-65 and MEDS and qSOFA (respectively p = 0.894 and p = 0.256).
When the 14th-day mortality in the patients with pneumonia is considered, the scoring system with the largest AUC, that is, showing the best performance in distinguishing dead and living patients, has been found as MEDS (>8). CURB-65 (>2) comes second while qSOFA (⩾2) comes third. According to the comparative analysis made between scoring systems, there is no statistical difference between MEDS, CURB-65, and qSOFA (respectively p = 0.425 and p = 0.295).
When we consider the 28th-day mortality in the patients with pneumonia, the scoring system with the largest AUC, that is, showing the best performance in distinguishing dead and living patients, has been found as MEDS (>6). CURB-65 (>2) comes second while qSOFA (⩾2) comes third. According to the comparative analysis made between scoring systems, no difference has been statistically found between MEDS and CURB-65 (p = 0.095). There is difference between MEDS and qSOFA (p = 0.016).
The second most frequent infection focus (19%) in patients we studied was skin and soft tissue infection. The number of patients who died in this group 3 was not analyzed because it was not sufficient to reach statistically significant results.
In the patients we studied, the third most frequent infectious disease was determined as urinary tract infection with a rate of 15.75%. When the scoring systems applied to these patients are analyzed, sensitivity, specificity, PPV, and NPV found according to the 14th-day mortality and p values determined with AUCs are specified in Table 11.
Analysis of scoring systems according to the 14th-day mortality in patients diagnosed with urinary system infection.
SIRS: Systemic Inflammatory Response Syndrome score; qSOFA: quick SOFA score; MEDS: Mortality in Emergency Department Sepsis score; MEWS: Modified Early Warning Score; PPV: positive predictive value; NPV: negative predictive value; AUC: area under the curve.
When we consider the 14th-day mortality in the patients with urinary system infection, the scoring system with the largest AUC, that is, showing the best performance in distinguishing dead and surviving patients, has been found as MEDS (>8). MEWS (>3) comes second while CCI (>2) comes third. The success rates of the SIRS and qSOFA score in predicting mortality were not found to be statistically significant in this group (respectively p = 0.292 and p = 0.613). According to the comparative analysis made between scoring systems, no difference has been statistically found between MEDS and MEWS (p = 0.168). There is a difference between the MEDS and the CCI (p = 0.047).
In the patients we included, the fourth most frequent infectious disease was identified as abdominal infections with a rate of 14.75%. When the scoring systems applied to these patients are analyzed, sensitivity, specificity, PPV, and NPV found according to the 14th-day mortality and p values determined with AUCs are specified in Table 12.
Analysis of scoring systems according to the 14th-day mortality in patients diagnosed with abdominal infection.
SIRS: Systemic Inflammatory Response Syndrome score; qSOFA: quick SOFA score; MEDS: Mortality in Emergency Department Sepsis score; MEWS: Modified Early Warning Score; PPV: positive predictive value; NPV: negative predictive value; AUC: area under the curve.
When we consider the 14th-day mortality in the patients with abdominal infection, the scoring system with the largest AUC, that is, showing the best performance in distinguishing dead and living patients, has been found as MEDS (>8). MEWS (>3) comes second and qSOFA (⩾2) comes third. According to the comparative analysis made between scoring systems, there is no statistical difference between MEDS, MEWS, and qSOFA (respectively p = 0.529 and p = 0.393).
Discussion
The ideal scoring system for emergency departments should include a small number of physiologic variables that can be easily collected from the time of admission and provide clinically important results (such as mortality, duration of hospitalization) correctly. 22 In this study, SIRS, qSOFA, MEDS, MEWS, and CCI used as sepsis or objective risk assessment scoring systems, and CURB-65 systems specially developed for the patients diagnosed with pneumonia were used; it was aimed to determine the power of these scores’ predictive mortality rates and their superiority to each other.
In the study of Ranniko et al., patients with infection in emergency department were addressed and sepsis-related death was assessed on 28th and 90th days. In this study, 95% of the patients died on the 28th day. As a result, in the following studies, it is recommended to use the 28th-day mortality as a base. 23 We also determined the primary endpoint of our study as 28th-day mortality. We also took patients who died on the 5th and 14th days into consideration in order to compare the early efficacy of the scores.
In the study of Freund et al., patients with suspected infection admitted in emergency departments were handled and when in-hospital mortality was assessed, qSOFA was found to be more successful than SIRS criteria. 24 In the study of Finkelsztein et al., in-hospital mortality was assessed in the patients (67% of the patients were transferred from the emergency department) hospitalized outside of intensive care unit and qSOFA was detected to be more successful than SIRS criteria. 25 In the study of Churpek et al., patients hospitalized in emergency services and clinics were evaluated and the area under the curve (ROC) of qSOFA was found to be superior to SIRS. 26 In our study, qSOFA score was found to be superior to SIRS in comparing the AUC and estimating 5th-, 14th-, and 28th-day mortalities in all patient groups. Our study supports the use of the sepsis III criteria for emergency department.
In the study of Wang et al., patients diagnosed with infection and hospitalized in emergency department were evaluated and MEDS score was found to be superior to qSOFA in predicting the 28th-day mortality. 27 The study of Çildir et al. revealed that MEDS score may be useful for prognosis in patients diagnosed with sepsis according to SIRS criteria in emergency department. 28 In our study, MEDS and qSOFA scores were found to be the most effective in predicting 5th- and 14th-day mortalities in all patient groups and inpatients and to be substitute for each other. The most effective score for predicting the 28th-day mortality was found to be MEDS.
In the study of Chen et al., power of qSOFA, Confusion, Respiratory rate ≥30/minute, Systolic blood pressure <90 mmHg or Diastolic blood pressure ≤60 mmHg (CRB), and Confusion, Respiratory rate ≥30/minute, Systolic blood pressure <90 mmHg or Diastolic blood pressure ≤60 mmHg, Age ≥65 years (CRB-65) scores predicting mortality in the patients with pneumonia in emergency service was compared, and their predictive power of 28th-day mortality and the AUCs were found to be similar. 29 In our study, we found the efficacy of MEDS, CURB-65, and qSOFA score similar in predicting 5th- and 14th-day mortalities, but efficacy of MEDS and CURB-65 score superior to qSOFA on the 28th-day mortality.
An another important point in the emergency department is not to ignore the patient who has high risk of mortality; when we look at the sensitivities of the scores in our study, MEDS is still in the first place. SIRS is detected to be superior to qSOFA in sensitivity. However, it is also important to note that SIRS is ⩾2 in 209 of 400 patients.
This was a study performed at single-center, and sample size was not big as multicenter studies. Patients with missing data were not included in the study. Patients who died within 1 h after being admitted in emergency department were also excluded from the study. Although all excluded patients are not a large group, they may have affected the results.
As a result, our study supports use of qSOFA, which can be applied easily and rapidly in determining early mortality (5th and 14th days) in emergency department. However, use of MEDS score with more variables in predicting the 28th-day mortality may give better results. These recommendations are valid in patients with pneumonia; CURB-65 score can also be used with equivalent effectiveness as the most effective score in predicting 5th-, 14th-, and 28th-day mortalities. The predictive power of SIRS used for identifying sepsis was detected to be lower.
Footnotes
Acknowledgements
All authors contributed to the development of the study protocol. Maruf Beğenen and Vahide Aslihan Durak were responsible for obtaining ethical approval for the study and collected all data for analysis. Halis Akalın undertook the data analysis. Erol Armağan, Maruf Beğenen, and Vahide Aslihan Durak contributed to preparation of the manuscript, and all take responsibility for its contents.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Availability of data and materials
All materials taken from other sources (including our own published writing) were clearly cited.
Ethical approval
This study was initiated in the emergency department of a university hospital following the ethics committee approval.
Human rights
Our work does not infringe on any rights of others, including privacy rights, and intellectual property rights. There is no human rights violation in the study.
Informed consent
Written informed consent was obtained from all the patients for their anonymized information to be published in this article.
