Comparison between XGboost model and logistic regression model for predicting sepsis after extremely severe burns

Abstract

Objective

To compare an Extreme Gradient Boosting (XGboost) model with a multivariable logistic regression (LR) model for their ability to predict sepsis after extremely severe burns.

Methods

For this observational study, patient demographic and clinical information were collected from medical records. The two models were evaluated using area under curve (AUC) of the receiver operating characteristic (ROC) curve.

Results

Of the 103 eligible patients with extremely severe burns, 20 (19%) were in the sepsis group, and 83 (81%) in the non-sepsis group. The LR model showed that age, admission time, body index (BI), fibrinogen, and neutrophil to lymphocyte ratio (NLR) were risk factors for sepsis. Comparing AUC of the ROC curves, the XGboost model had a higher predictive performance (0.91) than the LR model (0.88). The SHAP visualization tool indicated fibrinogen, NLR, BI, and age were important features of sepsis in patients with extremely severe burns.

Conclusions

The XGboost model was superior to the LR model in predictive efficacy. Results suggest that, fibrinogen, NLR, BI, and age were correlated with sepsis after extremely severe burns.

Keywords

XGboost Logistic Regression Extremely Severe Burn Sepsis Risk Factors

Introduction

Extremely severe burns result in widespread damage to skin and tissues, and cause serious physiological and metabolic disturbances that require special treatment and intensive care.^1–3 Patients with extremely severe burns have a high risk for developing sepsis and a high mortality rate.^4–6 However, there has been little research on risk factors associated with sepsis in patients with extremely severe burns, and the clinical management of these patients continues to pose many challenges.

In recent years, research has shown that certain biomarkers have the potential to predict the development and outcomes of sepsis. For example, indicators such as serum pre-albumin, prothrombin time, and neutrophil to lymphocyte ratio (NLR) have been shown to be useful for predicting and treating sepsis.^7–9 However, the pathological process of sepsis that occurs in patients with extremely severe burns differs from that caused by other factors. Therefore, identifying risk factors for sepsis development in patients with extremely severe burns could aid in screening and recognizing populations which are at high risk of sepsis, and ultimately reduce its incidence.

While generalized linear regression (LR) models have been widely used in the past to predict sepsis, they have limitations in extracting variables from high-dimensional data. However, machine learning models can avoid missing important variables in a regression analysis through the use of algorithms. The Extreme Gradient Boosting (XGBoost) model is a powerful machine learning algorithm for risk prediction. It has been successfully applied to research in several medical fields.^10–12 In this current study, we retrospectively analysed data from patients with extremely severe burns, and compared an XGBoost model with a multivariable LR model for their ability to predict sepsis after extremely severe burns.

Patients

For this observational study, consecutive patients with extremely severe burns who had been hospitalized at the Red Cross Hospital in Guangzhou, China between January 1st, 2016, and December 31st, 2022 were identified from patient records. Patients included were aged ≥18 years had total body surface area (TBSA) in burns ≥50% or third-degree burns ≥20%. Patients whose sepsis had occurred before admission, had refused treatment within four days of hospitalization, had received corticosteroids or immunosuppressants or, had died, were excluded from the study. Sepsis was defined according to Sepsis-3 Consensus Definitions.¹³

The reporting of this study conforms to STROBE guidelines.¹⁴ The study obtained formal approval from the Institutional Review Board of Guangzhou Red Cross Hospital. Written informed consent was not required due to the retrospective design of the study and patient data were anonymized prior to analysis.

Data collection

The following data were extracted from the patients’ medical records: sex; age; hospital admission time; TBSA in burns; third-degree burn area; Burn index (BI), presence of inhalation injury; blood levels of lactate, albumin, creatinine, fibrinogen, red blood cells (RBCs), neutrophils, platelets, lymphocytes, magnesium, monocytes, high-density lipoprotein (HDL), and C-reactive protein (CRP); neutrophil-to-lymphocyte ratio (NLR). Blood samples had been collected within 12 hours after admission and burn area and depth had been assessed by burn surgeons. The BI reflected the severity of the burns, and for this study was equal to the area of third-degree burns plus half the area of the second-degree burns.¹⁵

Statistical analysis

Statistical analysis was performed using SPSS software (version 27.0 for Windows®; IBM Corp, Armonk, NY, USA). A P-value <0.1 was considered to indicate statistical significance. Normally distributed data were presented as a mean ± SD, and independent sample t-tests were used to analyse inter-group differences. Non-normally distributed data were presented as a medians and interquartile ranges, and Mann-Whitney U tests were used to analyse inter-group differences. Categorical variables were presented as absolute counts and as frequencies (%) and χ² tests were used to compare inter-group differences.

A multivariable LR analysis was performed to identify risk factors for sepsis in patients with extremely severe burns, and regression coefficients, odds ratios (ORs), and 95% confidence intervals (CIs) were calculated. The R rms package was used to plot a nomogram and Hosmer-Lemeshow goodness of fit test was used to evaluate the predictive model. The XGboost model was constructed using the XGboost machine learning algorithm. A shapely additive explanation (SHAP) analysis was used to interpret and analyse the risk factors for sepsis in patients with extremely severe burns to provide a basis for the reliability of the model results. The two models were evaluated using area under curve (AUC) of the receiver operating characteristic (ROC) curve. Youden Index, was used as a summary measure of the ROC curve.

Results

A total of 106 cases were identified, of whom, three patients were excluded. Of the 103 remaining patients, 20 (19%) were identified for the sepsis group, and 83 (81%) for the non-sepsis group (Figure 1). Values for age, admission time >8 hours, area of third-degree burns, BI, fibrinogen, and CRP were all greater in the sepsis group compared with the non-sepsis group (Table 1). However, levels of albumin, lymphocytes and HDL were lower in the sepsis group compared with the non-sepsis group. In the multivariable analysis, we used BI because there was a positive correlation between third-degree burns and BI. In addition, we used NLR as a clinical meaningful index because it has been reported to be predictive for sepsis in its early stages.

Figure 1.

Flow chart of the study population.

Table 1.

Demographic characteristics of patients with extremely severe burns.

Variables	Sepsis group(n = 20)	Non-sepsis group(n = 83)	Statistical significance
Age, y	48 ± 12	42 ± 13	P = 0.063
Sex			ns
Female	3 (15)	20 (24)
Male	17 (85)	63 (76)
Admission time			P = 0.003
≤8 hours	8 (40)	64 (77)
>8 hours	12 (60)	19 (23)
TBSA in burns, %	75 (60, 91)	70 (50, 83)	ns
Third-degree burns	55 (38, 73)	37 (27, 60)	P = 0.015
Burn Index	65 (49, 83)	50 (38, 70)	P = 0.033
Inhalation injury			ns
No	3 (15)	16 (19)
Yes	17 (85)	67 (81)
Lactic acid, mmol/l	5.52 ± 2.27	5.59 ± 2.27	ns
Albumin, g/l	26.3 ± 6.7	32.8 ± 7.8	P < 0.001
Creatinine, mg/dl	84.4 (69.0, 99.5)	75.0 (67.0, 98.0)	ns
Fibrinogen, g/l	2.85 (2.19, 3.72)	2.39 (1.96, 2.83)	P = 0.084
RBCs, ×10¹²/l	5.64 (4.78, 6.41)	5.8 (5.2, 6.5)	ns
Neutrophils, ×10⁹/l	20.8 ± 8.4	21.6 ± 9.5	ns
PLT, ×10⁹/l	191 (142, 352)	283 (219, 366)	ns
Lymphocytes, ×10⁹/l	1 (0.8, 2.4)	1.8 (1.2, 2.7)	P = 0.057
NLR, %	15.7 (10.1, 25.9	13.4 (6.1, 20.0)	ns
Magnesium, mmol/l	0.72 (0.66, 0.82)	0.75 (0.66, 0.82)	ns
Monocytes, 10⁹/l	1.43 (0.98, 1.97)	1.23 (0.88, 1.8)	ns
HDL, mmol/l	0.76 (0.57, 1.14)	0.99 (0.8, 1.26)	P = 0.046
CRP, mg/l	34.1 (2.0, 69.0)	2.3 (1.0, 12.6)	P = 0.01

Data are expressed as, n (%), mean ± standard deviation, or medians (25th, 75th percentiles).

TBSA, total body surface area; RBCs, red blood cells; PLT, platelets; NLR, neutrophil-to-lymphocyte ratio (NLR), HDL, high-density lipoprotein; CRP, C reactive protein; ns, not statistically significant.

The results of a multivariable LR analysis showed that age, admission time, BI, fibrinogen, and NLR were independent risk factors for the occurrence of sepsis (Figure 2). BI was the most significant risk factor for sepsis (P = 0.007) and the risk for sepsis in patients with extremely severe burns admitted within 8 hours after a burn injury was 4.7-fold greater than in patients admitted >8 hours after a burn injury. In addition, patients with extremely severe burns and high fibrinogen levels had a 1.9-fold greater risk of developing sepsis than those with low fibrinogen levels.

Figure 2.

A Forest map shows the results of the multivariable logistic regression (LR) analysis. Age, admission time, Body Index (BI), fibrinogen, and neutrophil-to-lymphocyte ratio (NLR) were independent risk factors for the occurrence of sepsis. OR, odds ratio; CI, confidence interval.

A nomogram, constructed to illustrate the risk for sepsis in patients with extremely severe burns, indicated that, as the fibrinogen level increased by 0.5 units, the nomogram score increased by 7.5 points (Figure 3). An ROC curve showed that the AUC of the LR model was 0.88 (95% CI: 0.82–0.95) and the maximum Youden index score was 0.22 (Figure 4). The Hosmer-Lemeshow goodness-of-fit test (i.e., H-L test) was conducted on the predictive model, with χ² = 13.35 and P = 0.1, demonstrating that the model had a good fit.

Figure 3.

Individualized nomogram of the logistic regression (LR) model for the risk of sepsis in patients with severe burns. NLR, neutrophil-to-lymphocyte ratio; BI, body index. *P < 0.05; **P < 0.01.

Figure 4.

Receiver operating characteristic (ROC) curve of the logistic regression (LR) model. Area under the curve (AUC) was 0.88, (P < 0.001; 95% CI, 0.82, 0.95). Youden index was 0.22 (95% CI, 0.83, 0.90).

The training set (80% data) was iteratively incorporated into XGboost model to obtain the best parameters. The sensitivity of the XGboost model was 82%, its specificity was 100%, and the AUC of the ROC curve was 0.91 (95% CI: 0.79, 1.0) and the maximum Youden index score was 0.08 (Figure 5).

Figure 5.

Receiver operating characteristic (ROC) curve of the XGboost model. Area under the curve (AUC) was 0.91, (P < 0.001; 95% CI, 0.82, 0.95). Youden index was 0.08 (95% CI, 0.82, 1.0).

A SHAP algorithm was used to obtain the importance of each predictor variable to the predicted results of the XGboost model. The variable importance plot lists the most important variables in descending order (Figure 6). Fibrinogen had the strongest predictive value for all predicted levels, followed by TBSA in burns, NLR, BI, and age. In addition, to detect positive and negative relationships between predicted values and target outcomes, SHAP values were applied to show mortality risk factors (Figure 7). The horizontal position on the graph shows whether the effect of the value was associated with a higher or lower prediction, and the colour shows whether the variable was high (red) or low (blue) for that observation. Our analysis showed that an increase in fibrinogen, NLR, BI, age, and admission time had a positive effect and pushed the prediction towards the occurrence of sepsis.

Figure 6.

SHAP summary plot of the features of the XGBoost model. The higher the SHAP value of a feature, the higher the probability of a predictive value for sepsis. Fibrinogen had the strongest predictive value for all predicted levels, followed by total body surface area (TBSA) in burns, neutrophil-to-lymphocyte ratio (NLR), body index (BI), and age. PLT, platelets; CRP, C-reactive protein; III, area of third-degree burns; RBC, red blood cells.

Figure 7.

Analysis of XGboost model features using SHAP values to show mortality risk factors. An increase in fibrinogen, NLR, BI, age, and admission time had a positive effect and pushed the prediction towards the occurrence of sepsis. SHAP, shapely additive explanation analysis; TBSA, total body surface area in burns; NLR, neutrophil-to-lymphocyte ratio; TBSI, total burn surface involved; PLT, platelets; CRP, C-reactive protein; III, area of third-degree burns; RBC, red blood cells.

Discussion

Sepsis is a common complication in patients with extremely severe burns, and can lead to septic shock, immune suppression, multiple organ dysfunction, and even death.^4–6 Therefore, preventing sepsis in patients with extremely severe burns is of great clinical importance.^16–19 In this present study, the incidence rate of sepsis among 103 severely burned patients, was 19%. Identifying risk factors for sepsis in these patients will undoubtedly lead to significant improvements in their outcome. We showed that the XGboost model was superior to the LR model in its predictive efficacy as assessed by AUC of the ROC curves. Common risk factors screened included, fibrinogen, NLR, BI, and age. Previous studies have also found that age was a factor affecting the prognosis of sepsis in patients with extremely severe burns, and was positively correlated with burn mortality.²⁰

In our study, the LR model showed that BI was the most significant risk factor for sepsis in patients with extremely severe burns, which was consistent with findings from a similar study.²¹ In addition, the LR model showed that the risk for sepsis in patients with extremely severe burns admitted within 8 hours after a burn injury was nearly five times greater than in patients admitted >8 hours after a burn injury. The SHAP interpreter of the XGboost model also showed that the longer the length of admission time after a burn injury, the higher the risk for sepsis developing in the patient. Indeed, admission time following a burn injury is an important risk factor, and timely treatment of a burn injury, including effective fluid resuscitation, anti-shock therapy, early burn wound management, infection control, and organ protection, is important in preventing sepsis.^22,23

Fibrinogen is a protein in the liver composed of three pairs of peptide chains, and plays an important role in the coagulation/anticoagulation process. Fibrinogen is critical for blood clotting, and when its level is elevated, it may increase blood coagulation and become an important risk factor for thrombotic disease. Studies have found that fibrinogen has important diagnostic and predictive value in various diseases, such as cardiovascular disease, cancer, and diabetes.^24–26 Elevated fibrinogen levels are an independent risk factor for death after sepsis, suggesting that elevated fibrinogen levels may have prognostic value for sepsis.^27,28 The results of this current study showed that patients with extremely severe burns and high fibrinogen levels had nearly a two-fold greater risk of developing sepsis than those with low fibrinogen levels. However, the pathological mechanism by which elevated fibrinogen levels increase the risk for sepsis remains unclear.

NLR reflects the dynamic balance between neutrophils and lymphocytes in peripheral blood and in recent years has received attention as a new type of inflammatory index. For example, in a disease state, changes in NLR may reflect the severity and prognosis of the disease.^29–31 In addition, NLR is considered to be a reliable biomarker for diagnosing bacteraemia and sepsis.^32–34 Studies have shown that a high NLR was associated with a poor prognosis in sepsis patients and was an independent risk factor for predicting sepsis patient mortality.³⁵ The results of our study showed that NLR was an independent risk factor for sepsis in severely burned patients, and so provides a new direction for future treatment strategies.

The study had several limitations. For instance, its sample size was small and only data from 103 patients were included which may have led to unintentional bias. Additionally, the study was conducted at a single data centre in China, which may limit the generalizability of its findings. Importantly, although the study found that fibrinogen, BI, age, and NLR were high-risk factors for sepsis in patients with severe burns, the clinical significance of these findings has not been fully explored. Despite these limitations, the findings of this present study are important and may provide new directions for future research and clinical applications. Further, large scale, prospective, randomised, studies are required to confirm our results.

In summary, this study showed that the XGboost model was superior to the LR model in predictive efficacy. The results suggest that, fibrinogen, NLR, BI, age and time of admission were correlated with sepsis after extremely severe burns. The appropriate management of patients with abnormal indices upon admission may help to reduce their risk of sepsis.

Footnotes

Acknowledgements

We wish to thank our colleagues in the Departments of burns and plastic surgery for their cooperation with this study.

Declaration of conflicting interests

The authors declare that there are no conflicts of interest.

Funding

This work was supported by research grant 202102020481 (P.L.) from Guangzhou Science and Technology Plan Project, research grant 202102010074 (T.Z.) from Guangzhou Science and Technology Plan Project School (Institute) Joint Funding Project, research grant RHPG05 from Guangzhou Research Hospital Construction Project, research grant 2023A03J0524 (Y.H.H.) from Guangzhou Science and Technology Plan Project School (Institute) Joint Funding Project, and research grant from 2018 Project Funds of Guangzhou Red Cross Hospital.

ORCID iD

Peng Liu

References

Chua

Khoo

Tan

, et al. Skin tissue engineering advances in severe burns: review and therapeutic applications. Burns Trauma 2016; 4: 3.

Wang

Liu

Fan

, et al. A case report of extremely severe 93% total body surface area burns in a 5-year-old boy. Eur Rev Med Pharmacol Sci 2023; 27: 3931–3940.

Jin

Yang

Weng

, et al. Epidemiology and Early Bacteriology of Extremely Severe Burns from an LPG Tanker Explosion in Eastern China. J Epidemiol Glob Health 2022; 12: 478–485.

Torres

Peterson

Wolf

SE.

Detection of Infection and Sepsis in Burns. Surg Infect (Larchmt) 2021; 22: 20–27.

Yang

Tan

Yao

, et al. Long-Term Effects of Severe Burns on the Kidneys: Research Advances and Potential Therapeutic Approaches. J Inflamm Res 2023; 16: 1905–1921.

Cato

Wearn

Bishop

, et al. Platelet count: A predictor of sepsis and mortality in severe burns. Burns 2018; 44: 288–297.

Kriplani

Pandit

Chawla

, et al. Neutrophil-lymphocyte ratio (NLR), platelet-lymphocyte ratio (PLR) and lymphocyte-monocyte ratio (LMR) in predicting systemic inflammatory response syndrome (SIRS) and sepsis after percutaneous nephrolithotomy (PNL). Urolithiasis 2022; 50: 341–348.

Zahorec

Neutrophil-to-lymphocyte ratio, past, present and future perspectives. Bratisl Lek Listy 2021; 122: 474–488.

Ruan

Chen

Liu

, et al. The combination of procalcitonin and C-reactive protein or presepsin alone improves the accuracy of diagnosis of neonatal sepsis: a meta-analysis and systematic review. Crit Care 2018; 22: 316.

10.

Jaeyoung

Honggook

Wonhyeong

, et al. Development of a machine learning model for the prediction of the short-term mortality in patients in the intensive care unit. J Crit Care 2022; 71: 154106.

11.

Kim

Chang

Kim

, et al. Machine learning for prediction of septic shock at initial triage in emergency department. J Crit Care 2020; 55: 163–170.

12.

Pellegrini

Green-Saxena

, et al. Supervised machine learning for the early prediction of acute respiratory distress syndrome (ARDS). J Crit Care 2020; 60: 96–102.

13.

Singer

Deutschman

Seymour

, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 2016; 315: 801–810.

14.

Von Elm

Altman

Egger

, STROBE Initiativeet al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guide-Lines for reporting observational studies. Ann Intern Med 2007; 147: 573–577.

15.

Tan

Zhou

Yuan

, et al. A New Resuscitation Formula Based on Burn Index Provides More Reliable Prediction for Fluid Requirement in Adult Major Burn Patients. J Burn Care Res 2021; 42: 962–967.

16.

Devine

Trainor

DM.

Critical care management of patients with severe burns and inhalational injury. Anaesthesia & Intensive Care Medicine 2023; 24: 397–401. Available from: https://doi.org/10.1016/j.mpaic.2023.04.007

17.

Wang

, et al. Intestinal barrier dysfunction in severe burn injury. Burns Trauma 2019; 7: 24.

18.

Huang

Chen

Guo

, et al. Prospective study and validation of early warning marker discovery based on integrating multi-omics analysis in severe burn patients with sepsis. Burns Trauma 2023; 11: tkac050.

19.

Henry

TCL

Huei

Mathew

, et al. Survival analysis and mortality predictors of hospitalized severe burn victims in a Malaysian burns intensive care unit. Burns Trauma 2019; 7: 3.

20.

Yichao

Xinyuan

Xiaonan

, et al. Valuable prognostic indicators for severe burn sepsis with inhalation lesion: age, platelet count, and procalcitonin. Burns Trauma 2018; 6: 29.

21.

Steinvall

Elmasry

Fredrikson

, et al. Standardised mortality ratio based on the sum of age and percentage total body surface area burned is an adequate quality indicator in burn care: An exploratory review. Burns 2016; 42: 28–40.

22.

Rizzo

Coates

Serio-Melvin

, et al. Higher Initial Formula for Resuscitation After Severe Burn Injury Means Higher 24-Hour Volumes. J Burn Care Res 2023; 44: 1017–1022.

23.

Singer

Gabbe

Cleland

, et al. The association between out of hours burn centre admission and in-hospital outcomes in patients with severe burns. Burns 2023; 49: 1062–1072.

24.

Hao

Jie

Jian

, et al. Fibrinogen-to-albumin ratio predicts overall survival of hepatocellular carcinoma. World J Gastrointest Oncol 2023; 15: 1662–1672.

25.

Wenjie

Wencai

, et al. Prognostic significance of fibrinogen and neutrophil/lymphocyte ratio score and D-dimer/Albumin ratio for prognosis in patients with aneurysmal subarachnoid hemorrhage. J Stroke Cerebrovasc Dis 2023; 32: 107338.

26.

Shuiqiao

Wenqiao

Qinghui

, et al. Prognostic value of APTT combined with fibrinogen and creatinine in predicting 28-Day mortality in patients with septic shock caused by acute enteric perforation. BMC Surg 2023; 23: 274.

27.

Schupp

Weidner

Rusnak

, et al. Fibrinogen reflects severity and predicts outcomes in patients with sepsis and septic shock. Blood Coagul Fibrinolysis 2023; 34: 161–170.

28.

Negrón

Weggeman

Grimbergen

, et al. Fibrinogen γ’ Promotes Host Survival During Staphylococcus aureus Septicemia in Mice. J Thromb Haemost 2023; 21: 2277–2290.

29.

Ouyang

Xiao

Huang

, et al. Baseline and early changes in the neutrophil–lymphocyte ratio (NLR) predict survival outcomes in advanced colorectal cancer patients treated with immunotherapy. Int Immunopharmacol 2023; 123: 110703.

30.

Yao

Wang

Tang

, et al. Neutrophil-to-lymphocyte ratio (NLR), platelet-to-lymphocyte ratio (PLR), and systemic immune inflammation index (SII) to predict postoperative pneumonia in elderly hip fracture patients. J Orthop Surg Res 2023; 18: 673.

31.

Zhu

Deng

, et al. Predictive value of NLR and PLR for immune-related adverse events: a systematic review and meta-analysis. Clin Transl Oncol 2023; Sep 8.

32.

Xin

Shao

, et al. Accuracy of the neutrophil-to-lymphocyte ratio for the diagnosis of neonatal sepsis: a systematic review and meta-analysis. BMJ Open 2022; 12: e060391.

33.

Wang

Huang

, et al. Diagnostic and therapeutic value of biomarkers in urosepsis. Ther Adv Urol 2023; 15: 17562872231151852.

34.

Wei

Huang

Yang

, et al. Neutrophil-to-Lymphocyte ratio as a prognostic marker of mortality and disease severity in septic Acute kidney injury Patients: A retrospective study. Int Immunopharmacol 2023; 116: 109778.

35.

Liu

She

, et al. Predictive value of immune cell counts and neutrophil-to-lymphocyte ratio for 28-day mortality in patients with sepsis caused by intra-abdominal infection. Burns Trauma 2021; 9: tkaa040.