This meta-analysis aimed to assess the diagnostic performance of artificial intelligence for detecting sepsis in intensive care unit.
Methods
A thorough literature search was performed using PubMed, Embase, and Web of Science to locate relevant studies published through November 2024. The selected studies specifically examined the diagnostic accuracy of artificial intelligence in identifying septicemia. To estimate pooled sensitivity and specificity values, a bivariate random-effects model was employed, with results reported alongside 95% confidence intervals. Heterogeneity across studies was evaluated using the I2 statistic.
Results
Of the 1495 studies initially identified, 16 studies encompassing a total of 159,947 patients, met the inclusion criteria for this meta-analysis. For the internal validation set, the pooled results for sepsis detection showed a sensitivity of 0.76 (95% CI: 0.71-0.80), a specificity was of 0.85 (95% CI: 0.81-0.89), and an area under the curve (AUC) of 0.87 (95% CI: 0.84-0.90). In comparison, the external validation set yielded a sensitivity of 0.78 (95% CI: 0.65-0.87), a specificity of 0.82 (95% CI: 0.76-0.86), and an AUC of 0.87 (95% CI: 0.83-0.89). Deeks’ funnel plot and Egger's test indicated no significant publication bias in both the internal and external validation sets(P = .63,.89).
Conclusions
The findings of this meta-analysis indicate that artificial intelligence demonstrates a high diagnostic performance in identifying sepsis and septic shock. However, substantial heterogeneity across studies may impact the robustness of this evidence. Further research using external validation datasets is required to confirm these results and evaluate their applicability in clinical settings.
FleischmannCScheragAAdhikariNK, et al.Assessment of global incidence and mortality of hospital-treated sepsis. Current estimates and limitations, Am J Respir Crit Care Med. 2016;193(3):259-272. https://10.1164/rccm.201504-0781OC
2.
HalpernNAPastoresSM. Critical care medicine in the United States 2000-2005: An analysis of bed numbers, occupancy rates, payer mix, and costs. Crit Care Med. 2010;38(5):65-71. https://10.1097/CCM.0b013e3181b090d0
3.
RiversENguyenBHavstadS, et al.Early goal-directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med. 2001;345(19):1368-1377. https://10.1056/NEJMoa010307
4.
NguyenHBCorbettSWSteeleR, et al.Implementation of a bundle of quality indicators for the early management of severe sepsis and septic shock is associated with decreased mortality. Crit Care Med. 2007;35(4):1105-1112. https://10.1097/01.Ccm.0000259463.33848.3d
5.
KumarARobertsDWoodKE, et al.Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock. Crit Care Med. 2006;34(6):1589-1596. https://10.1097/01.Ccm.0000217961.75225.E9
6.
WangCXuRZengYZhaoYHuX. A comparison of qSOFA, SIRS and NEWS in predicting the accuracy of mortality in patients with suspected sepsis: A meta-analysis. PLoS One. 2022;17(4):e0266755. https://10.1371/journal.pone.0266755
7.
TanTLTangYJChingLJAbdullahNNeohHM. Comparison of prognostic accuracy of the quick sepsis-related organ failure assessment between short- & long-term mortality in patients presenting outside of the intensive care unit - A systematic review & meta-analysis. Sci Rep. 2018;8(1):16698. https://10.1038/s41598-018-35144-6
8.
Giamarellos-BourboulisEJTsaganosTTsangarisI, et al.Validation of the new sepsis-3 definitions: Proposal for improvement in early risk identification. Clin Microbiol Infect. 2017;23(2):104-109. https://10.1016/j.cmi.2016.11.003
9.
TsalikELJaggersLBGlickmanSW, et al.Discriminative value of inflammatory biomarkers for suspected sepsis. J Emerg Med. 2012;43(1):97-106. https://10.1016/j.jemermed.2011.05.072
10.
LinkFKrohnKBurgdorffAMChristelASchumannJ. Sepsis Diagnostics: Intensive Care Scoring Systems Superior to MicroRNA Biomarker Testing. Diagnostics (Basel). 2020;10(9):701. https://10.3390/diagnostics10090701
11.
UmscheidCABeteshJVanZandbergenC, et al.Development, implementation, and impact of an automated early warning and response system for sepsis. J Hosp Med. 2015;10(1):26-31. https://10.1002/jhm.2259
12.
NguyenSQMwakalindileEBoothJS, et al.Automated electronic medical record sepsis detection in the emergency department. PeerJ. 2014;2:e343. https://10.7717/peerj.343
13.
JiangFJiangYZhiH, et al.Artificial intelligence in healthcare: Past, present and future. Stroke Vasc Neurol. 2017;2(4):230-243. https://10.1136/svn-2017-000101
14.
HorngSSontagDAHalpernYJerniteYShapiroNINathansonLA. Creating an automated trigger for sepsis clinical decision support at emergency department triage using machine learning. PLoS One. 2017;12(4):e0174708. https://10.1371/journal.pone.0174708
15.
KhanSDasAMishraA, et al.Evaluation of three protocols for direct susceptibility testing for gram-negative rods from flagged positive blood culture bottles. Microbiol Spectr. 2024;12(4):e0308123. https://10.1128/spectrum.03081-23
16.
SalamehJPBossuytPMMcGrathTA, et al.Preferred reporting items for systematic review and meta-analysis of diagnostic test accuracy studies (PRISMA-DTA): Explanation, elaboration, and checklist. Br Med J.2020;370:m2632. https://10.1136/bmj.m2632
17.
SunBLeiMWangL, et al.Prediction of sepsis among patients with major trauma using artificial intelligence: a multicenter validated cohort study. Int J Surg. 2024;111(1):467-480. https://10.1097/js9.0000000000001866
18.
ShashikumarSPStanleyMDSadiqI, et al.Early sepsis detection in critical care patients using multiscale blood pressure and heart rate dynamics. J Electrocardiol. 2017;50(6):739-743. https://10.1016/j.jelectrocard.2017.08.013
19.
CalvertJSPriceDAChettipallyUK, et al.A computational approach to early sepsis detection. Comput Biol Med. 2016;74(July):69-73. https://10.1016/j.compbiomed.2016.05.003
20.
KamHJKimHY. Learning representations for the early detection of sepsis with deep neural networks. Comput Biol Med. 2017;89(November):248-255. https://10.1016/j.compbiomed.2017.08.015
21.
PerssonIOstlingAArlbrandtMSoderbergJBecedasD. A machine learning sepsis prediction algorithm for intended intensive care unit use (NAVOY sepsis): Proof-of-concept study. JMIR Form Res. 2021;5(9):e28000. https://10.2196/28000
22.
WangRZSunCHSchroederPH, et al.editors. Predictive Models of Sepsis in Adult ICU Patients. 6th IEEE International Conference on Healthcare Informatics (ICHI); 2018 Jun 04-07; New York, NY2018:390-391.
23.
NematiSHolderARazmiFStanleyMDCliffordGDBuchmanTG. An interpretable machine learning model for accurate prediction of sepsis in the ICU. Crit Care Med. 2018;46(4):547-553. https://10.1097/CCM.0000000000002936
24.
ShashikumarSPJosefCSSharmaANematiS. DeepAISE - an interpretable and recurrent neural survival model for early prediction of sepsis. Artif Intell Med. 2021;113:102036. https://10.1016/j.artmed.2021.102036
25.
RoimiMNeubergerAShrotAPaulMGeffenYBar-LavieY. Early diagnosis of bloodstream infections in the intensive care unit using machine-learning algorithms. Intensive Care Med. 2020;46(3):454-462. https://10.1007/s00134-019-05876-8
26.
HenryKEHagerDNPronovostPJSariaS. A targeted real-time early warning score (TREWScore) for septic shock. Sci Transl Med. 2015;7(299):299ra122. https://10.1126/scitranslmed.aab3719
27.
Joseph GuillénJL. Margaret Furr, Tianyao Wang, Stephen Strong, Christopher C. Moore, Abigail, Flower LEB, Predictive Models for Severe Sepsis in Adult ICU Patients, 2015 Systems and Information Engineering Design Symposium. 2015: 182-187. https://doi:10.1109/sieds.2015.7116970.
28.
LiuRGreensteinJLGraniteSJ, et al.Data-driven discovery of a novel sepsis pre-shock state predicts impending septic shock in the ICU. Sci Rep. 2019;9(1):6145. https://10.1038/s41598-019-42637-5
29.
LiliopoulosSGDejacoADimakopoulosVSGkouzionisIA. Development and Validation of the VIOSync Sepsis Prediction Index: A Novel Machine Learning Model for Sepsis Prediction in ICU Patients. 2024. https://10.1101/2024.02.22.24303211.
30.
YuanKCTsaiLWLeeKH, et al.The development an artificial intelligence algorithm for early sepsis diagnosis in the intensive care unit. Int J Med Inform. 2020;141:104176. https://10.1016/j.ijmedinf.2020.104176
31.
PerssonIMacuraABecedasDSjovallF. Early prediction of sepsis in intensive care patients using the machine learning algorithm NAVOY(R) sepsis, a prospective randomized clinical validation study. J Crit Care. 2024;80:154400. https://10.1016/j.jcrc.2023.154400
32.
KimTTaeYYeoHJ, et al.Development and Validation of Deep-Learning-Based Sepsis and Septic Shock Early Prediction System (DeepSEPS) Using Real-World ICU Data. J Clin Med. 2023;12(22):7156. https://10.3390/jcm12227156
33.
WhitingPFRutjesAWWestwoodME, et al.QUADAS-2: A revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155(8):529-536. https://10.7326/0003-4819-155-8-201110180-00009
34.
WolffRFMoonsKGMRileyRD, et al.PROBAST: A tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med. 2019;170(1):51-58. https://10.7326/m18-1376
35.
WalterSD. Properties of the summary receiver operating characteristic (SROC) curve for diagnostic test data. Stat Med. 2002;21(9):1237-1256. https://10.1002/sim.1099
36.
GallagherEJ. Clinical utility of likelihood ratios. Ann Emerg Med. 1998;31(3):391-397. https://10.1016/s0196-0644(98)70352-x
37.
HigginsJPThompsonSG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;21(11):1539-1558. https://10.1002/sim.1186
38.
DesautelsTCalvertJHoffmanJ, et al.Prediction of sepsis in the intensive care unit with minimal electronic health record data: A machine learning approach. JMIR Med Inform. 2016;4(3):e28. https://10.2196/medinform.5909
39.
RhodesAEvansLEAlhazzaniW, et al.Surviving sepsis campaign: International guidelines for management of sepsis and septic shock: 2016. Intensive Care Med. 2017;43:(3)304-377. https://10.1007/s00134-017-4683-6
40.
LauritsenSMKalørMEKongsgaardEL, et al.Early detection of sepsis utilizing deep learning on electronic health record event sequences. Artif Intell Med. 2020;104:101820. https://10.1016/j.artmed.2020.101820
41.
BomrahSUddinMUpadhyayU, et al.A scoping review of machine learning for sepsis prediction- feature engineering strategies and model performance: A step towards explainability. Crit Care. 2024;28(1):180. https://10.1186/s13054-024-04948-6
42.
FleurenLMKlauschTLTZwagerCL, et al.Machine learning for the prediction of sepsis: A systematic review and meta-analysis of diagnostic test accuracy. Intensive Care Med. 2020;46(3):383-400. https://10.1007/s00134-019-05872-y
43.
YangZCuiXSongZ. Predicting sepsis onset in ICU using machine learning models: A systematic review and meta-analysis. BMC Infect Dis. 2023;23(1):635. https://10.1186/s12879-023-08614-0
44.
GuoJLiB. The application of medical artificial intelligence technology in rural areas of developing countries. Health Equity. 2018;2(1):174-181. https://10.1089/heq.2018.0037
45.
ImreyPB. Limitations of meta-analyses of studies with high heterogeneity. JAMA Netw Open. 2020;3(1):e1919325. https://10.1001/jamanetworkopen.2019.19325
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
