Abstract
Introduction
After cardiac surgery, patients on venoarterial extracorporeal membrane oxygenation (VA-ECMO) have a higher risk of nosocomial infection in the intensive care unit (ICU). We aimed to establish an intuitive nomogram to predict the probability of nosocomial infection in patients on VA-ECMO after cardiac surgery.
Methods
We included patients on VA-ECMO after cardiac surgery between January 2011 and December 2020 at a single center. We developed a nomogram based on independent predictors identified using univariate and multivariate logistic regression analyses. We selected the optimal model and assessed its performance through internal validation and decision-curve analyses.
Results
Overall, 503 patients were included; 363 and 140 patients were randomly divided into development and validation sets, respectively. Independent predictors derived from the development set to predict nosocomial infection included older age, white blood cell (WBC) count abnormality, ECMO environment in the ICU, and mechanical ventilation (MV) duration, which were entered into the model to create the nomogram. The model showed good discrimination, with areas under the curve (95% confidence interval) of 0.743 (0.692–0.794) in the development set and 0.732 (0.643–0.820) in the validation set. The optimal cutoff probability of the model was 0.457 in the development set (sensitivity, 0.683; specificity, 0.719). The model showed qualified calibration in both the development and validation sets (Hosmer–Lemeshow test, p > .05). The threshold probabilities ranged from 0.20 to 0.70.
Conclusions
For adult patients receiving VA-ECMO treatment after cardiac surgery, a nomogram-monitoring tool could be used in clinical practice to identify patients with high-risk nosocomial infections and provide an early warning.
Introduction
Since the first successful clinical application of extracorporeal membrane oxygenation (ECMO) in 1972, 1 ECMO technology has been used clinically for nearly 40 years. Although ECMO is widely used in refractory cardiogenic shock, severe respiratory failure, and cardiac arrest, 2 the in-hospital mortality rate associated with venoarterial (VA)-ECMO is as high as 59.6%. 3 It is well known that nosocomial infection is the most common complication of ECMO treatment. 4 Based on data issued by the Extracorporeal Life Support Organization (ELSO), patients have a worse prognosis when complicated by nosocomial infection. 5,6 Prevention and control of nosocomial infections are very important for the survival of patients on ECMO.
However, it is very difficult to identify nosocomial infections (ignoring the source of infection 7 ) in patients with cardiogenic shock after cardiac surgery. 8 The most important challenge is to clinically differentiate between sepsis and systemic inflammatory response syndrome. 9 Hypotension, fever, and laboratory tests, such as those for C-reactive protein and procalcitonin, have limited value in the diagnosis of nosocomial infection. Moreover, septic shock can be easily neglected because of the presence of cardiogenic shock. 8
At present, there is still a lack of effective means to recognize early warning signs of nosocomial infections in patients receiving VA-ECMO treatment after cardiac surgery. Here, we reviewed the occurrence of nosocomial infections in adult patients who received VA-ECMO after cardiac surgery
Methods
Study design and internal validation
We performed a retrospective observational study on nosocomial infection in patients receiving VA-ECMO therapy after cardiac surgery between 1 January 2011, and 31 December 2020, at the Cardiac Intensive Care Unit, Beijing Anzhen Hospital, Beijing, China. We randomly selected approximately 70% of the cases as the development set and 30% as the validation set. We developed a nomogram from the development set to predict nosocomial infections in patients receiving VA-ECMO after cardiac surgery.
This study was approved by the Institutional Review Board of Beijing Anzhen Hospital, Capital Medical University (No. 2,022,064X) on 21 March 2022. The committee waived the requirement for informed consent from individual patients because of the retrospective study design. 10
Study population and definition
The inclusion criteria in this study were as follows: (1) age ≥18 years, (2) patients who did not have prior infections and received VA-ECMO support after cardiac surgery, and (3) ECMO support duration ≥48 h. The exclusion criteria were as follows: (1) patients on VA-ECMO treatment with incomplete medical records, (2) death or weaning for any reason within 48 h after ECMO support, (3) venovenous ECMO (VV-ECMO) support, and (4) VA-ECMO for patients who did not undergo cardiac surgery.
Based on the definition of nosocomial infection from the US Center for Disease Control and Prevention, lower respiratory tract infection (LRTI) was defined as the presence of clinical manifestations (two or more of the following criteria: fever [more than 38.0°C or less than 36.0°C]; white blood cell [WBC] count >12×109/L or less than 4×109/L; purulent excretion from the lower respiratory tract; new or consistent infiltration from chest photographs), positive sputum culture, and initiation of antimicrobial therapy under the guidance of intensivists or infectious disease specialists. 11–15 Bloodstream infection (BSI) was defined as the presence of any of the clinical manifestations (including fever >38°C, chills, or hypotension) accompanied by a positive blood culture. 13,16–19 Urinary tract infection (UTI) was defined as the presence of symptomatic signs or positive urine routine results with a positive urine culture. 11 Surgical site infections (SSI) were defined as infections occurring within 30 days of surgery involving the skin, subcutaneous tissue, or muscle layer, or any item of the following: purulent drainage, positive culture from the drainage fluid, and infection identified by the surgeon. 13,20 Skin and soft tissue infections (SSTI) are defined as purulent drainage in the skin and soft tissue, any microorganism isolated from the culture of skin or soft tissue, or a surgeon providing debridement confirming of infection. 13 Nosocomial infection during ECMO was defined as infection occurring from 24 h after ECMO initiation to 48 h after ECMO removal.14,15,21 Nosocomial infection after ECMO was defined as infection occurring from 48 h after ECMO removal to discharge from the hospital.
All sources and frequencies of infection were recorded; however, the first infection was regarded as the primary source when multiple infections occurred in the same patient.
ECMO support and infection control measures
The indication for VA-ECMO was determined by cardiac surgery and an extracorporeal life support physician. 15 VA-ECMO cannulas were surgically implanted by trained team members using the femoral vein-femoral artery approach. The ECMO flow rate titration was adjusted according to lactic acid levels and organ function to ensure adequate tissue perfusion. A systematic heparin anticoagulation strategy was performed so that the activated clotting time was 180–220 s.
All the patients received antibiotics to prevent nosocomial infections when ECMO was initiated. Third-generation cephalosporins were routinely administered. Once nosocomial infection was suspected, empirical combination antibiotics were administered to cover gram-negative bacilli and gram-positive cocci, according to the local microorganisms in the intensive care unit (ICU). When the results of pathogenic culture were clear, antibiotics were administered as the target treatment.
Data collection
We retrieved data from the electronic medical records, including demographic variables (age, sex, nationality, past medical history, and underlying disease), specific data related to VA-ECMO (exact operation duration, duration of ECMO support, use of intra-aortic balloon pump [IABP], continuous renal replacement therapy [CRRT], ventilator usage, ECMO environment, and complications of ECMO therapy), the worst results of laboratory tests within 48 h of ECMO initiation (WBC counts, platelets, procalcitonin [PCT], liver function test, renal function test, blood gas analysis, and microbiological data), maximal vasoactive-inotropic score (VIS), 22,23 Sequential Organ Failure Assessment (SOFA) score 24 ), the diagnosis date of nosocomial infection, and outcome after therapy.
Statistical analysis
We performed the Kolmogorov–Smirnov test to assess the distribution characteristics of the continuous parameters. All numerical variables showed a skewed distribution. Skewed distribution data were expressed as medians and interquartile ranges. Categorical data were reported as frequencies or ratios. Referring to the definition of early onset ventilator-associated pneumonia, we grouped all durations of medical device use into intervals of 5 days. 25 The optimal cutoff values for most of the other continuous variables were divided into two groups. We used the Mann–Whitney U test to analyze differences in continuous variables between groups in the univariate analysis, and the chi-square test or Fisher’s exact test for categorical variables. The collinearity of the covariates was assessed, and model fitting was based on the Hosmer–Lemeshow goodness-of-fit test. Variables from the univariate analysis with p values of less than 0.1 were selected for multivariate analysis. The final selection of the multivariate logistic regression model was based on regression using the Akaike information criterion (AIC). 26 We developed a nomogram based on the results of multivariate analysis in the development set to obtain nosocomial infection probability estimates. The discrimination ability of the model was assessed by calculating the area under the receiver operating characteristic (ROC) curve (AUC). The best cutoff value of the model was regarded as the boundary point of the diagnosis, and patients were stratified into low- or high-risk nosocomial infections. 27 Internal validation was performed on the validation set according to the established model. Calibration ability was assessed using the Hosmer–Lemeshow test. Decision curve analysis (DCA) was used to evaluate the clinical utility of model-assisted decisions. 28 An interactive regression plot based on the prediction model was constructed to calculate the probability of nosocomial infection.
SPSS for Windows, version 26 (SPSS, Chicago, IL, USA), and R statistical software version 4.1.2 (https://www.r-project.org/) were used to analyze the data. Packages “pROC” “calibrate,” “MASS,” “rms,” “rmda,” “regplot” were used to create plots in this study. Statistical significance was set at p < .05.
Results
Patient characteristics
A flowchart of the patient enrolment process is shown in Figure 1. A total of 503 patients were included in this study. Of these, 213 (42.3%) met the diagnostic criteria for nosocomial infection. Using the random sampling method, 363 (approximately 70% of all cases) patients were allocated to the development set, and 140 (approximately 30% of all cases) patients were included in the validation set. Nosocomial infection occurred in 164 (45.2%) patients in the development set and 49 (35.0%) patients in the validation set. The prevalences of LRTI, BSI, UTI, and other infections during ECMO were 97 (19.3%), 60 (11.9%), 1 (0.2%), and 8 (1.6%), respectively. The corresponding prevalences after ECMO were 55 (10.9%), 40 (8.0%), 3 (0.6%), and 32 (6.4%), respectively (Table S1). Flowchart for screening patients for the predictive model in the study.
The characteristics of the study population are summarized in Table S2. Among the 40 variables, compared with the validation set, patients in the development set accounted for a higher ratio in the VIS ≤40 and PCT >4 mg/L groups. The remaining variables have the same distribution.
Independent predictors of nosocomial infection
Logistic regression analysis of predictive factors for nosocomial infection in adult patients on VA ECMO after cardiac surgery.
Abbreviation: IABP: Intra-Aortic Balloon Pump; CRRT: Continuous Renal Replacement Therapy;MV: mechanical ventilation; ECMO: extracorporeal membrane oxygenation; WBC: white blood cell
Model assessment and internal validation
We performed the discrimination assessment in both the development and validation sets. The area under the curve (AUC) in the development set was 0.743 (95% CI, 0.692–0.794) (Figure 2(a)), and that in the validation set was 0.732 (95% CI, 0.643–0.820) (Figure 2(b)). The AUC values in the development and validation sets were greater than 0.70, indicating that the predictive model had good discrimination. The area under the curve (AUC), calibration curve and decision curve analysis in development set and validation set, respectively. (A) Receiver operating characteristic curve (ROC) in the development set. The area under the curve (AUC) for the model in the development set was 0.743 (95% confidence interval [CI] 0.692–0.794). (B) ROC in the validation set. The AUC for the model in the validation set was 0.732 (95% CI 0.643–0.820). (C) Calibration curve in the derivation set. (Hosmer–Lemeshow test, χ2=5.761, p = .764). (D) Calibration curve in the validation set. (Hosmer–Lemeshow test, χ2=12.419, p = .191). (E) Decision curve analysis in the development set. (F) Decision curve analysis in the validation set.
The calibration curve of the predictive model was shown by the Hosmer–Lemeshow test, which revealed no significant difference in either the development set (χ2 = 5.761; p = .764) or the validation set (χ2=12.419; p = .191). (Figure 2(c) and Figure 2(d)). The calibration curve in the validation set lied below the reference line, suggesting that the model might slightly overestimate the risk of nosocomial infections.
DCA was performed on the development and validation sets (Figures 2(e) and (f)). When threshold probability was in the range of 0.20–0.70, the net benefit would gain from the model compared to “treat all” or “treat none” strategies.
Nomogram for the predicting model
We created a nomogram to predict the probability of nosocomial infections in adult patients receiving VA-ECMO after cardiac surgery. The nomogram is shown in Figure 3. The optimal probability for predicting nosocomial infection was 0.457. At this cutoff value, the Youden index was 0.402, sensitivity was 71.9%, specificity was 68.3%, positive predictive value was 64.1%, negative predictive value was 75.5%. The corresponding total score for the model is approximately 85 points. Nomogram for predicting nosocomial infection in patients on venoarterial extracorporeal membrane oxygenation (VA-ECMO) after cardiac surgery. (A) Nomogram for the model. (B) Interactive nomogram for the patient as an example. Figure legend. For example, a 68-year-old patient who was admitted with coronary artery disease for coronary artery bypass grafting. After surgery, he was transferred to the intensive care unit (ICU). He received MV for 3 days after surgery, after which he received VA-ECMO treatment because of circulatory instability. The white blood cell (WBC) count was 15×109/L. Hence, his total score from the model of nosocomial infection was as follows: for age >65 years, the score was 26; for WBC count in the abnormal range, the score was 24; for ECMO environment being the ICU, the score was 37; and for an MV duration of 3 days, the score was 0. Hence, the total score was 26+24+37=87, and the probability of nosocomial infection was approximately 45.8%, according to the interactive nomogram (more than the cutoff point 0.457), which would suggest that the patient was in the infectious state. The total score calculated by R software was shown automatically with a red dot, and the corresponding probability was shown with a red arrow in the interactive nomogram (B).
Model clinical application
The model score for nosocomial infection in adult patients on VA ECMO after cardiac surgery.
Abbreviation: MV: mechanical ventilation; ECMO: extracorporeal membrane oxygenation; WBC: white blood cell
Discussion
To the best of our knowledge, this is the first model to predict the probability of nosocomial infection based on independent factors after multivariate logistic regression. In this study, for adult patients receiving VA-ECMO treatment after cardiac surgery, four independent risk predictors were included: older age (≥65 years), WBC abnormality, ECMO environment in the ICU, and MV duration. The predictive model conducted using these four variables had good discrimination and calibration, which can play a role in identifying high-risk clinical warnings of nosocomial infections.
Many studies have investigated the risk factors for nosocomial infections in patients receiving ECMO therapy. Most of these studies have suggested that ECMO support duration is an independent risk factor. 6,14,15,19,29–34 Additionally, other studies reported independent risk factors including SOFA score, 33,35,36 older age, 37–39 ICU length of stay, 40,41 preoperative creatinine level, 32 VV-ECMO mode, 42 ventilator duration before ECMO removal, 42 VA-ECMO mode, 43 and immunosuppression status. 43 These discrepancies were due to the different sources of the included patients, smaller sample sizes, and different definitions of nosocomial infection among these studies.
MV duration was an independent risk factor in our model, which was consistent with the findings of a study by Gao et al. 44 in which the risk of hospital-acquired infection increased by 18% per 24-hour increment in ventilator usage. In our study, the risk of nosocomial infection reached 2.1 fold and 7.6 fold, with 6–10 days and more >10 days of MV, respectively. Consistent with the findings of most studies, 37–39 we also found that the risk of nosocomial infection increased significantly in older age.
Unexpectedly, ECMO environment in the ICU was an independent risk factor in our model, contrasting that in the non-ICU. Compared with the operating room environment, the ICU environment was vulnerable to microorganism contamination. However, a more reasonable explanation might be that some patients received ECMO treatment in the ICU because deterioration was caused by nosocomial infection after surgery at that time.
Nomograms are more intuitive and convenient than other visualization methods. If the probability reaches the cutoff point of 0.457, the patient is most likely at a high-risk infectious state. As variables such as age, ECMO environment, and WBC count were unmodifiable clinical factors, only MV duration was a potentially adjustable factor. From the nomogram, the probability of nosocomial infection in patients who received MV for more than 10 days was over 50%, indicating that those patients were at high risk of infection. Therefore, it is better to extubate as soon as possible within 10 days after surgery to reduce nosocomial infection opportunities.
When a patient was thought to be at a high risk of nosocomial infection, some measures were recommended. Increasing the frequency of testing of various pathogens (blood, urine, drainage, sputum), screening via ultrasound and radiological bedside check-up, strengthening the anti-infection drugs, and assessing and removing unnecessary catheters or devices as soon as possible would help reduce nosocomial infections. “Awake ECMO” might reduce the incidence of ventilator-associated pneumonia, as reported by Montero S et al. 45
The prevalence of nosocomial infection in adult patients receiving ECMO treatment in our study was 42.3%, which was higher than that (20.9%) reported by ELSO. 5,6 A possible explanation is that these studies focused only on nosocomial infections during ECMO, whereas our study included nosocomial infections not only during ECMO support but also after ECMO removal.
Some limitations of this study should be considered when interpreting the results. First, as a retrospective study, the time span was up to 10 years; this may have been much more advanced, and the prevalence of infection is unstable over time. Thus, the prediction efficiency decreased with time. Second, the model is only suitable for patients without prior infection who received VA-ECMO after cardiac surgery. Our results cannot be generalized to other centers because we did not have external validation. Third, similar to other models for predicting nosocomial infection, 27,44,46 our model predicts the probability of nosocomial infection, but does not describe the exact infection site.
Conclusions
Four independent predictors of nosocomial infection in patients during and after VA-ECMO after cardiac surgery were older age (≥65 years), WBC count abnormality (<4 or >12×109/L), ECMO environment (ICU), and MV duration (>5 days or >10 days). The nomogram created using these risk factors showed reliable performance in predicting nosocomial infection in patients during and after VA-ECMO after cardiac surgery.
Supplemental Material
Supplemental Material - A nomogram to predict nosocomial infection in patients on venoarterial extracorporeal membrane oxygenation after cardiac surgery
Supplemental Material for A nomogram to predict nosocomial infection in patients on venoarterial extracorporeal membrane oxygenation after cardiac surgery by Xiyuan Li, Liangshan Wang, Chenglong Li, Xiaomeng Wang, Xing Hao, Zhongtao Du, Haixiu Xie, Feng Yang, Hong Wang, and Xiaotong Hou in Perfusion
Footnotes
Acknowledgements
We would like to thank senior engineer Wu Yuhao, Electronic Medical Record, and the Information Center of Anzhen Hospital.
Author contributions
XYL, LSW, CLL, XMW, XH, HW, ZTD, HXX, and FY contributed to the data acquisition. XYL and LSW wrote the manuscript’s initial draft. XYL, LSW, and XMW performed statistical analyses. XYL, LSW, HW, and XTH contributed to the study conception and design. HW, and XTH contributed to the revision of this paper and final approval of the version to be published. All authors have read and approved the final manuscript.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the 1. Beijing Hospitals Authority “Ascent Plan” (FDL20190601, to X Hou). 2. The “Sailing” Program of key medical specialty of Beijing Hospitals Authority in 2021: for critical care medicine (fund number for extracorporeal life support: ZYLX202111, to X Li). The funding body had no role in the design of the study; collection, analysis, and interpretation of data; or in writing the manuscript.
Data availability
The datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request.
Ethics approval
Ethical approval was granted according to national requirements. This study was approved by the Medical Ethics Committee of the Beijing Anzhen Hospital, Capital Medical University. The requirement for informed consent was waived because the study was performed retrospectively and no interventions were applied.
Supplemental Material
Supplemental material for this article is available online.
Appendix
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
