Abstract
Objective
To develop a new inflammatory biomarker-based and simple-to-use nomogram for the early identification of acute pulmonary embolism (PE) patients at a high-risk mortality in intensive care unit (ICU).
Methods
We randomly allocated 1083 acute PE patients from the Medical Information Mart for Intensive Care IV database to derivation and internal validation cohort. We used logistic regression analysis to determine independent risk factors and to construct a predictive nomogram. We subsequently evaluated the calibration, discrimination and clinical usefulness of the nomogram.
Results
Age>66, neutrophil-to-lymphocyte ratio (NLR) > 10.1, lymphocyte-to-monocyte ratio (LMR) < 1.5, red cell distribution width (RDW) > 14.35, respiratory rate (RR) > 26bpm, oxygen saturation (SPO2), vasopressor use and malignant cancer were detected as important determinants of 28-day mortality and included in our nomogram. The calibration plot revealed an adequate fit of the nomogram for predicting the risk of 28-day mortality. Regarding discriminative power, receiver operating characteristic curve analysis showed that the nomogram had an area under the curve of 0.772 (95% CI:0.732, 0.811,
Conclusions
This proposed simple-to-use nomogram based on age, NLR, LMR, RDW, vasopressor use, RR, SPO2 and malignant cancer provides accurate death prediction for acute PE patients in ICU.
Introduction
Pulmonary embolism (PE) is a prevalent cardiovascular emergency with an estimated annual incidence of with roughly 40-110 cases per 100,000 people per year. 1 Despite advancements in diagnostic tests and treatment leading to a decrease in mortality to less than 9%, PE remains the third leading cause of cardiovascular death in the United States, following acute myocardial infarction and stroke.2,3 The intensive care unit (ICU) is usually the first choice setting where high risk acute PE patients are treated. More frequently, patients develop PE as a complication during their ICU stay for several risk factors such as major trauma or spinal cord injury.4,5
Identifying patients in ICU who are at a high-risk of mortality in the early phase of acute PE could be beneficial in order to provide more timely and adequate interventions. The Pulmonary Embolism Severity Index (PESI) and Simplified Pulmonary Embolism Severity Index (SPESI) are the most widely validated and used risk stratification tools that do not require any ultrasonography or laboratory tests.6–8 They can reliably identify patients who are at low risk of short-term mortality and who are potential candidates for outpatient care. 9 However, A number of studies derived and validated them in outpatients (including the emergency department setting),10,11 and extrapolation of their use to ICU patients is questionable. In a retrospective study of 286 ICU patients with acute PE, the predictive performance of sPESI for 30-day mortality was notably poor, with an area under the receiver operating characteristic curve (AUC) of 0.568 (95% CI, 0.500-0.637). 12 Since no specific scoring systems exist for critically ill patients, developing a more accurate, objective, and simple clinical prediction model for the early identification of disease severity for acute PE patients in ICU is necessary.
Systemic inflammation is known to have a significant impact on the progression of thrombosis.13,14 Several inflammation-based markers, including neutrophil-to-lymphocyte ratio (NLR), platelet-to-lymphocyte ratio (PLR), lymphocyte-to-monocyte ratio (LMR), and red cell distribution width (RDW) have been explored to identify acute PE patients at high risk of death.15–18 All of these markers are easily obtained from the routine blood test. However, there is a lack of predictive models including these simple but effective markers for predicting mortality in acute PE patients.
Therefore, we designed this study to explore the association between inflammatory markers mentioned above and 28-day mortality in acute PE patients in ICU. Additionally, we constructed a nomogram that integrated some clinical bedside parameters with the inflammatory parameters to predict 28-day mortality.
Methods
Study Design
This was a retrospective analysis based on the Medical Information Mart for Intensive Care IV(MIMIC-IV) database, which contains electronic health records of over 50,000 patients admitted to ICUs at the Beth Israel Deaconess Medical Center (Boston, MA, USA) from 2008 to 2019. The use of this database was approved by the institutional review boards of Massachusetts Institute of Technology (Cambridge, MA, USA) and Beth Israel Deaconess Medical Center (Boston, MA, USA). One author (Hongzhuang Chen) obtained the access and was responsible for the data extraction (certification number: 52748910). The study complied with the Reporting of Studies Conducted using Observational Routinely Collected Health Data (RECORD) statement.
Study Population
All admissions diagnosed with acute PE in the MIMIC-Ⅳ database were eligible for inclusion in the present study. Acute PE was identified according to the International Classification of Disease (ICD)-9 code or Classification of Disease (ICD)-10 code. The exclusion criteria included (1) Patients who were younger than 18 years; (2) Patients who received extracorporeal membrane oxygenation (ECMO). Additionally, we analyzed only the first ICU stay for patients who were admitted to ICU more than once. The included patients were randomly assigned into two cohorts (7:3): the primary cohort and the validation cohort. Models were developed from the primary cohort and evaluated in the validation cohort
Data Extraction
The information extracted from the MIMIC-IV database included the following: (1) demographics including age, sex and history of malignant cancer; (2) severity of illness quantified by the Sequential Organ Failure Assessment (SOFA) score, the Simplified Acute Physiology Score II (SAPS II), Oxford Acute Severity of Illness Score (OASIS), Pulmonary Embolism Severity Index (PESI), and Simplified Pulmonary Embolism Severity Index (SPESI). (3) collected to calculate the Charlson Comorbidity Index (CCI); (4) vital signs including mean arterial pressure (MAP), heart rate, temperature (°F), and respiratory rate. If a variable was recorded more than once, the value associated with the greatest severity of illness was used.
Laboratory variables including white blood cell count (WBC), platelet count, neutrophil count, lymphocyte count, Monocyte count, RDW and creatinine in the first blood test results within 24 h before or after ICU admission were also extracted.Vital signs capture rapid fluctuations in a patient's clinical status; thus, selecting the most abnormal (ie, worst) values helps reflect peak illness severity. In contrast, laboratory test results are more susceptible to therapeutic interventions. Therefore, using the first available results obtained within 24 h before or after ICU admission minimizes treatment-related confounding and more accurately represents the patient's baseline physiological status upon ICU entry. If a variable was recorded more than once, the first value associated was used. NLR was calculated as the neutrophil count divided by the lymphocyte count; LMR was calculated as the lymphocyte count divided by the monocyte count, and PLR was calculated as the platelet count divided by the lymphocyte count. In this study, we regarded 28-day mortality as the outcome of interest, which was also extracted from the MIMIC-IV database.
Statistical Analysis
Data analysis was performed using the R statistical software package (version 3.6.0). Continuous variables were expressed as mean and standard deviation or median and interquartile range (IQR), according to their distribution. Student's t-test was employed for normally distributed continuous variables, and the Mann-Whitney U test was utilized for non-normally distributed variables. The Chi-square test or Fisher's exact test was used for categorical variables.
The optimal cutoff values for each independent parameter, including age, RR, RDW, NLR, and LMR were determined using receiver operating characteristic (ROC) curve analysis with the cutoff value chosen based on the Youden index (specificity + sensitivity-1) achieved the highest value. Subsequently, these parameters were converted into dichotomous variables. Multivariate logistic regression analyses were conducted to investigate the independent risk factors for 28-day mortality, including age, RR, SPO2, NLR, LMR, RDW, vasopressor use, and malignant cancer. To prevent overfitting of the model, a backward step-down process based on the Akaike Information Criterion (AIC) was employed.
A nomogram based on the results of the logistic regression analyses was constructed. The performance of the nomogram was evaluated by its calibration, discrimination and clinical usefulness. 19 A calibration curve was constructed to evaluate the agreement between the nomogram predictions and the actual outcomes. Regarding the ability of discrimination, ROC curve analyses were performed, and AUCs were calculated and compared between the nomogram and other established scores (PESI and SPESI).. Furthermore, to assess the clinical usefulness of the predictive nomogram, decision curve analysis (DCA) was conducted by quantifying the net benefits at different threshold probabilities.
Results
Clinical features and baseline characteristics
The patient screening process is presented in Figure 1. A total of 1083 acute PE patients were finally included after reviewing 76945 admissions in the MIMIC-IV databases. The baseline characteristics of patients in the primary and validation cohorts are shown in the Table 1. Among the 759 individuals comprising the primary cohort, 47.3% were males, with average age was 63.8 years. The overall 28-day mortality rate was 18.3% (139/759). In the validation cohort, there were 153 males (47.2%) and 171 females (52.8%), with average age of 62.0 years. The overall 28-day mortality rate in the validation cohort was 14.5% (47/324).
The optimal cutoff values and risk factors for 28-day mortality

Study flow diagram depicting exclusion criteria and outcomes. MIMIC-IV, Medical Information Mart for Intensive Care Ⅳ; PE, pulmonary embolism; ECMO, extracorporeal membrane oxygenation (ECMO).
Baseline Characteristics and 28-day Mortality in the Primary Cohort and Validation Cohort.
Abbreviations: SOFA, Sequential Organ Failure Assessment; SAPS II, Simplified Acute Physiology Score II; CCI, Charlson comorbidity index; OASIS, Overall Anxiety Severity and Impairment Scale; PESI, Pulmonary Embolism Severity Index; SPESI, Simplified Pulmonary Embolism Severity Index; MAP, mean arterial pressure; WBC, white blood cell count; PLT, platelet; NLR, neutrophil-to-lymphocyte ratio; PLR, platelet-to-lymphocyte ratio; LMR, lymphocyte-to-monocyte ratio; RDW, red cell distribution width.
Considering the availability of variables, we determined the optimal cutoff values for age, RR, RDW, NLR and LMR for 28-day mortality. The optimal cutoff value for age was 66 years. The optimal cutoff value for RR was 26 bpm. The optimal cutoff values for RDW, NLR and LMR were 14.35, 10.1 and 1.5, respectively. Univariate and multivariate logistic regression analyses showed that age > 66, RR > 26, RDW>14.35, NLR >10.1, LMR<1.5, SPO2, vasopressor use and malignant cancer were all important determinants of 28-day mortality (Table 2).
Prognostic nomogram for 28-day mortality
Univariate and Multivariate Analyses for the Relationship Between the Candidate Risk Factors and 28-day Mortality in the Primary Cohort.
Abbreviations: OR, odds ratio; CI, confidence interval; RR, respiratory rate; NLR, neutrophil-to-lymphocyte ratio; LMR, lymphocyte-to-monocyte ratio; RDW, red cell distribution width.
The detected risk factors in the multivariate analysis were used to construct a nomogram (Figure 2). The nomogram was generated by as-signing a weighted score on the point scale to each of the independent prognostic parameters.
Performance of the nomogram model

Nomogram predicting the 28-day mortality of acute PE patients in ICU based on the primary cohort. RR, respiratory rate; NLR, neutrophil-to-lymphocyte ratio; LMR, lymphocyte-to-monocyte ratio; RDW, red cell distribution width.
The calibration curves for the probability of 28-day mortality are shown in Figure 3. The calibration plot revealed an adequate fit of the nomogram for predicting the risk of 28-day mortality in both the primary cohort and validation cohort. The discriminatory power of our model was assessed using ROC curve analysis and the AUC for 28-day mortality prediction (Figure 4). The nomogram derived from the primary cohort for 28-day mortality had an AUC of 0.772 (95% CI:0.732, 0.811), which was outperforming both PESI (0.686, 95% CI: 0.644, 0.728), and SPESI (0.624, 95% CI: 0.579, 0.670) (Figure 3). The discriminatory power was similar in the validation cohort, which yielded an AUC of 0.863 (95%CI: 0.814, 0.911). Figure 5 shows that the DCA of the new model produced a net benefit for both the training and verification cohorts as threshold probabilities were 0-0.60 for the primary cohort and 0-0.55 for the validation cohort.

Calibration curves for predicting the 28-day mortality of acute PE patients in ICU in the primary cohort (A) and validation cohort (B). The x-axis represents the nomogram-predicted probability of 28-day mortality, and the y-axis represents the actual observed 28-day mortality.

Receiver operating characteristic curve analysis and comparison of the AUCs for the nomogram, PESI score, and SPESI score in the primary cohort (A) and validation cohort (B). AUC, area under the curve; CI, confidence interval; PESI, Pulmonary Embolism Severity Index; SPESI, Simplified Pulmonary Embolism Severity Index.

Decision curves for the primary cohort (A) and validation cohort (B) implicating the net benefit with respect to the use of the nomogram for predicting 28-day mortality of acute PE patients in ICU.
Discussion
Our study developed and validated a simple-to-use nomogram model for the early prediction of 28-day mortality in acute PE patients in ICU. Our nomogram model achieved an AUC of 0.772. The calibration, discriminatory power and clinical usefulness of the nomogram model were then confirmed by a validation cohort from the same data set. PESI and SPESI have been the most extensively validated and used stratification tools to date. However, their principal strength lies in the reliable identification of patients at a low risk for 28-day mortality, which means that they are appropriate for identifying patients suitable for outpatient treatment rather than high-risk patients who may die in the short time in ICU. Moreover, some scoring systems involve complex calculations based on multiple data points, unnecessary imaging examination or delayed predictive results.20,21 Therefore, we developed a simple-to-use nomogram model comprising age, NLR, LMR, RDW, vasopressor use, RR, SPO2 and malignant cancer, all of which could be quickly obtained and effortless to calculate in ICU. We compared various aspects of the new model with PESI and SPESI. These results suggest that the new model had a significantly better predictive performance.
The pathophysiology of acute PE is a complex process where the inflammatory activation plays an important role. Neutrophils, lymphocytes, monocytes, and platelets are the key cells that take part in the inflammatory response. Alterations in NLR, PLR, and LMR reflect an imbalance between any 2 types of these cells, which can be considered an imbalance in the inflammatory response or immune status. NLR, PLR, and LMR have been introduced as inflammation-based markers in critically ill patients. Patients with increased NLR or PLR and decreased LMR have demonstrated to have a poor outcome.22–24 A study by Ma Y et al demonstrated that for 1 unit of increase of NLR, the risk of 30-day mortality rose about 13% in patients with acute PE. 24 PLR is also as a novel marker of in-hospital and long-term adverse outcomes among patients with acute pulmonary embolism. 23
RDW, which represents the size variation in red blood cells (RBCs) and calculated as the Standard Deviation (SD) of RBC volume divided by the mean corpuscular volume (MCV), is also a common hematologic parameter. Inflammatory cytokines could result in insufficient erythropoiesis and alter iron metabolism, as well as inhibit erythrocyte maturation, thus promoting the release of immature RBCs into the circulation and then causing an elevation in RDW levels.
25
Furthermore, Previous research has linked an elevated RDW with increased risk of all-cause mortality in conditions such as right ventricular dysfunction, pulmonary arterial hypertension and chronic obstructive pulmonary disease.26–28 This model suggests that an increase in RDW is associated with a higher risk of 28-day all-cause mortality in acute PE patients in ICU (OR = 2.75, 95% CI: 1.97-3.84,
Previous studies have shown the significant predictive power of PLR in PE patients.17,24 However, the predictive power of PLR was observed to be weak in our model. Our cohort comprised a heterogeneous group of critically ill patients, not all of whom had pulmonary embolism as the primary diagnosis. In critically ill patients, especially those with conditions like septic shock, immune system dysregulation can manifest after an initial phase of hyperstimulation. The presence of other serious comorbidities in our heterogeneous ICU cohort is a key factor that likely influenced the mortality outcome and diluted the specific predictive utility of PLR for PE-related mortality. This heterogeneity, however, reflects the real-world clinical scenario in the ICU, and our findings underscore that PLR should be interpreted with caution in such complex patient. Several studies have demonstrated that the increase in platelet count is closely associated with acute-phase reactants and pro-inflammatory molecules, such as high-sensitivity C-reactive protein(hs-CRP), interleukin-1(IL-1), and interleukin-6(IL-6). 29 The rise in platelet count reflects heightened platelet activity, leading to destructive pro-inflammatory and pro-thrombotic responses. 23 As we know, thrombocytopenia is a common occurrence in ICU, with a prevalence ranging from 8% to 77% depending on case mix and definitions.30,31 In our study, the incidence of thrombocytopenia was 27.4%. Thrombocytopenia can further complicate the interpretation of PLR.
To evaluate the patients comprehensively, we considered for inclusion in the model some of the known risk factors obtained easily within minutes of a patient's arrival to ICU. In our analysis, age, vasopressor use, RR, SPO2 and malignant cancer have been identified to be independent predictors of mortality for acute PE patients. These factors align with those included in PESI scoring system, reflecting their importance predicting mortality in acute PE patients. Though the cutoff value was slightly different for the enrollment of different cohorts and death outcomes, the association between these factors and poor prognosis was definite.
There were still several limitations in the present study. Firstly, the variability in the timing of blood tests in a retrospective analysis can introduce inconsistencies in data collection. Utilizing blood test results obtained within a more standardized and immediate timeframe, such as within 1 h after ICU admission, could enhance the accuracy and clinical relevance of the predictive model. Secondly, Our patient population was heterogeneous, encompassing both patients admitted to the ICU for severe PE and those who developed PE as a complication of another critical illness. This heterogeneity may introduce spectrum bias. Furthermore, variables with missing and outlier data were common in the MIMIC-IV database, which reduced the comprehension ability of the model. Finally, it was a single-center, retrospective database analysis, and different conclusions (especially in the cutoff value) can be achieved because of the heterogeneity of different cohorts. Our findings must be validated using a multicenter, prospective survey with a larger sample size.
Conclusions
We proposed a nomogram model with easily obtainable parameters. This simple-to-use nomogram based on age, NLR, LMR, RDW, vasopressor use, RR, SPO2 and Malignant cancer provides accurate death prediction for acute PE patients in ICU. Its predictive performance was superior to that of the PESI and SPESI, and the new model contains fewer variables than the PESI, making it more convenient for clinicians to use.
Footnotes
Acknowledgements
We thank the Intensive Care Unit of The Cancer Hospital of Shantou University Medical College for their helpful and continuous support.
Ethic Approval and Consent to Participate
The establishment of this database was approved by the Massachusetts Institute of Technology (Cambridge, MA) and Beth Israel Deaconess Medical Center (Boston, MA) and consent was obtained for the original data collection.
CRediT Authorship Contribution Statement
Mingqin Zhou: Investigation, Data curation, Writing - original draft.
Lifeng Xiao: Validation, Data curation, Writing - original draft.
Hongzhuang Chen: Conceptualization, Methodology, Software.
Xiaoluan Lin: Data curation, Methodology.
Shaona Lin: Formal analysis, Data curation.
Ruiyun Lin: Supervision, Writing - review & editing, Resources.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study was supported by Shantou Medical and Health Technology Plan Project ([2022]81-110).
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
