Abstract
To external validate the risk assessment model (RAM) of venous thromboembolism (VTE) in multicenter internal medicine inpatients. We prospectively collected 595 internal medical patients (310 with VTE patients, 285 non-VTE patients) were from Beijing Shijitan Hospital, Beijing Chaoyang Hospital, and the respiratory department of Beijing Tsinghua Changgeng Hospital from January 2022 to December 2022 for multicenter external validation. The prediction ability of Caprini RAM, Padua RAM, The International Medical Prevention Registry on Venous Thromboembolism (IMPROVE) RAM, and Shijitan (SJT) RAM were compared. This study included a total of 595 internal medicine inpatients, including 242 (40.67%) in the respiratory department, 17 (2.86%) in the respiratory intensive care unit, 49 (8.24%) in the neurology department, 34 (5.71%) in the intensive care unit, 26 (4.37%) in the geriatric department, 22 (3.70%) in the emergency department, 71 (11.93%) in the nephrology department, 63 (10.59%) in the cardiology department, 24 (4.03%) in the hematology department, 6 (1.01%) in the traditional Chinese medicine department, 9 (1.51%) cases in the rheumatology department, 7 (1.18%) in the endocrinology department, 14 (2.35%) in the oncology department, and 11 (1.85%) in the gastroenterology department. Multivariate logistic regression analysis showed that among internal medicine inpatients, age > 60 years old, heart failure, nephrotic syndrome, tumors, history of VTE, and elevated D-dimer were significantly correlated with the occurrence of VTE (P < .05). The incidence of VTE increases with the increase of D-dimer. It was found that the effectiveness of SJT RAM (AUC = 0.80 ± 0.03) was better than Caprini RAM (AUC = 0.74 ± 0.03), Padua RAM (AUC = 0.72 ± 0.03) and IMPROVE RAM (AUC = 0.52 ± 0.03) (P < .05). The sensitivity and Yoden index of SJT RAM were higher than those of Caprini RAM, Pauda RAM, and IMPROVE RAM (P < .05), but specificity was not significantly different between the 4 models (P > .05). The SJT RAM derived from general hospitalized Chinese patients has effective and better predictive ability for internal medicine inpatients at risk of VTE.
Keywords
Introduction
Venous thromboembolism (VTE) is a major risk factor for morbidity and mortality in internal medicine hospitalized patients. VTE includes deep vein thrombosis (DVT) and pulmonary embolism (PE). Each year globally over 10 million population develop VTE in the United States of America. 1 Several assessment tools for internal medicine patients are available, including the Padua risk assessment model (RAM), Kucher RAM, Caprini RAM, Geneva RAM, and IMPROVE RAM. 2 Among them, Caprini RAM, Padua RAM, and IMPROVE RAM are the most widely used in clinical practice. We developed the VTE risk assessment model (SJT model) based on hospitalized patients in a comprehensive hospital and external validated prospectively. 3 We want to explore the clinical effectiveness of the SJT RAM in multicenter internal medicine patients.
Methods
Ethical Approval
This study was approved by the Medical Ethics Committee of Beijing Shijitan Hospital Affiliated with Capital Medical University (No. 2020-2).
Subjects
We prospectively consecutive collected 595 internal medical inpatients (310 with VTE patients and 285 non-VTE patients) were from Beijing Shijitan Hospital, Beijing Chaoyang Hospital, and the respiratory department of Beijing Tsinghua Changgung Hospital from January 2022 to December 2022, including the respiratory department, respiratory intensive care unit, neurology department, intensive care unit, geriatric department, emergency department, nephrology department, cardiology department, hematology department, traditional Chinese medicine department, rheumatology department, endocrinology department, oncology department, and gastroenterology department.
VTE group: The inclusion criteria were as follows: age ≥18 years and ≥ 48 hours of hospitalization. DVT was confirmed by color Doppler ultrasound and/or venography of the upper and lower limbs, with (or not) PE diagnosed by CT pulmonary angiography (CTPA) or pulmonary ventilation-perfusion (V/Q) scanning. The exclusion criteria were as follows: superficial venous thrombosis, anticoagulant (including oral and subcutaneous injection), or thrombolytic drug use at admission or after admission, and incomplete clinical data.
Non-VTE group: We selected patients who were admitted to the same departments during the periods mentioned above. The inclusion criteria were as follows: age ≥18 years and ≥ 48 hours of hospitalization. Non-DVT patients were diagnosed by color Doppler ultrasound and/or venography of the upper and lower limbs, with non-PE patients confirmed by CTPA or V/Q. The exclusion criteria were as follows: superficial venous thrombosis, anticoagulant (including oral and subcutaneous injection), or thrombolytic drug use at admission or after admission, and incomplete clinical data.
Statistical Analyses
Measurement data were expressed as `x ± s. If the distribution of data was normal, a parametric model was used to calculate the P value. For the nonnormal distribution of data, the nonparametric rank sum test was used to calculate the P value. The count data were expressed as percentages (%), and the χ2 test was used for comparisons between groups. The risk factors related to VTE were analyzed by logistic regression. The relative risk ratio and 95% confidence interval (CI) were calculated. The sensitivity, specificity, and Youden index were analyzed. Based on sensitivity and specificity, an ROC curve was drawn, and the AUC value was defined as the AUC. R statistical software (Version 3.4.1) was used for all statistical analyses. P < .05 denotes statistical significance.
Results
This study included a total of 595 hospitalized patients, including 242 (40.67%) in the respiratory department, 17 (2.86%) in the respiratory intensive care unit, 49 (8.24%) in the neurology department, 34 (5.71%) in the intensive care unit, 26 (4.37%) in the geriatric department, 22 (3.70%) in the emergency department, 71 (11.93%) in the nephrology department, 63 (10.59%) in the cardiology department, 24 (4.03%) in the hematology department, 6 (1.01%) in the traditional Chinese medicine department, 9 (1.51%) cases in the rheumatology department, 7 (1.18%) in the endocrinology department, 14 (2.35%) in the oncology department, and 11 (1.85%) in the gastroenterology department (Table 1).
Distribution of Departments Included Inpatients.
Abbreviation: VTE, venous thromboembolism.
It was found that the following risk factors were significantly increased in VTE patients with internal medicine diseases compared to non-VTE groups: age > 60 years old, female patients, bed rest, respiratory failure, coronary heart disease, cancer, CVC, VTE history, and elevated D-dimer levels (P < .05) (Table 2).
Characteristics of VTE Patients in the External Multicenter.
Abbreviations: COPD, chronic obstructive pulmonary disease; CVC, central vein catheterization; ULN, upper limit of normal; VTE, venous thromboembolism.
Through multiple logistic regression analysis of the included risk factors, it was found that among internal medicine inpatients over 60 years old, heart failure, nephrotic syndrome, cancer, history of VTE, elevated D-dimer were significantly correlated with the occurrence of VTE (P < .05). The incidence of VTE increases with the increase of D-dimer (Table 3).
Logistic Regression Analysis of VTE Risk Factors.
Abbreviations: COPD, chronic obstructive pulmonary disease; CVC, central vein catheterization; ULN, upper limit of normal; VTE, venous thromboembolism.
Comparison of different RAMs (Table 4) in hospitalized patients in multicenter internal medicine. We found that Caprini RAM, Padua RAM, and SJT RAM have good discrimination for high risk or highest risk in VTE patients, except for IMMPROVE RAM (Table 5). The effectiveness of the SJT model (AUC = 0.80 ± 0.03) in VTE risk screening for internal medicine inpatients was greater than that of the Caprini risk assessment model (AUC = 0.74 ± 0.03), Padua risk assessment model (AUC = 0.72 ± 0.03), and IMPROVE risk assessment model (AUC = 0.52 ± 0.03) (P < .05). For hospitalized patients in internal medicine, the sensitivity and Youden index of the SJT model were higher than those of the Caprini model, Pauda model, and IMPROVE model (P < .05), but there was no significant difference in specificity among the 4 models (P > .05) (Figure 1 and Table 6).

ROC curves of different VTE risk assessment models in hospitalized patients in multicenter internal medicine.
Risk Factors and Definition of the Risk Level in the 4 Different RAMs.
Abbreviations: RAM, risk assessment model; BMI, body mass index; HRT, hormone replacement therapy; COPD, chronic obstructive pulmonary disease; ICU, intensive care unit; CCU, coronary care unit; CVC, central vein catheterization; ULN, upper limit of normal; HIT, heparin-induced thrombocytopenia.
Comparison of the 4 Different RAMs.
Abbreviation: RAM, risk assessment model.
Comparison of Different RAMs in External Multicenter.
Abbreviations: RAM, risk assessment model; IMPROVE, The International Medical Prevention Registry on Venous Thromboembolism; SJT, Shijitan.
Discussion
VTE is a common complication in internal medicine patients, especially in emergency departments. The American College of Chest Physicians (ACCP) recommends that all internal medicine patients undergo risk assessment for VTE, and non-low-risk patients should receive corresponding prophylaxis based on the risk stratification of VTE. 4 At present, both ACCP and Chinese professional guidelines recommend that inpatients in internal medicine use the Padua risk assessment model. The American Society of Hematology recommends that inpatients in internal medicine use either the Padua risk assessment model or the IMPROVE risk assessment model. Researchers such as Michael B. Rothberg from the Cleveland Clinic in the United States have constructed a VTE risk assessment model for internal medicine inpatients. This model includes 13 risk factors (including action status, active tumor, mechanical ventilation, respiratory failure, inflammatory bowel disease, ulcers, recent surgery, ischemic stroke, hemophilia, central venous catheterization, VTE history, acute infection, and paralysis). The sensitivity of the Cleveland model was higher than that of the Padua risk assessment model (68.8% vs 61.9%, P = .02), and the specificity was higher than that of the IMPROVE risk assessment model (70.9% vs 65.0%, P < .001). The high-risk VTE patients screened using the Cleveland model were 5.7 times higher than low-risk patients, which can effectively distinguish between high-risk and low-risk VTE patients. 5 This was similar to the validation results of the SJT RAM we constructed in our study. The screening ability of the SJT RAM for VTE in internal medicine inpatients was also better than the Padua risk assessment model and IMPROVE risk assessment model.
Many scholars have conducted relevant studies on the risk factors for VTE in hospitalized patients in internal medicine. In Wehmeyer's study, it was found that risk factors for VTE include bed rest/restricted mobility > 72 hours and infection. 6 Abdullah Damin conducted a risk assessment of VTE in hospitalized internal medicine patients and found that the most common risk factors for VTE were infection (54.7%), age > 70 years old (38.5%), and heart and/or respiratory failure (21.3%). 7 Mulugeta Asmaw conducted a retrospective study on 219 hospitalized patients in the internal medicine department and found that the probability of VTE in women was more than 14 times higher than that in men (P = .003). A decrease in activity, trauma and/or surgery in less than one month, active tumors, chronic kidney disease, and hypertension are all significantly associated with the occurrence of VTE. 8 In Professor Zhai Zhenguo's study on venous thromboembolism in Chinese inpatients (DissolVE-2), it was found that the main risk factors for surgical and internal medicine inpatients were main surgery (52.6%) and acute infection (42.2%), respectively. 9 Andrea J Darzi conducted a systematic evaluation of the prognostic factors for VTE in hospitalized internal medicine patients and found that the factors with moderate levels of evidence for VTE occurrence were: advanced age, elevated D-dimer levels, and a history of VTE; thrombophilia; cancers; critical illness; and infectious diseases. 10 This is consistent with our research findings that over 60 years old, heart failure, nephrotic syndrome, cancer, history of VTE, and elevated D-dimer were significantly correlated with the occurrence of VTE. It is widely known that D-dimer is closely related to the occurrence and recurrence of VTE, so the SJT RAM incorporates D-dimer as a risk factor. The IMPROVEDD research showed that the value of D-dimer was more than 2 times the upper limit of normal and could effectively predict the occurrence of VTE. High-risk VTE patients with D-dimer levels greater than 3 times higher could benefit from extended anticoagulation. 11 In our study, considering the different lab methods and normal value range of D-dimer in the different hospitals, so we did not add a fixed value of D-dimer, only the level of elevation was considered, which was in agreement with the above IMPROVEDD study.
The Caprini VTE Risk Assessment Model was developed in 1986 by Professor Joseph Caprini in collaboration with a group of physicians, surgeons, nurses, and scientists to construct a VTE risk assessment model based on the risk factors related to VTE. In 2019, Song et al 12 applied the Caprini risk assessment model to a prospective study of 262 Chinese patients undergoing pulmonary resection. The incidence rate of VTE in low-risk (score 0-4) patients was 0%, the incidence rate of VTE in moderate-risk (score 5-8) patients was 12.3%, and the incidence rate of VTE in high-risk (score ≥ 9) patients was 40.0%. With the increase in VTE risk stratification, the incidence rate of VTE also increased. Some retrospective studies also support the Caprini risk assessment model showing good predictive value for VTE in other surgeries such as urology, gynecological surgery, trauma, and postoperative critically ill patients.13–16 Domestic and foreign scholars have also conducted validation work using the Caprini risk assessment model in internal medicine patients. A retrospective study by the University of Michigan on internal medicine patients showed that risk stratification for VTE based on the patient Caprini risk assessment model and corresponding prevention measures resulted in a 15% reduction in VTE risk for patients. 17 A case-control study on medical inpatients showed that the Caprini risk assessment model effectively predicted the VTE risk of patients. With the increase of scores, the VTE incidence rate increased, and the VTE incidence rate of high-risk patients increased significantly (OR 2.6, 95% CI 1.3, 5.5). 18 de Bastos et al 19 used the Caprini risk assessment model to stratify the nonsurgical tumor patients and confirmed that there was no VTE in the low-risk group, and the incidence rate of VTE events in high-risk patients was 4.2%. Several retrospective Chinese studies have also achieved satisfactory results in validating the effectiveness of the Caprini risk assessment model for VTE risk screening in hospitalized internal medicine patients. A retrospective case-control study showed that the incidence rate of VTE and the risk level of VTE increased linearly, and the incidence rate of high-risk patients was significantly higher.20,21 Luo and Zhang 22 analyzed the VTE incidence rate of medical patients screened by the Caprini risk assessment model. The incidence rate of VTE was 1% in low-risk patients, 1.8% in moderate-risk patients, 5.7% in high-risk patients, and 10.6% in extremely high-risk patients. The Caprini risk assessment model can effectively predict the risk of VTE occurrence in patients admitted to the internal medicine intensive care unit and cerebrovascular disease patients.23–25
The Padua risk assessment model was constructed by scholars such as Barbara based on the Kucher model. Scholars have found through reviewing imaging data of pulmonary embolism patients that the Padua score was higher in PE patients, with positive and negative predictive values of 22.6% and 88.2%, respectively. From this, it can be seen that the high specificity and negative predictive value of the Padua risk assessment model can prevent patients from undergoing unnecessary imaging examinations. 26 This is consistent with our research findings that the Padua risk assessment model has high specificity. During the COVID-19 pandemic, Chinese scholars used the Padua risk assessment model to stratify the risk of 143 COVID-19 patients. In subgroup analysis, it was found that a Padua ≥ 4 score was significantly correlated with the occurrence of DVT. 27 It is well known that surgical patients use the Caprini risk assessment model to screen high-risk populations for VTE. So is the Padua risk assessment model also effective for surgical patients. Scholars have compared the effectiveness of the Caprini model and Padua model in preventing thrombosis during the perioperative period in patients undergoing elective abdominal surgery. A total of 122 patients were treated with the Caprini risk assessment model, while 101 patients were treated with the Padua risk assessment model for thromboprophylaxis. In the Caprini group, 87 (71%) patients had a VTE risk assessment of ≥ 5 points, while in the Padua group, 38 (38%) patients had a VTE risk assessment of ≥ 4 points; The overall correlation between the 2 groups of scores is moderate (r = 0.619), and there is no difference in the incidence of adverse outcomes between the 2 groups. This study suggests that the Padua risk assessment model can also be used for surgical patients. 28 Research has shown that the high risk of the Padua and IMPROVE risk assessment models is associated with early mortality in internal medicine patients. Guido Giuseppe and other scholars compared the predictive ability of 2 evaluation models for mortality in internal medicine patients. The increase in the score of the Padua risk assessment model is significantly correlated with an increase in mortality rate, so the Padua risk assessment model can be used to evaluate the early mortality risk of internal medicine patients during hospitalization. 29
Spyropoulos enrolled 15 156 internal medicine inpatients in 2011 and constructed an IMPROVE risk assessment model. 30 In the later stage, the IMPROVEDD model was constructed by adding numerical values of D-dimer. S Greco compared the differences in the ability of IMPROVE and IMPROVEDD to predict VTE, including in-hospital mortality and 30-day mortality. The results showed that IMPROVE score > 4 was significantly correlated with an in-hospital mortality rate (P = .05), and IMPROVE score can serve as an independent predictor of in-hospital mortality rate and 30-day mortality rate in internal medicine patients. 31 Alex C Spyropoulos used IMPROVEDD for large-scale external validation among hospitalized patients in multi-center internal medicine. The sensitivity of IMPROVEDD in predicting VTE risk was 0.971, the specificity was 0.218, the positive predictive value was 0.036, and the negative predictive value was 0.996. The area under the ROC curve (AUC value) is 0.702. In this multicenter external validation study, the IMPROVEDD VTE risk score showed good predictive ability. 32
The limitation of this study was that Beijing Tsinghua Changgung Hospital only included data from the Department of Respiratory and Critical Care Medicine, and did not include data from other internal medicine departments. It is planned to further improve the relevant information of other internal medicine departments in the hospital before further analysis. However, the sample size of internal medicine inpatients included in the research institute was sufficient requirements of this study and explained its related results. The next step is to follow up on the incidence rate of VTE within 30 days and 90 days of medical inpatients to evaluate the effectiveness of SJT RAM in screening the risk of VTE within 30 days and 90 days of medical inpatients.
The SJT model has the least number of risk factors (only 6 risk factors) than the other 3 RAMs, so physicians could save much time in assessing VTE risk for hospitalized patients. The working burden of medical staff will be reduced. From our study, The AUC value of SJT model (0.80 ± 0.03) for internal medicine inpatients was greater than Caprini model (0.74 ± 0.03), Padua model (0.72 ± 0.03), and IMPROVE model (0.52 ± 0.03) (P < .05). The sensitivity and Youden index of the SJT model were also higher than those of the Caprini model, Pauda model, and IMPROVE model (P < .05). The SJT model has better predictive ability for VTE risk in hospitalized patients with multicenter internal medicine diseases than the Caprini RAM, Padua RAM, and IMPROVE RAM. Being able to effectively identify high-risk internal medicine inpatients with VTE has clinical efficacy and is worthy of further promotion and application.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Beijing Municipal Administration of Hospitals Incubating Program, National Multidisciplinary Cooperative Diagnosis and Treatment Capacity Project for Major Diseases: Comprehensive Treatment and Management of Critically Ill Elderly Inpatients (grant numbers PX2022029 and 2019.YLFW).
