Development and External Validation of a Machine Learning Model to Predict Venous Thromboembolism Risk in Hospitalized Chinese Patients

Abstract

Objectives

To develop and externally validate a machine learning (ML) model of VTE in hospitalized Chinese patients.

Methods

We retrospectively reviewed structured data from patients with and without VTE (N = 1126 in each group) at Beijing Shijitan Hospital between January 2012 and December 2021. ML algorithms, including logistic regression (LR), decision tree (DT), or gradient boosting (GBoost) were used to establish prediction models. Patients were prospectively enrolled at Beijing Shijitan Hospital (N = 2916) or Beijing Chaoyang Hospital (N = 1339) for internal and external model validation, respectively.

Results

Several clinical features were had high weighted correlation with VTE in ML models: age, D-dimer level, platelet count, hemoglobin level, coronary heart disease, cancer, male sex, and comorbidity. The two highest weighted features were age and D-dimer. Optimal cut-off values indicated that age >65 years, white blood cell count >8.69 (10⁹/L), hemoglobin level >126 (g/L), platelet count >197 (10⁹/L), neutrophil percentage >72%, and D-dimer level >965 (ng/ml, D-dimer units) were significantly correlated with VTE occurrence. The AUC value of the GBoost model (0.88 ± 0.03) was significantly higher than the DT (0.70 ± 0.03) or LR (0.64 ± 0.03) models. GBoost also had a higher sensitivity, specificity, Youden index and Matthews correlation coefficient (MCC) than DT or LR (P < 0.05). In the internal validation cohort, the AUC value of the GBoost model (0.85 ± 0.03) was higher than the DT (0.76 ± 0.03) and LR (0.67 ± 0.03) models and the Caprini Risk Assessment Model (RAM) (0.75 ± 0.03). GBoost also had a greater sensitivity, specificity, Youden index and MCC than DT and LR (P < 0.05). In the external validation cohort, the AUC value of the GBoost model (0.81 ± 0.03) was also better than the DT (0.72 ± 0.03) and LR (0.71 ± 0.03) models and the Caprini RAM (0.73 ± 0.03). The GBoost model also had a higher sensitivity, specificity, Youden index and MCC than DT and LR (P < 0.05).

Conclusion

The GBoost ML model was better at identifying hospitalized Chinese patients at risk of VTE than other ML algorithms and the Caprini RAM.

Keywords

machine learning venous thromboembolism risk model development external validation Chinese patients

Introduction

Venous thromboembolism (VTE), encompassing deep vein thrombosis (DVT) and pulmonary embolism (PE), is a significant cause of morbidity and mortality worldwide and is increasingly recognized as a major health burden in China.¹ Despite this growing awareness, most existing VTE risk prediction models were developed using non-Chinese populations, limiting their generalizability and accuracy in Chinese patients due to potential differences in clinical risk factors and healthcare practices. For Caucasian families, F5 mutations are reported most frequently, at 37.8% (85/225), while PROS1 mutations are most frequently reported, at 40.4% (21/52), for East Asian families. Factor V Leiden is reported more frequently in Caucasians than in East Asians.² In the USA, the average length of stay for VTE is 6–7.7 days,³ but in China, the average length of stay for VTE is almost 15 days⁴ The incidence of VTE also differs between the USA (1.17/1000 person year) and China (1.45/1000 person year).⁵ Consequently, there is an urgent need to develop tailored, accurate VTE risk prediction models for Chinese patients to optimize prevention strategies and improve outcomes. In our previous approach to this problem, we used the Caprini risk assessment model (RAM) which is used widely in China and has demonstrated a higher sensitivity and predictive ability than the Padua RAM among hospitalized Chinese patients.⁶

Machine learning (ML) offers a promising approach for developing personalized prediction models by leveraging large-scale clinical data and identifying complex patterns that traditional statistical models may overlook. Compared to conventional clinical scores, ML can analyze multidimensional data to capture nonlinear relationships. Thereby improving predictive accuracy and providing more individualized risk assessments. Automated integration of electronic medical records (EMR), and other data eliminates subjectivity and omissions inherent in manual scoring. It rapidly processes large-scale data, enabling real-time evaluation of thousands of patient cases and reducing assessment time. However, few studies have externally validated these models or systematically compared their performance to established clinical scores.

In this study, we developed and validated ML-based VTE risk prediction models specifically for hospitalized Chinese patients. We compared the performance of three ML algorithms—logistic regression (LR), decision tree (DT), and gradient boosting (GBoost)—to conventional clinical scoring methods in internal and external validation cohorts. This work represents a step toward creating precision-based VTE prevention strategies tailored to hospitalized Chinese surgical and medical patients.

Methods

Ethical Approval

This study was approved by the Medical Ethics Committee of Beijing Shijitan Hospital Affiliated with Capital Medical University (No. 2020-2).

Study Population

We collected data from the structured electronic medical record (EMR) system. A retrospective case-control dataset with a 1:1 allocation ratio of VTE and non-VTE patients (N = 1126 in each group) at Beijing Shijitan Hospital was collected from January 2012 to December 2021 and used for model training. A prospective cohort of 2916 patients were consecutively enrolled for internal validation between January 2023 and December 2023. A separate prospective cohort of 1339 patients who were admitted to Beijing Chaoyang Hospital from October 2021 to March 2022 were consecutively enrolled for external validation. The enrolled patients were from the following departments: gastrointestinal surgery, orthopedics, thoracic surgery, urologic surgery, obstetrics and gynecology, cerebral surgery, neurology, intensive care unit, geriatrics, respiratory and critical care medicine, cardiovascular, nephrologic, hematologic, gastroenterology, traditional Chinese medicine, rheumatologic and immunologic, endocrine, oncologic, and emergency. (Figure 1).

Figure 1.

Data Collection for Development and Validation of ML VTE Prediction Models.

Inclusion criteria and case definition: The training dataset included patients who were ≥18 years old with ≥2 days of hospitalization. At admission, all patients were screened for DVT with a baseline compression ultrasound; PE diagnostic imaging was done only for symptomatic patients. Patients who did not exhibit symptoms of PE and had a normal D-dimer level and imaging confirmation of no DVT, were clinically defined as having no acute PE. VTE cases were defined by a DVT and/or PE diagnosis before discharge. All patients had predischarge imaging, including diagnostic color Doppler compression ultrasound for DVT and CT pulmonary angiography (CTPA) or pulmonary ventilation perfusion (V/Q) scanning for symptomatic PE.

Cases were identified by trained medical students, and case adjudication was performed by senior consultant physicians. We excluded patients with superficial venous thrombosis, those on anticoagulant or thrombolytic drug use at admission or after admission (including recombinant tissue plasminogen activator or urokinase), and those with incomplete structured clinical data. Control patients had confirmed negative VTE imaging. Internal validation data from Beijing Shijitan Hospital patients and external validation data from Beijing Chaoyang Hospital patients were collected using the same case definitions as for the training dataset.

Sample Size：Based on the incidence rate of VTE in Beijing Shijitan hospital (5.7%), the necessary sample size, calculated by logistic regression (PASS software), was at least 244 cases.

Definition and Collection of Clinical Variables: After reviewing the literature, the content of the Caprini RAM, and the structured data extracted from the EMR system, we captured the following variables from patients at admission: age, sex, D-dimer level, white blood cell count, platelet count, hemoglobin count, percentage of neutrophils; whether patients were diagnosed with hypertension, coronary heart disease, tumor (history of, or active), diabetes; or had surgery (within the last month), a blood transfusion, were smokers, or had comorbidities.

Model Development

The VTE RAM was developed using ML methods, which included five steps: data preprocessing, correlation analysis, model training, model testing, and model evaluation. The process is shown in Figure 2 (Appendix 1).

Figure 2.

The Process of Developing Machine Learning Models.

Data Preprocessing

Many clinical characteristics were captured in the raw data obtained from the structured EM.The most commonly used methods to deal with missing data include the direct-deletion method, the mean-filling method, and the mode-filling method. In order to improve the accuracy of model prediction, the mean- filling method was selected. Therefore, preprocessing of the original data was performed before correlation analysis.

Correlation Analysis

In ML, correlation can be used to identify features that have a significant impact on the target variable. In this study, the weighted Pearson correlation coefficient was used to measure the correlation of features with VTE risk. For each clinical characteristic, a decision tree classifier was used to calculate the weight for each characteristic. During the construction of the decision tree, each internal node will select a characteristic that can minimize the impurity to divide the samples. The total contribution of each characteristic to reduce impurities at all internal nodes was calculated, and after normalization, the result was the weight of each clinical characteristic.

Establishment of Machine Learning Models

In this study, we selected LR, DT and GBoost algorithms according to comprehensive consideration of both the characteristics of medical data and the features of different algorithms.

①　 LR model

The core of logistic regression is the sigmoid function, whose mathematical expression is:

σ_{(z)} = \frac{1}{1 + e^{- z}}

where z is a linear combination, usually

z = ω^{T} x + b

where A is the weight vector, x is the input feature vector, and b is the bias term. The output value of the sigmoid function is between 0 and 1, which is interpreted as a number closer to 1 means a greater probability that the sample belongs to the class. This value represents the direction and degree of influence of each feature on the probability of disease.

②　 DT model

A classification decision tree learns and classifies data based on a tree structure and constructs a model through recursive division of data features, so as to classify and make predictions for new data.

③　 GBoost model

The core principle of the GBoost model is to improve overall predictive performance by iteratively training multiple weak learners and combining the accumulated predicted results of these weak learners into a strong learner. The gradient rise algorithm increases the value of the objective function by iteratively updating the parameters along the gradient direction of the objective function. For the general objective function $J_{(ω)}$ , at the t + 1 iteration, the $ω$ updated formula for the parameters is $ω_{t + 1} = ω_{t} + α \nabla J_{(ω_{t})}$ , where $α$ is the learning rate and $\nabla J_{(ω_{t})}$ is the gradient of the objective function $J_{(ω)}$ at $ω_{t}$ . The GBoost model effectively uses the gradient of the objective function to find the optimal parameter value.

Evaluation and Validation of Machine Learning Models

The receiver operating characteristic (ROC) curve, AUC value (area under ROC curve), Youden index, Matthews correlation coefficient (MCC), Kappa coefficient, and Jordan index were used to evaluate the classification performance of each prediction model. The ROC curve, drawn with false positive rate (FPR) as the horizontal coordinate and true positive rate (TPR) as the vertical coordinate, shows the performance of the model under different thresholds. The AUC value is the area under the ROC curve and ranges from 0.5 to 1, numbers closer to 1 indicate better performance. The MCC takes into account the number of true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN) in the classification model, providing a single value that summarizes the quality of the classification and reflects the ability of the classification model to distinguish positive and negative samples. The Kappa coefficient was used to evaluate agreement between the classification model predictions and the actual data, with values ranging from −1 to 1, Outcomes closer to +1 mean better agreement. The optimal cut-off values for risk factors were obtained using the Jordan Index method. ROC curves, sensitivity, specificity, Youden index and MCC were used to evaluate the performance of VTE prediction using ML models in internal and external validation cohorts. Python software 3.12 was used for all statistical analyses.

Results

Correlation and Weight of Clinical Characteristics with VTE

In this study, 15 clinical features associated with VTE were obtained from a structured EMR, including age, sex, D-dimer levels, white blood cell count, platelet count, hemoglobin count, percentage of neutrophils, diagnosis of hypertension, coronary heart disease, cancer (history of, or active), diabetes, whether they had surgery (with the last month), blood transfusion, other comorbidities, or were smokers. The correlation with VTE and the weight of each clinical characteristic in each model were calculated A correlation coefficient of more than 0.05 indicated a high correlation between the clinical feature and VTE. Clinical features that were highly correlated with VTE with a high weight were: age, male sex, D-dimer level, platelet count, hemoglobin count, presence of coronary heart disease, history of/active cancer, and comorbidity. The two clinical features with the highest weighted correlations were age and D-dimer level. (Table 1).

Table 1.

Correlation and Weight of Clinical Characteristics with VTE.

Clinical Characteristics	Correlation Coefficient	Weight	Correlation Coefficient > 0.05
Age	0.16229	0.19835	*
D-dimer (ng/ml DDU)	0.13289	0.19002	*
White blood cell count (10⁹/L)	0.03440	0.13342
Platelet count (10⁹/L)	0.06909	0.12586	*
Hemoglobin count (g/L)	0.08933	0.12431	*
Percentage of neutrophils (%)	0.04768	0.09189
Hypertension	0.03846	0.02323
Coronary heart disease	0.14468	0.02273	*
Tumor (history of, or active)	0.06028	0.01812	*
Male sex	0.06673	0.01596	*
Surgery (within the last month)	0.01524	0.01559
Diabetes	0.04086	0.01372
Blood transfusion	0.00092	0.01109
Smoking	0.03461	0.01007
Comorbidity	0.06877	0.00564	*

*indicates characteristic with weighted correlation coefficient >0.05.

Cut-off Value of Clinical Characteristics with VTE

The optimal cut-off value was obtained using the Jordan Index method, ROC curve method, and density estimation method for the data. The results showed that age >65 years, white blood cell count >8.69 (10⁹/L), hemoglobin count >126 (g/L), platelet count >197 (10⁹/L), neutrophil percentage >72%, and D-dimer level >965 (ng/ml, DDU) were significantly correlated with VTE occurrence, with D-dimer level >965 (ng/ml, DDU) having the highest predictive ability. We further stratified the patients into three age groups (≤65 years, 65–80 years, and ≥ 80 years) and found that the optimal D-dimer thresholds for VTE prediction varied across these groups. In patients aged ≤65 years, a significant increase in VTE incidence was observed when D-dimer >550(ng/ml, DDU). For those aged 65–80 years, the threshold was >750(ng/ml, DDU) for a marked rise in VTE occurrence. In the ≥ 80 years group, VTE risk increased significantly when D-dimer >965(ng/ml, DDU). (Table 2).

Table 2.

Cut-off Value and Correctness Rate of Clinical Characteristics with VTE.

Clinical Characteristics		Cut-off Value	Correctness Rate*
Age		65	0.591
White blood cell count (10⁹/L)		8.69	0.548
Hemoglobin count (g/L)		126	0.553
Platelet count (10⁹/L)		197	0.555
Percentage of neutrophils (%)		72	0.544
D-dimer (ng/ml DDU)	≤ 65 years	550	0.576
	65–80 years	750	0.724
	≥ 80 years	965	0.775

*Correctness rate means the correct classification rate.

Machine Learning Model Development

Comparing the ML models developed using retrospective data indicated that the AUC value of the GBoost model (0.88 ± 0.03) was higher than the DT (0.70 ± 0.03) and LR (0.64 ± 0.03) models. The GBoost model also had a higher sensitivity (0.79 ± 0.04), specificity (0.80 ± 0.04), Youden index (0.59 ± 0.04)and MCC (0.59 ± 0.05) than the DT (0.63 ± 0.04, 0.67 ± 0.04, 0.30 ± 0.04, 0.29 ± 0.05) or LR (0.60 ± 0.04, 0.59 ± 0.04, 0.19 ± 0.04, 0.20 ± 0.05) (P < 0.05) methods, meaning that the GBoost model had much better predictive value than DT or LR models (Figure 3).

Figure 3.

ROC Curve of Different Machine Learning Models in Development Cohort.

Internal Validation of Machine Learning Models

In internal validation, the AUC value of the GBoost model (0.86 ± 0.03) was also higher than the DT (0.76 ± 0.03) and LR (0.67 ± 0.03) models, as well as the Caprini RAM (0.76 ± 0.03). The GBoost model had a higher sensitivity (0.80 ± 0.04), specificity (0.76 ± 0.04), Youden index (0.56 ± 0.04)and MCC (0.56 ± 0.05) than DT (0.69 ± 0.04, 0.68 ± 0.04, 0.37 ± 0.04, 0.38 ± 0.05) or LR (0.63 ± 0.04, 0.60 ± 0.04, 0.23 ± 0.04, 0.22 ± 0.05) (P < 0.05) (Figure 4).

Figure 4.

ROC Curves of Different Machine Learning Models in Internal Validation Cohort.

External Validation of Machine Learning Models

In external validation, the AUC value of the GBoost model (0.81 ± 0.03) was higher than the DT (0.72 ± 0.03) and LR (0.71 ± 0.03) models, as well as the Caprini RAM (0.73 ± 0.03). The GBoost model had a higher sensitivity (0.73 ± 0.04), specificity (0.74 ± 0.04), Youden index (0.47 ± 0.04) and MCC (0.46 ± 0.05) than DT (0.59 ± 0.04, 0.64 ± 0.04, 0.23 ± 0.04, 0.33 ± 0.05) or LR (0.61 ± 0.04, 0.59 ± 0.04, 0.20 ± 0.04, 0.30 ± 0.05) (P < 0.05) (Figure 5).

Figure 5.

ROC Curves of Different Machine Learning Models in the External Validation Cohort.

Discussion

Artificial intelligence (AI) technology is a cost-effective method to promote precision health and prevent adverse patient outcomes. ML is a subset of AI that learns from available data, using recognized patterns and phenotypic groups, and makes decisions or predictions based on these data.

Our study documented that age and D-dimer levels had high weighted correlation with VTE risk. These two characteristics were also the best predictors for VTE in a clinical prediction model for VTE based on Support vector machine (SVM), developed and validated by Xu et al (2023)⁷ and in a model of cancer-related VTE.⁸ Age is usually included in the commonly used VTE RAMs in clinical practice, as it is the easiest characteristic to obtain. Vuilleumier et al reported that the 30-day mortality rate for acute PE patients under the age of 65 is 4%, while for older patients it is 11%.⁹ In the IMPROVE study, a significant correlation was found between being older than 60 and the occurrence of VTE (P < 0.001). The risk of VTE occurring within 3 months in hospitalized patients over 60 years old was twice that of younger patients.^10,11 This is consistent with our finding that age is a strong risk factor for VTE, especially in hospitalized patients >65 years old, and that age >65 years was the best cutoff value for high VTE risk.

D-dimer is closely related to the occurrence of VTE and is a commonly used biological marker for evaluating the coagulation status of patients in clinical practice. D-dimer is a degradation product of fibrin, and an increase in D-dimer levels indicates the presence of hypercoagulability and secondary fibrinolysis in the body. A multicenter, randomized, double-blind, parallel-controlled study conducted by Spyropoulos et al showed that a measurement of >2 times normal D-dimer levels can be used as a biological marker for predicting in- and out-of-hospital VTE occurrence, and that patients with D-dimer levels >3 times higher than normal levels are at high risk of VTE and can benefit from extended anticoagulation.¹² In the IMPROVE combined with D-dimer (IMPROVEDD) study, it was found that assigning a score of 2 to a D-dimer level that was greater than twice the normal upper limit significantly improved the ability to predict 77-day symptomatic VTE risk in hospitalized patients with acute disease (HR: 2.22, [95% CI: 1.38–1.58], P = 0.001). Therefore, IMPROVEDD can effectively screen hospitalized patients for VTE risk and need for thromboprophylaxis treatment.¹³ Our study showed that D-dimer levels >965 ng/ml (DDU) had the highest predictive ability, in agreement with these studies. Our study demonstrated that the optimal D-dimer threshold increases with advancing age, which aligns with the current Chinese guidelines recommending age-adjusted cut-off values for D-dimer interpretation.¹⁴ Furthermore, our research indicated that some clinical data, including platelet count and hemoglobin levels, were significantly correlated with the occurrence of VTE, although these indicators are not included in the Caprini RAM. Chen et al also identified several other new risk factors for VTE, including low basophil counts and percentages, a high anion gap, and high eosinophil counts and percentages.¹⁵ Including these biomarkers may enhance the predictive accuracy and comprehensiveness of current VTE risk evaluation tools, thereby contributing to more effective prevention and management strategies for at-risk individuals.

The difference between ML modeling and traditional modeling methods lies in their approach to data analysis. Traditional methods analyze linear data, such as linear regression and logistic regression, but ML methods can handle a broader range of data types. As such, ML methods may have obvious advantages in predicting VTE risk.¹⁶ Many studies have compared ML and traditional methods. Across these studies, ML models have shown an AUC ranging from 0.65 to 0.989. Ding et al, found that GBoost outperformed the traditional Geneva score in predicting VTE after hip arthroplasty, with an AUC of 0.982 compared to 0.552, respectively.¹⁷ GBoost also performed the best in predicting VTE after radical gastrectomy, with an AUC of 0.989.¹⁸ A meta-analysis showed that RF and LR achieved the highest AUC (0.90) in assessing the risk of VTE in patients with lung cancer.¹⁹ Comparing Khorana and Caprini RAMs with GBoost, GBoost exhibited an AUC of 0.99 in predicting postoperative VTE in patients with colorectal cancer.²⁰ Meng et al, found that the Gboost model had a better performance than traditional models in VTE prediction for hospitalized cancer patients.²¹ In a retrospective study, Sheng et al, compared LR, RF and GBoost in 3078 individuals. RF was found to have superior performance and great potential in clinical practice. The results showed that RF was effective in correcting the misclassification by Caprini risk score(CRS) in both the positive and negative groups.²² Comparing conventional scores and ML algorithms for predicting major bleeding after a VTE, Grdinic et al, showed that the ML predictive performance was higher than that of the conventional score and may potentially provide better estimates of bleeding risk than those obtained via classical statistical analysis.²³ Chiasakul et al, reviewed 20 studies using AI for prediction of VTE and found the mean AUC for AI versus conventional methods were 0.79 (95% CI: 0.74-0.85) versus 0.61 (95% CI: 0.54-0.68), respectively, suggesting that an AI-based clinical decision support system could reduce the rate of hospital-acquired VTE by 19%.²⁴ Hu et al, developed an RF model for colon cancer that had good predictive capability with an AUC of 0.745 for external validation.²⁵ Guan et al, established three ML models, including RF, XGBoost, and SVM, to predict the risk of VTE in critically ill patients based on ICU data. The results indicated that the RF model achieved the best performance, with an accuracy of 0.99.²⁶ These studies are consistent with our findings, which indicated that the GBoost algorithm demonstrated the strongest predictive capability for VTE risk among ML models and was also better than a traditional VTE RAM. In contrast, He et al, compared the prediction effects of different ML models with the Padua and Caprini RAMs in trauma patients. The Caprini RAM showed better discrimination ability in predicting VTE events (TPR = 0.667, FPR = 0.227, AUC = 0.773) than ML, because the face-to-face evaluation of patients after admission may provide more accurate data.²⁷

Lam et al, surveyed clinicians and found that a majority of respondents believed that ML can be helpful in the field of VTE prevention and management.²⁸ Chrysafi et al,²⁹ analyzed the application of ML for VTE prevention, diagnosis, and management. They found that the advantages of ML included accelerating clinical documentation,Identifying bleeding events, assisting with clinical decision-making, generating synthetic data, and designing novel anticoagulants. Park et al, developed a RF model derived from EMR data to identify patients at risk for 30-day VTE and readmission after discharge, helping to select patients who may benefit from extended thromboprophylaxis.³⁰

Most studies that have examined using ML for VTE RAMs are single center retrospective studies, in contrast to our prospective multicenter validation study. Deng et al, collected the data of 256 transplantation patients with VTE and developed a ML model that integrated 8 objective clinical and laboratory predictors. The AUC value of this ML model for VTE risk was 0.88.³¹ Liu et al, reviewed data collected between December 2019 and July 2022 for VTE in urological inpatients. They used Random forest (RF) and SVM for assessing the VTE risk; the accuracy of SVM was 92.6%. Danilatou et al, documented several retrospective studies of ML from a single institution, only a few of which performed external validation. Some ML models described excellent performance for VTE prediction internally, compared with traditional RAMs but their performance tended to be lower in external validation cohorts.³² Our study used both internal and external data to validate our models. In this study population, GBoost was better in the than other models and conventional RAM in both internal and external cohorts.

ML offers the following significant advantages, such as 1) ML through feature importance analysis, may uncover novel VTE-related biomarkers or clinical features. 2) Integration with hospital information systems (HIS) allows automatic triggering of alerts and preventive recommendations, enhancing clinical compliance. 3) Unlike traditional models that rely on fixed variables and weights with long update cycles, machine learning-based models can periodically incorporate new patient data for training, automatically adjusting parameter weights to adapt to evolving clinical scenarios. Their self-updating capability surpasses that of conventional models. The challenges of ML include hidden biases, clinical reluctance, data privacy, litigation and need for prospective testing.²⁹ An EMR contains multidimensional data of variable quality, thus while building large datasets from multiple institutions is essential, there are numerous challenges, including reconciling different test assays and interpretation of clinical variables.

The limitations of this study are as follows:1) Our study included all hospitalized patients and did not categorize them into internal medicine, surgical medicine or oncology patients. In the next step of our research, we will compare different ML models among patients in these different categories. 2) The ML analysis in this study was based solely on structured clinical variables available in the EMR system, though not all clinically relevant variables are captured in a structured format. Nevertheless, the structured variables included in this study encompassed key VTE-related predictors, and the constructed ML model demonstrated statistically significant predictive performance. As the structured data capture in EMR systems continues to improve, future incorporation of additional clinical variables may further enhance model performance. And we plan to embed ML model into EMR system and automatically capture structured data and assign risk scores upon admission. this would significantly reduce the time needed for physician evaluation, thus reducing workload and improving evaluation efficiency.

Supplemental Material

sj-docx-1-cat-10.1177_10760296251372942 - Supplemental material for Development and External Validation of a Machine Learning Model to Predict Venous Thromboembolism Risk in Hospitalized Chinese Patients

Supplemental material, sj-docx-1-cat-10.1177_10760296251372942 for Development and External Validation of a Machine Learning Model to Predict Venous Thromboembolism Risk in Hospitalized Chinese Patients by Xiaolan Chen, Peiran Wang, Xinyu Hu, Guilian Wang, Zhijia Zhang, Qibo Liu, Yuanhua Yang, Alfonso J Tafur and Lei Pan in Clinical and Applied Thrombosis/Hemostasis

Footnotes

Acknowledgments

The authors thank Dr Jawed Fareed, Dr Grigorios Gerotziafas, and Dr Joseph A. Caprini for their support and encouragement during this study as well as reviewing the manuscript and providing editorial support.

ORCID iDs

Xiaolan Chen

Alfonso J Tafur

Author Contributions

X.L.C. and L.P. designed the article. G.L.W. was responsible for methodology. X.Y.H., P.R.W. and Y.H.Y. collected data. Z.J.Z, P.R.W. and Q.B.L. analyzed the data. X.L.C wrote the initial draft, A.J.T reviesd the draft.

Funding

This work was supported by the Beijing Municipal Administration of Hospitals Incubating Program and Training Plan for High level Public Health Technical Talents Construction Project (grant numbers PX2022029 and 03-33).

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Supplemental Material

Supplemental material for this article is available online.

References

Zhang

Lei

Shao

, et al. China Venous Thromboembolism Study Group . Trends in hospitalization and in-hospital mortality from VTE, 2007 to 2016, in China. Chest. 2019;155(2):342–353.

Zhang

Shu

, et al. The genetics of venous thromboembolism: A systematic review of thrombophilia families. J Thromb Thrombolysis. 2021;51(2):359–369.

Fernandez

Hogue

Preblick

Kwong

. Review of the cost of venous thromboembolism. Clinicoecon Outcomes Res. 2015;28(7):451–462.

Zhai

Wang

Lei

, et al. China pUlmonary thromboembolism REgistry Study (CURES) investigators . Trends in risk stratification, in-hospital management and mortality of patients with acute pulmonary embolism: An analysis from the China pUlmonary thromboembolism REgistry Study (CURES). Eur Respir J. 2021;58(4):2002963.

Chinese Thoracic Society, Pulmonary Embolism and Pulmonary Vascular Disease Working Group, Chinese Medical Doctor Association, Respiratory Branch, Pulmonary Embolism and Pulmonary Vascular Disease Working Committee, National Cooperative Group for the Prevention and Treatment of Pulmonary Embolism and Pulmonary Vascular Disease . Guidelines for the diagnosis, treatment, and prevention of pulmonary thromboembolism. Chin Med J. 2018;98(14):1060–1087.

Chen

Pan

Deng

, et al. Risk assessment in Chinese hospitalized patients comparing the Padua and Caprini scoring algorithms. Clin Appl Thromb Hemost. 2018;24(9_suppl):127S–135S.

Lei

, et al. Machine learning predicts cancer-associated venous thromboembolism using clinically available variables in gastric cancer patients. Heliyon. 2023;9(1):e12681.

Drăgan

AŞ

. Novel insights in venous thromboembolism risk assessment methods in ambulatory cancer patients: From the guidelines to clinical practice. Cancers (Basel). 2024;16(2):458.

Vuilleumier

Simona

Méan

, et al. Comparison of cardiac and non-cardiac biomarkers for risk stratification in elderly patients with non-massive pulmonary embolism. PLoS One. 2016;11(5):e0155973.

10.

Spyropoulos

Anderson

FitzGerald

, et al. IMPROVE Investigators . Predictive and associative models to identify hospitalized medical patients at risk for VTE. Chest. 2011;140(3):706–714.

11.

Barbar

Noventa

Rossetto

, et al. A risk assessment model for the identifcation of hospitalized medical patients at risk for venous thromboembolism: The Padua prediction score. J Thromb Haemost. 2010;8(11):2450–2457.

12.

Spyropoulos

Lipardi

, et al. Modified IMPROVE VTE risk score and elevated D-dimer identify a high venous thromboembolism risk in acutely ill medical population for extended thromboprophylaxis. TH Open. 2020;4(1):e59–e65.

13.

Gibson

Spyropoulos

Cohen

, et al. The IMPROVEDD VTE risk score: Incorporation of D-dimer into the IMPROVE score to improve venous thromboembolism risk stratification. TH Open. 2017;1(1):e56–e65.

14.

Pulmonary Embolism and Pulmonary Vascular Disease Group of Chinese Thoracic Society, Pulmonary Embolism and Pulmonary Vascular Disease Working Committee of Chinese Association of Chest Physicians, National Cooperative Group for Prevention and Treatment of Pulmonary Vascular Disease . Guidelines for the diagnosis, prevention and management of pulmonary thromboembolism in China (2025 edition). Natl Med J China. 2025;105(26):2162–2194.

15.

Chen

Petrazzini

Malick

Rosenson

. Prediction of venous thromboembolism in diverse populations using machine learning and structured electronic health records. Arterioscler Thromb Vasc Biol. 2024;44(2):491–504.

16.

Liu

Yang

Cheng

Chien

Chuang

Jin

. Identification of key risk factors for venous thromboembolism in urological inpatients based on the caprini scale and interpretable machine learning methods. Thromb J. 2024;22(1):76.

17.

Ding

Zheng

, et al. Machine learning-based screening of risk factors and prediction of deep vein thrombosis and pulmonary embolism after hip arthroplasty. Clin Appl Thromb Hemost. 2023;29(12):10760296231186145.

18.

Office of the Surgeon General (US); National Heart, Lung, and Blood Institute (US) . The Surgeon General's Call to Action to Prevent Deep Vein Thrombosis and Pulmonary Embolism. Office of the Surgeon General (US); 2008.

19.

Franco-Moreno

Madroñal-Cerezo

Muñoz-Rivas

TorresMacho

Ruiz-Giardín

Ancos-Aracil

. Prediction of venous thromboembolism in patients with cancer using machine learning approaches: A systematic review and meta-analysis. JCO Clin Cancer Inform. 2023;7(8):e2300060.

20.

Qin

Liang

Xie

, et al. Development and validation of machine learning models for postoperative venous thromboembolism prediction in colorectal cancer inpatients: A retrospective study. J Gastrointest Oncol. 2023;14(1):220–232.

21.

Meng

Wei

Fan

, et al. Development and validation of a machine learning model to predict venous thromboembolism among hospitalized cancer patients. Asia Pac J Oncol Nurs. 2022;9(12):100128.

22.

Sheng

Wang

Hao

Zhang

. Development and validation of machine learning models for venous thromboembolism risk assessment at admission: A retrospective study. Front Cardiovasc Med. 2023;10(8):1198526.

23.

Grdinic

Radovanovic

Gleditsch

, et al. Developing a machine learning model for bleeding prediction in patients with cancer-associated thrombosis receiving anticoagulation therapy. J Thromb Haemost. 2024;22(4):1094–1104.

24.

Chiasakul

Lam

McNichol

, et al. Artificial intelligence in the prediction of venous thromboembolism: A systematic review and pooled analysis. Eur J Haematol. 2023;111(6):951–962.

25.

Yuan

Zhang

Lei

. Development and validation of machine learning models for predicting venous thromboembolism in colorectal cancer patients: A cohort study in China. Int J Med Inform. 2024;195(3):105770.

26.

Guan

Chang

. Interpretable machine learning models for predicting venous thromboembolism in the intensive care unit: An analysis based on data from 207 centers. Crit Care. 2023;27(1):406.

27.

Luo

Hou

, et al. Predicting venous thromboembolism in hospitalized trauma patients: A combination of the Caprini score and data-driven machine learning model. BMC Emerg Med. 2021;21(1):60.

28.

Lam

Dodge

Zerbey

, et al. The potential use of artificial intelligence for venous thromboembolism prophylaxis and management: Clinician and healthcare informatician perspectives. Sci Rep. 2024;14(1):12010.

29.

Chrysafi

Lam

Carton

Patell

. From code to clots: Applying machine learning to clinical aspects of venous thromboembolism prevention, diagnosis, and management. Hamostaseologie. 2024;44(06):429–445.

30.

Park

Kim

Lee

Zheng

Amin

. Personalized risk prediction for 30-day readmissions with venous thromboembolism using machine learning. J Nurs Scholarsh. 2021;53(3):278–287.

31.

Deng

Zhu

Zhang

, et al. Machine learning algorithm as a prognostic tool for venous thromboembolism in allogeneic transplant patients. Transplant Cell Ther. 2023;29(1):e1–e10.

32.

Danilatou

Dimopoulos

Kostoulas

Douketis

. Machine learning-based predictive models for patients with venous thromboembolism: A systematic review. Thromb Haemost. 2024;124(11):1040–1052.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB