Machine Learning to Dynamically Predict In-Hospital Venous Thromboembolism After Inguinal Hernia Surgery: Results From the CHAT-1 Study

Abstract

Background

The accuracy of current prediction tools for venous thromboembolism (VTE) events following hernia surgery remains insufficient for individualized patient management strategies. To address this issue, we have developed a machine learning (ML)-based model to dynamically predict in-hospital VTE in Chinese patients after hernia surgery.

Methods

ML models for the prediction of postoperative VTE were trained on a cohort of 11 305 adult patients with hernia from the CHAT-1 trial, which included patients across 58 institutions in China. In data processing, data imputation was conducted using random forest (RF) algorithm, and balanced sampling was done by adaptive synthetic sampling algorithm. Data were split into a training cohort (80%) and internal validation cohort (20%) prior to oversampling. Clinical features available pre-operatively and postoperatively were separately selected using the Sequence Forward Selection algorithm. Nine-candidate ML models were applied to the pre-operative and combined datasets, and their performance was evaluated using various metrics, including area under the receiver operating characteristic curve (AUROC). Model interpretations were generated using importance scores, which were calculated by transforming model features into scaled variables and representing them in radar plots.

Results

The modeling cohort included 2856 patients, divided into 2536 cases for derivation and 320 cases for validation. Eleven pre-operative variables and 15 combined variables were explored as predictors related to in-hospital VTE. Acceptable-performing models for pre-operative data had an AUROC ≥ 0.60, including logistic regression, support vector machine with linear kernel (SVM_Linear), attentive interpretable Tabular learning (TabNet), and RF. For combined data, logistic regression, SVM_Linear, and TabNet had better performance, with an AUROC ≥ 0.65 for each model. Based on these models, 7 pre-operative predictors and 10 combined predictors were depicted in radar plots.

Conclusions

A ML-based approach for the identification of in-hospital VTE events after hernia surgery is feasible. TabNet showed acceptable performance, and might be useful to guide clinical decision making and VTE prevention. Further validated study will strengthen this finding.

Keywords

algorithm machine learning logistic regression venous thromboembolism inguinal hernia risk factors

Introduction

Inguinal hernia repair is one of the most frequently performed surgical procedure worldwide. In China, over 1 million individuals undergo this operation annually.¹ While advancements in surgical techniques and materials have led to satisfactory outcomes for many hernia patients,² a proportion still experience severe postoperative complications, such as venous thromboembolism (VTE). Specifically, VTE occurs in 0.18% to 0.45% of patients within 30 to 90 days after undergoing inguinal hernia repair,^3,4 which can result in additional treatment and management, longer hospital stays, increased healthcare costs, and worse prognosis.⁵ Importantly, postoperative VTE is a preventable complication that can be mitigated by identifying risk factors early on, including patient information, treatment approaches, and surgical procedures.

To date, several prediction models for postoperative VTE have been developed and improved, but most are not specifically designed to estimate VTE risk following hernia surgery, and their discriminatory capacities remain modest.^6–8 In order to identify patients at high risk of VTE after hernia repair, our group has developed a practical 5-item tool, the Chinese Hernia Adult Thromboembolism (CHAT) score, using multivariate logistic regression (LR) analysis.⁹ Although the CHAT score exhibited accepted performance metrics, it did not address the potential existence of complex nonlinear relationships between clinical risk factors and VTE events. In addition, the model might be limited for the low incidence (0.14%) of VTE after hernia surgery, which is a common issue encountered when developing prediction models for clinical events.

To address complex analytic challenges, machine learning (ML) algorithms have emerged as an essential tool, enabling significant breakthroughs in various domains of smart cities, including healthcare, fitness, skill assessment, and personal assistants.¹⁰ In recent years, ML algorithms have been successfully utilized to develop novel models with impressive performance for disease prediction in the medical field.^11–13 In this study, our objectives were 2-fold: (1) to compare the performance of various ML models for predicting in-hospital VTE after hernia surgery using dynamic data (pre-operative and postoperative data) and (2) to identify clinically meaningful, model-agnostic interpretations to facilitate clinical decision making and care planning.

Methods

Study Design and Participants

The CHAT-1 study, a multicenter, retrospective, cross-sectional investigation (registration number: ChiCTR1900020853) was conducted at 58 medical centers (each with a minimum of 500 beds) in China between January 1, 2017 and December 31, 2017. The detailed methods and primary results of the CHAT-1 trial have been reported elsewhere.¹ Briefly, eligible patients were those who (1) underwent emergency or elective hernia surgery; (2) were 18 years or older; (3) had potential risk factors associated with in-hospital VTE, including demographics, diseases history, comorbidities, medication therapy, and procedure; and (4) reported an in-hospital VTE event or not during hospitalization. Patients were excluded if (1) they were outpatients and underwent an ambulatory operation, (2) their data was missing or illogical, or (3) they had recently suffered and were receiving anticoagulant therapy before admission.

The study protocol was approved by the Ethics Committee of Beijing Chaoyang Hospital, Capital Medical University, and the trial was done complied with the Declaration of Helsinki. The need for informed consent was waived because of CHAT-1's retrospective design and anonymized data. Present study followed the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) reporting guidelines.¹⁴

Predictors and Outcomes

Based on Caprini score,¹⁵ Padua score,¹⁶ and clinical relevance, we identified pre-operative and postoperative variables that were potentially related to in-hospital VTE after hernia surgery (Supplemental Table S1). Pre-operative variables involved patient characteristics that were available before the surgery, such as demographics, disease history, comorbidities, drug-related characteristics, and hernia type. In those with the intake of antithrombotic agents, the decision on whether to continue or interrupt the agents before the operation was primarily based on the selection of the responsible doctors. Postoperative variables included procedure and postoperative information. The aforementioned variables were collected from medical records and surgical anesthesia reports at each participating institution, using a predesigned form.

The target outcome was in-hospital VTE developed after hernia surgery. VTE events included deep vein thrombosis (DVT) and pulmonary embolism (PE): (1) DVT is determined using a combination of clinical symptoms (swelling and pain of the lower limbs, tenderness behind the lower leg and/or medial thigh) and auxiliary examination of color ultrasound for veins of both lower limbs or lower extremity venography¹⁷; (2) PE is warned by related clinical manifestations (dyspnea and shortness of breath) and laboratory values (plasma D-dimer), and diagnosed based on computed tomography (CT) pulmonary arteriography.¹⁸ Interruption of antithrombotic agents was defined as stopping antithrombotic agents including temporary replacement by heparin or not.

Data Processing

Patients with more than 30% missing data were excluded. For patients with partially missing data, Random Forest (RF) algorithm was used for data-level imputation.^19,20 Because of the proportional imbalance between patients with and without in-hospital VTE, sampling methods were taken to make up the shortage caused by the imbalanced sample size between different levels of the target variable.²¹ Patients without VTE were undersampled to meet modeling requirement that patients with VTE accounted for 1% of the total sample size. Whereafter, patients with VTE were oversampled using adaptive synthetic sampling algorithm (ADASYN),²² 1:1 matched with those without VTE.

Data Separation and Feature Filtering

The data separation process was conducted before oversampling. In this process, the data were randomly divided into 2 data subsets (named training set and test set) in a ratio of 8:2, which would be used for model establishment and verification, respectively.

The process of feature selection was conducted using the training set, employing the Sequence Forward Selection (SFS) algorithm based on LR, support vector machine (SVM), and RF. This process aimed to select feature subsets with the optimal performance and minimum size.^23–25 The input data were divided into 2 groups: pre-operative and combined (pre-operative and postoperative) variables. For each group, the algorithm started with an empty set of features and subsequently added variables. The first feature selected was the one with the most significant impact on improving fitness, and it was added to the feature set. Then, the algorithm sought the second feature whose combination with the first selected feature resulted in the best fitness improvement. This process continued until no additional feature improved the model's performance, which was evaluated using the F1-score and the area under the receiver operating characteristic curve (AUROC). F1-score is a function that calculated to balance precision and recall, with a larger value indicating better performance of the model.²⁶ If the performance of candidate feature subset was inferior to that of the feature subset in the previous round, the iteration was stopped, and the feature subset of the previous round was considered the optimal feature selection result.²³

ML Model Establishment, Evaluation, and Interpretation

Both linear and nonlinear ML models were applied to the pre-operative and combined datasets, including LR, RF, SVM, gradient boosting tree (GBT), and attentive interpretable Tabular learning (TabNet). SVM was implemented using linear and radial basis function (RBF) kernel, and GBT was implemented using XGBoost (extreme gradient boosting), LightGBM (light gradient boosting machine), Adaboost (adaptive boosting), and Catboost (gradient boosting with categorical features support) package. The prediction performance of all models was evaluated through 5 metrics: AUROC, precision, recall, F1-score, and accuracy (Supplemental Table S2). Ultimately, the acceptable-performing ML models were selected based on their AUROC for pre-operative and combined datasets, separately.

The importance of features refers to the degree to which each feature contributes to improving the predictive power of the entire model.²⁷ We calculated the importance scores of the features using selected algorithms. Features with higher importance scores were more closely related to the accurate prediction of in-hospital VTE. As importance scores are not fixed-scale values, we present the results as normalized importance scores. This means that the importance of a feature is scaled with respect to the feature with the highest importance value to obtain easy-to-read and comparable plots.²⁸ Finally, we identified the 50% features based on normalized importance score in selected models and depicted them in radar plots as major predictors.

Statistical Analyses

Categorical variables are presented as numbers and percentages. In this study, data were extracted and summarized using Excel 2016. Data processing and model development were done by Python (version 3.7.0). ML algorithms were built based on the scikit-learn package (version 0.22.2). Interaction analyses were used to compare predictive performance between ML models based on pre-operative and combined data. Statistics were performed employing STATA software (version13), and a P value of < .05 indicated a statistically significant difference. All authors had access to the study data and had reviewed and approved the final version of manuscript.

Results

Patient Enrollment Process and Patient Characteristics

As shown in Figure 1, a total of 11 305 patients enrolled in the CHAT-1 study met the inclusion criteria for the current study, with a median in-hospital length of stay of 4 days. Of these, 4477 patients were excluded due to more than 30% missing data, and remaining 6828 patients were eligible in further analysis. Among them, 16 cases developed in-hospital VTE after hernia surgery, while 6812 cases did not. Demographics and characteristics of these patients are presented in Supplemental Table S3. After randomly selecting 1584 non-VTE cases during undersampling process, a total of 1600 patients were included in model establishment, with a derivation cohort of 1280 cases (12 VTE and 1268 non-VTE patients) and a validation cohort of 320 cases (4 VTE and 316 non-VTE patients). Within the derivation cohort, patients with VTE were oversampled to 1268 cases using ADASYN. The demographics and characteristics of patients in both derivation and validation cohorts are summarized in Table 1. Approximately 90% of the patients were male and 20% were aged 75 years or older. Roughly one-third of patients had one or more comorbidities, including hypertension, atrial fibrillation, coronary heart disease, and diabetes. Most of the patients received surgery for reducible hernia (63.53% in derivation cohort and 93.12% in validation cohort) under general anesthesia (others were under local or spinal anesthesia).

Figure 1.

The model development strategy. Abbreviations: AUROC, area under the receiver operating characteristic curve; TabNet, attentive interpretable Tabular learning; VTE, venous thromboembolism.

Table 1.

Demographics and Characteristics of Patients in Modeling.

Variables	Derivation Cohort (n = 2536)	Validation Cohort (n = 320)
Demographics, n (%)
Male	2269 (89.47)	283 (88.44)
Age (years)
≤ 60	1215 (47.91)	131 (40.94)
61-74	907 (35.76)	125 (39.06)
≥ 75	414 (16.32)	64 (20.0)
Smoking	428 (16.88)	70 (21.88)
Body mass index (> 25 kg/m²)	501 (19.76)	83 (25.94)
Disease history, n (%)
Stroke or TIA (< 1 month)	3 (0.12)	0 (0.0)
Pulmonary disease (< 1 month)	2 (0.08)	0 (0.0)
Varicose veins of lower limb	46 (1.81)	1 (0.31)
History of cancer	56 (2.21)	8 (2.5)
History of heart valve replacement	10 (0.39)	2 (0.62)
History of percutaneous coronary intervention	81 (3.19)	4 (1.25)
History of inflammatory bowel disease	29 (1.14)	4 (1.25)
History of venous thromboembolism	116 (4.57)	6 (1.88)
Family history of thrombosis	90 (3.55)	1 (0.31)
Congenital or acquired thrombophilia	71 (2.80)	1 (0.31)
Comorbidities, n (%)
Hypertension	456 (17.98)	56 (17.5)
Atrial fibrillation	47 (1.85)	7 (2.19)
Coronary heart disease	298 (11.75)	25 (7.81)
Diabetes mellitus	90 (3.55)	16 (5.00)
Nephrotic syndrome	4 (0.16)	1 (0.31)
Chronic bronchitis	20 (0.79)	2 (0.62)
Agent therapy, n (%)
Interruption of antithrombotic agents	359 (14.16)	23 (7.19)
Oral contraceptives or hormone replacement	66 (2.60)	0 (0.0)
Hernia type, n (%)
Reducible hernia	1611 (63.53)	298 (93.12)
Recurrent hernia	160 (6.31)	13 (4.06)
Procedure, n (%)
Open operation	1074 (42.35)	153 (47.81)
Time of surgery > 45min	1973 (77.80)	268 (83.75)
Emergency surgery	39 (1.54)	15 (4.69)
General anesthesia	1686 (66.48)	210 (65.62)
Intraoperative blood loss > 10 mL	761 (30.01)	141 (44.06)
Postoperative information, n (%)
Compression time < 24 h	1095 (43.18)	198 (61.88)
Postoperative bleeding	46 (1.81)	11 (3.44)
Postoperative hematoma	185 (2.71)	5 (1.56)
Postoperative white blood cell count ≥ 10*10⁹/L	45 (1.77)	13 (4.06)

Abbreviation: TIA, transient ischemic attack.

Feature Filtering of In-Hospital VTE

According to characteristics listed in Table 1, 24 potential pre-operative predictors were included in feature filtering using SFS algorithm based on LR, SVM, and RF. Finally, a feature set consisting of 11 variables was determined as optimal one with a F1-score of 0.875 and an AUROC of 0.966 in the training set (Supplemental Table S4 and Figure S1A), including sex, age, history of VTE, family history of thrombosis, varicose veins of lower limb, stroke, or transient ischemic attack (TIA) (< 1 month), chronic bronchitis, interruption of antithrombotic agents, oral contraceptives or hormone replacement, recurrent hernia, and reducible hernia. Similarly, a feature set of 15 variables (F1-score of 0.952, AUROC of 0.994) was derived from the combined dataset with 32 potential predictors in the training set (Supplemental Table S4 and Figure S1B). The combined feature set contained aforementioned 10 features (except for chronic bronchitis) and additional 5 features (congenital or acquired thrombophilia, coronary heart disease, anesthesia type, operation type, and compression time).

Model Performance

Across all ML models, the predictive performance using combined data were numerically better than that using only pre-operative data in terms of AUROCs. However, statistical significance was only observed in LightGBM, Catboost, SVM_RBF (P < .05) (Figure 2). For pre-operative data, models with AUROC ≥ 0.60 were considered acceptable, including LR (AUROC: 0.62, 95% confidence interval (CI): 0.55-0.69), SVM_Linear (AUROC: 0.60, 95% CI: 0.53-0.65), TabNet (AUROC: 0.71, 95% CI: 0.68-0.73), and RF (AUROC: 0.60, 95% CI: 0.52-0.66). For combined data, LR (AUROC: 0.67, 95% CI: 0.61-0.72), SVM_Linear (AUROC: 0.65, 95% CI: 0.59-0.72), and TabNet (AUROC: 0.74, 95% CI: 0.70-0.77) demonstrated superior performance for predicting in-hospital VTE (AUROC ≥ 0.65 for each model). Detailed overview of the performance of the considered ML models is described in Supplemental Tables S5 and S6.

Figure 2.

AUROC and 95% confidence intervals for machine learning prediction models. Abbreviations: AUROC, area under the receiver operating characteristic curve; Adaboost, adaptive boosting; Catboost, gradient boosting with categorical features support; LightGBM, light gradient boosting machine; SVM_Liner, support vector machine with linearity kernel; SVM_RBF, support vector machine with radial basis function kernel; TabNet, attentive interpretable Tabular learning; XGBoost, extreme gradient boosting.

Model Interpretation

Feature importance varied across datasets and ML models (Supplemental Tables S7 and S8), and the scores were normalized to the range of 0-1 (Supplemental Tables S9 and S10). In models based on pre-operative data, the features of family history of thrombosis, interruption of antithrombotic agents, and recurrent hernia consistently had a positive impact on developing in-hospital VTE, whereas reducible hernia had a negative impact. More features with consistent results were observed in models based on combined data, including positive factors such as male, history of VTE, family history of thrombosis, varicose veins of lower limb, stroke, or TIA, interruption of antithrombotic agents, and oral contraceptives or hormone replacement. Negative factors included reducible hernia, general anesthesia, open operation, and compression time < 24 h. According to ranking of absolute values, the top 5 predictors were selected from 11 pre-operative variables, and the top 7 predictors were selected from 15 combined variables in each model. Therefore, 7 pre-operative predictors (age ≥ 75 years, history of VTE, family history of thrombosis, chronic bronchitis, interruption of antithrombotic agents, oral contraceptives or hormone replacement, reducible hernia) and 10 combined predictors (male, history of VTE, family history of thrombosis, varicose veins of lower limb, interruption of antithrombotic agents, oral contraceptives or hormone replacement, reducible hernia, open operation, general anesthesia, and compression time < 24 h) were displayed in radar plots (Figure 3).

Figure 3.

Radar plot for the most important predictors of postoperative VTE based on pre-operative (A) and combined data (B). Abbreviations: SVM_Liner, support vector machine with linearity kernel; TabNet, attentive interpretable Tabular learning; VTE, venous thromboembolism.

The patient studied was a male who underwent laparoscopic surgery for recurrent hernia under general anesthesia and received compression for 30 h after the operation. The patient had a history of VTE, varicose veins of lower limb, and a family history of thrombosis. Additionally, he had been taking aspirin for 3 years due to coronary stent implantation. When the TabNet model was applied using the 10 relevant variables for this patient, the predicted probability of developing VTE was 80.4%.

Discussion

Major Findings

In present study, we employed ML techniques to develop predictive models for the occurrence of in-hospital VTE after hernia surgery, using pre-operative data alone or in combination with postoperative data. Generally, models using combined data performed better than those using pre-operative data. Of all ML models, LR, SVM_Linear, and TabNet achieved acceptable performance using both pre-operative data and combined data, and RF also had an acceptable performance using pre-operative data. The predictors contributing most to the model's predictions varied depending on the dataset and the ML model used. Based on feature importance ranking, we identified 7 pre-operative predictors and 10 combined predictors as the most important clinical variables affecting the risk of in-hospital VTE after hernia surgery.

Comparison With the Current Models

Although the incidence of VTE after hernia surgery is relatively low, physicians’ awareness of the possibility of postoperative VTE is often insufficient. VTE episodes are a dangerous complication that can lead to increased healthcare burden and reduced quality of life for patients. Therefore, the development of a predictive model with high accuracy is of great importance. In 2015, Pannucci et al created a risk assessment tool for postoperative VTE based on 113 873 hernia patients from the database of the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP).²⁹ The investigators explored 14 variables by multivariate analysis and developed a model with a moderate performance (c-statistic: 0.79 ± 0.01). In this model, patients were identified from the ACS-NSQIP database, inevitably leaving out certain pivotal predictors that were not involved in database. Another study in 2018 used Cox regression analysis to explore risk factors of postoperative VTE after inguinal hernia repair but did not develop a forecasting model.³ Recently, our group developed the CHAT score using multivariate LR analysis for risk assessment of in-hospital VTE after hernia repair,⁹ which showed superior predictive performance and easier clinical application than the Caprini score. However, conventional multivariate regression has limitations in dealing with complex nonlinear relationship and data with insufficient sample size of positive events. Furthermore, prior studies did not account for the time of data availability in the perioperative continuum. Therefore, separate models derived by ML algorithms using dynamic data in the pre-operative and postoperative phase have the potential to overcome the limitations of existing models.

ML Model Performance

ML methods have gained attention for their ability to learn complex relationships and improve out-of-sample predictions, particularly for predicting postoperative complications.³⁰ For instance, Xue et al³¹ utilized surgical patients’ pre-operative characteristics and text-based clinical data to develop ML models for predicting 5 major postoperative complications, including DVT and PE. As more ML algorithms emerge, investigators are paying increasing attention to comparing the performance of various models. In a study of atrial fibrillation patients from 2 population-based registries, ML models (RF, GBT, and 2 neural networks) for clinical outcomes (death, major bleeding, and stroke) did not perform better than the stepwise LR model.³² Regarding the prediction performance of 9 models in our study, 3 models (LR, SVM_Linear, and TabNet) satisfactorily predicted postoperative VTE in both datasets. Among these models, the nonlinear algorithm TabNet was the best-performing model, with the highest AUROC (0.71, 95% CI: 0.68-0.73 in pre-operative data; 0.74, 95% CI: 0.70-0.77 in combined data), indicating a nonlinear relationship between predictors and VTE events. After further comparison based on datasets, we recognized that adding postoperative variables indeed improved the predictive performance of all ML models, although the difference was marginal. Therefore, predicting postoperative VTE events by using data available before hernia surgery is feasible. Practitioners can use these dynamic predictions to carry out perioperative therapeutic strategies and care management.

Pivotal Features for VTE Prediction After Hernia Surgery

ML models can be challenging to interpret and communicate due to the intricate relationship between feature set and outcome behind the prediction. In present study, we explored a model-agnostic interpretation technique to describe potential clinical factors that contribute to postoperative VTE in hernia patients. Although the feature importance calculated by model interpretation techniques is primarily aimed at data statistics, this study extended the interpretation techniques to facilitate meaningful use in clinical application. Instead of estimating the contributions of features extracted from the original clinical data, we used a 2-step systematic approach to map the features extracted from both pre-operative and intra-operative variables back to the clinical variable to generate clinically meaningful interpretation. Before modeling, the SFS algorithm was used to select relatively vital features from original clinical data. After modeling, importance scores of the selected features were calculated in each model, and pivotal features were subsequently selected and used for clinical interpretation. By leveraging feature-importance analysis and radar map, we generated a visualization format to interpret patient-associated risks based on the clinical variables.

By highlighting significant clinical variables that contribute to the VTE prediction, we detected several key features, including age ≥ 75 years, history of VTE, family history of thrombosis, varicose veins of lower limb, chronic bronchitis, and oral contraceptives or hormone replacement. These factors are also important risk factors in the Caprini score. Moreover, we found that interrupting antithrombotic agents before hernia operation significantly increased the risk of in-hospital VTE in all available models with pre-operative data or combined data. Since patients on antithrombotic agents are often at high risk for VTE, interrupting of anticoagulants could definitely increase the risk of VTE. Additionally, we identified that reducible hernia was associated with a lower risk of postoperative VTE, consistently in both datasets. We also found that open operation and compression time < 24 h were extremely key predictors for decreasing in-hospital VTE after hernia surgery in postoperative models. Previous work has shown that increased intra-abdominal pressure can lead to venous stasis in the lower extremities and dilation of the common femoral vein,^33,34 which can create vein wall microtears and activate the local clotting cascade, resulting in thrombus formation.³⁵ In patients undergoing hernia repair, complicated hernia procedures, laparoscopic surgery, and compression time ≥ 24 h identified as key factors that could increase intra-abdominal pressure. The significant features and their visualized plots can assist practitioners in identifying patients at high risk of developing VTE, enabling preemptive and early intervention to minimize the risk of postoperative VTE.

Clinical Implication and Future Application

Our previous study indicated that only 23.40% of patients with hernia underwent VTE risk assessment by Caprini score.¹ Therefore, developing a specific prediction model for this population would be desirable for clinical practice. Firstly, surgeons should conduct a comprehensive assessment of patient conditions to identify risk factors before the procedure. Patients aged ≥ 75 years, with a history of VTE, family history of thrombosis, those who have received oral contraceptives or hormone replacement, or diagnosed with complex hernia may be at high risk for postoperative VTE. Such patients often take antithrombotic agents routinely, and interrupting these medications could increase the risk of VTE, while continuing them could increase the risk of bleeding. Therefore, heparin bridging may be an option to balance the risk of VTE and bleeding. Secondly, clinical practitioners should tailor surgical and care planning for patients at high risk of in-hospital VTE, taking into account the surgical type, anesthetic mode, and compression time. Thirdly, reassessment for the risk of VTE should be carried out after surgery, and patients at high risk should receive pharmacological prophylaxis for 10 to 14 days after procedure, as guidelines recommended. Fourthly, with pharmacological prophylaxis, such as low-molecular-weight heparin, potential thrombosis, and bleeding events should be closely monitored using D-dimer detection and ultrasonic evaluation. In the future, external validation will be conducted using data from our completed CHAT-3 trial, a prospective multicenter randomized parallel-group study involving 1008 patients with hernia (registration number: ChiCTR2000033769).³⁶ Based on confirmed models, an online risk-assessment tool will be developed to facilitate estimation of risk for postoperative VTE in hernia patients. In conclusion, identifying patients at high risk is of prime importance, and preventive measures should be taken to improve prognosis and accelerate hospital discharge.

Strengths and Limitations

Several strengths warrant address in this study. First, the study adheres to the TRIPOD guidelines and utilizes robust modeling procedures, including thorough data processing, feature filtering, model establishment, and evaluation. Second, the study employs a large multicenter cohort, providing a rich source of data for the creation of a prediction model. Third, the use of ML algorithms allows for the exploration relationship between risk factors and VTE using dynamic data in pre-operative and postoperative phase, separately. However, several limitations also need to be acknowledged. First, as a retrospective study conducted in China, the generalizability of the predictive model to other geographical regions, such as the United States or Europe, is uncertain. Second, only in-hospital VTE events were assessed, inevitably underestimating the overall VTE burden in patients during the entire postoperative period. Third, due to the low incidence of VTE after hernia surgery, sample size between non-VTE and VTE patients is imbalanced. Although undersampling and oversampling were employed to balance the sample size on both side, external validation should be conducted in the future. Besides, the detailed data of the surgery were all obtained from the surgical records, which may contain some bias. The ongoing prospective CHAT-3 trial (registration number: ChiCTR2000033769), involving 1008 patients with hernia, will confirm this prediction model. Lastly, dependencies between different features are not considered in model interpretation approaches, which may introduce a correlation bias.

Conclusions

In summary, we have successfully developed and evaluated ML models for the dynamic prediction of in-hospital VTE events after hernia surgery. Among various algorithms, TabNet exhibited the best performance based on pre-operative or combined data. Based on acceptable-performing models, we have identified several important pre-operative features (such as history of VTE, family history of thrombosis, interruption of antithrombotic agents, oral contraceptives or hormone replacement, reducible hernia) and postoperative features (such as open operation, general anesthesia, and compression time < 24 h) as crucial predictors for VTE prediction. However, further external validation of these models is still needed to confirm their efficacy and generalizability.

Supplemental Material

sj-docx-1-cat-10.1177_10760296231171082 - Supplemental material for Machine Learning to Dynamically Predict In-Hospital Venous Thromboembolism After Inguinal Hernia Surgery: Results From the CHAT-1 Study

Supplemental material, sj-docx-1-cat-10.1177_10760296231171082 for Machine Learning to Dynamically Predict In-Hospital Venous Thromboembolism After Inguinal Hernia Surgery: Results From the CHAT-1 Study by Yi–Dan Yan, Ze Yu, Lan-Ping Ding, Min Zhou, Chi Zhang, Mang-Mang Pan, Jin-Yuan Zhang, Ze-Yuan Wang, Fei Gao, Hang-Yu Li, Guang-Yong Zhang, Hou-Wen Lin, Ming-Gang Wang and Zhi–Chun Gu in Clinical and Applied Thrombosis/Hemostasis

Footnotes

Author Contributions

Z-CG, Y-DY, CZ, M-MP, and H-WL were involved in conception, design, and manuscript writing. M-GW, H-YL, and G-YZ were involved in data acquisition. ZY, MZ, L-PD, J-YZ, Z-YW, and FG were involved in data analysis, model development, and interpretation. All the authors approved the final manuscript.

Data Availability Statement

The data that support the findings of this study are available on reasonable request from the corresponding authors.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical Approval

This study was conducted in accordance with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards, and was approved by the Ethics Committee of Beijing Chaoyang Hospital, Capital Medical University (2018-kd-315).

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by Research project on sustained improvement of evidence-based management of health care quality (YLZLXZ22K032), Clinical Science and Technology Innovation Project of Shanghai Shen Kang Hospital, Development Center (SHDC12021615), Clinical Research Innovation and Cultivation Fund of Ren Ji Hospital (RJPY-LX-008), Research Project of Drug Clinical Comprehensive Evaluation and Drug Treatment Pathway (SHYXH-ZP-2021-001), Ren Ji Boost Project of National Natural Science Foundation of China (RJTJ-JX-001), and Shanghai “Rising Stars of Medical Talent” Youth Development Program—Youth Medical Talents—Clinical Pharmacist Program (SHWJRS (2019) 072).

Informed Consent on Studies With Human Subjects

Written informed consent for hernia surgery was obtained from all patients before the procedure. The need for informed consent was waived because this study was a retrospective design and the patient's data were anonymized.

ORCID iDs

Zhi–Chun Gu

Chi Zhang

Supplemental Material

Supplemental material for this article is available online.

References

Wang

Zhang

Chen

, et al. Current prevalence of perioperative early venous thromboembolism and risk factors in Chinese adult patients with inguinal hernia (CHAT-1). Sci Rep. 2020;10(1):12667.

Dhanani

Olavarria

Holihan

, et al. Robotic versus laparoscopic ventral hernia repair: one-year results from a prospective, multicenter, blinded randomized controlled trial. Ann Surg. 2021;273(6):1076‐1080.

Humes

Abdul-Sultan

Walker

Ludvigsson

West

. Duration and magnitude of postoperative risk of venous thromboembolism after planned inguinal hernia repair in men: a population-based cohort study. Hernia. 2018;22(3):447‐453.

Shah

Wang

Bold

, et al. Nomograms to predict risk of in-hospital and post-discharge venous thromboembolism after abdominal and thoracic surgery: an American College of Surgeons National Surgical Quality Improvement Program analysis. J Surg Res. 2013;183(1):462‐471.

Friedman

. Venous thromboembolism and postoperative management of anticoagulation. Clin Geriatr Med. 2014;30(2):285‐291.

Tadesse

Kedir

Fentie

Abiye

. Venous thromboembolism risk and thromboprophylaxis assessment in surgical patients based on Caprini risk assessment model. Risk Manag Healthc Policy. 2020;13:2545‐2552.

Tian

Cui

Song

. A novel risk assessment model for venous thromboembolism after major thoracic surgery: a Chinese single-center study. J Thorac Dis. 2019;11(5):1903‐1910.

Wang

Wei

Zhou

. A nomogram model to predict the venous thromboembolism risk after surgery in patients with gynecological tumors. Thromb Res. 2021;202:52‐58.

Zhang

Yang

Wang

Zhang

. Prediction model of in-hospital venous thromboembolism in Chinese adult patients after hernia surgery: the CHAT score. Clin Appl Thromb Hemost. 2021;27:10760296211051704.

10.

Javed

Faheem

Asim

Baker

Beg

. A smartphone sensors-based personalized human activity recognition system for sustainable smart cities. Sustain Cities Soc. 2021;71:102970.

11.

Bharti

Khamparia

Shabaz

Dhiman

Pande

Singh

. Prediction of heart disease using a combination of machine learning and deep learning. Comput Intell Neurosci. 2021;2021:8387680.

12.

Hofer

Lee

Gabel

Baldi

Cannesson

. Development and validation of a deep neural network model to predict postoperative mortality, acute kidney injury, and reintubation using a single feature set. NPJ Dig Med. 2020;3:58.

13.

Churpek

Yuen

Winslow

Meltzer

Kattan

Edelson

. Multicenter comparison of machine learning methods and conventional regression for predicting clinical deterioration on the wards. Crit Care Med. 2016;44(2):368‐374.

14.

Collins

Reitsma

Altman

Moons

. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ (Clin Res ed). 2015;350:g7594.

15.

Golemi

Salazar Adum

Tafur

Caprini

. Venous thromboembolism prophylaxis using the Caprini score. Dis Mon. 2019;65(8):249‐298.

16.

Stuck

Spirk

Schaudt

Kucher

. Risk assessment models for venous thromboembolism in acutely ill medical patients. A systematic review. Thromb Haemostasis. 2017;117(4):801‐808.

17.

Wilbur

Shian

. Diagnosis of deep venous thrombosis and pulmonary embolism. Am Fam Physician. 2012;86(10):913‐919.

18.

Stein

Fowler

Goodman

, et al. Multidetector computed tomography for acute pulmonary embolism. N Engl J Med. 2006;354(22):2317‐2327.

19.

Tang

Ishwaran

. Random forest missing data algorithms. Stat Anal Data Min. 2017;10(6):363‐377.

20.

Petrazzini

Naya

Lopez-Bello

Vazquez

Spangenberg

. Evaluation of different approaches for missing data imputation on features associated to genomic data. BioData Min. 2021;14(1):44.

21.

Yang

Yuan

Long

Tong

. Predictive models of medication non-adherence risks of patients with T2D based on multiple machine learning algorithms. BMJ Open Diabetes Res Care. 2020;8(1):e001055.

22.

Coco

Neale

. A predictive model of recreational water quality based on adaptive synthetic sampling algorithms and machine learning. Water Res. 2020;177:115788.

23.

Hatamikia

Maghooli

Nasrabadi

. The emotion recognition system based on autoregressive model and sequential forward feature selection of electroencephalogram signals. J Med Signals Sens. 2014;4(3):194‐201.

24.

Gomez Hernandez

Starman

Davis

, et al. A distinguishing profile of chemokines, cytokines and biomarkers in the saliva of children with Sjogren's syndrome. Rheumatology. 2021;60(10):4765‐4777.

25.

Withanage

MHH

Hernandez

MPG

Starman

, et al. Dataset-chemokines, cytokines, and biomarkers in the saliva of children with Sjogren's syndrome. Data Brief. 2021;36:107139.

26.

Munir

Elahi

Ayub

Frezza

Rizzi

. Cancer diagnosis using deep learning: a bibliographic review. Cancers. 2019;11(9):1235.

27.

Zheng

, et al. Predicting blood concentration of tacrolimus in patients with autoimmune diseases using machine learning techniques based on real-world evidence. Front Pharmacol. 2021;12:727245.

28.

D'Ascenzo

De Filippo

Gallone

, et al. Machine learning-based prediction of adverse events following an acute coronary syndrome (PRAISE): a modelling study of pooled datasets. Lancet. 2021;397(10270):199‐207.

29.

Pannucci

Basta

Fischer

Kovach

. Creation and validation of a condition-specific venous thromboembolism risk assessment tool for ventral hernia repair. Surgery. 2015;158(5):1304‐1313.

30.

Shameer

Johnson

Glicksberg

Dudley

Sengupta

. Machine learning in cardiovascular medicine: are we there yet? Heart (British Cardiac Society). 2018;104(14):1156‐1164.

31.

Xue

, et al. Use of machine learning to develop and evaluate models using preoperative and intraoperative data to identify risks of postoperative complications. JAMA Netw Open. 2021;4(3):e212240.

32.

Loring

Mehrotra

Piccini

, et al. Machine learning does not improve upon traditional regression in predicting outcomes in atrial fibrillation: an analysis of the ORBIT-AF and GARFIELD-AF registries. Europace. 2020;22(11):1635‐1644.

33.

Losken

Carlson

Tyrone

, et al. The significance of intraabdominal compartment pressure after free versus pedicled TRAM flap breast reconstruction. Plast Reconstr Surg. 2005;115(1):261‐263.

34.

Huang

Bajaj

Gupta

Petersen

Miles

DAG

. Increased intraabdominal pressure in abdominoplasty: delineation of risk factors. Plast Reconstr Surg. 2007;119(4):1319‐1325.

35.

Kakkos

Nicolaides

Caprini

. Interpretation of the PREVENT study findings on the adjunctive role of intermittent pneumatic compression to prevent venous thromboembolism. Ann Transl Med. 2020;8(11):725‐725.

36.

Cao

Zhang

, et al Rationale and design of a prospective, multicenter, randomized controlled trial of postoperative venous thromboembolism prophylaxis in Chinese adult patients with inguinal hernia (CHAT-3 trial). Ann Palliat Med. 2021;10(10):11141-11147. doi:10.21037/apm-21-1594

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.13 MB