Machine learning algorithms for population-specific risk score in coronary artery bypass grafting

Abstract

Background

The aim of this study was to develop a new risk prediction score (NH Score) for patients undergoing coronary artery bypass grafting (CABG) specific to the Indian population and compare it to the Society of Thoracic Surgeon (STS) Score and the EuroSCORE II.

Method

The baseline features of adult patients who underwent CABG between the years 2015 and 2021 (n = 6703) were taken and split into training data (2015–2020; n = 5561) and validation data (2020–2021; n = 1142). The CatBoost algorithm was trained to predict risk scores (NH score), and the performance was tested on the validation set by Precision-Recall Curve and F1 Score. Model calibration was measured by the Brier Score, Expected Calibration Error and Maximum Calibration Error.

Results

The NH score outperformed both the STS and EuroSCORE II for all outcomes. For mortality, the PR AUC for NH Score was (0.463 [95% confidence interval [CI], 0.28–0.64]) compared to 0.113 [95% CI, 0.04–0.22] for the STS score and 0.146 [95% CI, 0.06–0.31] for the EuroSCORE II (p ≪ 0.0001). With respect to morbidity NH Score was superior to the STS score (0.43 [95% CI, 0.33–0.50]) vs. (0.229 [95% CI, 0.18–0.3, p < 0.0001). The observed to the predicted ratio for NH score was superior to the STS Score and similar to EuroSCORE II. NH Score was also more accurate at predicting the risk of prolonged ventilation compared to the STS Score.

Conclusion

NH score shows an excellent improvement over the performance of STS score and EuroSCORE II for modelling risk predictions for patients undergoing CABG in Indian population. It warrants further validation for larger datasets.