Abstract
Objectives
Chronic obstructive pulmonary disease (COPD) is a common respiratory disorder. Acute exacerbation of COPD (AECOPD) severely affects patients’ quality of life and prognosis. This study aimed to identify novel risk factors and develop an effective predictive model for AECOPD using machine learning (ML) models.
Methods
In this retrospective single-center study, clinical data and biomarkers from 565 participants were analyzed using ML algorithms. Feature selection employed least absolute shrinkage and selection operator regression. Eight ML models were trained and evaluated using receiver operating characteristic (ROC) and clinical decision curve analysis. The Shapley Additive explanations (SHAP) framework assessed feature contributions. An online personalized risk calculator was developed based on the optimal model and individual SHAP values.
Results
The XGBoost model demonstrated excellent discriminative performance, with areas under the ROC curve of 0.818 and 0.838 for the training and test sets, respectively. Key predictors identified by SHAP analysis included age, current smoking status, frequency of exacerbations in the previous year, albumin levels, sarcopenia index, and COPD Assessment Test score. These variables were integrated into an online calculator for research to illustrate individualized AECOPD risk estimation. However, external validation is still required before its clinical application.
Conclusions
We developed a preliminary ML model for predicting AECOPD, which provides a valuable tool for clinical risk assessment. The results also highlighted the correlation between sarcopenia and AECOPD risk.
1. Introduction
Chronic obstructive pulmonary disease (COPD) is a serious public health issue worldwide. Its prevalence and mortality rates remain persistently high, causing a huge economic and social burden. 1 Acute exacerbation of COPD (AECOPD) is a critical clinical complication in the natural progression of the disease and is strongly associated with deterioration in health status. 2 Previous studies have indicated that approximately 20% of patients with COPD experience frequent acute exacerbations requiring recurrent hospitalization for treatment. 3 Therefore, preventing AECOPD is one of the main treatment goals for patients with COPD.
The risk of AECOPD is influenced by various factors, including patient demographics, clinical characteristics, and biomarkers. 4 Numerous studies have shown that a considerable proportion of patients with COPD also present with sarcopenia, which is associated with poor clinical outcomes. 5 Notably, sarcopenia indicators (assessed using serum creatinine (Cr) and cystatin C (CyC) levels) have demonstrated predictive value for AECOPD occurrence and prognosis. 6 However, current clinical prediction still relies heavily on single-factor indicators, such as clinical parameters and pulmonary function tests, 7 which are inadequate for accurate and comprehensive risk assessment. Integrating sarcopenia indicators with conventional risk factors may facilitate early identification and prevention of exacerbations.
Interpretable machine learning (ML) models have been widely applied in disease prevention, diagnosis, and treatment, and have demonstrated superior predictive performance. 8 The advantage of ML techniques is their capacity to capture complex nonlinear relationships within datasets, thereby enhancing the accuracy and reliability of predictions. Moreover, ML enables feature-based interpretation of decisions for individual patients. 9 Applying ML to AECOPD prediction allows for more precise identification of high-risk individuals based on complex clinical data and biomarkers.
Therefore, this study aimed to establish a reliable ML model to assist clinicians in identifying patients at high risk of AECOPD and to provide an online personalized risk calculator.
2. Methods
2.1. Study design and participants
A schematic of the study methodology is presented in Figure 1. This retrospective study was approved by the Ethics Committee of the Affiliated Hospital of Gansu University of Chinese Medicine. Between January 2020 and November 2023, 565 patients with COPD from the respiratory department of our hospital were included. Research process flow diagram.
The inclusion criteria were as follows: (1) a confirmed diagnosis of COPD in accordance with the 2020 version of the Global Initiative for COPD (GOLD) guidelines; (2) participants who had complete medical records at the hospital and adhered to regular follow-up visits (within one year); (3) patients were included either in stable condition or during hospitalization for AECOPD (for the latter, the index exacerbation was not counted in the “previous year” history); and (4) no use of antibiotics or hormone-based medications within 1 month prior to admission. The exclusion criteria were as follows: (1) patients with incomplete clinical data and prognostic information; (2) patients with concurrent inflammatory diseases in other organs, such as bronchial asthma, autoimmune diseases, and inflammatory bowel disease; (3) patients who developed new symptoms during hospitalization and follow-up and were transferred to other departments due to these new symptoms; and (4) patients with contraindications for pulmonary function tests or those who were unable to undergo pulmonary function tests despite repeated instructions.
Sample size considerations: to fit a prediction model, each variable needs at least 10 events per variable (EPV). We evaluated 23 variables in the ML model, thus, the sample size in the derivation stage was at least 210 events.
2.2. Outcomes
The main outcome was the occurrence of AECOPD, as assessed using the 2022 GOLD guidelines. AECOPD was defined as an acute worsening of respiratory symptoms (e.g., increased dyspnea on exertion, worsened cough, increased sputum volume, or a change in sputum character) that necessitated a change in regular medication and/or hospitalization after exclusion of other known causes of respiratory deterioration, such as acute coronary syndrome, cardiac arrhythmia, pneumothorax, pleural effusion, or pulmonary thromboembolism. 10
2.3. Model predictors and feature selection
To identify potential predictive variables, we conducted a comprehensive literature review of previously identified variables, and those of significant importance were selected as predictive variables. In addition to conventional risk factors, we observed that sarcopenia is associated with the occurrence of AECOPD, which has been increasingly supported by numerous recent studies. 11 Therefore, we included assessment indicators of sarcopenia as potential variables (the ratio of serum Cr levels to serum CyC levels) 12 and classified the sarcopenia index (SI) into high and low groups based on the median value (92.45). To assess robustness, we further conducted sensitivity analyses by treating SI as a continuous variable. In summary, 21 features were included, covering patients’ sociodemographic characteristics, clinical features, biochemical tests, and pulmonary function examinations.
Missing data were present in only 6 cases (0.78%), and no outliers were observed. Given the very low proportion of missingness (<1%) and that data were missing completely at random, we excluding these 6 participants. The dataset was randomly partitioned, with 70% of the samples allocated to the training set for model development and learning, and 30% designated as the test set for the final evaluation of model performance. Baseline characteristics were compared between the training and test sets to assess distribution consistency. After randomly partitioning the data, least absolute shrinkage and selection operator (LASSO) regression was applied exclusively to the training set for feature selection. The selected features were then fixed and used to train all machine learning models on the training set, with performance evaluated on the independent test set. The LASSO method operates by adjusting the regularization parameter to control the intensity of the penalty term, thereby achieving coefficient shrinkage and automated feature selection. This process not only promotes sparsity in the model coefficients (i.e., making some coefficients tend to zero) but also automatically eliminates features with minor contributions to predictive performance, while retaining the most important features that enhance the model’s predictive capability.
2.4. Model development and evaluation
The applicability and effectiveness of eight ML algorithms were explored: logistic regression (LR), k-nearest neighbors (KNN), support vector machine (SVM), random forest (RF), extreme gradient boosting (XGBoost), neural network (NNET), linear discriminant analysis (LDA), and partial least squares (PLS). The hyperparameters were optimized using ten-fold cross-validation combined with a grid search to identify the optimal parameter combinations for each model. This process involved evaluating the predictive performance of each combination of hyperparameters in the training set, thereby gradually refining and optimizing the range of hyperparameter selection (Table S5). The selected algorithmic model was trained and fine-tuned using the training dataset to optimize its predictive capability. Where appropriate, synthetic minority over‐sampling technique (SMOTE) was applied to the training set to address class imbalance. The optimal model was selected based on its area under the receiver operating characteristic curve (AUROC), which evaluated the overall discriminative ability across classification thresholds. Additionally, key classification metrics, including sensitivity, specificity, precision, Youden’s index, positive predictive value (PPV), negative predictive value (NPV), and F1-score, were utilized to provide a nuanced understanding of model performance across different clinical scenarios. The calibration of the model was evaluated using the calibration curve and the Brier score. Clinical utility was quantitatively evaluated using decision curve analysis (DCA), which estimated the net benefit across different threshold probabilities to facilitate clinical decision-making. To obtain robust performance estimates and account for optimism, internal validation was performed using a 1000-times bootstrap procedure for the optimal model.
2.5. Variable importance and risk calculator
This study aimed to understand the impact of different variables on AECOPD occurrence. We compared the performance of the models constructed using the eight ML algorithms and selected the algorithm with the best performance as the optimal prediction model. To quantify the predictive value of the SI, we compared the full XGBoost model (including all six predictors) with a baseline logistic regression model that excluded SI. The net reclassification improvement (NRI) and integrated discrimination improvement (IDI) were calculated to quantify the incremental benefit of SI. To enhance the interpretability of this model, we used the Shapley Additive explanations (SHAP) framework to conduct an interpretability analysis. Finally, to convert the optimal model into a simple application tool, an online risk prediction model calculator was developed using the Shiny application-based web platform.
2.6. Statistical analysis
Continuous variables were expressed as mean ± standard deviation (SD) and compared using Student’s t-test or the Mann–Whitney U test for normally and non-normally distributed data, respectively. Categorical variables were presented as percentages (%) and analyzed using the Chi-square (χ2) or Fisher’s exact test. The ML models were developed in R using the ‘caret’ package. All statistical analyses were performed using SPSS (v25.0) and R (v4.5.0).
3. Results
3.1. Clinical features
Comparison of baseline characteristics in the Non-AECOPD and AECOPD groups.
Values are expressed as means ± standard deviations, or percentages.
BMI, Body Mass Index; CRP, C-reactive Protein; SI, Sarcopenia Index; FEV, Forced Expiratory Volume; FVC, Forced Vital Capacity; COPD, Chronic Obstructive Pulmonary Disease; GOLD, Global initiative for chronic Obstructive Lung Disease. AECOPD, Acute Exacerbation of COPD.
3.2. Key predictors of AECOPD
When λ.1SE in the LASSO model was set to 0.039, six variables with nonzero coefficients were identified by the LASSO algorithm (Figure 2(a) and (b)). This value of λ was chosen because, although the minimum binomial deviance corresponds to a lower λ, it results in too many included variables, potentially reducing the model’s generalization ability. The variables were considered key variables for AECOPD and used for subsequent ML modeling, including age, current smoking status, frequency of exacerbations in the previous year, ALB level, SI, and CAT score. Feature selection and model performance evaluation. (a, b) LASSO regression analysis for feature selection. (c, d) The AUC values demonstrating the model’s performance in the training set and the test set.
3.3. Comparison of multiple classification models
Performance metrics between eight machine learning models in test set.
PPV, Positive Predictive Value; NPV, Negative Predictive Value; LR, logistic regression; KNN, K-Nearest Neighbours; SVM, Support Vector Machine; RF, Random Forest; XGBoost, eXtreme Gradient Boosting; NNET, Neural Network; LDA, Linear Discriminant Analysis; PLS, Partial Least Squares.
3.4. Feature importance
The contribution of each input variable to the model output was assessed using the SHAP feature importance metric. Furthermore, adding SI to the model resulted in a significant incremental benefit than the baseline logistic regression model (NRI = 0.213, IDI = 0.037), indicating that SI provides clinically meaningful incremental predictive value beyond traditional risk factors. Sensitivity analysis using continuous SI yielded similar results (test AUROC 0.821 vs. 0.838) (Table S6). A scatter plot of SHAP values was used to illustrate the specific impact of each variable on the model output in the study population. The distribution of the SHAP values for each feature was also visualized (Figure 3 and Figure S2). Interpretation of the prediction model using SHAP. (a) Feature importance ranking based on mean absolute SHAP values. (b) SHAP beeswarm plot, where each point represents a Shapley value for a feature and an instance. The color indicates the feature value (red: high, blue: low), and the position shows the impact on the prediction. (c) SHAP force plot for a single representative sample, illustrating how each feature contributes to pushing the model’s output from the base value to the final prediction.
3.5. Online calculator
Based on the key predictors and the optimal XGBoost model, an online calculator was developed to assist clinicians in estimating individualized AECOPD risk in clinical practice (Figure 4; accessible at: https://haochen113.shinyapps.io/AECOPD/). The calculator visually displayed the SHAP values for each patient and explained the personalized risk. The personalized risk calculator and patient illustration. The online application included a panel for entering parameters and an interface for displaying the probability of AECOPD and the contributions of the predictors.
4. Discussion
In this study, we developed an interpretable ML model to assess the risk of AECOPD in older adults with COPD. The XGBoost model achieved the highest AUROC values, demonstrating consistent performance across both the training and test sets and the Brier scores and calibration curves indicated that the predicted risks were well-calibrated. Furthermore, a web-based calculation tool utilizing the XGBoost model was developed.
AECOPD is a critical event in COPD progression, 13 associated with accelerated lung function decline, 14 and high re-hospitalization rates (30–50% within 1 year). 15 The primary objective of this study was to investigate and identify the clinical factors and biomarkers closely associated with the occurrence of AECOPD. Aging is a well-established risk factor for AECOPD, 16 with every decade of age increasing the risk of exacerbation by 21%. 17 A large-scale study confirmed a dose-response relationship between accelerated biological age and the risk of AECOPD (hazard ratio = 1.18 per year). 18 Our finding that older patients with COPD are at a higher risk of subsequent acute exacerbations (77 vs. 70 years) is highly consistent with those of previous studies.
The occurrence and development of COPD and the risk of acute exacerbation are related to smoking. 19 A prospective study revealed that patients with COPD who smoked (>36 pack-years) had a higher risk of frequent AECOPD (odds ratio = 1.08). 20 Smoking cessation can alleviate the deterioration of lung function, reduce exacerbation frequency, and significantly improve the survival rate of patients with COPD. 21
Prior exacerbation frequency strongly predicts future AECOPD risk within 1 year of discharge. In a previous study, up to 70% and 90% of the research population in the GOLD groups C and D, respectively, experienced AECOPD, with >67% of patients in group D experiencing AECOPD or death within 5 years. 22 Lin et al. 23 used receiver operating characteristic (ROC) curves to analyze the risk of future acute exacerbations in patients with COPD and a history of AECOPD. Their model achieved an AUROC of 0.689, with a sensitivity of 75.3% and a specificity of 72.7%. Therefore, the management of patients with COPD and a history of AECOPD should be a key focus.
SHAP analysis identified the CAT score as the stronger predictor of AECOPD. A CAT score ≥ 15 can predict the risk of acute exacerbation in the future. 24 A previous study reported that patients in the frequent AECOPD group had significantly higher CAT scores than those in the control group (24.8 ± 6.7 vs. 17.5 ± 6.5). 25 Similarly, the present study showed that frequent exacerbators were characterized by elevated baseline CAT scores (15.94 ± 8.23 vs. 12.94 ± 7.70). A multicenter prospective study confirmed that high CAT scores shorten the time to the first exacerbation and increase the exacerbation risk (risk ratio = 1.50), independent of other factors. 26 For patients with COPD and a high CAT score at admission, the risk of future AECOPD is elevated, making personalized medical or non-pharmaceutical treatment particularly important, along with regular follow-up evaluations.
Sarcopenia is common in patients with COPD and is an important variable for assessing the risk of acute exacerbations, hospitalization, and mortality. 27 Degeneration of the respiratory muscles, especially the diaphragm and intercostal muscles, directly affects respiratory function. 28 Currently, the methods commonly used for measuring skeletal muscle mass include dual-energy X-ray absorptiometry, bioelectrical impedance analysis, and imaging-based estimations of the cross-sectional area of muscle using techniques such as computed tomography (CT) and magnetic resonance imaging. 29 However, these measurement tools have some limitations, such as radiation exposure, poor feasibility of longitudinal monitoring, and issues related to cost-effectiveness. Numerous recent studies have demonstrated that SI can reflect the degree of skeletal muscle atrophy and is significantly correlated with muscle mass measured using CT imaging. 30
The SI is a cost-effective and easily applicable indicator. Patients with COPD who have a low SI are more likely to experience frequent exacerbations and a worse prognosis. Hirai et al. 31 suggested that the SI is more effective (AUROC = 0.87) than other biomarkers for predicting sarcopenia. The present study results clearly demonstrated significant NRI and IDI, further proving that SI provides additional prognostic information compared to traditional factors. We recognize that a median split may result in loss of some information, but we chose it for clinical simplicity and comparability with other studies. Low SI is significantly associated with the risk and clinical prognosis of severe AECOPD. Sensitivity analysis result supported the robustness of our findings, future studies should validate an optimal SI cut-off. Moreover, serum ALB levels are closely related to nutritional status. A serum ALB level below 36.15 g/L can lead to impaired immune function and heightened inflammatory responses, resulting in an increased risk of acute exacerbations. 32
This study has some limitations. First, the study was conducted retrospectively with a small sample size and could not establish a cause-effect inference relationship between key predictors and AECOPD, therefore, our findings should be considered exploratory. Future research should adopt a longitudinal design to overcome this limitation. Second, although cross-validation and internal testing indicated a low risk of overfitting, larger samples are required to optimize the generalizability of the model. Third, the ML model has not been validated with external data, and multicenter prospective studies are warranted to verify its efficacy. Furthermore, the inclusion of model variables depends on the completeness of clinical data and literature screening. Although the LASSO method can effectively reduce redundant features, there remains a possibility that some important factors may be overlooked, potentially affecting the interpretability of the model. Finally, the influence of different diagnostic criteria for sarcopenia on the results was not compared.
5. Conclusions
In conclusion, this exploratory study incorporated sarcopenia and other factors into a risk-prediction model for AECOPD. Through a systematic comparison of multiple mainstream ML algorithms, we developed a stable and highly discriminative XGBoost prediction model, along with a visual online risk calculator. Future studies should validate this model in larger-scale, multicenter prospective cohorts to further optimize its accuracy and applicability.
Supplemental material
Supplemental material - Explainable machine learning model for predicting acute exacerbations of COPD combining sarcopenia index and traditional risk factors: A retrospective single-center exploratory study
Supplemental material for Explainable machine learning model for predicting acute exacerbations of COPD combining sarcopenia index and traditional risk factors: A retrospective single-center exploratory study by Ai-Bin Zhang, Li-Wen Zhou, Yu-Fen An, Qing-Qing Qin, Jian-Tong Wei and Hao Chen in Chronic Respiratory Disease.
Supplemental material
Supplemental material - Explainable machine learning model for predicting acute exacerbations of COPD combining sarcopenia index and traditional risk factors: A retrospective single-center exploratory study
Supplemental material for Explainable machine learning model for predicting acute exacerbations of COPD combining sarcopenia index and traditional risk factors: A retrospective single-center exploratory study by Ai-Bin Zhang, Li-Wen Zhou, Yu-Fen An, Qing-Qing Qin, Jian-Tong Wei and Hao Chen in Chronic Respiratory Disease.
Footnotes
Acknowledgements
We appreciate the editor and reviewers for their valuable feedback, which has contributed to improving this paper.
Ethical considerations
Given the retrospective nature of the study, written consent was not obtained. However, all patient records were anonymized prior to analysis. Then, related data were extracted from the hospital’s electronic and written medical records. The study was reviewed and obtained the approval from Ethics Committee of the Affiliated Hospital of Gansu University of Chinese Medicine.
Author contributions
Hao Chen designed the study. Ai-Bin Zhang wrote the assay. Jian-Tong Wei and Qing-Qing Qin carried out data analyses. Yu-Fen An and Li-Wen Zhou contributed to gather the data and helped with the data analysis again. All authors agree to be fully accountable for ensuring the integrity and accuracy of the work, and read and approved the final manuscript. The corresponding author supervised and reviewed the manuscript.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funded by Science and Technology Planning Project of Gansu Province (25JRRG006, 25JRRA841) and Healthcare Industry Research Projects of Gansu Province (GSWSQN2024-21).
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability Statement
The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
