Machine learning–based prediction of mortality in lung cancer: Application of severity-adjustment method

Abstract

Background

Lung cancer is a leading cause of cancer-related deaths globally, affecting 2.2 million people and causing approximately 1.8 million deaths annually. The 5-year survival rate remains low due to risk factors such as aging, smoking, and air pollution, which also impose significant financial burdens on healthcare systems. Early mortality prediction can improve patient outcomes and reduce costs.

Objective

This study aimed to develop mortality prediction models for lung cancer patients using machine learning techniques by comparing severity adjustment tools—Charlson Comorbidity Index (CCI), Age-adjusted Charlson Comorbidity Index (ACCI), and Elixhauser Comorbidity Index (ECI).

Methods

Data from the National Hospital Discharge Injury Survey (2006–2018) were analyzed for 24,472 lung cancer patients diagnosed with ICD-10 codes C33–C34. Machine learning methods, including logistic regression, decision trees, random forests, XGBoost, support vector machines, and neural networks, were applied. Models were evaluated using area under the curve (AUC), sensitivity, specificity, accuracy, and F1 score.

Results

Random forests performed best in the training set, while XGBoost and neural networks showed superior performance in the test set. Key predictors of mortality included admission route, length of stay, and surgery status.

Conclusions

XGBoost and neural network models demonstrated strong performance in predicting mortality among lung cancer patients. This study represents one of the first comparative analyses to evaluate machine learning–based mortality prediction models incorporating multiple comorbidity severity indices (CCI, ACCI, and ECI), providing evidence to support the selection of optimal severity adjustment tools and machine learning algorithms. Implementing these models, in combination with primary care and remote monitoring strategies after hospital discharge, may help reduce emergency room visits and hospital stays, thereby improving survival outcomes and alleviating healthcare costs.

1. Introduction

Lung cancer affects 2.2 million people annually and results in approximately 1.8 million deaths, making it one of the cancers with the highest incidence and mortality worldwide. ¹ The overall 5-year survival rate for lung cancer is considerably low (<20 %), making it a major cause of cancer-related deaths. ² As risk factors such as population aging, smoking, and air pollution increase, the number of lung cancer patients will continuously increase, which adds a considerable cost burden to the global medical system. ³ Lung cancer is a representative multifactorial disease in which genetic susceptibility, environmental exposures, and lifestyle factors interact, resulting in substantial clinical and biological heterogeneity among patients. Owing to this heterogeneity, considerable variability in prognosis and treatment response is observed even among patients with the same disease stage or treatment strategy, and conventional clinical indicators alone are insufficient to adequately explain mortality risk. Although treatment strategies for lung cancer have expanded with the introduction of targeted therapies and immunotherapies, treatment responses remain limited in a substantial proportion of patients, and survival rates tend to decline as the disease is diagnosed at more advanced stages.

To address these challenges, the application of artificial intelligence (AI) technologies, including machine learning and deep learning, has recently increased in lung cancer research. Recently, various studies have validated the performance of early detection and precise classification of lung cancer by utilizing ensemble learning frameworks and optimized deep learning models,^4,5 Furthermore, these AI technologies have proven effective in identifying complex pulmonary lesions, such as interstitial lung disease. ⁶ In this context, AI enables the integrated analysis of large-scale clinical and genomic data, allowing the identification of complex biological patterns and risk factors that are difficult to capture using traditional approaches.^7–9 In particular, machine learning–based methods can reflect the complex pathophysiology of lung cancer and inter-patient heterogeneity, thereby enabling more precise prediction of mortality risk and early identification of high-risk patients. Therefore, the development and validation of machine learning-based mortality prediction models.

Mortality is an important measure of health outcomes within a community and is used as a representative indicator of the country’s health level. Reducing the mortality rate can lower medical expenses by enabling disease prevention and early diagnosis, thereby facilitating the establishment of more effective policies. ¹⁰ In particular, in diseases such as lung cancer, in which the survival rate decreases to less than 5% and medical expenses increase as the disease progresses to the final stage, early management through mortality prediction can reduce the burden on patients and caregivers. Comorbidities, which are individual patient characteristics and important predictors of their health outcomes, are used to measure mortality rates more accurately.^11,12 The survival of patients with lung cancer can vary depending on comorbidities, such as cardiovascular disease and chronic lung disease, in addition to age and smoking. Lung cancer patients with comorbidities may not be eligible for surgery, which can reduce treatment effectiveness. Thus, comorbidities are important predictors of survival for lung cancer patients.^13,14

Representative methods for estimating morbidity and mortality rates by evaluating comorbidities include the CCI, ACCI, and ECI. The CCI was developed in 1984 to evaluate mortality rates and uses a compensation method by assigning weights to 19 comorbidities. ¹² Subsequently, Deyo (1992) ¹⁵ revised the 19 disease categories into 17 categories, and Quan et al. (2005) ¹⁶ converted these 17 modified diagnostic codes into an algorithm based on ICD-10 codes. The ACCI employs a method that includes the patient’s age in the CCI value, which more accurately predicts the postoperative mortality rate as it considers both age and comorbidities.^17–19 The ECI was developed to measure 30 comorbidities using the Elixhauser comorbidity system. ²⁰ Van Walraven et al. (2009) ²¹ modified this to the Elixhauser comorbidity index, a single-weight score that has since been used as the Thompson and AHRQ ECI.^22,23

We developed a mortality prediction model for each comorbidity severity compensation tool. Thus far, we have applied machine learning techniques, which are more effective analysis methods for prognosis prediction than traditional statistical methods.^24–26 In the medical field, machine learning analysis techniques are currently utilized for accurate diagnosis, severity risk prediction, and prognosis prediction; however, there is a lack of verification of mortality prediction, which implies patient survival.^27,28 In particular, few studies have been conducted that compared machine learning analysis techniques to propose a prediction model suitable for patients with lung cancer. Previous studies have applied machine learning approaches to predict survival outcomes, treatment response, and postoperative prognosis in patients with lung cancer, and have demonstrated improved predictive performance compared with traditional statistical models.^29,30 However, many of these studies have focused on specific clinical settings or limited sets of predictors, which restricts systematic comparisons across different machine learning algorithms and comprehensive evaluations of mortality prediction models that incorporate comorbidity severity. Widely used comorbidity severity adjustment tools, such as the CCI, ACCI, and ECI, have primarily been applied within conventional regression-based frameworks to predict mortality or healthcare utilization. In contrast, studies that apply these comorbidity severity indices to machine learning models and comparatively evaluate their predictive performance for mortality among lung cancer patients remain very limited. Accordingly, in this study, we used machine learning analysis techniques to identify the factors affecting mortality by comparing the mortality prediction models for each severity-compensating tool (CCI, ACCI, and ECI). Thus, we suggest excellent severity-compensating tools and prediction model techniques to predict the mortality of lung cancer patients.

2. Materials and methods

2.1. Study design and database

This study used a total of 13 years of data from the Korea National Hospital Discharge Injury Survey (KNHDIS) hosted by the Korea Disease Control and Prevention Agency from 2006 to 2018. The KNHDIS is operated by the Korea Disease Control and Prevention Agency, which surveys all patients discharged from medical institutions with more than 100 beds nationwide, including general hospitals, hospitals, and medical centers, as part of the domestic injury surveillance project. The database is managed by the Korea Disease Control and Prevention Agency (KDCA), and the raw data were obtained after submitting a data use request through the National Injury Information Portal (https://www.kdca.go.kr/injury/biz/injury/main/mainPage.do). The study subjects were patients with lung cancer diagnosed with C33-C34 of ICD-10-based main diagnosis codes. Among them, 24,472 patients aged 14 or older who responded to all items of Sex, Age, Admission Route, Surgery, Type of Insurance, LOS, Number of Beds, and Hospital Location were selected as the final research subjects (Figure 1).

Figure 1.

Flowchart of the study cohort from the KNHDIS.

2.2. Comorbidity assessment

To account for comorbidity-based severity, secondary diagnosis variables were utilized. Rather than simply listing individual complications, we calculated standardized comorbidity indices—the CCI, ACCI, and ECI—to comprehensively reflect patient health status. These three indices were not limited to diagnostic codes from a single discharge point; instead, they incorporated all ICD-10 codes for primary and secondary diagnoses recorded throughout the patient’s relevant clinical period. Weights were assigned based on the standardized algorithms originally proposed by Charlson et al. (1987) ³¹ and subsequently refined by Quan et al. (2005, 2011).^32,33 For precise calculation, all indices were derived using the validated ICD-10-based Excel dataset calculator developed by Prommik et al. (2022). Due to the nature of claims data, cases where specific ICD codes were absent were regarded as the absence of the corresponding condition. ³⁴

The CCI is a validated tool for quantifying comorbidity burden to predict 10-year mortality. ³⁵ This study applied the updated weighting system by Quan et al. (2011), which recalibrates historical weights using contemporary clinical data. ³³ In this system, weights ranging from 0 to 6 are assigned to 17 comorbidity categories based on their association with mortality risk. Total scores were calculated as the sum of these weights and categorized into 0, 1, 2, and ≥3 points, following conventional classification methods. ³⁶

The ACCI measures mortality risk by adding age-related weights to the baseline CCI. Starting from age 40, one additional point was assigned for every 10-year increase in age (≤49: 0; 50–59: 1; 60–69: 2; 70–79: 3; 80–89: 4; 90–99: 5; and ≥100: 6 points). ³⁷

The ECI is based on Elixhauser’s 31 comorbidity categories and demonstrates superior predictive power for in-hospital mortality. ³⁸ Unlike the CCI, it includes psychological comorbidities such as depression, as well as cardiac arrhythmias and valvular diseases. Weights were assigned according to the system by Moore et al. (2017), ³⁹ with individual conditions assigned weights ranging from -7 (drug abuse) to +14 (metastatic cancer) to determine the final composite score.^40,41

2.3. Data processing

As for the dependent variable of this study, the ‘treatment outcome’ variable was utilized among the data provided by the KNHDIS regarding the presence or absence of mortality upon discharge. Improved (complete or light recovery) was defined as not dying, while hopelessness (discharge for dying) and mortality were defined as death. As independent variables, comorbidity severity compensation tools (CCI, ACCI, and ECI), sex, age, admission route, presence of surgery, type of insurance, LOS, number of beds, and hospital location were used. The comorbidity severity compensating tools and age were defined as a ‘Continuous’ type. The admission route was divided into Emergency and Outpatient groups, and surgery was classified based on the type of surgical resection performed (Pneumonectomy, Bilobectomy, Lobectomy, Segmentectomy, and Wedge resection). Insurance types were divided into medical and health insurance subscribers. In addition, the LOS was calculated by subtracting the admission date from the discharge date, which was defined as a ‘Continuous’ type. Hospital location was classified into Seoul, metropolitan cities (including Gyeonggi-do), and others, based on the region where the hospital is located. The number of beds was divided into 100-299 beds, 300-499 beds, 500-999 beds, and over 1,000 beds (Table 1).

Table 1.

Classification and definition of variables.

Variable	Definition
Death	0 = No
	1 = Yes
Sex	0 = Male
	1 = Female
Age	Continuous
Admission Route	0 = Emergency
	1 = Outpatient
Surgery	0 = No
	1 = Yes
Type of Insurance	0 = Assistance
	1 = National Health Insurance
LOS a	Continuous
Hospital Location	0 = Greater Seoul
	1 = Metropolitan City (including Gyeonggi-do)
	2 = Other
Number of Beds	0 = 100-299
	1 = 300-499
	2 = 500-999
	3 = ≥1000
CCI	Continuous
ACCI	Continuous
ECI	Continuous

^aLOS: Length of Stay.

2.4. Statistical analysis

Data preprocessing was performed using SPSS version 23.0 and Excel, while the development and validation of mortality prediction models for lung cancer patients were conducted using R version 4.0.5. All analyses were carried out on a Windows 11-based workstation equipped with an Intel i7-9700 CPU and 32GB of RAM. Memory usage was optimized during model training to ensure the stability and reproducibility of the large-scale administrative claims data analysis.

First, frequency analysis was performed to identify the general characteristics of the study population. Second, to develop the mortality prediction models, we utilized six algorithms: logistic regression (LR), decision tree (DT), random forest (RF), XGBoost, support vector machine (SVM), and neural network (NN). The total dataset was randomly split into a training set (70%) and a testing set (30%). Cross-validation was not applied in this study; this was a strategic choice given that the large-scale administrative claims data provided a sufficient sample size, and it allowed for a consistent and reproducible comparison of the predictive performance across the three comorbidity indices (CCI, ACCI, and ECI) under identical data-splitting conditions. Hyperparameters for each model were initially set based on standard default values validated in prior literature and subsequently optimized through performance comparisons using an internal validation subset.^35,42 Detailed hyperparameter configurations are presented in Supplemental Tables S1–S3.

Third, to preserve clinical relevance and reflect the real-world environment of administrative claims data, all available variables—including demographic characteristics, comorbidities, and healthcare utilization indicators—were included without additional feature selection. Due to the structural characteristics of the claims data, missing values were minimal and were handled using complete-case analysis. For scale-sensitive models (LR, SVM, and NN), continuous variables were normalized, whereas tree-based models (DT, RF, and XGBoost) were trained using raw values.

Fourth, as the class imbalance associated with the mortality rate (14.37%) reflects the natural distribution in clinical practice, we focused on verifying the models’ real-world clinical applicability rather than modifying the data through artificial resampling techniques such as SMOTE. To objectively evaluate potential performance bias resulting from this imbalance, we assessed the models’ discriminative power using both the Area Under the Curve (AUC) and the F1-score, the latter of which is particularly sensitive to class imbalance. Finally, feature importance analysis was conducted for the models exhibiting superior predictive performance (XGBoost and NN). For the XGBoost model, feature importance was derived using its built-in importance function, while for the neural network, the relative importance of each feature was evaluated based on contribution scores derived from the model.

2.5. Machine learning model

2.5.1. Algorithm selection

LR, DT, RF, XGBoost, SVM, and NN were used to develop a mortality prediction model for patients with lung cancer. LR is a traditional statistical method that is used for binary classification. It is widely used to predict the event generation probability by applying data to a logistic function. Unlike a linear regression analysis, it utilizes a multivariate methodology to establish a functional relationship between two or more independent variables and one dependent variable. ⁴³ Here, the dependent variable was dichotomous, with a value of one or 0. ⁴⁴ A DT is a supervised learning method mainly used in classification and regression analysis and is a model with a hierarchically arranged structure. First, the parent node entropy was calculated. Subsequently, the information gain was computed by subtracting the weighted sum of the child-node entropy from the parent entropy. The root node has the highest information gain. This process is repeated until the classification is completed. ⁴⁵

RF is a learning method used for classification and regression analyses. It is an ensemble algorithm that trains multiple decision trees and synthesizes the results to make predictions. It is currently used in the medical field because it is fast and highly accurate. ⁴⁶ XGBoost is commonly used in classification and regression analysis and is an ensemble technique that adds new models to correct errors in previous models. ⁴⁷ In other words, the learning errors of weak prediction models are weighted and sequentially reflected in the next learning model to create a strong prediction model. XGBoost, based on the gradient-boosted decision tree (GBDT), is further optimized for constructing and searching trees compared to the GBDT, which is more efficient and has higher prediction accuracy. ⁴⁸

An SVM based on logistic regression was utilized for data classification and regression. ⁴³ An SVM is a binary classifier that can separate two distinct groups. When the data belonging to one of two categories are given, a model is created to classify which category the new data must belong to using a hyperplane based on the input data set. The optimal hyperplane is the hyperplane in which the margin is maximized, that is, the distance from the boundary between the two categories to the data point is maximized.^49,50 The NN algorithm outperformed existing regression analyses and is widely used to show high accuracy in predicting various health conditions. A NN mimics the shape in which the neurons are connected and cannot function without other connected neurons. ⁵¹

2.5.2. Model evaluation

For model evaluation, area under the ROC curve (AUROCC), sensitivity, specificity, accuracy, and F1 score were utilized: (1) AUC refers to the area under the ROC curve and is a value that shows the correlation between overall sensitivity and specificity, which is evaluated as the closer it is to one, the better the performance is ⁵² ; (2) Sensitivity means the ratio of those evaluated as positive among those actually evaluated as positive, for which the performance is better as it is closer to one ⁵³ ; (3) Specificity refers to the ratio of those evaluated as negative among those actually evaluated as negative, for which the performance is better as it is closer to 0 ⁵⁴ ; (4) Accuracy is the most basic performance evaluation index in machine learning and is utilized to compare and evaluate models, referring to the ratio of the number of corresponding to correct answers compared to the overall data ⁵⁵ ; and (5) F1 score is an evaluation index commonly used for imbalanced data because it can consider both Precision and Recall, ⁵⁶ which represents the better performance as being closer to 1. Meanwhile, the F1 score in this study was relatively lower than other performance metrics, which may reflect the inherent class imbalance of the dataset and the limited number of mortality events. Given the low mortality rate in the analyzed data, even small changes in false-positive or false-negative classifications can substantially affect the F1 score. However, because the primary objective of this study was risk stratification and identification of high-risk patients rather than precise case-level classification, discrimination-oriented metrics such as AUC and sensitivity were considered more appropriate for evaluating clinical usefulness. The five model performance evaluation indices utilized were calculated using the following formulas:

$\left(1\right)\hspace{0.17em}Area\hspace{0.17em}Under\hspace{0.17em}the\hspace{0.17em}ROC\hspace{0.17em}Curve\left(\text{AUC}\right)=1-\left(\frac{True\hspace{0.17em}negative}{True\hspace{0.17em}negative+False\hspace{0.17em}positive}\right)$

$\left(2\right)\hspace{0.17em}\text{Sensitivity}=\frac{True\hspace{0.17em}positive}{True\hspace{0.17em}positive+False\hspace{0.17em}negative}$

$\left(3\right)\hspace{0.17em}\text{Specificity}=\frac{True\hspace{0.17em}negative}{True\hspace{0.17em}negative+False\hspace{0.17em}positive}$

$\left(4\right)\hspace{0.17em}\text{Accuracy}=\frac{True\hspace{0.17em}positive+True\hspace{0.17em}negative}{True\hspace{0.17em}positive+True\hspace{0.17em}negative+False\hspace{0.17em}positive+False\hspace{0.17em}negative}$

$\left(5\right)\hspace{0.17em}\mathrm{F}1\hspace{0.17em}\text{Score}=2*\hspace{0.17em}\frac{Precision*Recall}{Precision+Recall}$

3. Results

3.1. Demographic characteristics

Among the 24,472 patients with lung cancer who were the subjects of this study, 20,956 survived and 3,516 died. First, the mean age of the survivors was 66.12 years old. In terms of gender, males were 14,946 (71.32%), and more than 6,010 females (28.68%). Regarding the admission route, emergencies accounted for 4,610 (22%) patients, whereas the number of outpatients was 16,346 (78%); patients who did not undergo a surgical operation, 18,196 (86.83%) did not undergo a surgical accounted, for 2,760 (13.17%) did not undergo surgery. In terms of the type of insurance, medical beneficiaries (assistance) were 1,822 medical beneficiaries (8.69%), and health insurance subscribers accounted for 19,134 (91.31%), most of whom were health insurance subscribers. The mean LOS was 10.7 days. Regarding hospital locations, metropolitan cities accounted for the most, with 8,165 (38.96%), followed by Seoul with 7,340 (35.03%) and other areas with 5,451 (26.01%). Regarding the number of beds, 500-999 beds accounted for the largest number with 11,335 (54.09%), followed by 1,000 or more beds with 7,009 (33.45%), 300-499 beds with 1,388 (6.62%), and 100-299 beds with 1,224 (5.84%). In terms of comorbidity severity compensation tools, survivors’ CCI was found to be an average of 2.86 units, ACCI was an average of 5.03 units, and ECI was 6.39 units.

The mean age of deceased patients was 68.06 years old. In terms of gender, males were 2,664 people (75.77%), more than 852 females (24.23%). For the admission route, emergencies accounted for 2,083 (59.24%), while outpatients accounted for 1,433 (40.76%), showing that, unlike survivors, for the deceased, the patients who were hospitalized as emergencies accounted for a higher ratio. A total of 3,472 patients (98.75%) did not undergo surgery and 44 patients (1.25%) underwent surgery, most of whom did not undergo surgery. In terms of the type of insurance, medical beneficiaries were 385 (10.95%) and health insurance subscribers accounted for 3,131 (89.05%), most of whom were health insurance subscribers; the mean LOS was 21.62 days, which was twice that of survivors. Regarding hospital locations, metropolitan cities (including Gyeonggi-do) accounted for the most, with 1,515 (43.09%), followed by other areas with 1,243 (35.35%) and Seoul with 758 (21.56%). Regarding the number of beds, 500-999 beds accounted for the largest proportion 1,870 (53.19%), followed by 1,000 or more beds 703 (19.99%), 100-299 beds 545 (15.5%), and 300-499 beds 398 (11.32%). In terms of the comorbidity severity compensation tools, survivors’ CCI was found to be an average of 3.67 units, ACCI was an average of 6.06 units, and the ECI average of 8.81 units, showing that the number of comorbidities in the deceased was higher than that of survivors (Table 2).

Table 2.

Characteristics of the selected participants.

Variable	Survival (N=20,956)	Death (N=3,516)	P-value
Age (years), mean (SD)	66.12 (10.61)	68.06 (10.9)	<0.001
Sex, n (%)	​	​	<0.001
Male	14,946 (71.32)	2,664 (75.77)	​
Female	6,010 (28.68)	852 (24.23)	​
Admission Route, n (%)	​	​	<0.001
Emergency	4,610 (22)	2,083 (59.24)	​
Outpatient	16,346 (78)	1,433 (40.76)	​
Surgery, n (%)	​	​	<0.001
N	18,196 (86.83)	3,472 (98.75)	​
Y	2,760 (13.17)	44 (1.25)	​
Type of Insurance, n (%)	​	​	<0.001
Assistance	1,822 (8.69)	385 (10.95)	​
National Health Insurance	19,134 (91.31)	3,131 (89.05)	​
LOS a	​	​	<0.001
Mean (SD)	10.7 (15.56)	21.62 (30.71)	​
Hospital Location, n (%)	​	​	<0.001
Greater Seoul	7,340 (35.03)	758 (21.56)	​
Metropolitan City (including Gyeonggi-do)	8,165 (38.96)	1,515 (43.09)	​
Other	5,451 (26.01)	1,243 (35.35)	​
Number of Beds, n (%)	​	​	<0.001
100-299	1,224 (5.84)	545 (15.5)	​
300-499	1,388 (6.62)	398 (11.32)	​
500-999	11,335 (54.09)	1,870 (53.19)	​
≥1000	7,009 (33.45)	703 (19.99)	​
CCI, mean (SD)	2.86 (2.97)	3.67 (3.06)	<0.001
ACCI, mean (SD)	5.03 (3.06)	6.06 (3.08)	<0.001
ECI, mean (SD)	6.39 (7.13)	8.81 (7.78)	<0.001

^aLOS: Length of Stay.

Table 3 presents the distribution of comorbidity indices (CCI, ACCI, and ECI) within the study cohort. In the overall cohort, the proportions of patients with CCI ≥3, ACCI ≥3, and ECI ≥3 were 44.53%, 76.37%, and 53.24%, respectively. Significant differences in the distribution of all three indices were observed between the survival and death groups (P < 0.001). Notably, the death group exhibited a substantially higher prevalence of high-severity scores (≥3) compared to the survival group, indicating that these indices effectively reflect the clinical severity and mortality risk of the patients.

Table 3.

Distribution of comorbidity indices (CCI, ACCI, and ECI) according to survival status.

Variable	Survival (N=20,956)	Death (N=3,516)	Total (N=24,472)	P-value
CCI, n (%)	​	​	​	<0.001
0	8,405 (40.11)	1,109 (31.54)	9,514 (38.88)	​
1–2	3,614 (17.25)	446 (12.68)	4,060 (16.59)	​
≥3	8,937 (42.65)	1,961 (55.77)	10,898 (44.53)	​
ACCI, n (%)	​	​	​	<0.001
0	612 (2.92)	38 (1.08)	650 (2.66)	​
1–2	4,645 (22.17)	487 (13.85)	5,132 (20.97)	​
≥3	15,699 (74.91)	2,991 (85.07)	18,690 (76.37)	​
ECI, n (%)	​	​	​	<0.001
0	9,822 (46.87)	1,247 (35.47)	11,069 (45.23)	​
1–2	347 (1.66)	28 (0.80)	375 (1.53)	​
≥3	10,787 (51.47)	2,241 (63.74)	13,028 (53.24)	​

3.2. Performance comparison of ML model

Performance evaluation of each machine learning model revealed that RF was the most suitable for all severity-compensating tools (CCI, ACCI, and ECI) in the training set. In the CCI, the RF model showed the predictive power of AUC (0.977, 95% CI 0.975-0.98), sensitivity (0.936, 95% CI 0.925-0.945), specificity (0.898, 95% CI 0.893-0.903), accuracy (0.904, 95% CI 0.899-0.908), and F1 Score (0.736). In the ACCI, the RF model exhibited a predictive power of AUC (0.982, 95% CI 0.981-0.984), sensitivity (0.92, 95% CI 0.908-0.93), specificity (0.932, 95% CI 0.928-0.936), accuracy (0.93, 95% CI 0.926-0.934), and F1 score (0.791). For ECI, the RF model displayed the prediction power of AUC (0.983, 95% CI 0.981-0.985), sensitivity (0.944, 95% CI 0.935-0.953), specificity (0.91, 95% CI 0.906-0.915), accuracy (0.915, 95% CI 0.911-0.919), and F1 Score (0.762), suggesting that the RF model was the most suitable for predicting mortality in lung cancer.

In the Test set, all the CCI, ACCI, and ECI models were statistically significant, and the XGBoost and NN models showed the best performance. For CCI, the XGBoost model exhibited the predictive power of AUC (0.819, 95% CI 0.806-0.831), sensitivity (0.819, 95% CI 0.794-0.842), specificity (0.672, 95% CI 0.66-0.684), accuracy (0.693, 95% CI 0.682-0.704), and F1 Score (0.434). The NN model displayed a predictive power of AUC (0.817, 95% CI 0.804-0.83), sensitivity (0.83, 95% CI 0.806-0.852), specificity (0.66, 95% CI 0.648-0.672), accuracy (0.685, 95% CI 0.674-0.695), and F1 score (0.431), suggesting that the XGBoost model exhibited slightly better performance than the NN model.

In the ACCI, the XGBoost model showed the prediction power of AUC (0.82, 95% CI 0.807-0.832), Sensitivity (0.817, 95% CI 0.792-0.84), Specificity (0.681, 95% CI 0.669-0.692), Accuracy (0.7, 95% CI 0.69-0.711), and F1 Score (0.439). Additionally, the NN model displayed the predictive power of AUC (0.82, 95% CI 0.808-0.833), sensitivity (0.813, 95% CI 0.788-0.836), specificity (0.685, 95% CI 0.673-0.696), accuracy (0.703, 95% CI 0.693-0.714), and F1 Score (0.44), suggesting that both models had the same performance.

In addition, in the ECI, the XGBoost model exhibited a predictive power of AUC (0.821, 95% CI 0.809-0.834), sensitivity (0.767, 95% CI 0.74-0.792), specificity (0.731, 95% CI 0.72-0.742), accuracy (0.736, 95% CI 0.726-0.746), and F1 Score (0.455). The NN model showed the predictive power of AUC (0.82, 95% CI 0.808-0.833), sensitivity (0.835, 95% CI 0.811-0.857), specificity (0.653, 95% CI 0.641-0.665), accuracy (0.679, 95% CI 0.668-0.69), and F1 Score (0.428), suggesting that, similar to CCI, the XGBoost model exhibited slightly better performance than the NN model. In conclusion, regarding the mortality prediction of patients with lung cancer, the XGBoost model showed the best performance for the CCI, ACCI, and ECI (Figure 2, Table 4).

Table 4.

Performance comparison of the lung cancer mortality machine learning models.

Model		Training set					Test set
		AUC (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)	Accuracy (95% CI)	F1 score	AUC (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)	Accuracy (95% CI)	F1 score
CCI	Logistic Regression	0.803 (0.795-0.812)	0.777 (0.76-0.793)	0.702 (0.695-0.71)	0.713 (0.706-0.72)	0.437	0.809 (0.796-0.822)	0.791 (0.765-0.815)	0.697 (0.685-0.708)	0.71 (0.7-0.721)	0.44
	Decision Tree	0.759 (0.749-0.77)	0.748 (0.731-0.765)	0.726 (0.719-0.734)	0.73 (0.723-0.736)	0.443	0.761 (0.746-0.776)	0.763 (0.736-0.788)	0.723 (0.712-0.734)	0.729 (0.719-0.739)	0.447
	Random Forest	0.977 (0.975-0.98)	0.936 (0.925-0.945)	0.898 (0.893-0.903)	0.904 (0.899-0.908)	0.736	0.786 (0.772-0.8)	0.602 (0.572-0.632)	0.792 (0.782-0.802)	0.765 (0.755-0.775)	0.424
	XGBoost	0.834 (0.827-0.842)	0.84 (0.825-0.854)	0.677 (0.669-0.684)	0.7 (0.693-0.707)	0.446	0.819 (0.806-0.831)	0.819 (0.794-0.842)	0.672 (0.66-0.684)	0.693 (0.682-0.704)	0.434
	Support Vector Machine	0.743 (0.731-0.754)	0.532 (0.512-0.552)	0.879 (0.874-0.884)	0.829 (0.824-0.835)	0.473	0.7 (0.682-0.719)	0.485 (0.454-0.515)	0.854 (0.845-0.862)	0.801 (0.791-0.81)	0.411
	Neural Network	0.824 (0.817-0.832)	0.835 (0.82-0.85)	0.668 (0.66-0.675)	0.692 (0.685-0.699)	0.438	0.817 (0.804-0.83)	0.83 (0.806-0.852)	0.66 (0.648-0.672)	0.685 (0.674-0.695)	0.431
ACCI	Logistic Regression	0.803 (0.795-0.812)	0.829 (0.813-0.843)	0.649 (0.642-0.657)	0.675 (0.668-0.682)	0.423	0.809 (0.797-0.822)	0.837 (0.813-0.859)	0.643 (0.631-0.655)	0.671 (0.66-0.682)	0.422
	Decision Tree	0.697 (0.686-0.708)	0.588 (0.568-0.608)	0.78 (0.773-0.786)	0.752 (0.746-0.759)	0.405	0.703 (0.686-0.719)	0.602 (0.572-0.632)	0.781 (0.77-0.791)	0.755 (0.745-0.765)	0.414
	Random Forest	0.982 (0.981-0.984)	0.92 (0.908-0.93)	0.932 (0.928-0.936)	0.93 (0.926-0.934)	0.791	0.786 (0.771-0.8)	0.556 (0.525-0.586)	0.826 (0.817-0.836)	0.787 (0.778-0.797)	0.429
	XGBoost	0.834 (0.827-0.842)	0.831 (0.816-0.846)	0.684 (0.676-0.692)	0.705 (0.698-0.712)	0.448	0.82 (0.807-0.832)	0.817 (0.792-0.84)	0.681 (0.669-0.692)	0.7 (0.69-0.711)	0.439
	Support Vector Machine	0.738 (0.726-0.749)	0.525 (0.505-0.545)	0.871 (0.865-0.876)	0.821 (0.815-0.827)	0.458	0.702 (0.684-0.721)	0.491 (0.461-0.522)	0.853 (0.844-0.862)	0.801 (0.792-0.81)	0.415
	Neural Network	0.822 (0.814-0.83)	0.811 (0.795-0.826)	0.691 (0.684-0.699)	0.709 (0.702-0.715)	0.444	0.82 (0.808-0.833)	0.813 (0.788-0.836)	0.685 (0.673-0.696)	0.703 (0.693-0.714)	0.44
ECI	Logistic Regression	0.805 (0.796-0.813)	0.801 (0.785-0.817)	0.677 (0.67-0.685)	0.695 (0.688-0.702)	0.43	0.811 (0.799-0.824)	0.811 (0.786-0.834)	0.672 (0.66-0.683)	0.692 (0.681-0.702)	0.43
	Decision Tree	0.697 (0.686-0.708)	0.588 (0.568-0.608)	0.78 (0.773-0.786)	0.752 (0.746-0.759)	0.405	0.703 (0.687-0.719)	0.602 (0.572-0.632)	0.781 (0.77-0.791)	0.755 (0.745-0.765)	0.414
	Random Forest	0.983 (0.981-0.985)	0.944 (0.935-0.953)	0.91 (0.906-0.915)	0.915 (0.911-0.919)	0.762	0.79 (0.776-0.804)	0.602 (0.571-0.631)	0.799 (0.789-0.809)	0.771 (0.761-0.78)	0.43
	XGBoost	0.839 (0.831-0.847)	0.78 (0.763-0.796)	0.737 (0.73-0.744)	0.743 (0.736-0.75)	0.466	0.821 (0.809-0.834)	0.767 (0.74-0.792)	0.731 (0.72-0.742)	0.736 (0.726-0.746)	0.455
	Support Vector Machine	0.747 (0.736-0.758)	0.532 (0.513-0.552)	0.876 (0.87-0.881)	0.826 (0.821-0.832)	0.468	0.699 (0.68-0.717)	0.489 (0.458-0.519)	0.844 (0.835-0.853)	0.793 (0.784-0.802)	0.404
	Neural Network	0.826 (0.818-0.833)	0.843 (0.828-0.857)	0.659 (0.651-0.666)	0.685 (0.678-0.692)	0.435	0.82 (0.808-0.833)	0.835 (0.811-0.857)	0.653 (0.641-0.665)	0.679 (0.668-0.69)	0.428

Figure 2.

Precision-recall curves of six different machine learning models for predicting lung cancer mortality. (A) CCI model performance in the training and test sets. (B) ECI model performance in the training and test sets. (C) ACCI model performance in the training and test sets.

3.3. Feature importance in lung cancer mortality

Predictive factors for the mortality of lung cancer patients were identified in XGBoost and NN, which had the best performance among the machine learning models (Figure 3). In the CCI XGBoost and ACCI XGBoost models, LOS was the most important factor affecting the mortality of lung cancer patients, followed by admission route and surgery. However, for the ECI XGBoost models, the Admission Route was an important factor affecting the mortality of lung cancer patients, followed by LOS and Surgery. Meanwhile, in all the CCI neural network models, the ACCI NN model, and the ECI NN model, the admission route acted as an important factor affecting the mortality of lung cancer patients, whereas Surgery, LOS, and medical institutions with 1,000 or more beds belonged to other important factors. In summary, LOS and admission routes were identified as the top factors predicting mortality in patients with lung cancer.

Figure 3.

Feature importance of the XGBoost and NN model.

4. Discussion

In this study, we developed a mortality prediction model for patients with lung cancer for each severity-compensating tool by applying machine learning techniques to the KNHDIS data of the Korea Disease Control and Prevention Agency. Although studies on severity-compensating tools such as the CCI, ACCI, and ECI are actively underway, very few studies have proposed prediction models based on severity-compensating tools for lung cancer patients. Therefore, this study suggests a mortality prediction model for lung cancer patients with high predictive power and identifies the major factors for predicting mortality in lung cancer patients. We developed a mortality prediction model for lung cancer patients by utilizing LR, DT, RF, XGBoost, SVM, and ANNA, and comprehensively evaluated its performance with AUROCC, sensitivity, specificity, accuracy, and F1 score, confirming that the XGBoost and NN models had superior prediction power.

Examining precedent studies, the XGBoost model showed excellent prediction power in establishing an early lung cancer prediction model, ⁵⁷ and in terms of machine learning-based lung cancer risk prediction, XGBoost exhibited the best performance among LR, Naïve Bayes, RF, and XGBoost algorithms. ⁵⁸ In addition, the XGBoost-based AI model showed the best predictive power for predicting the postoperative prognosis of patients with NSCLC. ⁵⁹ Meanwhile, when applying LASSO, RF, XGBoost, NN, and Super Learner to predict in-hospital sepsis mortality, XGBoost, and NN showed the best performance. ⁶⁰ Furthermore, the NN model was found to have the highest accuracy in predicting the mortality of stroke patients, ⁶¹ whereas the XGBoost model had better predictive power than LR and RF models in predicting the mortality rate of severe influenza patients. ⁶² In other words, it could be confirmed that the XGBoost model is an excellent model for the early detection and prognosis prediction of lung cancer and that the NN model exhibits excellent performance in predicting mortality. The results of this study also showed that the XGBoost and NN models, which have been shown to have the best prediction power and accuracy in previous studies, exhibited the best performance. This can be the basis for supporting the possibility of utilizing XGBoost and NN models to predict the mortality rate of lung cancer patients in the future. Therefore, the adoption of machine learning models in clinical settings can provide healthcare professionals with better decision-making tools and contribute to improving the survival rate of lung cancer patients. ⁶³

In the XGBoost and NN models based on the severity-compensating tools CCI, ACCI, and ECI, the key variables for predicting mortality in lung cancer patients were admission route, LOS, and surgery. Treatment methods for lung cancer include surgery, chemotherapy, and radiation therapy, which are performed within one year of diagnosis. For lung cancer, the course of the disease after initial diagnosis and the likelihood of visiting a medical institution are inversely proportional, and the side effects caused by cancer treatment may result in emergency room visits. Acute pain due to drug treatment, the risk of infection, inflammatory disease due to decreased immune function, side effects, and complications due to radiation therapy can lead to emergency room visits by patients with lung cancer. As such, patients with lung cancer may visit the emergency room more frequently, and an increase in emergency room visits can lead to an increase in the LOS. In general, 54.5% of patients with lung cancer who visit the emergency room end up during hospitalization or mortality, and as the number of emergency room visits increases, their symptoms may worsen or their complications may increase. Most patients with lung cancer who visit the emergency room require treatment in the intensive care unit (ICU), and hospitalization in the ICU can increase the LOS. Decreased levels of consciousness due to pneumonia, pleural effusion, pulmonary embolism, and brain metastasis act as a factor in increasing the LOS in the ICU.^64,65 Regarding the admission route, the results of this study also showed that the mortality rate was higher for patients who were hospitalized through the emergency room than through the outpatient route, in which the LOS for deceased patients was found to be more than twice that of survivors. When a patient is hospitalized, the severity is assessed to determine the LOS, and through management of the LOS, the timing of treatment intervention can be determined in consideration of situations that may occur in the patient. In particular, an increase in LOS can act as a factor in increasing medical expenses, leading to increasing the burden on patients. ⁶⁶ Accordingly, it is necessary to prevent emergency room visits through continuous monitoring of patients with lung cancer and reduce the LOS by identifying the risk factors for hospitalization.

In patients with lung cancer, there is a close relationship between the treatment period and mortality rate. The longer the treatment duration, the greater the impact on the patient mortality rate. Although surgical resection is an effective treatment method for lung cancer patients, the mortality rate within 5 years is high in patients who receive surgical treatment. It seems that the mortality rate could increase because of complications caused by recurrence and concomitant diseases. Surgical resection performed in the early stages of cancer can increase the survival rate of patients with lung cancer; however, the mortality rate is high for patients who receive surgical treatment 30 days after diagnosis. In other words, the mortality rate of patients with lung cancer increases as they wait longer for treatment, and concomitant diseases may increase their complications after surgery, negatively affecting prognosis. Advances in cancer diagnosis and treatment technologies have enabled cancer patients to survive longer if diagnosed and treated early. In patients with lung cancer, as the severity of the disease increases, functional recovery cannot be achieved after surgery, thus increasing the disease burden. To overcome this problem, minimally invasive surgery, which can reduce the length of hospital stay, has recently been adopted for lung cancer resection. In conclusion, because the mortality rate of lung cancer patients increases due to the occurrence of concomitant diseases rather than the cancer itself as the disease progresses, continuous management and treatment of lung cancer patients after surgery seems to be necessary. Since the most important goal of postoperative treatment is a patient’s normal life, integrated management, including regular checkups, medical treatment, and rehabilitation treatment, is necessary. Furthermore, it is necessary to continuously maintain patients’ health, reduce their emergency room visits, and prevent an increase in the LOS by minimizing side effects that may occur during the treatment process.^13,67–70

This study has the following limitations. No comorbidities or complications were observed. Since the data in this study were based on diagnosis at the time of discharge, the temporal relationship between the occurrence of comorbidities is unknown. In future studies, if temporal relationships could be identified and analyzed by distinguishing comorbidities and complications separately, a more accurate mortality prediction model for lung cancer patients could be built. Additionally, because of the nature of the research data, it did not contain clinical information such as cancer stage; therefore, it could not be reflected in the model. If clinical variables were included in future studies, more specific mortality-predictive factors could be identified. Furthermore, this study did not employ ensemble learning approaches (e.g., stacking) or advanced interpretation techniques such as SHAP or LIME. The primary objective of this study was to compare and evaluate predictive performance using structured panel data, rather than image-based deep learning models, and to focus on model benchmarking across comorbidity severity indicators. Future research could apply ensemble methods and post hoc explainability analysis to enhance model robustness and provide more interpretable evidence for the pathways associated with mortality risk.

5. Conclusion

In this study, three commonly used comorbidity indices (CCI, ACCI, ECI) were applied to machine learning-based mortality prediction models for lung cancer patients and compared. The analysis revealed consistent key predictors—namely, admission route, length of stay, and surgical status—regardless of the comorbidity index used, demonstrating the structural validity of claims data-based prediction models. Although the study utilized data from a single country, the results suggest that machine learning models can effectively identify patients at elevated risk of mortality by leveraging standardized clinical data, including comorbidity severity. Given the limitations in generalizability from single-country data, external validation using multinational datasets is warranted. As the variables used in this study are standardized across most countries, similar predictive performance is expected in external validations.

These findings indicate that machine learning can provide a robust evidence base for managing lung cancer patients throughout their treatment journey. Continuous post-diagnosis management is critical, as frequent emergency visits and prolonged hospital stays—often due to post-surgical recurrence or comorbidities—are major risk factors for mortality. The XGBoost and Neural Network models developed in this study can facilitate early identification of high-risk patients. Linking these patients to primary care and digital health-based remote monitoring systems allows healthcare providers to receive timely information about abnormal signs, helping prevent emergency visits and hospital readmissions. In essence, machine learning can support safe at-home management post-discharge, contributing to reduced mortality and improved therapeutic outcomes.

While further model refinement and external validation incorporating additional clinical variables (e.g., cancer stage, detailed treatment information) are necessary, this study provides a methodological foundation for improving risk assessment in lung cancer patients. Moreover, these predictive models can help establish more systematic post-discharge management strategies and enable earlier interventions for high-risk patients.

Supplemental material

Supplemental material - Machine learning–based prediction of mortality in lung cancer: Application of severity-adjustment method

Supplemental material for Machine learning–based prediction of mortality in lung cancer: Application of severity-adjustment method by Sewon Park, Selin Woo, Ji-Hyun Park, Bumhee Park and Munjae Lee in Digital Health.