Performance of machine learning-based coronary computed tomography angiography for selecting revascularization candidates

Abstract

Background

Limited studies have investigated the accuracy of therapeutic decision-making using machine learning-based coronary computed tomography angiography (ML-CCTA) compared with CCTA.

Purpose

To investigate the performance of ML-CCTA for therapeutic decision compared with CCTA.

Material and Methods

The study population consisted of 322 consecutive patients with stable coronary artery disease. The SYNTAX score was calculated with an online calculator based on ML-CCTA results. Therapeutic decision-making was determined by ML-CCTA results and the ML-CCTA-based SYNTAX score. The therapeutic strategy and the appropriate revascularization procedure were selected using ML-CCTA, CCTA, and invasive coronary angiography (ICA) independently.

Results

The sensitivity, specificity, positive predictive value, negative predictive value, accuracy of ML-CCTA and CCTA for selecting revascularization candidates were 87.01%, 96.43%, 95.71%, 89.01%, 91.93%, and 85.71%, 87.50%, 86.27%, 86.98%, 86.65%, respectively, using ICA as the standard reference. The area under the receiver operating characteristic curve (AUC) of ML-CCTA for selecting revascularization candidates was significantly higher than CCTA (0.917 vs. 0.866, P = 0.016). Subgroup analysis showed the AUC of ML-CCTA for selecting percutaneous coronary intervention (PCI) or coronary artery bypass graft (CABG) candidates was significantly higher than CCTA (0.883 vs. 0.777, P < 0.001, 0.912 vs. 0.826, P = 0.003, respectively).

Conclusion

ML-CCTA could distinguish between patients who need revascularization and those who do not. In addition, ML-CCTA showed a slightly superior to CCTA in making an appropriate decision for patients and selecting a suitable revascularization strategy.

Keywords

Machine learning coronary artery disease computed tomography angiography coronary angiography myocardial revascularization

Introduction

Coronary computed tomography angiography (CCTA) has been introduced as a reliable method for the first-line diagnostic test in patients with suspected or known coronary artery disease (CAD) in the outpatient, inpatient, or emergency department setting (1,2). Multiple studies have demonstrated that CCTA has a high diagnostic accuracy for obstructive CAD when using invasive coronary angiography (ICA) as a reference standard (3 –5). In addition, the characteristics based on CCTA have been proven to be independent risk predictors of cardiovascular events (6).

According to the 2018 European Society of Cardiology/European Association for Cardio-Thoracic Surgery (ESC/EACTS) guidelines, it is critical to identify patients with obstructive CAD who are likely to benefit from myocardial revascularization (7). Yet, limited studies have investigated the accuracy (70%–89.8%) of therapeutic decision-making using CCTA compared with ICA (8 –10). Recently, machine learning (ML) algorithms have been developed to both automate the processes and improve the diagnostic performance of cardiovascular imaging (11,12). Moreover, ML can assist doctors with increasing efficiency, reducing errors, and achieving objectives by providing streamlined, time-saving workflows in treatment decision-making (13). We have reported the clinical evaluation of an ML-based CCTA postprocessing and reporting system in patients with suspected CAD (14). The SYNTAX score has been widely advocated as an aided evaluation method in both the U.S and European revascularization guidelines for revascularization strategy (7,15). However, the performance of ML-based CCTA and SYNTAX for therapeutic decision-making requires further investigation. Thus, we sought to investigate the accuracy of ML-CCTA for selecting revascularization candidates compared with CCTA.

Material and Methods

The present study was approved by the local institutional review board and was conducted in compliance with the Health Insurance Portability and Accountability Act (HIPAA) of 1996.

Patients

CCTA and ICA examinations between July 2017 and July 2019 were retrospectively searched in the institution's Picture Archive and Communication System (PACS). Written informed consent was obtained from all participants. The study population consisted of patients with stable CAD with an interval of <30 days between CCTA and ICA were included (Fig. 1). General exclusion criteria were as follows: poor quality images; images that could not be recognized or analyzed by ML (scans affected by stair-step artifacts, respiratory motion artifacts, excessive image noise, or artifacts caused by arrhythmia); incomplete records; previously underwent percutaneous coronary intervention (PCI), coronary artery bypass graft (CABG), or other cardio surgery. Demographic and clinical information on cardiovascular risk factors, including age, sex, body mass index (BMI), symptoms, hypertension, diabetes mellitus (DM), chronic kidney disease (CKD), dyslipidemia, serum creatinine, urea nitrogen, EF, history of chronic obstructive pulmonary disease (COPD), and peripheral vascular disease (PVD), were collected from electronic medical records. History of COPD was defined as the long-term use of bronchodilators or steroids for lung disease. History of PVD was defined as exercise-related claudication or angiographic stenosis of >50% in the aorta or other arteries compared with that in the coronary arteries. Creatinine clearance calculated by using the Cockroft Gault formula was also acquired.

Fig. 1.

Flow diagram showed the study population. CABG, coronary artery bypass graft; CAD, coronary artery disease; CCTA, coronary computed tomography angiography; ICA, invasive coronary angiography; ML-CCTA, machine learning-based CCTA; PCI, percutaneous coronary intervention.

Criteria for revascularization

We established eligibility criteria for revascularization according to the 2018 ESC/EACTS guidelines (7). Revascularization therapeutic decision-making (PCI or CABG) for the patients with stable CAD was based on the level of evidence and the strength of recommendation. We considered the higher class of recommendation and level of evidence was the better revascularization therapeutic decision in this study. For example, CABG (I B) was the better revascularization therapeutic decision for a two-vessel CAD patient with proximal left anterior descending coronary artery (LAD) stenosis rather than PCI (I C). If the class of recommendation and level of evidence were both the same for CABG and PCI, either type would be the revascularization therapeutic decision. Based on these considerations, the therapeutic recommendation decision for the type of revascularization in patients with stable CAD is shown in Table 1. The SYNTAX score was an anatomical complexity of coronary lesions scoring system to evaluate the extent of CAD (16). The SYNTAX score was calculated by online calculator (www.syntaxscore.com/calculator/start.htm; version 2.28).

Table 1.

Therapeutic recommendation decision for the type of revascularization in patients with stable CAD.

Recommendation according to extent of CAD	Revascularization
Recommendation according to extent of CAD	PCI	PCI/CABG	CABG
One-vessel disease with proximal LAD stenosis		I A
One-vessel or two-vessel disease without proximal LAD stenosis	I C
Two-vessel disease with proximal LAD stenosis			I B
Left main disease with low SYNTAX score (0–22)		I A
Left main disease with intermediate (23–32) and high SYNTAX score (≥33)			I A
Three-vessel disease with low SYNTAX score without DM		I A
Three-vessel disease with intermediate or high SYNTAX score without DM			I A
Three-vessel disease with DM			I A

CABG, coronary artery bypass graft; CAD, coronary artery disease; DM, diabetes mellitus; LAD, left anterior descending coronary artery; PCI, percutaneous coronary intervention.

CCTA and ML-CCTA studies

Multidetector-row CT imaging was performed with a dual-source CT scanner (Somatom Definition, Siemens Medical Solutions, Forchheim, Germany) and a 256-slice CT scanner (Brilliance iCT; Philips Healthcare, Best, the Netherlands) which has been reported previously (17,18). Heart rate control (HR ≥65 beats/min) was performed with beta-blockers before the scan. For contrast enhancement, 60–80 mL of iopromide (370 mgI/mL, Bayer Schering Pharma, Leverkusen, Germany) followed by 30–40 mL of pure saline with a flow rate of 4–5 mL/s. The scan will automatically commence during an inspiratory breath hold of 8–14 s after a delay of 2 s when the density in the region of interest (descending aorta) rises to a preset value (100 HU).

CCTA images were sent to one of the two workstations for postprocessing: Intelligence space portal (Version 6.0.4, Philips Medical Systems, Best, the Netherlands) and SyngoVIA (Siemens Medical Solutions, Forchheim, Germany). The postprocessing includes transaxial 2D image stacks, multiplanar reformations (MPR), maximum intensity projections (MIP), curved multiplanar reformations (cMPR), and volume rendering technique (VRT) reconstructions according to the JCCT guideline (19). Three radiologists (blinded to the patients’ history, ML-CCTA and ICA results, the dataset split up between them) with 5 (YD), 10 (SCZ), and 16 (JWX) years of experience in cardiac CT evaluated the CCTA images (Fig. 2). The coronary arteries with a diameter ≥2 mm were evaluated for luminal narrowing. The standardized Coronary Artery Disease-Reporting and Data System (CAD-RADS) classification was reported in our recent researches (14,18,20).

Fig. 2.

(a–e) Lesions showed in ML-CCTA, multidetector-row CCTA, and ICA. ML-CCTA showed mild stenosis in LAD (7 segments), severe stenosis in LCX (11 segments), and occlusion in RCA (2 segments). (f–j) Multidetector-row CCTA showed the same pictures and these coronary lesions were confirmed on (k, l) ICA. CCTA, coronary computed tomography angiography; ICA, invasive coronary angiography; LAD, left anterior descending coronary artery; LCX, left circumflex coronary artery; ML-CCTA, machine learning-based CCTA; RCA, right coronary artery.

ML-CCTA calculations were performed by one radiologist (ZFH) using a software prototype (CoronaryDoc, version 1.5, Shukun Technology, Beijing, China), which has been described previously (14,18,21). The software is based on an artificial intelligence ML platform for standardized postprocessing and structured reporting using CCTA data (Supplementary Figures 1–3).

Fig. 3.

Correlation between SYNTAX scores. (a) Correlation between the ML-CCTA-based SYNTAX score and the ICA-based SYNTAX score. (b) Correlation between the CCTA-based SYNTAX score and the ICA-based SYNTAX score. CCTA, coronary computed tomography angiography; ICA, invasive coronary angiography; ML-CCTA, machine learning-based CCTA; SYNTAX, The Synergy between Percutaneous Coronary Intervention with TAXUS and Cardiac Surgery.

Invasive coronary angiography

ICA was performed according to standard practice by a group of experienced interventional cardiologists. Coronary stenosis was evaluated by visual estimation (stenosis ≥50%, XW was blinded to the patients’ history, CCTA and ML-CCTA results) and the decision of whether or not to proceed with revascularization and which revascularization to perform were based on invasive angiographic assessment and the ICA-based SYNTAX score.

Therapeutic decision-making

Standardized postprocessing and structured reporting including CAD-RADS using CCTA data was created based on the ML algorithm. The SYNTAX score was calculated with an online calculator based on ML-CCTA results. Therapeutic decision-making was determined by a team with experienced cardiologists based on the ML-CCTA results and ML-CCTA-based SYNTAX score. Then, another team was asked to determine the treatment strategy based on the CCTA results and CCTA-based SYNTAX score, which was blinded to the treatment strategy of ML-CCTA. The treatment strategy was recorded as coronary revascularization (PCI or CABG) and/or optimal medical treatment for ML-CCTA, CCTA, and ICA, respectively.

Statistical analysis

Continuous variables were presented as mean ± SD. Categorical variables were presented as percentages or absolute values. We used either the chi-square test or Fisher's exact test, as appropriate, for categorical variables and one-way ANOVA test for continuous variables. The correlation between the SYNTAX scores calculated from ML-CCTA, CCTA, and ICA was analyzed using Spearman’s correlation coefficient. Designed as a diagnostic test, the performance of ML-based CCTA for selecting revascularization candidates was determined as sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy. P < 0.05 was considered as statistically significant. All statistical analysis was performed using SPSS version 18 (SPSS, Inc., Chicago, IL, USA) and MedCalc Statistical Software version16.8.4.0 (Ostend, Belgium).

Results

The study population included 190 men and 132 women (mean age = 63.8 years; age range = 35–87 years). The baseline characteristics of the study patients were summarized in Table 2. There was no significant difference among the patients who fulfilled CABG surgery criteria based on CCTA, ML-CCTA, or ICA. In addition, there was no significant difference among the patients who fulfilled PCI criteria based on CCTA, ML-CCTA, or ICA.

Table 2.

Basic characteristics of the study patients.

Characteristics	Total (n = 322)	CABG			Pvalue	PCI			Pvalue
Characteristics	Total (n = 322)	ICA(n = 86)	ML-CCTA(n = 79)	CCTA(n = 71)	Pvalue	ICA (n = 68)	ML-CCTA(n = 61)	CCTA(n = 82)	Pvalue
Age (years)	63.8 ± 9.8	64.3 ± 10.2	64.4 ± 10.2	65.4 ± 9.8	0.744	64.0 ± 9.2	63.1 ± 9.5	63.1 ± 10.0	0.818
Male	190 (59)	66 (76.7)	60 (75.9)	51 (71.8)	0.758	52 (76.5)	45 (73.8)	56 (75)	0.520
BMI (kg/m²)	24.8 ± 2.7	24.7 ± 2.6	24.8 ± 2.5	25.1 ± 2.5	0.666	24.9 ± 2.9	24.7 ± 2.8	24.6 ± 3.0	0.833
COPD	9 (2.8)	4 (4.7)	4 (5.1)	3 (4.2)	0.971	4 (5.9)	2 (3.3)	3 (5.7)	0.722
DM	94 (29.2)	26 (30.2)	20 (25.3)	16 (22.5)	0.538	24 (29.2)	23 (37.7)	26 (33.6)	0.750
CKD	40 (12.4)	13 (15.1)	12 (15.2)	10 (14.1)	0.978	7 (10.3)	7 (11.5)	10 (14.3)	0.935
PVD	5 (3.1)	4 (4.7)	4 (5.1)	3 (4.2)	0.971	1 (1.5)	1 (1.6)	2 (3.6)	0.897
Hypertension	196 (60.9)	60 (69.8)	54 (63.3)	51 (71.8)	0.898	38 (55.9)	31 (50.8)	44 (69.3)	0.848
CrCl	111.7 ± 43.8	85.8 ± 26.3	85.8 ± 25.8	84.2 ± 23.0	0.911	104 ± 94.1	105.3 ± 99.0	89.4 ± 29.7	0.382
TG	2.0 ± 1.7	2.0 ± 1.2	2.0 ± 1.2	1.9 ± 1.0	0.910	2.4 ± 2.5	2.4 ± 2.4	2.0 ± 1.9	0.541
TC	4.7 ± 1.1	4.7 ± 1.0	4.6 ± 1.0	4.8 ± 0.9	0.707	4.5 ± 1.1	4.7 ± 1.1	4.8 ± 1.1	0.468
HDL-C	1.2 ± 0.3	1.2 ± 0.3	1.2 ± 0.3	1.2 ± 0.3	0.577	1.1 ± 0.3	1.1 ± 0.3	1.2 ± 0.3	0.518
LDL-C	2.8 ± 0.9	2.9 ± 1.0	2.8 ± 0.9	2.9 ± 0.9	0.824	2.7 ± 0.8	2.7 ± 0.8	2.9 ± 0.9	0.408
Apo-A	1.3 ± 0.2	1.3 ± 0.2	1.3 ± 0.2	1.3 ± 0.2	0.571	1.3 ± 0.2	1.3 ± 0.3	1.3 ± 0.3	0.980
Apo-B	1.0 ± 0.2	1.0 ± 0.2	1.0 ± 0.2	1.0 ± 0.2	0.921	1.0 ± 0.3	1.0 ± 0.3	1.0 ± 0.3	0.893
SCR	73.1 ± 38	78.4 ± 20.0	78.7 ± 19.8	77.1 ± 16.8	0.853	71.9 ± 19.1	71.6 ± 20.4	80.5 ± 68.5	0.400
BUN	5.6 ± 1.9	5.9 ± 2.3	5.7 ± 1.6	5.7 ± 1.3	0.714	5.3 ± 1.5	5.3 ± 1.6	5.6 ± 2.2	0.360
FFA	0.5 ± 0.3	0.5 ± 0.3	0.5 ± 0.3	0.5 ± 0.2	0.751	0.5 ± 0.3	0.5 ± 0.3	0.5 ± 0.3	0.799
EF (%)	59.2 ± 5.8	57.4 ± 6.0	57.5 ± 6.0	57.7 ± 6.8	0.947	60.1 ± 5.2	59.3 ± 5.7	60.0 ± 5.1	0.643
SYNTAX score		19 (15.6–24.9)	19 (15–25)	20 (15.5–25.3)	0.734	9.3 (7–14)	8 (7–14)	11.5 (7–15)	0.448

Values are given as n (%), mean ± SD, or median (IQR).

Apo-A, apolipoprotein A; Apo-B, apolipoprotein B; BMI, body mass index; BUN, blood urea nitrogen; CABG, coronary artery bypass graft; CCTA, coronary computed tomography angiography; CKD, chronic kidney disease; COPD, chronic obstructive pulmonary disease; CrCl, creatinine clearance rate; DM, diabetes mellitus; EF, ejection fraction; FFA, free fatty acids; HDL-C, high density lipoprotein-C; IQR, interquartile range; LDL-C, low density lipoprotein-C; ML-CCTA, machine learning-based CCTA; PCI, percutaneous coronary intervention; PVD, peripheral vascular disease; SCR, serum creatinine; TC, total cholesterol; TG, triglyceride.

Correlation of SYNTAX score and CAD-RADS classification

There was a strong correlation between the ML-CCTA-based SYNTAX score and the ICA-based SYNTAX score (Spearman’s correlation coefficient, r = 0.809; P < 0.001) (Fig. 3a). Similarly, a strong correlation was found between the CCTA-based SYNTAX score and the ICA-based SYNTAX score (Spearman’s correlation coefficient, r = 0.777; P < 0.001) (Fig. 3b) (Supplementary Figure 4). In addition, we also found a strong correlation between the ML-CCTA-based CAD-RADS classification and CCTA-based CAD-RADS classification (intraclass correlation coefficient = 0.879; P < 0.001) (Supplementary Table 1).

Diagnostic accuracy of ML-CCTA and CCTA for selecting revascularization candidates

A total of 154 patients fulfilled the revascularization criteria using ICA as standard diagnostic tool, 140 patients met the criteria using ML-CCTA, and 153 patients met the criteria using CCTA. The sensitivity, specificity, PPV and NPV, and accuracy of ML-CCTA for selecting revascularization candidates were 87.01%, 96.43%, 95.71%, 89.01%, and 91.93% respectively. The sensitivity, specificity, PPV and NPV, and accuracy of CCTA for selecting revascularization candidates were 85.71%, 87.50%, 86.27%, 86.98%, and 86.65%, respectively. The area under the receiver operating characteristic (ROC) curve (AUC) of ML-CCTA for selecting revascularization candidates was significantly higher than CCTA (0.917, 95% confidence interval [CI] = 0.882–0.945 vs. 0.866, 95% CI = 0.824–0.901; P = 0.016) (Fig. 4). The data were summarized in Table 3. The clinical workup would be changed in 10.1% (17/169) of patients from optimal medical treatment to revascularization and 19.6% (30/153) of patients from revascularization to optimal medical treatment if they only used ML-CCTA when compared with CCTA. As a result, it would slightly underestimate the number of patients who need revascularization compared to CCTA.

Fig. 4.

ROC curves for selecting revascularization candidates. AUC, area under curve; CCTA, coronary computed tomography angiography; ML-CCTA, machine learning-based CCTA; ROC, receiver operating characteristic.

Table 3.

Diagnostic accuracy of CCTA and ML-based CCTA for selecting revascularization candidates.

Variable	TP	TN	FP	FN	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)	Accuracy (95% CI)	AUC (95% CI)
ML-CCTA	134	162	6	20	87.01 (80.42–91.70)	96.43 (92.04–98.54)	95.71 (90.50–98.25)	89.01 (83.32–93.00)	91.93 (88.82–95.03)	0.917 (0.882–0.945)
CCTA	132	147	21	22	85.71 (78.95–90.64)	87.50 (81.31–91.92)	86.27 (79.56–91.11)	86.98 (80.74–91.49)	86.65 (82.65–90.64)	0.866 (0.824–0.901)
P value										0.016

AUC, area under the curve; CCTA, coronary computed tomography angiography; FN, false negative; FP, false positive; ML, machine learning; ML-CCTA, machine learning-based coronary computed tomography angiography; NPV, negative predictive value; PPV, positive predictive value; TN, true negative; TP, true positive.

Diagnostic accuracy of ML-CCTA and CCTA for selecting the method of revascularization with candidates

Table 4 shows the diagnostic accuracy of the two procedures for selecting the method of revascularization with candidates. A total of 68 patients fulfilled the PCI criteria using ICA as a standard diagnostic tool, 61 patients met the criteria using ML-CCTA, and 82 patients met the criteria using CCTA. The sensitivity, specificity, PPV and NPV, and accuracy of ML-CCTA for selecting PCI candidates were 79.41%, 97.24%, 88.52%, 94.64%, and 93.48%, respectively. The sensitivity, specificity, PPV and NPV, and accuracy of CCTA for selecting PCI candidates were 69.12%, 86.22%, 57.32%, 91.25%, and 82.61%, respectively. The AUC of ML-CCTA for selecting PCI candidates was significantly higher than CCTA (0.883, 95% CI = 0.843–0.916 vs. 0.777, 95% CI = 0.727–0.821; P < 0.001) (Fig. 5a).

Fig. 5.

ROC curves for selecting method of revascularization with candidates. (a) ROC curves for selecting PCI candidates. (b) ROC curves for selecting CABG candidates. AUC, area under curve; CCTA, coronary computed tomography angiography; ML-CCTA, machine learning-based CCTA; ROC, receiver operating characteristic.

Table 4.

Diagnostic accuracy of CCTA and ML-based CCTA for selecting method of revascularization with candidates.

Variable	TP	TN	FP	FN	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)	Accuracy (95% CI)	AUC (95% sCI)
PCI
ML-CCTA	54	247	7	14	79.41 (67.55–87.90)	97.24 (94.16–98.79)	88.52 (77.17–94.88)	94.64 (90.97–96.92)	93.48 (90.69–96.27)	0.883 (0.843–0.916)
CCTA	47	219	35	21	69.12 (56.60–79.46)	86.22 (81.22–90.09)	57.32 (45.92–68.02)	91.25 (86.75–94.37)	82.61 (78.05–87.16)	0.777 (0.727–0.821)
P value										<0.001
CABG
ML-CCTA	73	230	6	13	84.88 (75.17–91.39)	97.46 (94.28–98.96)	92.41 (83.61–96.87)	94.65 (90.81–97.00)	94.10 (91.45–96.75)	0.912 (0.875–0.940)
CCTA	60	225	11	26	69.77 (58.78–78.96)	95.34 (91.59–97.53)	84.51 (73.54–91.65)	89.64 (85.03–93.00)	88.51 (84.81–92.21)	0.826 (0.780–0.865)
P value										0.003

AUC, area under the curve; CABG, coronary artery bypass graft; CCTA, coronary computed tomography angiography; FN, false negative; FP, false positive; ML, machine learning; ML-CCTA, machine learning-based CCTA; NPV, negative predictive value; PCI, percutaneous coronary intervention; PPV, positive predictive value; TN, true negative; TP, true positive.

In total, 86 patients fulfilled the CABG surgery criteria using ICA as a standard diagnostic tool, 79 patients met the criteria using ML-CCTA, and 71 patients met the criteria using CCTA. The sensitivity, specificity, PPV and NPV, and accuracy of ML-CCTA for selecting CABG candidates were 84.88%, 97.46%, 92.41%, 94.65%, and 94.10%, respectively. The sensitivity, specificity, PPV and NPV, and accuracy of CCTA for selecting CABG candidates were 69.77%, 95.34%, 84.51%, 89.64%, and 88.51%, respectively. The AUC of ML-CCTA for selecting CABG candidates was significantly higher than CCTA (0.912, 95% CI = 0.875–0.940 vs. 0.826, 95% CI = 0.780–0.865; P = 0.003) (Fig. 5b).

Treatment with follow-up

In addition to the accuracy investigation, we further followed the real treatment of the included patients to provide more details for the outcome of the patients. During the follow-up (mean = 4.6 months; range = 0–16 months), 15 of the 322 patients underwent CABG surgery, 122 underwent PCI, and 185 received optimal medical treatment. However, there are significant differences between treatment decisions based on ML-CCTA, CCTA, or ICA and real treatment (Supplementary Table 2).

Discussion

The present study demonstrated that ML-based CCTA could distinguish between patients who need revascularization and those who do not. In addition, ML-based CCTA showed a better performance than CCTA in making an appropriate decision for patients and selecting a suitable revascularization strategy following the 2018 ESC/EACTS Guidelines on myocardial revascularization.

Previous studies showed that CCTA-based CABG candidate selection exhibited high diagnostic accuracy with a sensitivity of 83.3%–96.5%, specificity of 90.9%–97.3%, PPV of 88.3%–93.8%, and NPV of 90.7%–98.6% (10,22 –24). The specificity, NPV, and accuracy of CCTA- or ML-CCTA-based CABG candidate selection in the present study were comparable to those of previous studies. However, the sensitivity and PPV of CCTA were relatively low. This may have been partly because of the different CABG surgery guidelines used (ACC/AHA 2004 guideline update for coronary artery bypass graft surgery, 2011 ACCF/AHA CABG surgery guidelines, and 2014 ESC/EACTS guidelines). CABG surgery selection criteria in the 2018 ESC/EACTS guidelines we used had some changes: three-vessel CAD with diabetes mellitus was recommended for CABG surgery. Moreover, two-vessel disease with proximal LAD stenosis was also recommended for CABG surgery in the present study. The results of the present study demonstrated that ML-CCTA performed more accurately in identifying patients with or without the need for revascularization compared to CCTA, using ICA as the reference standard.

Recently, Tesche et al. (25) reported the use of a machine learning algorithm-based CT-FFR value to assess the functional significance of CAD and make the therapeutic decision. They showed excellent performance (sensitivity of 97%, specificity of 100%, PPV of 100%, NPV of 97%, and accuracy of 99%) of ML-based CT-FFR with CCTA in determining the appropriate treatment strategy. However, the use of FFR or CT-FFR remains limited in China, France, and the United States (26 –28). In China, FFR measurement costs an extra 10,000 Yuan and financial support is still lacking (26). Moreover, as the only clinical and commercial CT-derived FFR evaluation product that has been proved by the FDA, HeartFlow FFR_CT requires transmission of CT datasets to its offsite core laboratory, which requires 8 h for processing and analysis and costs up to $1500 without any guarantee of reimbursement (29). In addition, the requisite offsite handling of sensitive and confidential patient information and associated medical conditions is a highly delicate issue involving IT security, potential for data abuse, etc. (30).

ML has been increasingly applied across the diagnostic and prognostic applications of CAD (31 –34). However, most of those previous studies did not consider decisions regarding the selection of a treatment strategy. The principal focus of our study was to compare our ML-CCTA-based decision-making with CCTA-based decision-making. We found that the ML-CCTA-based program was superior to the CCTA-based method for both appropriateness of revascularization and determining the treatment strategy when considering ICA as the reference standard.

The prognostic value of the SYNTAX score has been extensively discussed. However, the low intra- and inter-observer reproducibility of the SYNTAX score has been clearly reported (35). ML assistance could result in higher inter-observer reproducibility (36). The correlation between ML-CCTA-based SYNTAX score and ICA-based SYNTAX score was strong, and slightly higher than that between CCTA-based SYNTAX score and ICA-based SYNTAX score in the present study and previous studies (23,37,38). To the best of our knowledge, the present study is the first to analyze the value of ML-CCTA-based SYNTAX score for selecting revascularization strategy candidates who underwent diagnostic imaging.

CAD-RADS can now provide guidance on clinical decision-making through non-invasive imaging and may play an important role in the bridge between test abnormalities and optimal patient care. The recent multinational CONFIRM study suggested that incorporating CAD-RADS into CCTA reports might provide a novel opportunity to promote evidence-based care after CCTA (39). A strong correlation was shown between the ML-CCTA-based CAD-RADS classification and CCTA-based CAD-RADS classification in the present study. In addition, the CAD-RADS scoring system based on ML was faster, more objective, and repeatable in determining CAD-RADS grades by measuring stenosis degree compared to manual measurement. A previous study demonstrated that a structured reporting platform with automated calculation of the CAD-RADS score might improve data quality and support the standardization of clinical decision-making (40). However, this reporting platform automatically determined the CAD-RADS category that should be based on the data provided by the readers, although that remained hidden to the readers.

The present study has some limitations. First, revascularization candidates who were selected on the basis of ICA, ML-CCTA, or CCTA did not reflect real revascularization data according to the results of the follow-up. Setting up which decision is the right one would be more challenging but would actually provide the value of the decision made by comparison with patient outcomes. Some patients who fulfilled the revascularization criteria underwent optimal medical treatment instead of revascularization, possibly because of the patient's or physician's preference, cost, co-morbidity issues, or other clinical considerations, such as the burden of ischemia. In addition, the functional information of the coronary artery is needed in decision-making for the revascularization strategy. Multidisciplinary decision-making in a Heart Team is needed to analyze and interpret the available diagnostic evidence, determine the need for myocardial revascularization, and assess the relative short- and long-term safety and efficacy of the percutaneous and surgical options (7). Second, we did not include patients with stents, grafts, or other cardio surgeries. Third, hemodynamic evaluation was not performed in the present study and the long-term prognostic value of ML-CCTA-based decision-making remains to be further investigated. Forth, two different teams made the decisions for CCTA and ML-CCTA, respectively, without taking into account the inter-reader variability. Finally, in this study, only patients with CCTA referred for ICA were included, meaning that they had an abnormal CCTA or other characteristics that increases the a priori risk of CAD requiring revascularization. This does not represent the general population who undergo CCTA in whom the a priori risk of revascularization is lower. To properly evaluate the accuracy of ML-CCTA, all patients who underwent CCTA should be included. Therefore, the current numbers will be biased.

In conclusion, ML-CCTA could distinguish between patients who need revascularization and those who do not. In addition, ML-CCTA was slightly superior to CCTA in making an appropriate decision for patients and selecting a suitable revascularization strategy.

Supplemental Material

sj-docx-1-acr-10.1177_02841851231158730 - Supplemental material for Performance of machine learning-based coronary computed tomography angiography for selecting revascularization candidates

Supplemental material, sj-docx-1-acr-10.1177_02841851231158730 for Performance of machine learning-based coronary computed tomography angiography for selecting revascularization candidates by Zengfa Huang, Yi Ding, Yang Yang, Shengchao Zhao, Shutong Zhang, Jianwei Xiao, Chengyu Ding, Ning Guo, Zuoqin Li, Shiguang Zhou, Guijuan Cao and Xiang Wang in Acta Radiologica

Supplemental Material

sj-docx-2-acr-10.1177_02841851231158730 - Supplemental material for Performance of machine learning-based coronary computed tomography angiography for selecting revascularization candidates

Supplemental material, sj-docx-2-acr-10.1177_02841851231158730 for Performance of machine learning-based coronary computed tomography angiography for selecting revascularization candidates by Zengfa Huang, Yi Ding, Yang Yang, Shengchao Zhao, Shutong Zhang, Jianwei Xiao, Chengyu Ding, Ning Guo, Zuoqin Li, Shiguang Zhou, Guijuan Cao and Xiang Wang in Acta Radiologica

Supplemental Material

sj-docx-3-acr-10.1177_02841851231158730 - Supplemental material for Performance of machine learning-based coronary computed tomography angiography for selecting revascularization candidates

Supplemental material, sj-docx-3-acr-10.1177_02841851231158730 for Performance of machine learning-based coronary computed tomography angiography for selecting revascularization candidates by Zengfa Huang, Yi Ding, Yang Yang, Shengchao Zhao, Shutong Zhang, Jianwei Xiao, Chengyu Ding, Ning Guo, Zuoqin Li, Shiguang Zhou, Guijuan Cao and Xiang Wang in Acta Radiologica

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Scientific Research Project of the Wuhan Municipal Health Commission (WX2019B02 and WX20D49).

ORCID iD

Xiang Wang

Supplemental Material

Supplemental material for this article is available online.

References

Knuuti

Wijns

Saraste

, et al. 2019 ESC guidelines for the diagnosis and management of chronic coronary syndromes. Eur Heart J 2020;41:407–477.

Kelion

Nicol

. The rationale for the primacy of coronary CT angiography in the National Institute for Health and Care Excellence (NICE) guideline (CG95) for the investigation of chest pain of recent onset. J Cardiovasc Comput Tomogr 2018;12:516–522.

Chang

Lin

Gebow

, et al. Selective referral using CCTA versus direct referral for individuals referred to invasive coronary angiography for suspected CAD: a randomized, controlled, open-label trial. JACC Cardiovasc Imaging 2019;12:1303–1312.

Takagi

Tanaka

Nagata

, et al. Diagnostic performance of coronary CT angiography with ultra-high-resolution CT: comparison with invasive coronary angiography. Eur J Radiol 2018;101:30–37.

Budoff

Dowe

Jollis

, et al. Diagnostic performance of 64-multidetector row coronary computed tomographic angiography for evaluation of coronary artery stenosis in individuals without known coronary artery disease: results from the prospective multicenter ACCURACY (Assessment by Coronary Computed Tomographic Angiography of Individuals Undergoing Invasive Coronary Angiography) trial. J Am Coll Cardiol 2008;52:1724–1732.

Andreini

Pontone

Mushtaq

, et al. A long-term prognostic value of coronary CT angiography in suspected coronary artery disease. JACC Cardiovasc Imaging 2012;5:690–701.

Neumann

Sousa-Uva

Ahlsson

, et al. 2018 ESC/EACTS guidelines on myocardial revascularization. Eur Heart J 2019;40:87–165.

Lin

Dunning

Narula

, et al. Impact of an automated multimodality point-of-order decision support tool on rates of appropriate testing and clinical decision making for individuals with suspected coronary artery disease: a prospective multicenter study. J Am Coll Cardiol 2013;62:308–316.

Williams

Hunter

Shah

ASV

, et al. Use of coronary computed tomographic angiography to guide management of patients with coronary disease. J Am Coll Cardiol 2016;67:1759–1768.

10.

Kim

Lee

Kim

, et al. Coronary computed tomography angiography for selecting coronary artery bypass graft surgery candidates. Ann Thorac Surg 2013;95:1340–1346.

11.

Singh

Al'Aref

Van Assen

, et al. Machine learning in cardiac CT: basic concepts and contemporary data. J Cardiovasc Comput Tomogr 2018;12:192–201.

12.

van Rosendael

Maliakal

Kolli

, et al. Maximization of the usage of coronary CTA derived plaque information using a machine learning based algorithm to improve risk stratification; insights from the CONFIRM registry. J Cardiovasc Comput Tomogr 2018;12:204–209.

13.

Brandt

Emrich

Schoepf

, et al. Ischemia and outcome prediction by cardiac CT based machine learning. Int J Cardiovasc Imaging 2020;36:2429–2439.

14.

Huang

Xiao

Wang

, et al. Clinical evaluation of the automatic coronary artery disease reporting and data system (CAD-RADS) in coronary computed tomography angiography using convolutional neural networks. Acad Radiol 2022. DOI: 10.1016/j.acra.2022.05.015.

15.

Patel

Calhoon

Dehmer

, et al. ACC/AATS/AHA/ASE/ASNC/SCAI/SCCT/STS 2017 appropriate use criteria for coronary revascularization in patients with stable ischemic heart disease: a report of the American College of Cardiology Appropriate Use Criteria Task Force, American Association for Thoracic Surgery, American Heart Association, American Society of Echocardiography, American Society of Nuclear Cardiology, Society for Cardiovascular Angiography and Interventions, Society of Cardiovascular Computed Tomography, and Society of Thoracic Surgeons. J Am Coll Cardiol 2017;69:2212–2241.

16.

Sianos

Morel

Kappetein

, et al. The SYNTAX score: an angiographic tool grading the complexity of coronary artery disease. EuroIntervention 2005;1:219–227.

17.

Huang

Xiao

, et al. Evaluation of LVDD by CCTA with dual-source CT in type 2 diabetes Mellitus patients. Current Med Imaging 2020;16:214–223.

18.

Huang

Xiao

Xie

, et al. The correlation of deep learning-based CAD-RADS evaluated by coronary computed tomography angiography with breast arterial calcification on mammography. Sci Rep 2020;10:11532.

19.

. 2014 SCCT guidelines for the interpretation and reporting of coronary CT angiography: a report of the Society of Cardiovascular Computed Tomography Guidelines Committee. J Cardiovasc Comput Tomogr 2015;9:e3.

20.

Huang

Zhang

Jin

, et al. Prognostic value of CAD-RADS classification by coronary CTA in patients with suspected CAD. BMC Cardiovasc Disord 2021;21:476.

21.

Han

Liu

Sun

, et al. Deep learning analysis in coronary computed tomographic angiography imaging for the assessment of patients with coronary artery stenosis. Comput Methods Programs Biomed 2020;196:105651.

22.

Suh

Hong

Lee

, et al. Accuracy of CT for selecting candidates for coronary artery bypass graft surgery: combination with the SYNTAX score. Radiology 2015;276:390–399.

23.

Lee

Han

Hur

, et al. Accuracy of computed tomography for selecting the revascularization method based on SYNTAX score II. Eur Radiol 2018;28:2151–2158.

24.

Lee

Kim

, et al. Diagnostic accuracy of 64-slice multidetector computed tomography for selecting coronary artery bypass graft surgery candidates. J Thorac Cardiovasc Surg 2011;141:571–577.

25.

Tesche

Vliegenthart

Duguay

, et al. Coronary computed tomographic angiography-derived fractional flow reserve for therapeutic decision making. Am J Cardiol 2017;120:2121–2127.

26.

Tang

Song

, et al. Fractional flow reserve guided percutaneous coronary intervention improves clinical outcome with reduced cost in contemporary clinical practice. Chin Med J (Engl) 2015;128:2000–2005.

27.

Puymirat

Muller

Sharif

, et al. Fractional flow reserve: concepts, applications and use in France in 2010. Arch Cardiovasc Dis 2010;103:615–622.

28.

Dattilo

Prasad

Honeycutt

, et al. Contemporary patterns of fractional flow reserve and intravascular ultrasound use among patients undergoing percutaneous coronary intervention in the United States: insights from the National Cardiovascular Data Registry. J Am Coll Cardiol 2012;60:2337–2339.

29.

Benton

Jr. Tesche

De Cecco

, et al. Noninvasive derivation of fractional flow reserve from coronary computed tomographic angiography: A review. J Thorac Imaging 2018;33:88–96.

30.

Zhong

Zhang

, et al. Application of patient-specific computational fluid dynamics in coronary and intra-cardiac flow simulations: challenges and opportunities. Front Physiol 2018;9:742.

31.

Betancur

Otaki

Motwani

, et al. Prognostic value of combined clinical and myocardial perfusion imaging data using machine learning. JACC Cardiovasc Imaging 2018;11:1000–1009.

32.

Al'Aref

Maliakal

Singh

, et al. Machine learning of clinical variables and coronary artery calcium scoring for the prediction of obstructive coronary artery disease on coronary computed tomography angiography: analysis from the CONFIRM registry. Eur Heart J 2020;41:359–367.

33.

Sparapani

Dabbouseh

Gutterman

, et al. Detection of left ventricular hypertrophy using Bayesian additive regression trees: the MESA. J Am Heart Assoc 2019;8:e009959.

34.

Johnson

Zhao

, et al. Scoring of coronary artery disease characteristics on coronary CT angiograms by using machine learning. Radiology 2019;292:354–362.

35.

Garot

Tafflet

Kumar

, et al. Reproducibility and factors influencing the assessment of the SYNTAX score in the left main Xience study. Catheter Cardiovasc Interv 2012;80:231–237.

36.

Bien

Rajpurkar

Ball

, et al. Deep-learning-assisted diagnosis for knee magnetic resonance imaging: development and retrospective validation of MRNet. PLoS Med 2018;15:e1002699.

37.

Papadopoulou

Girasis

Dharampal

, et al. CT-SYNTAX score: a feasibility and reproducibility study. JACC Cardiovasc Imaging 2013;6:413–415.

38.

Kerner

Abadi

Abergel

, et al. Direct comparison between coronary computed tomography and invasive angiography for calculation of SYNTAX score. EuroIntervention 2013;8:1428–1434.

39.

Xie

Cury

Leipsic

, et al. The coronary artery disease-reporting and data system (CAD-RADS): prognostic and clinical implications associated with standardized coronary computed tomography angiography reporting. JACC Cardiovasc Imaging 2018;11:78–89.

40.

Szilveszter

Kolossvary

Karady

, et al. Structured reporting platform improves CAD-RADS assessment. J Cardiovasc Comput Tomogr 2017;11:449–454.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.21 MB

0.02 MB