Abstract
Background
Coronary angiography is fundamental for the diagnosis and treatment of coronary artery disease. Manual quantitative coronary angiography (QCA) is accurate and reproducible; however, it is time-consuming and labor-intensive. However, recent advancements in artificial intelligence (AI) have enabled automated and rapid analysis of medical images, addressing the need for real-time quantitative coronary analysis.
Aims
This study aimed to evaluate the accuracy of AI-based QCA (AI-QCA) compared with that via manual QCA and clinician acceptance.
Methods
This retrospective, single-center study was conducted in two phases. Phase 1 was a pilot study comparing AI-QCA with manual QCA and visual estimation. It involved 15 patients who underwent coronary angiography at Seoul National University Bundang Hospital between September 2011 and July 2021. Phase 2 included a larger cohort of 762 patients, with 1002 coronary angiograms analyzed between May 2020 and April 2021.
Results
In phase 1, AI-QCA and manual QCA consistency varied among the observers, with AI-QCA showing superior consistency compared with visual estimation. However, a strong correlation between AI-QCA and manual-QCA was found in phase 2. AI-QCA accurately identified and quantitatively analyzed multiple lesions in the major vessels, providing results comparable with those of manual QCA.
Conclusions
AI-QCA demonstrated high concordance with manual QCA, offering real-time analysis and reduced workload. Therefore, AI-QCA has the potential to be a valuable tool for diagnosing and treating coronary artery disease, necessitating further studies for clinical validation.
Keywords
Introduction
Coronary angiography is a fundamental tool for the diagnosis and treatment of coronary artery disease.1,2 Identifying the severity of stenosis using angiography is usually the starting point for assessing ischemia.3,4 The choice of a stent size is frequently based on only angiography.5,6 Manual quantitative coronary angiography (QCA) has proven to be accurate and reproducible for assessing vessel size and stenosis severity.7–9 Seldom is QCA used even in recent clinical trials, in which visual estimation has become the norm. 10 Additionally, its use in clinical practice is currently limited because it is time-consuming and labor-intensive. 5 Another reason is the introduction of intracoronary imaging, despite the relatively low penetration of invasive imaging methods.
Therefore, there is an unmet clinical need for a convenient, real-time quantitative coronary analysis. Recent advances in artificial intelligence (AI) have enabled automated and rapid analysis of medical images without human error.11–13 Furthermore, understanding coronary angiography requires a long training period, even for cardiology specialists, making it a good candidate for AI applications.14,15 Therefore, in this study, we tested the accuracy of AI-based QCA (AI-QCA) compared with that via conventional manual QCA and evaluated its acceptance by clinicians.
Methods & materials
This retrospective analysis was conducted at a single center and included two separate phases. Phase 1 was a pilot study with a small sample size in which AI-QCA was compared with manual QCA and visual estimation. Phase 2 was conducted using a larger sample size to compare AI-QCA with manual QCA.
Enrollment criteria: Phase 1 pilot study
The phase 1 pilot study enrolled 15 patients aged ≥18 years who underwent coronary angiography and had significant stable coronary stenosis between September 2011 and July 2021 at the Seoul National University Bundang Hospital. Patients with total occlusion, acute myocardial infarction, or lesions located in a vessel with a diameter of <2.0 mm were excluded. The inclusion criteria were stratified by diseased coronary vessels and lesion length (LL). The coronary vessel included the left anterior descending artery, 4 left circumflex artery (LCX), or right coronary artery (RCA). However, the focal lesion was defined as <10 mm in LL and the diffuse one as ≥20 mm. Finally, the included patients were those with a single focal lesion at the LAD (n = 3), a diffuse lesion at the LAD (n = 3), a single focal lesion at the LCX (n = 3), a single focal lesion at the RCA (n = 3), or a diffuse lesion at the RCA (n = 3). The study protocol was approved and the mandate for obtaining informed consent was waived by the Seoul National Bundang Hospital Institutional Review Board (approval number: B-2109-708-112), because the research was deemed to present no more than minimal risk of harm to the participants.
Enrollment criteria: Phase 2 study
Phase 2 study was designed as a retrospective evaluation of coronary angiography with a larger sample size. Patients aged ≥18 years who underwent coronary angiography between May 2020 and April 2021 at Seoul National University Bundang Hospital and received at least one percutaneous coronary intervention were included in this study. Patients without full angiographic images before percutaneous coronary intervention, lesions in the left main coronary artery, vessels with a reference diameter of < 2 mm, and complete occlusion with thrombolysis in myocardial infarction flow grade 0 were excluded from the study. Patients with lesions unsuitable for AI-QCA or manual QCA were also excluded. Finally, 762 patients with 1002 coronary angiograms were selected out of the 1086 potentially eligible patients (Supplemental Figure 1). The study protocol was approved by the Seoul National Bundang Hospital Institutional Review Board (approval number B-2406-906-106). The Board determined that the research involved no more than minimal risk, and waived the requirement for informed consent.
Study variables and angiographic analysis
The study variables were the reference diameter (RD), proximal RD, distal RD, minimum lumen diameter (MLD), percent diameter stenosis (%DS), and LL. The investigator selected the best angle after reviewing all the coronary angiography images of each patient.
AI-QCA was performed using MPXA-2000 software (Medipixel, Seoul, Korea) following the vendor's protocol, which was approved by the Korean Ministry of Food and Drug Safety and the US Food and Drug Administration (regulation number: 21 CFR 892.2050). A brief overview of the AI-QCA process is shown in Figure 1. The software employs an ensemble of three neural networks—DeepLabV3+, U-Net++, and U2-Net—to perform vessel recognition and stenosis detection. 16 The deep-learning models were trained using 7658 images from 3129 patients who underwent coronary angiography at Asan Medical Center and Chungnam National University Hospital between February 2016 and November 2016.The software automatically chose the best frame and provided study variables without human intervention. Manual QCA was performed by an independent dedicated core laboratory using the CAAS software (Pie Medical, the Netherlands). An experienced interventional cardiologist performed the visual estimation.

Steps of AI-QCA process. (A) DICOM upload page for web-based AI-QCA version. (B) The best frame for QCA is recommended: 42nd frame out of 111 frames in this case. One can play the video by clicking the play button, or switch to other frames by clicking the forward/backward buttons or dragging the slide. When “Analyze” button is clicked, (C) AI-assisted QCA is performed. In this case, the most severe lesion in the mid-left anterior descending artery is shown first. P and D indicate proximal and distal lesion margins, respectively. Lesion length, diameter stenosis (DS), minimum lumen diameter (MLD), proximal reference diameter (D), distal reference diameter (D), and reference diameter (RD) are shown. Estimated lumen diameter along the vessel is shown in the diagram. (D) By clicking “S#1” tab, analysis on diagonal branch is presented. AI-QCA: artificial intelligence-based quantitative coronary angiography; DICOM, Digital Imaging and Communications in Medicine; QCA, quantitative coronary angiography.
Statistical analysis
In the phase 1 study, the results of AI-QCA were compared with those of manual QCA and visual estimation. Manual QCA and visual estimation were performed by multiple observers: manual QCA by three independent dedicated core labs and visual estimation by 4 independent cardiologists (2 with <5 years of independent experience and 2 with 5–15 years of experience). In addition, the cardiologists were requested to respond to the survey. Two QCA images were obtained: one acquired using AI-QCA and the other using manual QCA. The cardiologists were asked two questions: (1) Which image do you think was analyzed using AI-QCA? and (2) Which image do you agree with more? Interrater agreement for manual QCA and visual assessment was evaluated using intraclass correlation coefficients (ICC). The correlation between the two measurement methods was assessed using Pearson's correlation coefficient. The values from the three core laboratories and four cardiologists for each study variable were averaged and treated as a single value. ICC values were interpreted as poor, moderate, good, or excellent. 17
Phase 2 study compared the results of AI-QCA and manual QCA performed by a single observer. The correlation between the AI-QCA and manual QCA measurements was represented by a scatter plot and evaluated using Pearson's correlation coefficient. The Bland-Altman plot was used to visualize the agreement between the two methods. A subgroup analysis was performed based on vessel involvement.
Continuous variables are expressed as mean ± standard deviation, and categorical variables are expressed as numbers and percentages. The ICC and Pearson correlation coefficients and 95% confidence intervals were provided. All statistical analyses were performed using the R version 4.1.2 (http://www.R-project.org; R Foundation for Statistical Computing, Vienna, Austria).
Results
Phase 1: A pilot study
The phase 1 study was a pilot test with a small sample size that enrolled 15 patients with 15 simple lesions (1 lesion per 1 patient), stratified based on lesion location and length. The mean age of the 15 study participants was 70.5 years, and 8 (53.3%) were male patients. The ICC for manual QCA indicated moderate consistency across the three independent core laboratories, except for poor consistency for LL (Supplemental Table 1). Visual estimation by the four independent interventional cardiologists showed better consistency between each other for MLD, %DS, and LL.
The ICC between AI-QCA and manual-QCA varied among the observers (Table 1). In addition, different human observers showed agreement between 0.43 and 0.73 with each other. The correlation coefficients between AI-QCA and manual QCA were good or very good, except for LL (Table 2). Visual assessment also showed good correlation with manual QCA. The scatter plot and Bland-Altman plot for AI-QCA and manual QCA are shown in Supplemental Figures 2 and 3, respectively. The agreement level was numerically higher for AI-QCA and visual estimation, particularly for MLD and %DS (Supplemental Table 1).

Scatter plot analysis between AI-QCA and manual QCA (phase 2). The plot shows strong correlations across all measurements: proximal RD, distal RD, RD, MLD, %DS, and LL. Gray regions indicate 95% confidence intervals for each measurement. AI-QCA, artificial intelligence-based quantitative coronary angiography; %DS, percent diameter stenosis; LL, lesion length; MLD, minimal lumen diameter; QCA, quantitative coronary angiography; RD, reference diameter.

Bland-Altman plot between AI-QCA and manual QCA (phase 2). The points are distributed around the central line (black solid line) and fall within the limits of agreement (red dashed lines), indicating a high concordance between the two methods. AI-QCA, artificial intelligence-based quantitative coronary angiography; QCA, quantitative coronary angiography.
Intraclass correlation coefficient indicating the consistency of lesion parameters between AI-QCA versus manual QCA by different observers in phase 1 pilot study (n = 15).
AI-QCA: artificial intelligence-based quantitative coronary angiography; ICC: intraclass correlation coefficients.
Correlation coefficients and p values of lesion parameters of AI-QCA and visual assessment with manual QCA calculated using Pearson's method.
AI-QCA: artificial intelligence-based quantitative coronary angiography.
A survey of participating cardiologists was conducted to determine how well they could distinguish between and agree on the two different QCA modalities. Notably, 43 of the 60 respondents (71.7%) correctly distinguished which images were created using AI-QCA or manual QCA; 42 (70.0%) replied that they agreed more with manual QCA than AI-QCA.
Phase 2: A retrospective study
Based on the results of the phase 1 pilot study, a retrospective analysis was conducted with a larger sample size. In total, 1002 coronary angiograms from 762 patients were included in this study. The mean age of the study population was 71.4 years (Table 3), and 73.4% were men. Notably, 23.5% of those who responded (533 participants) were current smokers. Hypertension, diabetes, and chronic kidney disease were prevalent in 57.1%, 36.5%, and 4.3% of the participants, respectively. The clinical diagnoses in the study population included acute coronary syndrome (41.6%) and stable coronary disease (58.4%). The lesion locations were 50.4%, 19.6%, and 30.5% in the left anterior descending, left circumflex, and right coronary arteries, respectively. The mean RD, MLD, %DS, and LL were 2.80 mm, 1.06 mm, 62%, and 23.6 mm, respectively (Table 4).
Clinical characteristics of the study population.
HDL: high-density lipoprotein; LDL: low-density lipoprotein; NSTEMI: non-ST-elevation myocardial infarction; STEMI: ST-elevation myocardial infarction.
Characteristics of the study lesion.
AI-QCA showed strong to very strong correlations with statistical significance across the study variables, including proximal and distal RD, RD, MLD, and LL (Figure 2). The correlation coefficient for %DS was 0.55, indicating moderate correlation (Table 5). The Bland-Altman plots in Figure 3 reveal high concordance between the two methods for the study variables. The mean values of the proximal and distal RD, RD, MLD, %DS, and LL measured using AI-QCA were similar to those measured using manual QCA. The measurements were evenly distributed around the central line and were within the limits of agreement. Correlation was consistent across lesion locations (Supplemental Table 2).
Pearson's correlation coefficients of lesion parameters measured by AI-QCA and manual QCA in phase 2 study (N = 1002).
AI-QCA: artificial intelligence-based quantitative coronary angiography; CI: confidence interval.
Discussion
This study demonstrated that lesion descriptions acquired using AI-QCA could correlate well with those acquired using manual QCA by human experts. We also found that manual QCA observers showed imperfect agreement and that the agreement between AI-QCA and manual QCA was similar to that between the observers. AI-QCA showed good agreement with the eyeball estimation of experienced cardiologists. Also, AI-QCA was highly practical because it could provide QCA results upon only a single click.
Recent advances in machine learning technology have boosted the clinical applications of AI in healthcare. However, understanding and interpreting coronary angiography images is challenging for several reasons. Complex three-dimensional cardiac structures can be represented by two-dimensional images. 18 Coronary angiography is performed as a video encompassing more than two to three heartbeats. Furthermore, a single still image does not contain sufficient information owing to heartbeats, respiratory movements, and patient motion artifacts. 19 Other reasons include the high variation in normal personal anatomy and discrepancies between anatomical and physiological abnormalities. 20 The AI-QCA application tested in this study was powered by machine learning algorithms that enabled the identification and segmentation of coronary arteries. Every QCA step, such as the best frame choice and lesion analysis, is fully automated to resemble manual QCA performed by human analysts.
These findings are consistent with those of previous studies. In Kim et al. 16 demonstrated a strong correlation between AI-QCA and manual QCA for identifying lumen diameters. Similarly, Moon et al. 21 showed a moderate to strong correlation between AI-QCA and intravascular ultrasound when analyzing coronary lesions with significant stenosis.
In addition, recent studies have suggested that the use of an AI system could help operators estimate lesion severity and reduce interobserver variability. Du et al. 22 also built a deep neural network model for the identification of coronary artery segments and recognition of lesion morphology, suggesting that it might help cardiologists flag and diagnose lesion severity and morphology during coronary intervention. Avram et al. 23 showed that multiple purpose-built neural networks accomplished automated analysis of coronary angiograms and suggested that it could increase standardization and reproducibility in angiographic coronary stenosis assessment. The same group developed a video-based AI model for coronary artery disease analysis demonstrating enhanced precision and reliability. 24
Our data suggest that AI-QCA has the potential to replace manual QCA in supporting clinicians in decision making. Lesion assessments made using AI-QCA were generally consistent with those made using manual QCA. The correlation between AI-QCA and manual-QCA was comparable with that of visual estimation by experienced human experts. Moreover, its advantages over manual QCA include the ability to provide real-time analysis that can be visualized within seconds and the reduction in workload on trained personnel.
The finding that the correlation between AI-QCA and manual-QCA was not perfect requires further discussion. In this study, we found that the agreement between human observers was far from perfect. This finding suggests that variations in manual QCA exist even among human experts in lesion selection and contour delineation. Notably, %DS by manual QCA is frequently used as a ground truth in studies23,24; however, earlier studies have highlighted the issue of significant interobserver bias in interpreting coronary angiography images. 15 The core machine learning algorithm of the AI-QCA was trained to replicate the manner in which human experts estimated coronary lesions. Manual QCA is not a perfect ground truth that every human observer agrees. However, AI-QCA can hardly resemble every manual QCA.
The present study has some limitations. First, this was a single-center study, and the findings cannot be generalized to other populations. Second, a priori sample size calculation was not performed in the two studies. Phase 1 study was planned with a small sample size and served as a pilot study. Phase 2 study intended to include approximately 1000 angiograms and enrolled all comers during a specific period with minimal exclusion criteria.
Conclusions
This study demonstrated that AI-QCA was highly concordant with manual QCA. Prompt and accurate lesion estimation using AI-QCA can aid clinicians in making better clinical decisions during coronary angiography and angioplasty. However, further studies are needed to validate its role in diagnosing and treating cardiovascular diseases when it is integrated into clinical practice.
Supplemental Material
sj-docx-1-dhj-10.1177_20552076241306937 - Supplemental material for Validation of artificial intelligence-based quantitative coronary angiography
Supplemental material, sj-docx-1-dhj-10.1177_20552076241306937 for Validation of artificial intelligence-based quantitative coronary angiography by Do-Hyun Kim, Sun-Hwa Kim, Hyun-Wook Chu, Si-Hyuck Kang, Chang-Hwan Yoon, Tae-Jin Youn and In-Ho Chae in DIGITAL HEALTH
Footnotes
Abbreviations
Author contributions
H-WC was involved in conceptualization, methodology, and Software; H-WC in data curation and writing—original draft preparation; D-HK and S-HK in visualization and investigation; S-HK in supervision; DHK and S-HK in software and validation; and S-HK, D-HK, and S-HK in writing—reviewing and editing. All authors reviewed the text and approved the final version of the manuscript for publication.
Consent to participate
The IRB waived the requirement for informed consent as the study was assessed to present only minimal risk to participants.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethical considerations
Both study protocols were approved by the Institutional Review Board (IRB) of Seoul National University Bundang Hospital (Approval number: B-2406-906-106, B-2406-906-106). The IRB determined that this research involved no more than minimal risk to the participants.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Medipixel Inc. supported phase 1 of this study, and the Bio&Medical Technology Development Program of the National Research Foundation (NRF) funded by the Korean government (MSIT) (No. RS-2023-00222910) supported phase 2. We extend our gratitude to the team at the Seoul National University Bundang Hospital for their assistance with data collection. We also thank Medipixel Inc. for their support with the AI-QCA software and the independent core laboratories and interventional cardiologists for their contributions to manual QCA and visual estimations.
Data availability
The datasets generated during and/or analyzed during the current study are not publicly available due privacy issues but are available from the corresponding author on reasonable request.
Impact on daily practice
AI-QCA improves diagnostic accuracy and efficiency through real-time analysis, significantly reducing the workload on clinicians. This technology enables rapid decision making and provides standardized, consistent results, making it a beneficial tool for the diagnosis and treatment of coronary artery disease. Integrating AI-QCA into clinical practice could enhance the quality of patient care and assist clinicians in making informed decisions, potentially leading to better patient outcomes.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
