Abstract
Background:
The no-reflow (NRF) phenomenon is the “Achilles heel” of interventionists after performing percutaneous coronary intervention (PCI) in patients with ST-segment elevation myocardial infarction (STEMI). No definitive treatment has been proposed for NRF, and preventive strategies are central to improving care for patients who develop NRF.
Objectives:
In this study, we aim to investigate the clinical prediction models developed to predict NRF in STEMI patients undergoing primary PCI.
Design:
Systematic review.
Data sources and methods:
Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were observed. Studies that developed clinical prediction modeling for NRF after primary PCI in STEMI patients were included. Data extraction was performed using the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies (CHARMS) checklist. The Prediction Model Risk of Bias Assessment Tool (PROBAST) tool was used for critical appraisal of the included studies.
Results:
The three most common predictors were age, total ischemic time, and preoperative thrombolysis in myocardial infarction flow grade. Most of the included studies internally validated their developed model via various methods: random split, bootstrapping, and cross-validation. Only three studies (18%) externally validated their model. Six studies (37%) reported a calibration plot with or without the Hosmer–Lemeshow test. The reported area under the curve ranged from 0.648 to 0.925. The most common biases were in the statistical domain.
Conclusion:
Clinical prediction models aid in individualizing care for STEMI patients with NRF after primary PCI. Of the 16 included studies, we report four to have a low risk of bias and low concern with regard to our research question, which should undergo external validation with or without updating in future studies.
Jargon summary
- Search Results: Out of 7,095 citations, 16 studies were reviewed.
- Study Locations: Studies were mainly conducted in China, Turkey, and a few other countries.
- Predictors: Common predictors of NRF included age, total ischemic time, and preoperative blood flow. The models used various data like patient demographics, lab results, and clinical observations.
- Model Performance: Models showed varying levels of accuracy in predicting NRF. Most models used logistic regression for development. Internal validation was done in several studies, but external validation was limited.
- Clinical Use: Predictive models can help in timely decision-making during PCI. However, current models need more validation and improvement in their design to be widely accepted in clinical practice.
- Future Research: More robust, prospective studies are needed, especially in diverse populations, to develop and validate better prediction models.
Keywords
Introduction
Acute myocardial infarction (MI) is one of the most severe complications of coronary artery disease.1,2 ST-segment elevation myocardial infarction (STEMI) is a type of acute coronary syndrome that is associated with significant morbidity and mortality. Therefore, prompt myocardial reperfusion is central to preventing major adverse cardiovascular events in STEMI patients.3,4 Primary percutaneous coronary intervention (PCI) is the reperfusion strategy of choice for patients presenting with STEMI.2,5
Despite the high success rates of achieving normal coronary flow after PCI, optimal blood flow cannot be established in all cases. 6 The presence of microvascular obstruction despite a patent epicardial vessel is called the no-reflow (NRF) phenomenon. 6 Its incidence has been estimated at 2%–60% with varying diagnostic criteria.6,7 NRF has been repeatedly associated with short- and long-term poor outcomes after PCI.5,7 NRF is linked to larger infarct size, left ventricular failure, malignant arrhythmias, cardiac rupture, and higher 5-year mortality.5,6,8 NRF can be diagnosed with several imaging techniques, the most common being coronary angiography at the catheterization lab. Thrombolysis in myocardial infarction (TIMI) flow grade < 3 or 3 with myocardial blush grade (MBG) 0 to 1 is referred to as angiographic NRF. 9 A more accurate diagnosis of NRF can be established via noninvasive imaging modalities like myocardial contrast echocardiography and contrast-enhanced cardiac magnetic resonance (CMR).9,10 Currently, there is no definite treatment for an established no-reflow, highlighting a critical role for prevention in the management of NRF.11,12 Remote ischemic postconditioning, early administration of IIb/IIIa inhibitors, thrombus aspiration, and intracoronary high-dose adenosine are some available preventive measures that can be taken according to the clinician’s experience. 11 To aid interventionists in decision-making with regard to the prevention of NRF, a bedside clinical risk score or nomogram seems essential. 12
Prediction modeling studies are on the rise in clinical medicine to aid in guiding individual care. These studies develop, validate, or update multivariable models that can predict whether an individual is currently having a particular disease or outcome (diagnostic prediction models) or will have a special outcome in the future (prognostic prediction models).13,14 Many prediction modeling studies tend to report their prediction model in the form of an easy-to-use scoring system known as a “bedside model.” 15 These simplified models are highly informative in the case of patient risk assessment and the adoption of a proper management strategy. In the literature, many researchers have developed and validated various prognostic models with clinical, demographic, or angiographic variables that can be used preoperatively to predict the risk of NRF. However, no model has been widely accepted in clinical practice to predict NRF in STEMI patients undergoing primary PCI so far.
In this systemic review, we aimed to review and critically appraise the current evidence regarding prediction models for predicting NRF in patients with STEMI undergoing primary PCI.
Results
Search results and study selection
After searching the databases, a total of 7095 citations were obtained (Figure 1). A number of 4191 duplicates were removed; 2904 articles remained for title and abstract screening. Finally, 152 articles underwent full-text screening, and 16 studies were included in this systematic review.

PRISMA flow diagram.
Study characteristics and participants
Studies were included from 2012 16 to 2022.17–21 Studies were performed in China (n = 8),17,21–27 Turkey (n = 4),16,18,28,29 France, 30 Pakistan, 19 Serbia, 20 and the United Kingdom. 31 The included studies were prospective 22 and retrospective cohorts.16–21,23–26,28–31 Patients were enrolled between 2003 and 2021.
Outcomes and predictors
A number of eight articles defined angiographic no-reflow as TIMI flow grade < 3 after revascularization16,17,19–21,26,29,31 (Table 1). Four articles defined the no-reflow phenomenon as TIMI flow grade < 3 or TIMI flow grade = 3 with a myocardial blush grade < 2.22,23,27,28 One article used CMR techniques to define no-reflow (microvascular obstruction) after PCI. 30 One article used TIMI frame count to detect no-reflow. 18 The number of candidate predictors considered for models ranged from 722,27,31 to 29, 19 including demographic, laboratory, angiographic, electrocardiographic, and clinical data, with the final included predictors ranging from 3 to 7. Age, total ischemic time, and preoperative TIMI flow grade were the most frequent predictors in the final models (Supplemental Table S2).
Model characteristics.
AUC, area under curve; EPV, events per variable; HL: Hosmer–Lemeshow; N/A, not available.
Model development
The sample size of studies varied from 112 30 to 3205, 18 and the EPV from 3.4 31 to 47.3. 27 All studies used logistic regression for model development.16–31 Six studies used stepwise selection16,17,19,25,28,30 and six used backward elimination as a method for selecting predictors in multivariable modeling.18,22–24,27,31 Only one study considered the shrinkage of predictor weights and regression coefficients. 17
Model performance
A number of 13 articles assessed the performance of the model in terms of discrimination through reporting C-statistics area under the curve (AUC), ranging from 0.648 to 0.925.17,19–24,26–31 Six models were assessed using calibration plots,17,18,21,24–26 and the Hosmer–Lemeshow test was reported in seven articles17,19–21,24,26,28; with four studies reporting both.17,21,24,26 Ten models were internally validated: random split data,17,19,20,22,24,27 cross-validation,25,30 and bootstrapping method.17,18,21 External validation was done geographically in two models23,28 and temporally in one model. 25 Five models were finally presented as nomograms,18,21,24–26 while other models were presented as risk scores.16,17,19,20,22,23,27–31
Risk of bias and applicability
One model had a high risk of bias regarding the target populations (6.2%), 26 seven had a high risk of bias in the predictor domain (43.7%),22,23,25–27,29 and two had a high outcome risk of bias (12.5%)24,25 (Table 2). In the analysis domain, seven articles were considered to have a high risk of bias (43.7%)16,22,23,27–31 and three had an unclear risk of bias (18.7%).25,26,28 Some reasons for considering studies to have a high analysis risk of bias included having a low sample size or EPV, a lack of calibration or discrimination, inappropriate handling of continuous predictors, a lack of internal validation, and the wrong selection method for candidate predictors. Five studies had a low overall risk of bias.17–21 Two studies had high applicability concerns in the participant domain (12.5%).24,26 One study used predictors that were measured 90 min after PCI and did not apply to our pre-interventional predictor measurement concern (6.2%). 30 Two studies used definitions of NRF using parameters other than TIMI and/or MBG,18,30 and one study did not define its outcome. 24 Hence, three studies had high applicability concerns with regard to the outcome domain (18.7%).18,24,30 Four studies had a low risk of bias in all domains of risk of bias and applicability assessment17,19–21 (Figure 2).
Quality appraisal of included studies.
A, analysis; App, applicability; O, outcome; P, participants; Pr, predictors; ROB, risk of bias.Light red or pink: Might represent some concerns or moderate risk of bias.Orange (“?”): Often indicates unclear risk of bias or insufficient information tomake a definitive judgment.

Representation of PROBAST results. (a) PROBAST risk of bias. (b) PROBAST applicability.
Discussion
In this systematic review, we performed a comprehensive assessment of all clinical prediction models that have been developed to predict the no-reflow phenomenon in STEMI patients undergoing primary PCI. We used two validated and reliable tools (Prediction Model Risk of Bias Assessment Tool (PROBAST) 14 and Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies (CHARMS) 13 ) to systematically review and critically appraise the models and studies. Of the 16 included studies, 4 were of low concern regarding the risk of bias (ROB) and applicability, according to the PROBAST tool. The most common biases in other studies were in the statistical analysis domain, mainly a small sample size, a lack of calibration tests, and internal or external validation.
The included studies used a variety of demographic, laboratory, clinical, and angiographic data in their final models. Age, total ischemic time, and preoperative TIMI flow grade were the most frequent predictors in the final models. In a recent systematic review, 32 STEMI patients older than 60 years old and with a total ischemic time of >4–6 h had higher odds of developing NRF after PCI. A meta-analysis showed that initial TIMI flow ⩽1 and high thrombus burden are the most powerful risk factors for NRF in STEMI patients undergoing PCI. 33
It is of utmost importance to use variables as readily accessible as possible for an emergency PCI. 34 Many studies used laboratory data such as neutrophil counts or D-dimer levels, which were not available at the time that the model was intended to be used.21,22,26 Therefore, such studies are not candidates to be applicable in clinical practice. There are numerous novel predictors of NRF emerging. Morphologic parameters such as lipid index, plaque burden, and thin-capped fibroatheroma assessed by optical coherence tomography and intravascular ultrasound are promising a more accurate prediction of NRF.35,36 Proposed molecular biomarkers of NRF such as micro-RNA-208a, 37 perilipin, 38 and paraoxonase-1 39 not only help in better understanding the pathophysiology of the NRF to devise better prevention and treatment strategies but could also help in prediction modeling. However, these novel markers may not be cost-effective to measure or not be available at the time the model is to be used.
The models that have been developed for the prediction of NRF have been mainly in the STEMI patient population. However, few studies have been conducted to develop NRF prediction models in patients presenting with non-STEMI or stable angina.40,41 Our outcome of interest was angiographic no-reflow, defined by TIMI flow grade with or without MBG. CMR with contrast is the gold standard for the characterization of myocardial perfusion after infarction.42,43 The predictive value of impaired myocardial reperfusion as shown by CMR was greater than angiographic parameters for functional variables such as end-systolic volume, end-diastolic volume, and ejection fraction when assessed in a 4-month follow-up. 44 Late MVO, as indicated by CMR, is a strong predictor of major adverse cardiac events, cardiac death, recurrent MI, and heart failure hospitalization. 45 One of the included studies used CMR to document NRF and therefore did not fulfill its applicability.
Model performance evaluation is central to the development of a clinical prediction model. Model discrimination is the ability of the model to distinguish between patients with or without the outcome. 46 An AUC of >0.75 is considered to have decent discriminative ability for a model. Of the four studies with a low ROB, only one had an AUC of less than 0.75. 21
Calibration has been labeled as the “Achilles hill” of clinical prediction modeling, and research has shown that most of the diagnostic and prognostic studies have poor calibration. 47 Despite good discrimination, models should be checked for calibration to determine if risk estimates are accurate. 47 The HL test is not a reliable measure of calibration by itself; the calibration plot and slope must be reported in conjunction with it. 13 Six studies reported the model’s calibration plot, and none had a calibration slope.
A large proportion of clinical prediction models developed annually in medical literature lack an adequate sample size, leading to inaccurate predictions and inappropriate decision-making for many individuals. 48 Currently, EPV is an acceptable way to assess the adequate sample size, with EPV > 10 being a rule-of-thumb lower limit to avoid overfitting in a binary logistic regression. 49 Six of our included studies had an EPV of more than 10. However, not all studies with an EPV lower than 10 were considered to have a high ROB due to shortcomings of the criterion, and some authors have argued that it is not the only parameter affecting the correctness of model estimation, highlighting the need to promote better rules for sample size evaluation in clinical prediction research. 50
Internal and external validation are cornerstones of the model development process before being implemented in clinical practice. 51 Internal validation is evaluating the performance of a clinical prediction model with the data used to develop it. 52 On the other hand, external validation tests the performance of the model in another population and is of utmost importance for the assessment of model generalizability. 52 There are three widely used methods for internal validation of clinical prediction models: random split-sample, cross-validation, and bootstrapping, with the latter two being of higher accuracy. 53 All four studies with a low ROB internally validated their models, but their external validation is yet to be performed in future studies.
Limitations
We were not able to perform a meta-analysis due to the heterogeneity of the data. Furthermore, given the lack of model stability data and the EPV to fortify this review’s findings, making any meta-model from the existing models requires the full equation of the model that was not reported in any of the currently available models. 54 We did not include non-regression model developments in our systematic review because they are at a high risk of overfitting when the sample size is inadequate, and the potential lack of transparency limits their clinical applicability. 34 Future studies must attempt to clearly characterize calibration measurements as they are keys to model performance evaluation.
Further implications
We strongly encourage adhering to TRIPOD reporting guideline 15 to help better report the findings of clinical prediction research. Since most studies were performed in Eastern Asia and the Eastern Mediterranean regions, the validity of their models should be assessed in other geographical and temporal situations. Angiographic assessment of NRF is quicker and more feasible in a situation where “time is the myocardium,” but MVO assessed by CMR later after the PCI can provide a more sensitive assessment of the infarction area, for which future studies are recommended to develop models. Almost all of the included studies had a retrospective study design, which hinders their utility in clinical prediction modeling. 55 Future multi-center prospective cohort studies by independent researchers are needed to externally validate and update these models for better reliability and application in clinical practice. Clinical impact studies, which are deemed to be phase III trials for clinical prediction modeling, should be performed to assess the cost-effectiveness and long-term outcome benefits for patients.
Conclusion
In a systematic review of all clinical prediction models developed to predict NRF in STEMI patients following primary PCI, we report four of them to be of low ROB and low concern with regard to their applicability to our research question. Certain studies with high bias risks significantly impact the ability to generalize models, a crucial factor that requires close attention. Research with significant biases toward specific target populations, predictors, and outcomes may result in models that excel in specific, controlled settings but struggle to apply to wider, more varied populations. Furthermore, worries regarding the suitability of the model, such as employing atypical outcome definitions or predictors assessed after the intervention, also reduce its applicability to clinical settings. Researchers must make sure to have rigorous study designs, adequate sample sizes, standardized predictor measurements, and thorough internal validation processes to reduce these potential problems.
Materials and methods
The protocol for this study was registered on PROSPERO in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines under registration number CRD42023454447.
(Available at: https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42023454447)
Search strategy
A literature search was performed in MEDLINE (through PubMed), Scopus, Web of Science, and Embase (through Elsevier) on July 4, 2023. The search query consisted of three major parts. The first part represents terms related to “score” or “prediction model.” The second part consisted of terms related to the “no-reflow phenomenon.” The last part included terms pertaining to “percutaneous coronary intervention” and “ST-segment elevation myocardial infarction.” The detailed search strategy is available in Supplemental Table S1.
Eligibility criteria
We included studies that developed a prognostic model for the prediction of no-reflow after primary PCI and met our inclusion criteria. Studies fulfilling the following criteria were included: (i) conducted on people with STEMI undergoing PCI; (ii) had an outcome defined as the angiographic NRF after primary PCI; and (iii) developed a clinical prediction model with a simplified presentation.
Exclusion criteria were reviews, case reports, case series, conference abstracts, animal studies, non-English literature, neural network-developed models, non-STEMI, or stable angina patient population, and PCI on bypass grafts. Three authors (RE, MR, and PF) individually assessed titles and abstracts, while two authors (RE and MR) independently evaluated full texts for eligibility. Discrepancies were resolved through consensus.
Data extraction
Data extraction was performed by two authors (RE and MR). We used the CHARMS standardized data extraction form for extracting data. Studies were investigated for study information (author, publication year), source of data, enrollment period, modeling method, selection method for final predictors, number of events, candidate predictors, final predictors, events per variable (EPV), sample size, type of validation, performance measures (calibration, discrimination), AUC, and presentation format.
ROB assessment
We used a standardized form of PROBAST (Prediction Model Risk of Bias Assessment Tool) to evaluate the ROB and the applicability of the included studies. Applicability was of high concern when the study population, predictors, or outcome differed from our review question. Each study was evaluated in four major domains for ROB and applicability: participants, predictors, outcome, and analysis. Two independent reviewers filled out the PROBAST checklist (RE and MR), and disagreements were settled by consensus.
Descriptive analysis
Variables were defined according to CHARMS instructions. 13 The source of data can be a prospective or retrospective cohort, a (nested) case–control study, an existing registry, or a randomized control trial. The modeling method could be a logistic regression or any extension of it. Candidate predictors are deemed to be independent variables before entering univariate or multivariate regression analysis. The number of patients with NRF divided by the total number of candidate predictors yields EPV. Candidate predictors can be selected based on prior knowledge or by univariate analysis. Final model predictors are determined by various approaches, such as forward selection, backward elimination, or stepwise selection during multivariate logistic regression. Sample sizes are reported as the number of patients in the development dataset. Internal validation is usually performed in three ways: random-split sampling, bootstrapping, or cross-validation. External validation could be temporal, geographical, or done by other investigators in different settings. Model performance evaluations include discrimination (AUC graph, AUC), calibration (calibration plot, calibration slope, and the Hosmer–Lemeshow test), and overall performance measures. Presentation formats are usually in the form of a risk score or nomogram. Several features of the statistical analysis domain, such as EPV, type of validation, and performance evaluation (calibration and discrimination), were of significant importance in our critical appraisal of the included studies. 34
Supplemental Material
sj-docx-1-tak-10.1177_17539447241290438 – Supplemental material for Predicting the no-reflow phenomenon in ST-elevation myocardial infarction patients undergoing primary percutaneous coronary intervention: a systematic review of clinical prediction models
Supplemental material, sj-docx-1-tak-10.1177_17539447241290438 for Predicting the no-reflow phenomenon in ST-elevation myocardial infarction patients undergoing primary percutaneous coronary intervention: a systematic review of clinical prediction models by Reza Ebrahimi, Mahdi Rahmani, Parisa Fallahtafti, Amirhossein Ghaseminejad-Raeini, Alireza Azarboo, Arash Jalali and Mehdi Mehrani in Therapeutic Advances in Cardiovascular Disease
Supplemental Material
sj-docx-2-tak-10.1177_17539447241290438 – Supplemental material for Predicting the no-reflow phenomenon in ST-elevation myocardial infarction patients undergoing primary percutaneous coronary intervention: a systematic review of clinical prediction models
Supplemental material, sj-docx-2-tak-10.1177_17539447241290438 for Predicting the no-reflow phenomenon in ST-elevation myocardial infarction patients undergoing primary percutaneous coronary intervention: a systematic review of clinical prediction models by Reza Ebrahimi, Mahdi Rahmani, Parisa Fallahtafti, Amirhossein Ghaseminejad-Raeini, Alireza Azarboo, Arash Jalali and Mehdi Mehrani in Therapeutic Advances in Cardiovascular Disease
Footnotes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
