Abstract
Triage of patients with acute coronary syndrome (ACS) at high risk of in-hospital complications is essential. In this study, we evaluated the quick sepsis organ failure assessment (qSOFA) score as a tool for predicting the prognosis of 964 patients admitted to the cardiovascular intensive care unit (CICU) with ACS over a 4-year period. In total, out of 964 patients included, with a percentage of 4.6% for 30-day mortality. The risk of 30-day mortality was independently associated with qSOFA ≥ 2 at admission (hazard ratio = 2.76, 95% CI 1.32-5.74, p = 0.007). For MACEs, qSOFA ≥ 2 at admission was a predictive factor with (odds ratio = 2.42, 95% CI 1.37–4.36, p = .002). A qSOFA ≥ 2 on admission had an AUC of 0.729 (95% CI [0.694, 0.762]), with a good specificity of 91.6%. For 30-day mortality, an AUC of 0.759 (95%CI [0.726, 0.792]) for cardiogenic shock with specificity of 92.5%. For MACEs, an AUC of 0.702 (95% CI [0.64, 0.700] with a specificity of 95%. Concerning the different scores tested, we found no significant difference between the Zwolle score and the qSOFA score for predicting prognosis, whereas the CADILLAC score was better than qSOFA for predicting 30-day mortality (AUC = 0.829 and De long test = 0.03). However, there was no difference between qSOFA and CADILLAC scores for predicting cardiogenic shock (De Long test at 0.08). This is the first study to evaluate qSOFA as a predictive score for 30-day mortality and MACEs, and the results are very encouraging, particularly for cardiogenic shock.
Keywords
Introduction
The management of acute coronary syndrome (ACS) is based on several components, firstly myocardial revascularization, management of cardiovascular risk factors, comorbidities, and above all the management of possible complications in the acute phase, which are varied and complex, from simple dysfunction of left ventricular systolic function to cardiogenic shock and rhythmic storm.
Given this clinical polymorphism of ACS patients, the level of cardiovascular intensive care unit (CICU) hospitalization and the complexity of procedures will change according to the complications observed. 1
Therefore having patient triage scores on admission remains a very important tool to predict the level of management required according to the expected complications in the acute phase. 2
In 2016, the Sepsis Survival Campaign recommended the use of the sepsis organ failure assessment (SOFA) score to classify patients admitted to intensive care with sepsis according to severity. 3
Although the SOFA score has a high sensitivity to predict in-hospital mortality in patients with sepsis or septic shock, it is still a complex tool to apply at the time of admission to the emergency department to triage patients with infection who will subsequently develop an unfavorable outcome. 4
For this purpose, the quick SOFA (qSOFA) has been proposed, due to its simplicity of application during admission to the intensive care unit (ICU). 4
Our hypothesis in this paper is to evaluate the qSOFA score on admission to the CICU in patients with ACS to predict in-hospital mortality and major adverse cardiovascular events (MACE) following ACS during hospitalization.
Materials and Methods
Rationale and Design of the Study
Several scores have been studied to predict the prognosis of patients with ACS, principally the Global Registry of Acute Coronary Events (GRACE) score, 5 the thrombolysis end flow in myocardial infarction (TIMI) score, 6 the Zwolle score, 7 and the Controlled Abciximab and Device Investigation to Lower Late Angioplasty Complications (CADILLAC) score. 8 These scores include biological markers, which may be falsely elevated in other conditions, and angiographic elements, which cannot be known prior to coronary angiography and percutaneous coronary intervention (PCI). For these reasons, these scores, although having a good predictive value, cannot be used as admission triage scores to predict the prognosis of patients admitted with ACS who have undergone PCI. Our aim in this study is to evaluate a purely clinical score, on admission to the CICU, on the prediction of in-hospital outcome of patients with ACS (with or without ST-segment elevation): the qSOFA score.
We retrospectively included all patients admitted to the CICU of Oujda University Hospital for ACS, over a 4-year period, from January 2018 to December 2021. All demographic, clinical, ultrasonographic, and angiographic data, as well as intra-hospital evolution, were obtained from patients’ electronic medical records.
Definition and Scores
Quick Sequential Organ Failure Assessment 9
The quick sequential organ failure assessment (qSOFA) was defined in accordance with the Surviving Sepsis Campaign 2016. Patients were awarded 1 point for a Glasgow Coma Scale of 14 or less, 1 point for a systolic blood pressure of 100 mmHg or less, and a respiratory rate of 22/min or more. A score of 2 or more is considered positive (Supplemental material; Table 1)
The Zwolle Score 10
The Zwolle risk score distinguishes ST elevation myocardial infarction (STEMI) patients with a higher risk of in-hospital adverse events after PCI. This score consists of 6 clinical and angiographic variables (age, Killip class, myocardial ischemia time, number of vessels with significant stenosis, anterior location of ischemia, and thrombolysis flow in myocardial infarction after PCI), each variable is scored with a weighted point value. Scores of 0–3 are considered low risk of 30-day mortality or cardiac arrest after discharge, while scores ≥4 are considered high risk (Supplemental material; Table 2).
The CADILLAC Score 11
The CADILLAC risk score was developed to identify patients at low risk of adverse cardiovascular events after STEMI treated by primary PCI. This risk score assesses several prognostic factors: age, anemia, creatinine clearance, Killip stage, left ventricular ejection fraction, TIMI, and the presence of multiple coronary diseases. Using this score, patients were stratified into low (CADILLAC score 0–2), intermediate (CADILLAC score 3–5), and high (CADILLAC score ≥6) risk groups. (Supplemental material; Table 3).
Cardiogenic Shock 12
Cardiogenic shock was defined according to the Society for Cardiovascular Angiography and Interventions consensus statement on the classification of cardiogenic shock. It combines a decrease in cardiac output with signs of tissue hypoxia, in the absence of signs of hypovolemia.
The hemodynamic criteria for cardiogenic shock are
- Systolic blood pressure <90 mmHg, for at least 30 min. - Low cardiac output (index <2.2 L/min/m2). - Pulmonary artery occlusion pressure > 15 mmHg.
Major Ischemic Events
Major ischemic events are defined as any ischemic event that occurs during hospitalization including cerebral magnetic resonance imaging confirmed ischemic stroke, early sentinel stenosis, or acute limb or mesenteric ischemia.
Major Bleeding Events
Major bleeding events are defined as any bleeding event that occurred during hospitalization including cerebral hemorrhage, severe bleeding at the coronary puncture site, internal or external bleeding that required discontinuation of antithrombotic therapy, and intervention to stop the bleeding.
Acute Coronary Syndromes and Coronary Lesions
We defined ACS as STEMI or non-STEMI (NSTEMI). For coronary lesions, we defined it as a stenosis of more than 50% in the left main (LM) and 70% in the other arteries.
Outcomes
We first determined the predictors of 30-day all cause mortality after admission to CICU as primary outcome, then the predictors of MACEs during hospitalization as secondary composite outcomes of acute heart failure, major ischemic events, major bleeding events, and cardiogenic shock during hospitalization.
Statistical Analysis
Normally distributed quantitative variables were described as mean and standard deviation, and analyzed between the 2 groups using the t-student test. Nonnormally distributed quantitative variables were described as median and interquartile range, and analyzed by nonparametric tests. Categorical variables were described as numbers and analyzed between the 2 groups using Pearson's chi-square test or Fisher's exact test.
A Cox proportional hazards regression model was used to estimate hazard ratios (HRs) and 95% confidence intervals (CIs) for 30-day mortality, and logistic regression analysis was performed to investigate predictors of MACE with associated 95% CI and 2-tailed p-value indicated. A univariate approach was first performed for all covariates, then a multivariate analysis was performed for all covariates that showed a statistically significant result in the univariate analysis, with an associated 95% CI and 2-tailed p-value indicated. The Kaplan–Meier method was used to estimate survival, and the logrank test was used to assess hypotheses concerning differences observed in survival curves. For all tests, a p-value of <.05 was considered statistically significant.
Receiver characteristic curves were generated for predictive accuracy of in-hospital mortality and MACE using the qSOFA score. The area under the curve (AUC) of the score is provided and compared, using the De Long test, with that of the CADILLAC and Zwolle scores. Data were collected and processed using IBM SPSS 26.0 statistical software.
Ethical Approval
This study did not require formal ethics approval. Access to patient data was authorized and approved by the Mohammed VI University Hospital. Given the retrospective design of this study. The requirement for patient consent was waived by the local ethics committee. Data anonymity was respected in accordance with national and international guidelines.
Results
General Characteristics
We included a total of 964 patients in this study who met the inclusion criteria. A predominance of males was noted, with a total proportion of 74.4%. Patients with a qSOFA score of 2 or 3 at admission had a higher mean age than patients with a qSOFA score of 0 or 1, with a mean age of 68.5 years versus 61.931 days, with a significant difference (p < .001). The same observation was made for the length of hospital stay (5.83 days for qSOFA 2–3 vs 3.61 days for qSOFA 0–1) (p < .001).
For cardiovascular risk factors, diabetes and hypertension were the most frequent. Previous coronary heart disease was present in 10% of patients (p = .002), and previous heart failure in 2.6% (p = .041). The qSOFA 2–3 group had delayed consultations, with a mean time from symptom onset to consultation of 19 h versus 15.1 h in the qSOFA 0–1 group (p < .001).
Regarding the type of ACS on admission, 72.2% were presented as STEMI, and the remainder as NSTEMI. On echocardiography, the qSOFA 2–3 group had significantly more altered left and right ventricular systolic function with respective p-values (p < .001; p = .004), with a reduced ejection fraction in the same group (43% vs 48%; p < .001).
Coronary angiography showed multivessel involvement in 48% of patients, with the left anterior descending (LAD) artery the most pathological artery in 72.5%. As for the culprit artery, the middle or distal LAD was the most frequent, followed by the left circumflex artery (LCX), then the right coronary artery, with no difference between the qSOFA 0–1 or 2–3 groups. On the other hand, there was a difference concerning the LM and ostio-proximal LAD as the culprit artery (p = .032).
PCI was performed in 71.5%, surgery in 11.6%, and medical treatment in 16.9%, with no difference between the 2 groups. There was no difference between the 2 groups in the time between admission and PCI. During hospitalization in CICU, patients with qSOFA 2–3 group benefited more from central venous access (p < .001), as well as more recourse to vasoactive drugs (p < .001), and the performance of noninvasive ventilation with positive pressure (p = .001).
MACEs were more frequent in the qSOFA 2–3 group, with a significant difference for cardiogenic shock (p < .001), ischemic events (p = 0.001), and hemorrhagic events (p = 0.006), but not for acute heart failure (p = 0.058). The same was observed for life-threatening ventricular arrhythmia, severe conduction disorders, and mechanical complications. For 30-day all-cause mortality, 45 events were observed, with a significant difference between the 2 groups (p < .001).
Table 1 summarizes the general characteristics of our patients according to the qSOFA score on admission.
Comparison of Basic Characteristics of Patients With ACS According to qSOFA Score at Admission in CICU.
Abbreviations: ACS, acute coronary syndrome; qSOFA, quick sequential organ failure assessment; LBBB, left bundle branch block; RBBB, right bundle branch block; EF, ejection fraction; LV, left ventricle; RV, right ventricle; LM, left main; LAD, left anterior descending; LCX: left circumflex artery; RCA, right coronary artery; Hb, hemoglobin; GFR: glomerular filtration rate; CRP, C-reactive protein; WBC, White blood cell; CICU, cardiovascular intensive care unit; NIPPV, no invasive positive pressure ventilation; PCI, percutaneous coronary intervention; VA, ventricular arrhythmia; CD, conduction disorder; MACE, major adverse cardiovascular events;
EKG, electrocardiogram; AVR, lead aVR; MV, mechanical ventilation.
Results in bold mean significant results, i.e. p-value < .05.
Primary and Secondary Outcomes Analysis
In multivariate analysis, and after adjustment, the risk of mortality at 30 days was independently associated with the qSOFA ≥ 2 at admission (HR = 2.76, 95% CI 1.32–5.74, p = .007), cardiogenic shock in CICU (HR = 4.08, 95% CI 1.91–8.71, p < 0.001), LM and ostial–proximal LAD as culprit artery (HR = 3.67, 95% CI 1.721–7.55, p = .001), acute kidney injury with a glomerular filtration rate (GFR) < 30 mL/min/1.73 m2 (HR = 3.04, 95% CI 1.24–7.26, p = .015) In addition, an ejection fraction <40%, a mechanical complication, a major bleeding event, and a C-reactive protein > 300mg/L (Table 2).
Cox Proportional Hazards Regression Analyses for 30-Day all-Cause Mortality.
Abbreviations: CAD, coronary artery disease; EF, ejection fraction; LAD, left anterior descending; CRP, C-reactive protein; GFR, glomerular filtration rate; HR: hazard ratio; LM, left main; qSOFA: quick sepsis organ failure assessment.
Results in bold mean significant results, i.e. p-value < .05.
The Kaplan–Meier event curve (Figure 1) showed that a qSOFA ≥ 2 score at admission was significantly associated with an increased incidence of all-cause mortality at 30 days of admission (logrank < .001).
Kaplan–Meier curve for 30-day all-cause mortality. Patients-based qSOFA score on admission (A) and in-hospital MACE (B).
A multivariate logistic regression analysis identified the following MACE risk, qSOFA ≥ 2 at admission (OR = 2.42, 95% CI 1.37–4.36, p = 0.002), prior atrial fibrillation with CHADS2-VA2SC > 1 (OR = 4.92, 95% CI 3.08–7.73, p = .041), Killip stage > II on admission (OR = 4.92, 95% CI 2.76–6.76, p < .001), LM or ostia-proximal LAD as culprit artery (OR = 2.939, 95% CI 1.92–5.19, p = .001). Adding to this, ejection fraction < 40%, diastolic dysfunction, and acute kidney disease with a GFR < 30 mL/min/1.73 m2 (Table 3).
Logistic Regression Analysis of Risk Factors in the MACE Group.
Abbreviations: CAD, coronary artery disease; STEMI, ST elevation myocardial infarction; LVSF, left ventricular systolic function; LV, left ventricle; RV, right ventricle; LAD, left artery descending; LM, left main; Hb, hemoglobin; GFR, glomerular filtration rate; WBC, white blood cell; CRP, C-reactive protein; CD, conduction disorders; MACE, major adverse cardiovascular events; qSOFA, quick sepsis organ failure assessment; AVR, lead aVR.
Results in bold mean significant results, i.e. p-value < .05.
*Significant values in univariate analysis.
The Kaplan–Meier event curve showed that a qSOFA ≥ 2 score on admission was significantly associated with an increased incidence of in-hospital MACEs (Figure 1), cardiogenic shock, and ischemic and hemorrhagic events. However, the incidence of acute heart failure was not statistically significant between the 2 groups (Supplemental material; Figures 1–4).
Analysis of the qSOFA Score to Predict the Outcomes Studied
Receiver operating characteristic (ROC) curve analysis (Figure 2) found that the AUC for admission qSOFA for 30-day all-cause mortality is 0.729 (95% CI [0.694, 0.762]), with a cut-off of 1, with good specificity of 0.916, but sensitivity of 0.523. For cardiogenic shock AUC = 0.759 (95% CI [0.726, 0.792] with a cut-off of 1, with a good specificity of 0.925, a sensitivity of 0.57, with a positive predictive value of 0.36 and a negative predictive value of 0.95 (Figure 3). For MACEs during hospitalization, AUC = 0.702 (95% CI [0.64, 0.700] with a cut-off of 1, with a good specificity of 0.95, a sensitivity of 0.44, and a positive predictive value of 0.90 (Table 4).
Comparison of receiver operating characteristic (ROC) curves for predicting 30-days all-cause mortality (A) and in-hospital MACE (B), respectively, for qSOFA, Zwolle, and CADILLAC score.
Comparison of ROC curves for predicting in CICU cardiogenic shock, respectively, for qSOFA, Zwolle, and CADILLAC scores.
Prediction of 30-Day All-Cause Mortality, in-Hospital MACE Events, and in CICU Cardiogenic Shock by qSOFA Score, Zwolle Score, and CADILLAC Score.
Abbreviations: AUC, area under the curve; PPV, positive predictive value; NPV, negative predictive value; MACE, major adverse cardiovascular events; CICU, cardiovascular intensive care unit; CADILAC, Controlled Abciximab and Device Investigation to Lower Late Angioplasty Complications; qSOFA, quick sepsis organ failure assessment.
Results in bold mean significant results, i.e. p-value < .05.
The comparison of the qSOFA at admission with the Zwolle score on predictive value found no significant difference in mortality at 30-days with an AUC = 0.766 (95% CI [0.733, 0.797] for the Zwolle score (p = .310), as well as for cardiogenic shock with an AUC = 0.802 (95% CI [0.770, 0.831] for the Zwolle score (p = .253), and finally even for MACEs, no difference is observed between the 2 scores with an AUC = 0.750 (95% CI [0.715, 0.7382] for the Zwolle score (p = .0.06) (Table 4, Figures 2 and 3).
For the comparison between the qSOFA and the CADILLAC score, the difference was significant for 30-day mortality, with an AUC = 0.829 (95% CI [0.799, 0.857] for the CADILLAC score (p = .03), as well as for in-hospital MACEs with an AUC = 0.816 (95% CI [0.785, 0.844] for the CADILLAC score (p ≤ .001). However, there was no difference between the qSOFA and the CADILLAC score for the predictive value of cardiogenic shock in the CICU with an AUC = 0.842 (95% CI [0.813, 0.869] for the CADILLAC score (p = .08) (Table 4, Figures 2 and 3).
Discussion
Risk scores are an essential tool in the daily practice of CICU practitioners, as CICUs have since developed from simple units managing patients with ACS to more specialized units, with different phenotypes of patients admitted, as well as more complex illnesses to manage, not limited to coronary emergency, 13 risk scores are becoming more important in the different steps of patient management, from triage on admission to the various diagnostic strategies to be carried out, to the choice of treatment and the timing of its implementation, and finally to the prediction of short- and long-term prognosis. 14
For ACS, 4 major scores are currently validated: the TIMI score, the GRACE score, the Zwolle score, and the CADILLAC score. 15
The TIMI score was developed from the databases of 2 large randomized controlled trials comparing the efficacy and safety of enoxaparin versus unfractionated heparin in patients with unstable angina or NSTEMI.16,17
This score is purely prognostic and is of no use for triage or diagnostic orientation, as it predicts all-cause mortality, recurrent myocardial infarction, or a severe ischemic event requiring revascularization over a 14-day period after admission using 7 parameters, 6 of which are purely clinical, making it easy to use, 18 but limited in the ACS spectrum, especially in view of the known progress in risk stratification of mortality in patients with unstable angina or NSTEMI. 19
The GRACE score represents the most important score to date. It was derived from data from the GRACE, which included 17 142 patients from 94 hospitals in 14 different countries. 20
It is a prognostic score, which predicts 6-month mortality, with the advantage that it can be applied to all patients on the ACS spectrum, but with the disadvantage that it is complex in practice, with a set of 8 clinical, hemodynamic, and biological parameters. 21 It should be noted that the GRACE score was considered for a long time as a score for therapeutic orientation and not only prognosis, and this in NSTEMI patients, since it was considered as a score to define the time to perform PCI in NSTEMI patients, 22 before it was downgraded in this sense since the European Society for Cardiology 2020 and 2023 guidelines on the management of NSTEMI. 23
The Zwolle score is the only currently validated triage score. 24 It was developed to triage STEMI patients who are eligible for early discharge after PCI, while maintaining their prognosis. 7 The aim is to reduce the length of hospital stay, as well as the cost of care for these patients. 25 In validation cohorts, a low Zwolle score (<4) in STEMI patients undergoing PCI has been shown to be associated with very low mortality at 2 days (0.1%) and between 3 and 10 days (0.2%). 26 But the main limitation of this score is that it is only applicable to STEMI patients who have undergone PCI, and no angiographic or echocardiographic variables are included in these items. 10
Given the limitations of the Zwolle score, the CADILLAC score was developed in the same patient population as the Zwolle score, but with the integration of ejection fraction, TIMI flow at the end of PCI, and number of diseased vessels, derived from the database of the Controlled Abciximab and Device Investigation to Lower Late Angioplasty Complications (CADILLAC) trial, 27 and validated in the database of the Stent-Primary Angioplasty in Myocardial Infarction trial, 28 but with the aim of predicting mortality at 30 days and 1 year, and not of triaging patients according to their risk during hospitalization.8,11
As we have seen, the Zwolle score remains the only score that can be used to triage patients according to their risk of developing a complication during their stay in the CICU.
Our study is the first to evaluate the use of the qSOFA score at admission as a predictive risk score for adverse outcomes after ACS, and the results are very encouraging for the use of the qSOFA as a triage score for ACS patients, in the sense of providing guidance on the type of hospitalization unit and the level of ICU needed to improve short- and long-term mortality.
Jacob was the first to evaluate the SOFA score to predict prognosis, first in 2015 in a population of 300 consecutive patients admitted to the CICU at Mayo Clinic Hospital, St Mary's Campus, a tertiary care hospital in Rochester, MN, of whom 206 had ACS. The comparison between the SOFA and APACHE II scores showed no difference with good discriminatory power (AUC between 0.84 and 0.92). 29 Then in 2018, a population of 9961 patients, without diagnostic individualizations, were admitted to the same unit. The primary endpoint was in-hospital all-cause mortality, and secondary endpoints included ICU mortality, 30-day mortality, and postdischarge mortality. The first-day SOFA score predicted in-hospital mortality, with an area under the receiver characteristic curve (AUROC) of 0.83, with no difference in the receiver characteristic curve of the APACHE-III and APACHE-IV scores, and patients with a first-day SOFA score of <2 had a low risk of mortality. 30
Finally, in 2019, an analysis was conducted but with an analysis of the organ SOFA subscores for risk stratification of in-hospital mortality in CICU. 30 The SOFA score at day 1 predicted in-hospital mortality with an AUROC of 0.72, thus each SOFA subscore predicted in-hospital mortality (all p < .01), with AUROC values between 0.53 and 0.67, and the simplified SOFA score, including the subscore of the cardiovascular system, central nervous system, and the respiratory and renal systems, which is close in items to the qSOFA, is significantly predictive of in-hospital mortality, with a similar discriminatory ability to the day 1 SOFA score for in-hospital mortality in the study population (AUROC = 0.73). Although these results are drawn from a heterogeneous population with respect to admission diagnosis, given the frequency of ACS as a reason for admission to CICU, we can note such similarity between our results and those published by Jacob and us.
As far as the qSOFA score is concerned, to our knowledge, there is only 1 published study that has evaluated the use of qSOFA to predict prognosis in 1296 ACS patients, using the same design as ours. The authors defined high-risk patients whose qSOFA was 2 or 3 on admission. This group was independently associated with higher in-hospital mortality (28% vs 8%; OR: 3.8; 95% CI 2.4–6.2; p < .01), as well as higher 1-year mortality in the Kaplein–Meier analysis, further confirming our results and our hypothesis that qSOFA can be performed first as a triage score and then as a prognostic score for the whole ACS population.31
Conclusion
Our study shows the effectiveness of using qSOFA as a triage score for ACS patients admitted to a CICU, with the aim of individualizing levels of care, length of hospital stay, and cost of care. Comparing this admission score with the Zwolle score, which is the only triage score developed in this sense, no difference was observed, with the advantage of speed and ease of use of the qSOFA score, and this is already validated in the only study published in this sense. Therefore, we strongly recommend the evaluation of this score in other cohorts to be able to integrate it as a simple triage score in CICU for patients with ACS.
Supplemental Material
sj-docx-1-cat-10.1177_10760296231218705 - Supplemental material for Usefulness of the Quick-Sepsis Organ Failure Assessment Score in Cardiovascular Intensive Care Unit to Predict Prognosis in Acute Coronary Syndrome
Supplemental material, sj-docx-1-cat-10.1177_10760296231218705 for Usefulness of the Quick-Sepsis Organ Failure Assessment Score in Cardiovascular Intensive Care Unit to Predict Prognosis in Acute Coronary Syndrome by Amine Bouchlarhem, Zakaria Bazid, Nabila Ismaili and El Ouafi Noha in Clinical and Applied Thrombosis/Hemostasis
Footnotes
Author Contributions
AB developed the project idea, writing and revising the proofs. ZB, NI, and NE supervised the project. All authors contributed to the article and approved the submitted version.
Data Availability Statement
The data underlying this article are available in the article and its online Supplemental material.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
