The low accuracy of the non-ST-elevation myocardial infarction electrocardiograph criteria of the fourth universal definition of myocardial infarction

Abstract

Background:

The electrocardiograph has been integral to the diagnosis of acute coronary syndromes since the mid-20th Century and is an important initial investigation that chest pain patients undergo on presentation to the Emergency Department. The Fourth Universal Definition of Myocardial Infarction recommends using dichotomous cut-offs to identify ischaemic electrocardiographs.

Objectives:

We aimed to summarise the existing knowledge to inform emergency clinicians about the diagnostic accuracy of the new guidelines.

Methods:

We performed a systematic review and a narrative analysis due to the heterogeneity of the studies.

Results:

We were able to obtain diagnostic characteristics for 10 papers. The ST-depression criteria were highly specific but poorly sensitive in five papers, with a specificity of 97.2%–99.3% and a sensitivity of 16.6%–20.0%. The remaining papers reported a higher sensitivity of 25.7%–58.6% but a lower specificity of 86.0%–91.2%. T wave inversion demonstrated poor specificity; the papers that looked at 0.1 mV T wave inversion demonstrated a sensitivity of 26.9%–46.8% and a specificity of 68.6%–86.4%.

Conclusion:

The heterogeneous evidence database demonstrates that the Fourth universal definition’s diagnostic performance varies wildly. Apart from two outlying papers, ST-depression has suboptimal sensitivity but high specificity. T wave inversion appears to be more sensitive yet less specific.

Keywords

Coronary occlusion/diagnosis Coronary occlusion/physiopathology electrocardiography heart/physiopathology humans

Introduction

Acute chest pain is the second most common cause of unplanned admission to hospital in England.¹ The majority of these patients present to the Emergency Department (ED), and accurate recognition of acute coronary syndromes (ACS) in this cohort is integral to emergency medicine diagnostics. The mortality rate of patients with missed acute myocardial infarction (AMI) is twice that of patients who are diagnosed accurately.²

A possible solution would be to admit all patients for serial electrocardiograph (ECG) measurements and serial biomarker studies but this comes with its own pitfalls such as hospital acquired infections and an extraordinary demand on finite resources. With a prevalence of ACS in previous cohorts being as low as 7%,³ increasing focus has been devoted to early rule-out strategies that combine elements from the history, clinical examination, an ECG and the initial cardiac biomarker.^4–7

Recently, in August 2018, the Fourth Universal Definition of Myocardial Infarction was published and recommended the use of objective cut-offs in much the same way as the ST-elevation myocardial infarction guidelines.⁸ Thygesen and colleagues attempted to provide strict criteria for the diagnosis of myocardial injury based on changes in troponin concentration and their findings can be summarised in the ‘Ten Commandments’.⁹ One can make a diagnosis of non-ST-elevation myocardial infarction (NSTEMI) if the cardiac troponin value is above the 99th percentile of the upper reference limit and there is evidence of cardiac symptoms or a culprit lesion on angiography or ECG changes. The guidelines define ECG signs of non-ST-elevation acute coronary syndrome (NSTEACS) to be 0.05 mV or more of ST-depression in two contiguous leads and T wave inversion of 0.1 mV or more in two contiguous leads. The overall effect of these new guidelines in the acute care setting is not clear; this is an important question given the clinical importance in recognising AMI, and low prevalence of the condition in chest pain patients. To this end, we have performed a systematic review of the literature to demonstrate how these guidelines perform in an ED setting.

Methods

Search strategy and eligibility criteria

This systematic review adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines for diagnostic test accuracy reviews.¹⁰ A three-part search strategy was used, combining free type, database-specific terms and Boolean operators, focusing on the ECG, the ED and suspected ACS. There was no time limit and all the references of relevant papers were examined. The search strategy explored the libraries of Medline, Embase and all Evidence-Based Medicine reviews, including Cochrane, using the Ovid interface. A grey literature search was also performed. Authors were contacted to seek further diagnostic raw data or further clarification regarding a papers method.

All diagnostic studies, whether they are retrospective or prospective, observational or interventional, were eligible if they were based in the ED. To be included in the analysis, studies had to report the diagnostic characteristics of the ECG in identifying NSTEMI or major adverse cardiac events (MACE). If the diagnostic characteristics were not available, we contacted the authors. We restricted our search to English language papers. A full breakdown of the search strategy is available on Prospero (CRD42018111749).

Index test and outcome measures

The ECG is a cornerstone of ACS diagnosis and its use has remained relatively static. ST-depression and T wave inversion are the classic signs of ischaemia on an ECG in the NSTEACS cohort.⁸ The ECG has varying diagnostic performance based on the dichotomous cut-off chosen and the place where it is measured. We have reported all the various cut-offs used by the studies alongside their diagnostic characteristics.

The primary outcome of interest was a diagnosis of AMI. The third universal definition of myocardial infarction (which was in use until very recently) was considered to be the optimal reference standard.¹¹ This definition requires a rise and or/fall in the cardiac biomarker, with one value above the 99th percentile, and signs or symptoms suggestive of myocardial ischaemia. If diagnostic accuracy for AMI was not reported, we also considered studies that evaluated the incidence of MACE at 30 days. MACE is a composite diagnosis of AMI, revascularisation and all-cause mortality, in an attempt to take account of unstable angina (UA). It is customary to report this as a 30-day outcome but we did not require the incidence of MACE to be reported as part of our inclusion criteria.

Data extraction

Database-specific syntax was used to search the electronic databases. All of the papers abstracts were searched and screened independently by two investigators (N.M. and C.R.) for eligibility. The investigators were middle grade emergency physicians with 8 and 4 years of experience, respectively, and both are doctoral emergency medicine research fellows with numerous ACS publications. Both investigators then undertook a full-text review, including a review of the references, to establish which papers were suitable for data extraction. After this was performed, a meeting was held to discuss findings and resolve discrepancies. Authors were contacted for missing data and possible new material. We independently analysed the papers using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool.¹² This was performed over a 2-month period. Discrepancies were resolved by consensus between the investigators. In the event of any irreconcilable differences in opinion, a third investigator who is a professor of emergency medicine (R.B.) was set to provide a final adjudication. There were no disagreements.

Statistical analysis

After extracting data, we considered the appropriateness of pooling the results in a meta-analysis. Unfortunately, there was no consistency between studies with regard to the measurement of the ECG (index test), the adjudication of the primary outcome varied widely, and the patient selection was strikingly different. We had planned to formally assess the heterogeneity of the papers using the Cochrane Q chi-square test and the I² statistic but this was also deemed unnecessary as there was overt evidence of clinical heterogeneity between studies, rendering meta-analysis inappropriate.

Results

The literature review identified 19 potential papers from 979 eligible articles. A search of the grey literature and manual inspection of references managed to find five more articles.^13–17 We are incredibly grateful for those who provided further information on request.^4,13,18–20 We were able to obtain diagnostic characteristics for 10 papers.^{4,3,7,13,14,18–22} We summarise the review process in Figure 1 and the characteristics of the included studies are described in Tables 1 –4.

Figure 1.

A flow diagram summarising the paper selection process.

Table 1.

QUADAS-2.

	Risk of bias			Applicability concerns
	Patient selection	ECG	Outcome	Flow and timing	Patient selection	ECG
Six et al.¹⁴	Good	Good	Bias	Bias	Bias	Bias
Ngako et al.¹⁸	Bias	Bias	Good	Good	Bias	Bias
Fleischmann et al.²¹	Bias	Good	Unclear	Good	Bias	Bias
Hess et al.⁷	Bias	Good	Good	Good	Good	Bias
Backus et al.⁴	Good	Good	Good	Good	Good	Bias
Six et al.¹⁹	Bias	Good	Good	Good	Bias	Good
Melki and Jernberg³	Good	Unclear	Bias	Bias	Good	Bias
Visser et al.²²	Good	Good	Good	Good	Good	Good
Liu et al.¹³	Bias	Good	Bias	Bias	Bias	Good
Ong et al.²⁰	Bias	Good	Bias	Bias	Bias	Good

ECG: electrocardiograph; QUADAS-2: Quality Assessment of Diagnostic Accuracy Studies-2.

Table 2.

Characteristics of included studies.

References	Inclusion criteria	Exclusion criteria	Summary of the index test employed in the paper
Six et al.¹⁴	Patients had chest pain	Patients with evidence of an STEMI	0.05 mV ST-depression in contiguous leads measured at the J point by two blinded cardiologists using visual estimation
Ngako et al.¹⁸	Patients had symptoms suggestive of an ACS (NSTEMI, STEMI, UA)	Patients over 65 years of age	0.05 mV ST-depression or 0.1 mV T wave inversion in contiguous leads. ST-depression was measured at the J point Adjudication was performed by two sets of two blinded investigators
Fleischmann et al.²¹	Patients with symptoms suggestive of an ACS	Patients with LBBB or ventricular pacing, LVH STEMI, or ST-depression greater than 0.1 mV in two contiguous leads	0.05 mV ST-depression or 0.1 mV T wave inversion in two contiguous leads. ST-depression was measured at the J point. All measurements were made by a single blinded cardiologist
Hess et al.⁷	Patients were older than 24, with chest pain suggestive of ACS	Patients with STEMI, haemodynamic instability, cocaine use, traumatic injury, pregnancy, or an inability to receive 30-day follow-up	0.05 mV ST-depression or 0.2 mV TWI in contiguous leads ST-depression was measured at 60 ms from the J point. All measurements were made by two blinded board certified EM Drs
Backus et al.⁴	Patients who presented to a cardiology emergency room with chest pain	Patients presenting with only dyspnoea or palpitations or clear evidence of STEMI	0.05 mV ST-depression in contiguous leads measured at the J point by blinded cardiologists by visual estimation
Six et al.¹⁹	Patients’ chest discomfort of at least 5-min duration for which the clinician suspected ACS	Unable to provide consent, transferred from another hospital, pregnant, or unable to be contacted after discharge STEMI	0.05 mV ST-depression in any lead measured at the J point The treating clinician analysed the ECG
Melki et al.³	Patients presenting with chest pain for whom the clinician suspected ACS	Presence of STEMI	0.05 mV ST-depression It was unclear whether the leads were contiguous or where the ST-depression was measured
Visser et al.²²	Patients admitted to the ED with chest pain, 18 years old	Patients presenting with only syncope, shortness of breath, dyspnoea, palpitations, or atypical symptoms. Patients with clear evidence of STEMI or inter-hospital referrals	0.05 mV ST-depression in contiguous leads There was no comment on where the ST-depression was measured. Measurements were made using visual estimation by the treating clinician
Liu et al.¹³	Patients with non-traumatic chest pain	Patients were excluded if they were less than 30 years old or found to be in a non-sinus rhythm	ST-depression was judged by the treating clinician and measured at the J point It did not have to be in contiguous leads. The amount of T wave inversion was unspecified The treating clinician was blinded to the primary outcome
Ong et al.²⁰	Patients with non-traumatic chest pain	Patients who were less than 25 years old, or had: new Q waves, STEMI, ST-depression greater than 0.1 mV in two or more contiguous leads	0.1 mV ST-depression and 0.1 mV T wave inversion ST-depression was measured at the J point Contiguous leads were used and measurements were made by the treating clinician blinded to the primary outcome

STEMI: ST-elevation myocardial infarction; ACS: acute coronary syndromes; NSTEMI: non-ST-elevation myocardial infarction; UA: unstable angina; ECG: electrocardiograph; ED: Emergency Department LBBB: left bundle branch block; LVH: left ventricular hypertrophy; EM: Emergency Medicine; TWI: T wave inversion.

Table 3.

The diagnostic characteristics of studies that looked at ST-depression.

References	Number of patients	Patients with ACS (%)	Sn	Sp	PPV	NPV
Six et al.¹⁴	120	29 (24)	58.6 (38.9–76.5)	91.2 (83.4–96.1)	68.0 (50.6–81.5)	87.4 (81.7–91.5)
Ngako et al.¹⁸	399	124 (31)	17.7 (11.5–25.6)	89.1 (84.8–92.5)	42.3 (30.6–54.9)	70.6 (68.7–72.5)
Fleischmann et al.²¹	120	37 (30.8)	29.7 (15.9–47.0)	98.8 (93.5–100.0)	91.7 (59.6–98.8)	75.9 (71.9–79.6)
Hess et al.⁷	2718	336 (12)	17.3 (13.4–21.7)	97.2 (96.4–97.8)	46.4 (38.3–54.7)	89.3 (88.8–89.7)
Backus et al.⁴	2388	407 (17)	42.8 (37.9–47.7)	86.0 (84.4–87.5)	38.5 (34.9–42.3)	88.0 (87.0–88.8)
Six et al.¹⁹	410	30 (7)	20.0 (7.7 –38.5)	98.6 (97.0–99.6)	54.6 (28.0–78.7)	94.0 (92.9–94.9)
Melki et al.³	2906	374 (12.9)	25.7 (21.3–30.4)	88.7 (87.4–89.9)	25.1 (21.5–29.2)	89.0 (88.4–89.6)
Visser et al.²²	255	75 (29.4)	20.0 (11.7–30.8)	97.2 (93.6–99.1)	75.0 (53.1–88.8)	74.5 (72.2–76.6)
Liu et al.¹³	648	223 (34)	16.6 (12.0–22.1)	99.3 (98.0–99.9)	92.5 (79.4–97.5)	69.4 (68.1–70.7)
Ong et al.²⁰	1690	169 (10.0)	31.4 (24.5–38.9)	89.0 (87.3–90.6)	24.1 (19.6–29.3)	92.1 (91.3–92.8)

ACS: acute coronary syndromes; PPV: positive predictive value; NPV: negative predictive value.

Table 4.

The diagnostic characteristics of studies that looked at T wave inversion.

References	Number of patients	Patients with ACS (%)	Sn	Sp	PPV	NPV
Ngako et al.¹⁸	399	124 (31)	34.7 (26.4–43.8)	73.1 (67.4–78.2)	36.8 (29.9–44.2)	71.3 (68.2–74.2)
Hess et al.⁷	2718	336 (12)	14.9 (11.3–19.1)	93.9 (92.9–94.8)	25.6 (20.3–31.8)	88.7 (88.2–89.1)
Liu et al.¹³	648	223 (34)	26.9 (21.2–33.2)	86.4 (82.7–89.5)	50.9 (42.8–58.8)	69.3 (67.3–71.1)
Ong et al.²⁰	1690	169 (10)	46.8 (39.0–54.6)	68.6 (66.2–71.0)	14.2 (12.2–16.5)	92.1 (90.9–93.1)

ACS: acute coronary syndromes; PPV: positive predictive value; NPV: negative predictive value.

There were some concerns regarding patient selection in the included studies. Hess and Ong excluded patients under the age of 24 and 25, respectively, while Liu and Ngako went further, excluding those under the age of 30 and 65, respectively.^7,13,18,20 Fleischmann and Ong also excluded patients that had more than 1 mm of ST-depression in two or more contiguous leads.^20,21 There was also concern regarding how representative the patient populations were with some papers having a prevalence between 24% and 34% or as low as 7%.^{3,13,14,18,21,22}

The method for analysing the index test, the ECG, was unclear in several papers but the majority of queries were cleared up thanks to beneficial responses via email.^{4,13,14,18–20} Only four papers had the treating clinician reviewing the ECG; the remainder were evaluated by independent investigators affecting the applicability of their findings.^13,19,20,22

A number of the papers used low sensitivity troponins or failed to define AMI using the universal definition of myocardial infarction, meaning that the reference standard may incorrectly diagnose the target condition, although once again most queries were resolved via email. The use of different assays affected the applicability of the reference standard in several papers. The ST-depression was measured in many ways, using different criteria. We outlined the exact cut-offs in the diagnostic characteristics table, describing how certain researchers used the J point while other used 60 ms past the J point. Other papers left it up to the treating clinician’s discretion. This variability was similar in the T wave inversion articles, with only two papers using T wave inversion of 0.1 mV in two contiguous leads.^18,20

The ST-depression criteria appear to be highly specific but poorly sensitive in 4 papers, with a specificity of 97.2%–99.3% and a sensitivity of 16.6%–20.0%.^3,7,13,²² The other papers reported a higher sensitivity of 25.7%–58.6% but a lower specificity of 86%–91.2%.^{3,4,14,18–20} One article had an impressive combination of sensitivity and specificity, but this was a small study with a remarkably high prevalence.²¹ The vast disparities may be due to certain papers with high sensitivity using Cardiologist assessment with less stringent diagnostic criteria.^4,14,19 Apart from the paper from Hess et al., T wave inversion demonstrates poor specificity; this is likely due to Hess et al.⁷ using 0.2 mV of T wave inversion. The papers that looked at 0.1 mV demonstrated a sensitivity of 26.9%–46.8% and a specificity of 68.6%–86.4%.^13,18,20

We do not feel that the Fourth Universal Definition will be the final iteration. As Sambola et al. note there are likely to be future changes in the ECG diagnostic criteria: global ST-depression with ST-elevation in aVR may be used to diagnose a left main coronary artery stenosis.²³ Furthermore, the way clinician’s judge ST-depression may change: Valentine et al. note that tachydysrhythmias may develop diffuse ST-segment depression resembling myocardial ischaemia that in fact represents non-ischaemic repolarization changes that the Fourth Universal Definition term ‘cardiac memory’.²⁴ The inclusion of this term in the new Universal Definition may affect the diagnostic performance of clinicians in the future. In light of these possible changes, we aim to prospectively assess the diagnostic accuracy of these guidelines in a multi-centre study using explicit inclusion and exclusion criteria.

Limitations

The primary outcome for this study was NSTEMI, and we considered that the optimal reference standard was adjudication according to the universal definition, but we also reviewed studies that reported the composite result of MACE. We made an a priori plan to synthesise these outcomes in two separate analyses. Unfortunately, we were unable to produce a meta-analysis due to unclear definitions and different troponin assays. The issues with synthesis were further compounded by the studies’ different ways of measuring ST-depression and T wave inversion. It appears that as one digs deeper into the literature, there is no standardised measurement of ST-depression despite the universal definition for NSTEACS being in its current incarnation for the past 6 years.^8,13 This finding has concerning implications for the utility of existing clinical prediction models: while there is no consensus in the literature for what defines ischaemic ST-changes, one should not suppose that clinicians will be able to use these models accurately and reliably. This proposition is given credence by the variability in the diagnostic performance of the History, ECG, Age, Risk factors, and initial Troponin (HEART) Pathway.²⁵

Conclusion

Current Emergency Department research into the diagnostic accuracy of the ECG for NSTEACS demonstrates a sensitivity and specificity that varies greatly between studies. The main issue appears to be inconsistent definitions of the primary outcome and the index test. The extent of between-study heterogeneity in the approach to interpreting ST-changes on the ECG clearly identifies a pressing need for greater standardisation of the approach. This also emphasises the potential value of a prediction model using objective parameters to optimise the approach to interpreting ECGs for evidence of ischaemia.

Footnotes

Authorship

All three authors contributed to all the phases of this research.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethical approval

This article was a systematic review of previously published literature and did not require ethical approval.

Human rights

This article was a systematic review of previously published literature and did not infringe anyone’s human rights.

Informed consent

This paper did not use patient data and thus no consent was required.

ORCID iD

Niall Morris

References

NHS. Hospital admitted patient care activity, 2016-17: diagnosis, https://digital.nhs.uk/data-and-information/publications/statistical/hospital-admitted-patient-care-activity/2016-17

Pope

Aufderheide

Ruthazer

, et al. Missed diagnoses of acute cardiac ischemia in the emergency department. N Engl J Med 2000; 342(16): 1163–1170.

Melki

Jernberg

HEART score: a simple and useful tool that may lower the proportion of chest pain patients who are admitted. Crit Pathw Cardiol 2013; 12(3): 127–131.

Backus

Six

Kelder

, et al. A prospective validation of the HEART score for chest pain patients at the emergency department. Int J Cardiol 2013; 168(3): 2153–2158.

Body

Carley

McDowell

, et al. The Manchester acute coronary syndromes (MACS) decision rule for suspected cardiac chest pain: derivation and external validation. Heart 2014; 100(18): 1462–1468.

Cullen

Greenslade

Than

, et al. The new Vancouver CHEST PAIN RULE using troponin as the only biomarker: an external validation study. Am J Emerg Med 2014; 32(2): 129–134.

Hess

Brison

Perry

, et al. Development of a clinical prediction rule for 30-day cardiac events in emergency department patients with chest pain and possible acute coronary syndrome. Ann Emerg Med 2012; 59(2): 115–125. e1.

Thygesen

Alpert

Jaffe

, et al. Fourth universal definition of myocardial infarction. Eur Heart J. 2018; 25: ehy462.

Thygesen

‘Ten commandments’ for the fourth universal definition of myocardial infarction 2018. Eur Heart J 2019; 40(3): 226–226.

10.

Liberati

Altman

Tetzlaff

, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Plos Med 2009, http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2707010/

11.

Thygesen

Alpert

Jaffe

, et al. Third universal definition of myocardial infarction. Circulation 2012; 126(16): 2020–2035.

12.

Whiting

Rutjes

AWS

Westwood

, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011; 155(8): 529–536.

13.

Liu

Lee

MAB

AFW

, et al. Risk stratification for prediction of adverse coronary events in emergency department chest pain patients with a machine learning score compared with the TIMI score. Int J Cardiol 2014; 177(3): 1095–1097.

14.

Six

Backus

Kelder

JC.

Chest pain in the emergency room: value of the HEART score. Neth Heart J 2008; 16(6): 191–196.

15.

Herlitz

Karlson

Karlsson

, et al. A description of the characteristics and outcome of patients hospitalized for acute chest pain in relation to whether they were admitted to the coronary care unit or not in the thrombolytic era. Int J Cardiol 2002; 82(3): 279–287.

16.

Karlson

Herlitz

Wiklund

, et al. Early prediction of acute myocardial infarction from clinical history, examination and electrocardiogram in the emergency room. Am J Cardiol 1991; 68(2): 171–175.

17.

Tierney

Fitzgerald

McHenry

, et al. Physicians’ estimates of the probability of myocardial infarction in emergency room patients with chest pain. Med Decis Making 1986; 16(1): 12–17.

18.

Ngako

Santin

Hemery

, et al. Prediction of myocardial infarction risk in older patients with acute coronary syndrome. Am J Emerg Med 2009; 27(6): 675–682.

19.

Six

Cullen

Backus

, et al. The HEART score for the assessment of patients with chest pain in the emergency department: a multinational validation study. Crit Pathw Cardiol 2013; 12(3): 121–126.

20.

Ong

MEH

Hao

Yap

, et al. Validation of the new Vancouver chest pain rule in Asian chest pain patients presenting at the emergency department. CJEM 2017; 19(1): 18–25.

21.

Fleischmann

Zegre-Hemsey

Drew

BJ.

The new universal definition of myocardial infarction criteria improve electrocardiographic diagnosis of acute coronary syndrome. J Electrocardiol 2011; 44(1): 69–73.

22.

Visser

Wolthuis

Breedveld

, et al. HEART score and clinical gestalt have similar diagnostic accuracy for diagnosing ACS in an unselected population of patients with chest pain presenting in the ED. Emerg Med J 2015; 32(8): 595–600.

23.

SEC Working Group for the 2018 ESC Fourth Universal Definition of Myocardial Infarction; Expert Reviewers for the 2018 ESC Fourth Universal Definition of Myocardial Infarction; SEC Guidelines Committee. Comments on the 2018 ESC fourth universal definition of myocardial. Rev Esp Cardiol 2019; 72(1): 10–15.

24.

Valentine

Tcheng

Waites

Translating the translation. J Am Coll Cardiol 2018; 72(21): 2668.

25.

Van Den Berg

Body

. The HEART score for early rule out of acute coronary syndromes in the emergency department: a systematic review and meta-analysis. Eur Heart J Acute Cardiovasc Care 2018; 7(2): 111–119.