Analytical assays and bootstrap resampling method to validate performance of the Roche Elecsys STAT highly sensitive troponin T assay and its application for the ‘rule-out’ part of ESC guidelines for NTSTEMI

Abstract

Background

The European Society of Cardiology (ESC) guidelines recommend a dynamic (0-1h) cardiac troponin (cTn) determination for non-ST elevation myocardial infarction diagnosis. For patients with low cTn levels, a discharge from emergency can be considered. Nevertheless, cTn cutoffs for discharge are lower than the limits of quantification proposed by laboratory reagent suppliers.

Aim

Validate cTn assay on the Elecsys STAT kit.

Materials and methods

Precision, trueness, repeatability and within-laboratory variability were calculated from internal quality control and plasma pooled at 5.78 and 10.73 ng/L. Accuracy was calculated from external quality control. Uncertainty of measurement was calculated from (i) the uncertainty of the standard and control values and (ii) by precision from pooled plasma. Distribution of precision results from pooled plasma has been evaluated by bootstrap simulations. Dilution linearity tests with patient plasma were performed to evaluate the method for values near 5 ng/L.

Results

Precision and trueness ranged from 1.35 to 4.45% and from 0.14 to −3.74%, respectively. Accuracy results ranged from 101.40 to 104.90%. Within laboratory variability was 2.91%. Uncertainty ranged from 3.66% to 19.90% for higher (2188) to lower values (5.78 ng/L). Bootstrap simulations allowed utilization of precision data from pooled plasma to evaluate cTn assay. The method was linear from 4.48 to 39.80 ng/L. A linear regression model best described the data.

Conclusion

Elecsys STAT method provides accurate cTn results, including patients with cTn results categorizing them as ‘rule-out’ in the ESC guidelines.

Keywords

Cardiac troponin emergency bootstrap NSTEMI ESC

Introduction

High-sensitivity cardiac troponin (hs-cTn¹,²),¹ I or T, is recommended for the diagnosis and management of patients with chest pain suspected of non-ST elevation myocardial infarction (NSTEMI³).² This assay is critical for emergency departments as it involves the ability to rule out a large proportion of patients. The algorithm is relevant in the context of overwhelmed emergency medical services, partly by successive waves of Covid.

Thus, for the Roche Elecsys® hs-TnT⁴ kit,³ patients with concentrations at first sampling (T0⁵) < 5 ng/L or T0 <12 ng/L and with a delta⁶ T0/T1 (or 0h/1h rule-out and rule-in algorithm as named by the ESC) less than 3 ng/L where T1⁷ as a second sampling at 60 +/− 10 min from the first sampling can be ruled out.² In this way, in terms of length of stay, resource utilization and predicted diagnostic accuracy,⁴ it is important that the laboratory's hs-cTn methods can quantify low cTn⁸ concentrations, following ESC⁹ guidelines.

However, for the Elecsys® hs-TnT kit, but also for other hs-cTn suppliers, concentrations of interest to categorize ‘rule-out’ patients are lower than the limit of quantification (LLOQ)¹⁰ proposed by the supplier (<13 ng/L for Elecsys Roche®).³ While cTn quantification is likely possible, little data describes kit performance (e.g. precision and trueness) at low concentrations. Besides, to achieve the shortest possible turnaround time (Tat¹¹), suppliers provide STAT¹²(Short Turnaround Time)⁵ kit, reducing quantification delay (9 instead of 18 min for Elecsys®). However, there is limited data comparing STAT and non-STAT reagents, especially for conditions not recommended by the supplier.

Consequently, to fulfil the ESC recommendations,² the objective of our study was to evaluate the performance of the hs-TNT STAT Elecsys® assay, especially for cTn concentrations categorizing patients as ‘rule-out’.

Material and methods

Immunoassays

Assays were performed at Montauban Hospital, on a Cobas® Pro e801 module (Roche Diagnostics®, Meylan, France), on a single measuring cell. The used kit was Elecsys® TnT Gen 5 STAT, with a TaT of 9 min. Briefly, the reaction implies ruthenium and biotin-labelled antibodies. They form sandwich complexes with TnT from the sample, corresponding to an electrochemiluminescence sandwich immunoassays.⁶ According to instructions shift from the manufacturer, following guidelines from CLSI EP17-A2^{13
7} lower limit of quantification (LLOQ¹⁴) and limit of detection (LOD)¹⁵ are 13 and 5 ng/L, respectively. The manufacturer performed an internal study to evaluate LOD and LLOQ with a precision coefficient of variation (CV) lower to 20% for hs-TnT STAT Elecsys®. LLOQ for a CV¹⁶ below 10 and 20%, and LOD were 3.22, 1.00 and 2.72 ng/L. The manufacturer data was used to compare the reagent evaluation results for concentrations <13 ng/L, as no further data is available in the literature.

Samples

Assays to evaluate hs-TnT quantification performances were conducted on two types of samples, Roche® internal quality controls (IQC) and patient samples. IQCs were PreciControl® (PC) TN1 and 2¹⁷, for targets (peer) at 25.7 and 2115 ng/L, respectively.

Because IQCs do not evaluate the performance of the method for values <13 ng/L, assays were performed on patient samples for this range of concentrations. Plasma samples were all collected from lithium heparin tubes (Alloga®, France) and used within 8 h from patient sampling. On arrival at the laboratory, the samples were centrifuged at 2000g¹⁸ for 10 min. Samples were first analyzed according to the physician's prescription and then stored for subsequent use in this study. To assess within-day and interday precision and bias, plasma samples were pooled to achieve concentrations close to 5 and 10 ng/L (pool plasma 1 and 2 [PP1 (5,78) and PP2 (10,73)]).¹⁹

To assess linearity, plasmas from patients with cTn (i) <3, (ii) 15, (iii) 20 and (iv) 150 ng/L were selected. Samples with cTn <3 ng/L were used to dilute plasmas with higher concentrations. After appropriate dilution, the samples were pooled to achieve a suitable volume for processing. The concentration from the resulting pool was measured to determine the initial concentration.

Troponin quantification was part of routine patient care, without any additional procedures. Therefore, no institutional review board or ethics-committee approval was required, in accordance with the French regulations on biomedical research.^8,9

External quality controls (EQC²⁰) were also used to evaluate the accuracy of the method for concentrations >13 ng/L.¹⁰ The EQC were provided by Probioqual®, a French supplier and member of EQALM²¹ (European organization for external Quality Assurance providers in Laboratory Medicine).

Precision, trueness, accuracy, and uncertainty of measurement

Roche® IQC were performed on hs-TnT method as recommended by the supplier. For pooled plasma between-run assays, samples were stored at −20°C, as recommended by kit supplier. Before the assay, the plasma was thaw at room temperature (23°C), vigorously shaken and then centrifuged at 10,000g for 5 min.

Trueness and precision were evaluated from within-day and between-run assays. For within-day validation, 56 samples from the four levels (PCTN1/2 26,3 and 2188 ng/L, respectively, and the two levels of plasma 5,78 and 10,73 ng/L, respectively) were analyzed 2 times per day. For between-days validation was determined one sample by level from PCTN IQC during 7 days and 2 samples by level for pooled plasma also during 7 days. Trueness was defined as the percentage deviation from the nominal level (trueness = [(concentration_n - target concentration)/concentration_n]*100, and precision as the coefficient of variation, calculated by the equation CV = (sd/m)*100; where CV= coefficient of variation, sd²² = standard deviation, m²³ = mean.

Within-laboratory reproducibility was calculated by the following equation:

R w = [(R S D i + \dots + R S D n) / n]

(1)

where Rw: within-laboratory reproducibility and RSD: relative standard deviation.

Within-run precision (repeatability) was calculated and was used as a target to evaluate between-run precision (between run precision = 0.75*repeatability precision).^24,2511,12

For IQCs, trueness was obtained from the TIQCON® software¹³ and inter laboratory IQC system.

The method accuracy was evaluated using the EQC program from Probioqual® (France; member of EQALM).¹⁴ The accuracy was expected to be within 80%–120% range.¹⁵

Uncertainty of measurement (UM²⁶)^16–19 was calculated according to two equations, one considering IQCs and the other precision data from pooled plasma. From the IQC, UM calculation was based on (i) UM associated to calibration standard (supplier data) and (ii) IQC (PCTN) data analyzed using TIQCON® inter laboratory quality control. Then, UM was calculated according to the following equation:

u (C) = {[u^{2} (C Q I) + u^{2} (s t a n d a r d)]}^{1 / 2}

(2)

As suggested by international guidelines, if no calibration UM is available, precision data generated from pooled plasma was used to calculate the UM by the following equation:

u (C) = {[u^{2} (C Q I)]}^{1 / 2}

(3)

For both methods to calculate uncertainty, the expanded value (U) can be predicted as:

U = 2 \times u (C)

(4)

To evaluate the evolution of UM with cTn concentration, concentration was plotted against UM.

Linearity

To evaluate the linearity²⁰ of the method on concentrations <13 ng/L, four pooled plasma samples were diluted with samples free of cTn (initially measured <3 ng/L). The selected model (e.g. linear and quadratic), better describing the data, allowed to conclude about the linearity of the method.

Each dilution point obtained was repeated (n = 10). The mean achieved during repeatability was used as the theoretical target. The cTn quantified during repeatability was compared to this theoretical target to evaluate the trueness of the repeatability. Repeatability precision was assessed as described in Precision, Trueness, Accuracy, and Uncertainty of Measurement section.

cTn from patients

T0, T1 and deltaT0/T1 of patients from emergency department were recorded for 6 months in order to compare biological variability of cTn to UM. Data analysis focused on samples with cTn results <10 ng/L, at which value (i) patients are often at low risk for NSTEMI and (ii) UM is strongly increasing. The variation of T1–T0 was calculated by the following equation: variation (%) = (DeltaT1-T0/T0)*100.

Statistics

The distribution of the plasma pooled results (normal distribution) was evaluated by a Shapiro–Wilk test. If the distribution was normal, the data was used for parametric bootstrapping or variance comparison of precision results.

A parametric bootstrap method was applied to precision data from pooled plasma to assess the distribution and dispersion of results. A simulation of n=100,000 was used on the initial data (n = 10 and 10 for PP1 and PP2, respectively). The simulation results were expressed as mean and 95% prediction interval (m; [IP5;95]). Precision results were considered valuable in assessing the method if the bootstrap simulations led to a small dispersion of results. Bootstrap simulations were assessed using wessa® online software²¹ and R software (R4.3.1; ‘function [bootstrap_result, alpha]’). To evaluate the linearity of the method up to low cTn values (<13 ng/L), dilution data was combined into a single dataset. To determine whether the model that best describes the method is a line or a curve, a linear regression was compared to a quadratic regression. The model selected was the one with the highest r^2 and adjusted r^2 and lowest Akaike information criterion (AIC)/Bayesian Information Criterion (BIC) scores. Goodness of fit were assessed using Mycurvefit²² online software and R software (R4.3.1; function ‘lm’, ‘ggplot2’).

The level of significance was set at p < 0.05.

Results

Precision, trueness, accuracy, and uncertainty of measurement

Data from IQC and pooled plasma for trueness, precision, repeatability, within-laboratory variability and uncertainty were described in Table 1. Accuracy values from external quality control were ranged from 101.4% to 104.9% for 4 samples with cTn values which ranged from 19.3 to 1832 ng/L.

Table 1.

Precision, trueness, accuracy, and uncertainty of measurement for IQC and pooled plasma.

Level	hs-cTnT target (ng/L)	Trueness IQC (%)	Precision IQC (%)	Accuracy EQC (%)	Rw (%)	Uncertainty measurement (%)
PPP1	5.78	−3.74	4.45	NA	2.91	19.80
PPP2	10.73	−1.93	2.49	NA		9.96
PCTN1	26.3	−0.272	2.56	−13.44		6.37%
PCTN2	2188	0.142	1.35	−6.22		3.66%

cTn: cardiac troponin; IQC: internal quality control; EQC: external quality control; NA: Non applicable; Rw: within-laboratory reproducibility; PPP1 and PPP2: pooled plasma 1 and pool plasma 2; PCTN: precicontrol troponin (Roche® internal quality control).

Precision results from bootstrap simulation were 5.75 ng/L IP95 [5.70; 5.81] and 10.52 ng/L IP95 [10.43; 10.63] from wessa software and 5.75 ng/L IP95 [5.68;5.82] and 10.53 ng/L IP95 [10.43; 10.63] for pooled plasma 1 and 2, respectively.

Uncertainty measurement was plotted against cTn concentration in Figure 1.

Figure 1.

Uncertainty of measurement applied to cTn concentrations.

Linearity

Results of dilution and precision/trueness of repeatability of each dilution point are detailed in Table 2.

Table 2.

Results of dilution to assess the linearity of the method.

	Dilution 11/2		Dilution 21/4		Dilution 31/9	Dilution 41/10
Initial concentration (calculated; ng/L)	12.27	15.92	7.86	9.68	7.69	4.79
Measured diluted concentration (mean; ng/L)	12.08	16.56	8.02	10.11	7.29	4.48
Repeatability (CV%)	1.88	1.25	3.09	2.27	3.27	5.16
Trueness (%)	1.57	−3.86	−1.99	−4.25	5.62	7.14

CV: coefficient of variation.

A linear regression (y= −0.36 + 1.01x + ε with ε describing random error; r^2 = 0.9985 and adjusted r^2 = 0.9981 for Mycurvefit software; y = 0.348 + 0.989*x + ε with r^2 = 0.9985, adjusted r^2 = 0.9981 from R) was the model that best described the data, compared to a quadratic equation (y = −0.3739347 + 1.011,893*x - 0.00004518025*x² + ε for Mycurvefit software; y = 0.00,287 + 1.04*x −0.00,127*x² + ε with r^2 = 0.9986, adjusted r^2 = 0.9976 from R), based on AIC and BIC scores²⁷ (2.952 and 3.346 for AIC and BIC; 4.947 and 5.539 for AIC and BIC, for linear and quadratic models, respectively, from Mycurvefit and 7.364 and 6.739 for AIC and BIC; 9.129 and 8.296 for AIC and BIC, for linear and quadratic models, respectively, from R). The regression was linear between 4.48 and 39.8 ng/L.

Discussion

The aim of our study was to explore the performance of the hs-TnT²³ assay by the Roche Elecsys® STAT kit, especially for low cTn values (<13 ng/L). Indeed, the latter are of particular interest for the rule-out strategy of the ESC recommendations.² ESC algorithm require quantification of low cTn concentrations (<5 and delta T0-T1<3 ng/L for Elecsys Roche®).² However, few studies detail analytical validations for cTn kit within these measured ranges.

Thus, the linearity was validated over a range from 4.48 to 39.80 ng/L. The best model to describe the data was a linear plot like Chenevier-Gobeaux et al.²⁰ Such data is relevant to interpret sandwich reaction signal measurements. Indeed, measurement range linearity on low values is not consistent for the sandwich immunoassay.

Quantification performance for low cTn values is supported by precision and trueness data, both from IQC and PP. Indeed, the latter are consistent with those observed in other studies.^20,24 Between-run assay data (precision, trueness) interpretation, obtained from pooled plasma, could have been limited by the number of samples used. This limitation has been overcome by data simulated by bootstrap,¹² which confirms the validity of the results obtained in these tests. By resampling the observed data, bootstrap provides a way to estimate the variability of an analytical method. It allows to generate multiple ‘bootstrap samples’ by randomly selecting observations from the original dataset with replacement. Besides variability, bootstrap is of particular interest to assess the precision of an analytical method for small sample sizes. Indeed, this method generates additional synthetic datasets by resampling the available data. It provides more robust estimates of variability. Few teams applied this simulation tool in biochemistry. Nevertheless, it has been validated in several fields, including haematology,¹² and is useful to study the distribution of results on small samples, as in our study.

In fact, different analytical performances between instruments could explain discrepancies in sensitivity/specificity for the diagnosis of NSTEMI between methods. Varying performance, and therefore UM, would lead to a variation in cTn around the ‘rule-out’ threshold at T0 but also in the delta T0/T1. A knowledge of UM at such levels would allow a grey zone to be established in order to avoid misrecognized patients requiring extended management (e.g. measurement of cTn at T3²⁸). However, a proper estimation of UM is required. Indeed, given the performance of our method (UM = 20%) and ESC algorithm,² one should observe many values in T1<T0 where physiologically cTn does not vary. Therefore, such performance could lead to an error in management, with an inappropriate rule-out. Nevertheless, (i) data from clinical trials are reassuring² and (ii) only 11% of T0–T1 pairs observed in our laboratory have a cTn 20% different at T1, compared to T0, with no accounting for cTn biological variability in these results. Consequently, UM, but also precision, increase seems analytically consistent with decreasing cTn concentrations. Considering precision data detailed from other studies, one can expect UM (equation (2) prediction) to be close to those calculated in our study. However, these performances cannot be achieved for all cTn measurement methods.

Indeed, point-of-care (POCT)²⁹ systems^2,25 are (i) not sufficiently sensitive and (ii) precision data may lead to large UM, to effectively distinguish patients in rule-out situations. Nevertheless, POC technologies are improving^26–28 and recently proposed systems have analytical performance close to that of laboratory instruments. For our activity, this approach is still not appropriate because (i) the management of POCs³⁰ with clinical departments can be challenging, (ii) they are expensive compared to automated measurements and (iii) our system can deliver a result with an optimal TaT (pneumatic transport of samples, centrifugation of Barricor™ tubes in 3 min on a dedicated centrifuge and a STAT reagent that delivers results in 9 min; average turnaround time in our laboratory of 30 min after receipt of the sample). Furthermore, the TaT cannot be an argument on its own, as the ESC recommends sampling at T1 without waiting for the T0 result.

The ability to provide a result with a short TaT relies mainly upon the configuration of STAT to quantify cTn.^5,29 This involves merging the two reaction steps into one. However, limited data is available on the analytical performance of this reagent configuration, from Roche® or other suppliers. Available Roche® data for cTn suggests that performance is not quite the same with or without STAT.^25,30 No comparison was made between the two approaches in our study. But based on (i) data described in our study, with performance close to that observed for values > LLOQ and (ii) Roche Elecsys® reagent data without STAT, it could be assumed that the STAT method is robust for low concentrations (<13 ng/L) of cTn.

Conclusions

In conclusion, our study supports that the Elecsys® STAT hs-TnT assay to accurately determine cTn, even for values below the LLOQ documented by the supplier. From an analytical point of view, it is therefore possible to follow the ESC recommendations, especially for ‘rule-out’ orientation of patients.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship and/or publication of this article.

Ethical approval

None (part of laboratory activity).

Guarantor

MD.

Contributorship

All authors contributed to the design and execution of the study. All authors reviewed and edited the manuscript and approved the final version.

ORCID iD

David Metsu

Notes

References

Mueller

Egger

Peer

, et al. 5th generation cardiac troponin I and T assays in clinical routine – A head-to-head comparison with data from the Linz troponin (LITROP) study. Clin Chim Acta 2018; 485: 195–204.

Collet

Thiele

Barbato

, et al. 2020 ESC Guidelines for the management of acute coronary syndromes in patients presenting without persistent ST-segment elevation. Eur Heart J 2021; 42(14): 1289–1367.

Elecsys® Troponin T hs cobas® Technical data sheet Roche Diagnostics.

Ambavane

Lindahl

Giannitis

, et al. Economic evaluation of the one-hour rule-out and rule-in algorithm for acute myocardial infarction using the high-sensitivity cardiac troponin T assay in the emergency department. Zeller T, éditeur. PLoS One 2017; 12(11): e0187662.

Kavsak

Andruchow

McRae

, et al. Profile of Roche’s Elecsys Troponin T Gen 5 STAT blood test (a high-sensitivity cardiac troponin assay) for diagnosing myocardial infarction in the emergency department. Expert Rev Mol Diagn 2018; 18(6): 481–489.

Zhelev

Hyde

Youngman

, et al. Diagnostic accuracy of single baseline measurement of Elecsys Troponin T high-sensitive assay for diagnosis of acute myocardial infarction in emergency department: systematic review and meta-analysis. BMJ 2015; 350(jan12 11): h15–h15.

CLSI . Evaluation of detection capability for clinical laboratory measurement procedures; approved guideline—Second Edition. CLSI document EP17-A2. Wayne, PA: Clinical and Laboratory Standards Institute, 2012.

French code of public health, article L1121-1

[Internet]. Disponible sur: https://www.legifrance.gouv.fr/codes/article_lc/LEGIARTI000006685827/2006-04-19

French code of public health, article L1121-2. Disponible sur: https://www.legifrance.gouv.fr/codes/article_lc/LEGIARTI000025457486/

10.

Apple

, Collinson PO, for the IFCC task force on clinical applications of cardiac biomarkers. Analytical characteristics of high-sensitivity cardiac troponin assays. Clin Chem 2012; 58(1): 54-61.

11.

Kin Tekce

Tekce

Aktas

, et al. The role of the uncertainty of measurement of serum creatinine concentrations in the diagnosis of acute kidney injury. Ren Fail 2016; 38(2): 305–310.

12.

Cui

Wang

, et al. Combining Nordtest method and bootstrap resampling for measurement uncertainty estimation of hematology analytes in a medical laboratory. Clin Biochem 2017; 50(18): 1067–1072.

13.

Tiqcon [Internet]. Disponible sur. https://www.tiqcon.com/

14.

External quality controls, Probioqual®

[Internet]. Disponible sur: https://www.probioqual.com/02_EEQ/01_BIOCHIMIE/0215_EEQ_CARD_FR.php

15.

Vassault

Grafmeyer

de Graeve

, et al. Analyses de biologie médicale: spécifications et normes d’acceptabilité à l’usage de la validation de techniques. Ann Biol Clin 1999; 57(6): 685–695.

16.

Braga

Panteghini

. The utility of measurement uncertainty in medical laboratories. Clin Chem Lab Med 2020; 58(9): 1407–1413.

17.

Ćelap

Vukasović

Juričić

, et al. Minimum requirements for the estimation of measurement uncertainty: Recommendations of the joint Working group for uncertainty of measurement of the CSMBLM and CCMB. Biochem Med 2017; 27(3): 030502.

18.

Farrance

Badrick

Frenkel

. Uncertainty in measurement: a review of the procedures for determining uncertainty in measurement and its use in deriving the biological variation of the estimated glomerular filtration rate. Practical Laboratory Medicine 2018; 12: e00097.

19.

Rigo-Bonnin

Díaz-Troyano

García-Tejada

, et al. Estimation of the measurement uncertainty and practical suggestion for the description of the metrological traceability in clinical laboratories. Biochem med (Online) 2021; 31(1): 15–26.

20.

Chenevier-Gobeaux

Meune

Blanc

, et al. Analytical evaluation of a high-sensitivity troponin T assay and its clinical assessment in acute coronary syndrome. Ann Clin Biochem 2011; 48(5): 452–458.

21.

Wessa

, Free statistics software, office for research development and education, version 1.2.1 [Internet]. Disponible sur, 2022. https://www.wessa.net

22.

Mycurvefit OCF . Disponible sur. https://mycurvefit.com/

23.

Ren

Huang

Liu

, et al. High-sensitive cardiac troponin for the diagnosis of acute myocardial infarction in different chronic kidney disease stages. BMC Cardiovasc Disord 2021; 21(1): 100.

24.

Koerbin

Tate

Hickman

. Analytical characteristics of the Roche highly sensitive troponin T assay and its application to a cardio-healthy population. Ann Clin Biochem 2010; 47(6): 524–528.

25.

Collinson

Saenger

Apple

on behalf of the IFCC C-CB . High sensitivity, contemporary and point-of-care cardiac troponin assays: educational aids developed by the IFCC committee on clinical application of cardiac bio-markers. Clin Chem Lab Med 2019; 57(5): 623–632.

26.

Boeddinghaus

Nestelberger

Koechlin

, et al. Early diagnosis of myocardial infarction with point-of-care high-sensitivity cardiac troponin I. J Am Coll Cardiol 2020; 75(10): 1111–1124.

27.

Sörensen

Neumann

Ojeda

, et al. Diagnostic evaluation of a high-sensitivity troponin I point-of-care assay. Clin Chem 2019; 65(12): 1592–1601.

28.

Gunsolus

Schulz

Sandoval

, et al. Diagnostic performance of a rapid, novel, whole blood, point of care high-sensitivity cardiac troponin I assay for myocardial infarction. Clin Biochem 2022; 105–106: 70–74.

29.

Diagnostics

. Elecsys troponin T gen 5 STAT. Indianapolis, IN: Roche Diagnostics. 2017.

30.

High-Sensitivity* Cardiac Troponin I and T Assay Analytical Characteristics Designated by Manufacturer IFCC Committee on Clinical Applications of Cardiac Bio-Markers (C-CB) v122019.