Traditional Interference Experiments vs. Method Comparison Interference Experiments

Abstract

Two papers have appeared evaluating interferences in glucose meters. These studies are method comparisons with the added information of the medication(s) taken by the subjects. This paper contrasts a traditional interference study with the method comparison protocols. Unlike the advice in CLSI EP7, a substance that interferes should be reported even if the level of interference is clinically acceptable. The evidence of no clinically important interference in the method comparison protocol is very weak, and there is no possibility to detect statistically significant interferences. I provide an example where vitamin C at a therapeutic level was within clinical error limits, but when the concentration was at levels used to treat cancer, there was bias well above clinically acceptable limits.

Introduction

Two recent papers have evaluated interferences in glucose meters.^1,2 These are method comparisons with the added information of the medication(s) taken by the subjects. These studies called here; method comparison interference protocols (MCIP) deviate from traditional interference studies.

The purpose of this paper is to summarize a traditional interference study such as that described in CLSI (Clinical Laboratory Standards Institute) EP7, ³ and to contrast the differences between it and the MCIP.

An interference is a cause of bias in an analyte’s measurement due to a compound other than the analyte, or a property of the sample. An example interference caused by a compound could be a medication. An example interference caused by the property of the sample could be lipemia. An interference experiment has the same elements as all experiments: a goal, protocol, analysis, and report.

Goals

To understand the goal for an interference study, it is helpful to conceptually review the possible errors for a glucose meter result. ⁴ They are as follows:

• Imprecision, which affects every result

• Fixed bias, which also affects every result and may be zero. Examples are calibration and nonlinearity bias.

• Protocol dependent bias, which also may be zero. Here, the protocol refers to a dependence on how the sample is analyzed. For example, if a reagent strip loses linearity near its expiration date, a bias will occur only for samples assayed during that time.

• Sample specific bias. This bias is caused by an interference in a specific patient sample. If the assay is repeated, the error will persist. All samples have bias due to the net effect of all interferences in the sample. This bias is small and is part of the fixed bias. Sample specific bias refers to a larger bias usually due to 1 interference.

• Other errors include user error and manufacturing defects.

As a goal, there are two questions that can be answered by an interference experiment: (1) has interference been detected, and (2) if interference is detected, is it clinically significant.

Statistical Significance vs. Clinical Significance

Throughout all aspects of medicine, there are hundreds of articles about statistical versus clinical significance. One of the points often made is that statistically significant findings may be clinically unimportant. This also applies to interference in glucose meters where the amount of error that is clinically considered unimportant is rather large as represented by the width of the A (and perhaps B) zones in a Parkes glucose meter error grid. It is my opinion that a statistically significant interference which is clinically unimportant should be so reported (e.g., statistically significant interference detected and clinically unimportant). This differs from CLSI EP7 guidance which states that compounds that meet a clinical goal are not considered interference. An example from the literature following the EP7 guidance states, “substances frequently reported to interfere with enzymatic, electrochemical-based transcutaneous CGM systems, such as acetaminophen and ascorbic acid, did not affect Eversense readings.” ⁵

Yet in their table of interference results they show:

At 74 mg/dL of glucose, interference from 3 mg/dL of acetaminophen is -8.7 mg/dL

At 77 mg/dL of glucose, interference from 2 mg/dL of ascorbic acid is 7.7 mg/dL

Their goal for clinical significance was the ISO 15197 (total error) standard.

My rationale for reporting a compound that has been shown to interfere even though the level of interference is not clinically significant is that throwing away information is antithetical to good data analysis.

The goal of the MCIP is the ISO 15197 clinical acceptance goal of 95% of values within ±15 mg/dL (<100 mg/dL) and ± 15% (≥100 mg/dL). This total error goal accounts for errors from all sources; hence, any observed error cannot be assigned solely to an interference. And the fact that only 95% of the results must meet the ±15 mg/dL or ±15% limits is rather weak evidence to claim no interference. A previous traditional examination of glucose meter interferences used a goal of ±6 mg/dL. ⁶ Because other errors could add to an interference bias, the goal for an interference must be less than a total error goal.

Protocol

The interference protocol has several parts: the selection of interference candidates, the concentrations of glucose and the candidate interference, and the way the experiment is carried out. The selection of interfering substances can be daunting since the number of substances in blood is in the thousands. Suggested tables of candidates are in EP7 with the recommendation to use a concentration of interfering substance at or near a toxic level. The MCIP involved ambulatory subjects that likely had therapeutic doses of medications. A recent study showed that an extremely high level of vitamin C, used to treat cancer patients caused substantial glucose meter interference but a therapeutic level caused a clinically unimportant bias. ⁷ Remember that in reference 5, a therapeutic level of vitamin C was tested, and it was claimed not to affect the sensor. Endocrinologists and users would benefit from knowing the effect of vitamin C at therapeutic levels even if the bias is clinically within limits.

The authors of the MCIP suggest that their protocol has the advantage of evaluating the cumulative effects of several medications. However, the likelihood is low that several medications will synergistically add to yield a substantial interference. Every method comparison is like the MCIP with the difference that in a non-MCIP method comparison, medications are unknown.

Many protocol variations are described in CLSI EP7. A simple example follows. One aliquots a sample into two portions. One portion has no interference and the other is spiked with a suitably high (often toxic) level of candidate interference. If desired, one can then make a series of other concentrations of interfering substances by mixing.

The control in the interference experiment is the patient’s blood without the candidate interfering substance. Both the control and blood with the candidate interference are tested on the same glucose meter. In the MCIP, the control is a reference method. But there is no guarantee that the reference is free from interference—it is not a definitive reference method. Moreover, there is no way to distinguish error sources in the difference between the glucose meter result from reference.

Analysis

To ensure reliable results, sufficient replicates must be run for the control and test samples. EP 7 provides formulas for sample size calculations. To determine if interference has been detected, one calculates the average difference between test and control and computes a 95% confidence interval (CI) for this t-test. A simple Excel formula for the lower and upper 95% CI of the average bias found is

$$\begin{array}{l}\mathrm{Lower}95\%\mathrm{CI}=\mathrm{average}-\mathrm{CONFIDENCE}.\mathrm{NORM}\\ \left(0.025,\mathrm{standard}\mathrm{deviation,}\mathrm{number}\mathrm{of}\mathrm{replicates}\right)\end{array}$$

$$\begin{array}{l}\mathrm{Upper}95\%\mathrm{CI}=\mathrm{average}+\mathrm{CONFIDENCE}.\mathrm{NORM}\\ \left(0.025,\mathrm{standard}\mathrm{deviation},\mathrm{number}\mathrm{of}\mathrm{replicates}\right)\end{array}$$

Reporting

Figure 1 shows the results of 4 candidate interfering substances and how the results could be reported. Here, the clinically significant interference was set at ±6 mg/dL.

Figure 1.

Results for 4 candidate interfering substances. Abbreviation: CI, confidence interval.

In case 1, no interference has been detected because the 95% CI includes 0 mg/dL, but interference could be as low as -2.5 mg/dL. If desired, the CI could be reduced by running more replicates.

In case 2, interference of 3.7 mg/dL has been detected. The 95% CI does not include 0 but is within the clinical goal ± 6 mg/dL. Interference could be as high as 5.7 mg/dL.

In case 3, interference of 8.6 mg/dL has been detected, which exceeds the clinical goal. There is uncertainty in this finding because the 95% CI includes 6 mg/dL.

In case 4, interference of 16.8 mg/dL has been detected, which exceeds the clinical goal.

In reference 1, the authors claim no interference for the 500 prescription medicines in the study.

CLSI EP7 provides an example like case 2 for a compound for which interference has been detected, but states that the compound does not interfere (because the clinical significance goal has been met). If interference has been detected, it should be reported, even if the interference is not clinically significant.

Strength of Evidence

Many papers have been written about what constitutes strong or weak evidence. ⁸ The traditional interference experiment is moderately strong evidence that interference for an assay has been characterized. The test of a substance with and without the candidate interference is strong evidence. The only limitation is one may not have tested sufficient candidate interferences.

The MCIP protocol is very weak evidence. The goal allows 5% of the results to be beyond acceptance limits. The concentrations tested are not at toxic levels as recommended by EP7. And errors are due to a variety of error sources.

Conclusions

In an experiment designed to detect interferences, it is recommended to report statistically significant interferences even if they are clinically unimportant. The interference experiment itself is simple—test a sample with and without the candidate interference on the same glucose meter. Whereas one might argue that a MCIP is complementary to an EP7 protocol (which is required by manufacturers), the evidence of no clinically important interference is very weak and there is no possibility to detect statistically significant interferences.