Why the Details of Glucose Meter Evaluations Matters

Abstract

In an article in the Journal of Diabetes Science and Technology, Macleod and coworkers describe an evaluation of LifeScan glucose meters that focus on the effects of sample types and comparison methods. They make a valid point that these factors influence the accuracy observed in evaluations and recommend the comparison method be the one recommended by the manufacturer for the sample type in the intended use statement. Yet, the recommended comparison method is not a reference method. The accuracy hierarchy of definitive, reference, and field methods originally described by Tietz should remind one that virtually all glucose meter evaluations use commercially available field methods as the comparison method. Finally, one should not neglect the FDA adverse event database as a way to assess glucose meter performance.

Evaluations of glucose meters appear often as one wants to know whether the meter in question meets performance standards, how several meters compare, and (by regulation) the performance of meters after release for sale. It is natural to treat the “reference method” results as truth. Or putting things another way, differences in results are considered to be errors in the meter(s) under evaluation. The authors of a recent study in this journal make an important point; namely, that the details of glucose meter evaluations are important. ¹

“Reference method” is in quotes because rarely is an actual reference method used. Almost 40 years ago, Tietz described an accuracy hierarchy ² for laboratory medicine. The highest accuracy level is a definitive method (isotope dilution mass spectrometry for glucose), followed by a reference method (hexokinase for glucose), followed by field methods (eg, commercially available methods). Unfortunately, this description has fallen out of favor as the ponderous and unhelpful metrology language seems to dominate these days. ³ Thus, the frequently used term “reference method” is incorrectly used in evaluation articles because the comparison method is almost never the glucose reference method. Note that whereas commercial laboratory methods are often hexokinase based, the reference hexokinase method is a manual method using a protein free filtrate. ⁴

An interesting disclosure by the authors is that their meters are calibrated to the YSI method. While not surprising, this information is usually not readily available. The typical calibration process (and this is true for many other substances measured in blood) is to measure both representative clinical samples on the glucose meter and a chosen comparison method over several days and multiple reagent lots. A calibration algorithm is developed that provides equivalent results—on average—between the glucose meter and the comparison method.

The authors’ additional point of sticking to the sample type(s) listed in the intended use statement is valid. In their case, insufficient oxygen in venous samples is thought to have caused a bias and is irrelevant because a venous sample is not intended for their meter.

As to the comparison method being used, the authors’ request to use the manufacturer’s recommended comparison method, while reasonable, requires manufacturers to state the comparison method. But at a minimum, for the evaluations that use both the YSI and a hospital laboratory method, a plot of the differences between these two methods seems warranted as provided by the authors.

User error can affect performance, ⁵ and the authors have included the effect of users versus hospital technicians as part of their study.

The authors’ plea for paying attention to evaluation details will call attention to their own study since in a bit of tortured perhaps regulatory reasoning, they have evaluated one meter against the ISO 15197 2003 standard and the other meter against the ISO 15197 2015 standard, which makes little sense.

Whereas it is important to know the location of most of the glucose differences between the candidate meter and its comparison method, some other events albeit rare need to be examined:

1. Are there any results beyond the A and B regions of a glucose meter error grid?

2. Are there any instances when a result could not be obtained?

3. The ISO 15197 2015 standard is a total error goal. Given that it has been met, is there an average bias that is high enough to cause diabetes complications? ⁶

As the accuracy of glucose meters improves, and the lack of use of the real glucose reference method in evaluations, some errors attributed to the glucose meter probably belong to the comparison method. This makes it all the more important to follow the authors’ recommendations.

Finally, regarding the performance of glucose meters after release for sale, besides these evaluations, there is a database of adverse events reported to the FDA. ⁷ Performing a search of this database for LifeScan glucose meters for the period July 1 through July 31, 2018, yielded 284 events! Are these events real? Is the rate increasing or decreasing? How does the manufacturer deal with such events? This issue has previously appeared in this journal. ⁸ Perhaps it’s time to answer these questions.