Comment on “Do We Need the Replacement of YSI 2300? A View from the Clinical Laboratory” by Spanou and Makris

In their article, Spanou and Makris ¹ present their view on the topic of a suitable comparator method for performance studies of systems for self-monitoring of blood glucose (SMBG). They conclude that a laboratory-based “true reference method” should be used, and detailed analytical performance specifications should be reported for both the SMBG system and the comparison method used.

We wholly agree with reporting performance specifications for both the test system and the comparator: In any performance evaluation by method comparison, the selected comparator’s level of accuracy affects the perceived accuracy of the investigated test system. Therefore, it should also be reported so that performance data can appropriately be interpreted. However, we would like to comment on the phrase “true reference method.” This topic is also highly relevant for the working group on continuous glucose monitoring of the International Federation of Clinical Chemistry and Laboratory Medicine. ²

Isotope-dilution gas-chromatography mass-spectrometry (MS) has been accepted as a higher-order reference method for glucose concentrations in whole blood and in plasma by the Joint Committee for Traceability in Laboratory Medicine, ³ but measurements are quite expensive and time-consuming. In an ISO 15197 study with several hundred samples or in CGM studies with several thousand samples, this approach is not feasible. Even with other MS analyzers that are designed for more rapid measurements, ⁴ issues might arise. MS analyzers tend to require more elaborate pre-analytical steps than common lab analyzers. Costs per measurement are likely considerably higher than those for common lab analyzers. Furthermore, there might be a need for substantial documentation before such “self-developed” measurement procedures can be used in studies if the study results are intended to be used for regulatory approval of the test system.

Ideally, any comparison method used in performance evaluations should fulfill acceptance criteria to ensure that it is “accurate enough.” The question, however, seems to be what “accurate enough” is. Specific criteria could be set for bias and imprecision, like stated in the guideline POCT12 published by the Clinical and Laboratory Standards Institute. ⁵ When combined with the use of well-defined control samples, like reference materials with MS-certified target values, this would allow for more stringent quality control than current common practice. Alternatively (or additionally), acceptance criteria for the analyzer’s measurement uncertainty could be defined.

Even with lab analyzers, measures can be taken to improve the accuracy of measurement results, for example, in case the above-mentioned acceptance criteria for comparator methods might be hard to meet. The impact of imprecision, for example, can be reduced by performing multiple measurements from the same sample and averaging their results, as the standard error of the mean decreases with increasing number of measurements. Outliers, for example, caused by device malfunction, can be detected more reliably if at least duplicate measurements are performed. Systematic measurement differences (ie, bias) could be minimized by re-calibration of measurement results. ⁶ An approach like this might also contribute to minimizing differences in values reported by different types and models of lab analyzer. Depending on the extent of additional steps taken with lab analyzers, throughput might still be higher (and associated costs per sample lower) than when using MS analyzers.

In summary, Spanou and Makris make some good points in their article. ¹ Well-characterized comparator methods should be used in performance studies, and both their and the investigated test system’s performance should be reported. Further steps could be taken to improve the performance of lab analyzers when they are used as comparator methods.