Comparing Clinical Accuracy and Analytical Accuracy Between Continuous Glucose Monitors

Abstract

This study compares performance between two continuous glucose monitors (CGMs). The study design contains a mix of laboratory results (CGM vs YSI) and home results (CGM vs glucose meter). Analysis is provided for both clinical accuracy and analytical accuracy of CGM glucose measurements. Both types of accuracy are important. Error grid analysis informs about clinical accuracy. Analytical error is important as most users would prefer a CGM with a smaller spread of CGM versus reference differences. The authors provide the percentage of time that no result was obtained. Study design, data analysis, and editorial support were provided by a manufacturer of one of the products studied. This study provides a template for comparisons.

This important study ¹ compares performance between two continuous glucose monitors (CGMs). This type of study provides useful information beyond the fact that both products are U.S. Food and Drug Administration (FDA)-approved. The study design is excellent as it contains a mix of laboratory results (CGM vs YSI) and home results (CGM vs glucose meter). The later comparison is representative of real-world data. It is implied that these CGM users are making treatment decisions as opposed to hybrid closed-loop systems, where insulin pump software makes treatment decisions.

Analysis is provided for both clinical accuracy and analytical accuracy of CGM glucose measurements. Both types of accuracy are important. Error grid analysis informs about clinical accuracy, especially for acute injury. A CGM value of 400 mg/dL with a reference of 320 mg/dL is a 25% error (e.g., a fairly large analytical error), but this data point is in the A zone of the Parkes error grid, which suggests no clinical risk. Error grids inform about the clinical accuracy of larger analytical errors that increase the risk that an incorrect treatment decision will be made. For example, a CGM that reads 170 with a reference of 55 is a 210% analytical error and in the D zone of a Parkes error grid. The authors provide results for the Surveillance Error Grid ² which show that both meters have infrequent values with high clinical risk.

Analytical error is also important because even if there is a low risk of clinical inaccuracy, most users would prefer a CGM with a smaller spread of CGM versus reference differences. Authors use two common metrics: mean absolute relative difference (MARD), and the percentage of data within various differences from reference (15, 20, 40 mg/dL or 15%, 20%, and 40%). The mean bias of each CGM is also given. Mean bias is important because consistent, elevated glucose can lead to diabetes complications. The results of the metric CG-DIVA ³ provide a tolerance interval for percentage deviations. The value of this metric and how it was calculated is not clear.

One often neglected metric that these authors provide is the percentage of time that no result was obtained (their Table 3). A CGM that fails to provide a result is a serious clinical risk.

One would also like to know if any data were deleted (and or repeated), but this information was not provided.

The authors state that there was no bias in spite of industry funding. But bias can be subtle and very hard to detect. Study design, data analysis, and editorial support were provided by a manufacturer of one of the products studied. Conflict of interest bias is well documented.^4,5 From reference 5, “Aggregating the results of these articles showed a statistically significant association between industry sponsorship and pro-industry conclusions.” Bias does not invalidate results, but it does increase the uncertainty of any conclusions.

Finally, as another data source, users can query the FDA’s adverse event database. ⁶ These are real-world complaint data from users, but results must be interpreted with caution as the data are unverified, and there is no guarantee that different manufacturers report and properly classify events the same.

To summarize, the excellent study design, estimation of clinical and analytical accuracy will help users to choose among different CGMs and provides a template for other glucose-measuring device comparisons.