Using Radar Plots to Demonstrate the Accuracy and Precision of 6 Blood Glucose Monitoring Systems

Abstract

Background:

Previously, fingertip capillary blood glucose measurements from the CONTOUR^®NEXT (CN) blood glucose monitoring system (BGMS) and 5 other BGMSs were evaluated in comparison with measurements from a reference YSI glucose analyzer. Here, we use Radar Plots to graphically represent the accuracy and precision results from the previous study, including whether they met ISO 15197:2013 accuracy criteria.

Method:

A Radar Plot, a new method for capturing a distinct, single visualization of BGMS analytical performance, is a collection of concentric circles, each representing a particular magnitude of error. The center of the plot represents zero error (BGMS result is equivalent to reference result); as points are more distant from the center, the error increases, expressed in units of mg/dL or percentage for YSI values <100 and ≥100 mg/dL, respectively. The position of the data point above or below the horizontal line bisecting the plot indicates whether the BGMS measurement error was positive (BGMS result > YSI result) or negative (BGMS result < YSI result). Points within the “15-15 Zone,” representing ±15 mg/dL or ±15% error, satisfy ISO 15197:2013 accuracy criteria.

Results:

The percentage of results within the 15-15 Zone ranged from 83.6% to 99.8% for the 6 BGMSs (99.6% for CN).

Conclusions:

Radar Plots provide a different method for visually comparing the analytical performance of multiple BGMSs. The tight clustering of data points at the center of the CN Radar Plot illustrates the analytical performance of CN compared with 5 other BGMSs.

The presentation of analytical performance data for blood glucose monitoring systems (BGMSs) has been approached in distinct and familiar ways. For example, regression plots, ¹ modified Bland-Altman plots, ² and error grids (Clarke, ³ Parkes, ⁴ or surveillance ⁵ ) provide visual representations of analytical performance. The accuracy of a BGMS is one aspect of analytical performance, and the International Organization for Standardization (ISO) provides a standard for the acceptable accuracy of a BGMS. Specifically, the accuracy of a BGMS can be represented using ISO 15197:2013 accuracy criteria, which require that ≥95% of results fall within ±15 mg/dL or ±15% of reference at glucose concentrations of <100 mg/dL and ≥100 mg/dL, respectively. ⁶

Mean absolute difference (MAD) or mean absolute relative difference (MARD) may be used to represent accuracy as a single numeric value. Although, traditionally, this type of analysis has been used with continuous glucose monitoring systems, ⁷ these measures may also be useful for comparing multiple BGMSs in a single study using analysis of variance. ⁸ However, MARD is not a sufficient statistic and cannot be used by itself to describe meter system quality. ⁹

Because there is no single best visual representation of analytical performance data, a new way of looking at these data was developed that may help to better visualize analytical performance and the differences in analytical performance between BGMSs. A Radar Plot is a new way to plot the differences between BGMS values and reference instrument values using polar rather than Cartesian coordinates. In a previous study, the accuracy of the CONTOUR^®NEXT (CN) BGMS was compared with 5 other BGMSs. ⁸ Here, we present Radar Plots, which represent accuracy and precision of BGMS results relative to laboratory reference results, as a new way to visually represent data from the previous study of the 6 BGMSs. Radar Plots represent BGMS result accuracy in terms of both average error and ISO 15197:2013 accuracy criteria. The objectives of this short report are to demonstrate a novel way to visualize BGMS analytical performance and to visually compare analytical performance using Radar Plots.

Methods

For each BGMS, a Radar Plot was constructed comparing BGMS results with YSI glucose analyzer (YSI; YSI Life Sciences, Inc., Yellow Springs, OH) reference results. The 6 BGMSs evaluated in the previous study ⁸ were the following: CN (Ascensia Diabetes Care, Parsippany, NJ), Accu-Chek^® Aviva Nano (ACAN; Roche Diagnostics, Indianapolis, IN), FreeStyle Lite^® (FSL; Abbott Diabetes Care, Inc, Alameda, CA), OneTouch^® Ultra^®2 (OTU2; LifeScan Inc., Milpitas, CA), OneTouch^® Verio^® Pro (OTVP; LifeScan Inc., Milpitas, CA), and Truetrack^® (TT; Trividia Health [formerly Nipro Diagnostics Inc.], Fort Lauderdale, FL).

A Radar Plot is a collection of concentric circles, with each circle representing a particular magnitude of error. The direction of a ray extending from the center of the plot to the point indicates the laboratory blood glucose value, with laboratory values increasing from left to right. The points above the horizontal line bisecting the plot indicate positive errors (overestimate; the BGMS result is greater than the YSI result), and the points below the horizontal line bisecting the plot indicate negative errors (underestimate; the BGMS result is less than the YSI result). The dashed magenta lines indicate the boundary for laboratory values <100 mg/dL and ≥100 mg/dL. Single points on a Radar Plot represent error (difference) in units of mg/dL for YSI glucose values <100 mg/dL (the region of the plot within the magenta dashed lines) and in units of percentage for YSI glucose values ≥100 mg/dL (region of the plot outside of the magenta dashed lines). The center of the plot represents zero error (ie, the BGMS result is equivalent to the reference result); with increasing error, the points become more distant from the center of the plot.

Points within the outer green circle (bolder line) representing ±15 mg/dL or ±15% error for samples with YSI glucose concentrations of <100 mg/dL and ≥100 mg/dL, respectively, satisfy the ISO 15197:2013 accuracy limits. ⁶ This region of the plot is hereafter referred to as the 15-15 Zone. Additional degrees of accuracy are indicated by concentric circles that represent contours of error in 5 mg/dL (<100 mg/dL) or 5% (≥100 mg/dL) increments. The mean difference (for YSI values <100 mg/dL) and mean relative difference (for YSI values ≥100 mg/dL) of BGMS results from the YSI reference results were also computed. Precision is represented as the standard deviation (SD) of the difference and the relative difference. On the Radar Plot, precision is indicated by the consistency of difference between BGMS and laboratory values. A description of the geometry used to produce a Radar Plot can be found in Appendix 1 in the Supplementary Materials available online.

Results

As Radar Plots are a new method of evaluating BGMS analytical performance, it is useful to compare this visual representation with traditional graphic descriptions of the same dataset. In the case of CN, the mean difference (YSI <100 mg/dL) was 1.0145 mg/dL and the mean relative difference (YSI ≥100 mg/dL) was 1.7958% (see Table 1 for a more detailed description of mean difference and mean relative difference results). As shown in Figure 1A, the corresponding Radar Plot for CN showed that most data points clustered around the center of the plot, and 99.6% of these results were within the 15-15 Zone (the outer green circle).

Table 1.

Mean Differences and Mean Relative Differences of BGMS Results From the Reference Results.

Meter system	YSI values <100 mg/dL		YSI values ≥100 mg/dL
	n	Mean (SD) difference (mg/dL)	n	Mean (SD) relative difference (%)
CN	173	1.0145 (2.1463)	365	1.7958 (3.413)
ACAN	173	2.4972 (2.7308)	365	0.5105 (4.2802)
FSL	173	–2.2572 (3.5331)	365	–11.3312 (4.642)
OTU2	173	–9.5694 (4.6688)	364	–6.4371 (5.7214)
OTVP	172	3.8828 (3.0838)	364	–1.7167 (4.3612)
TT	173	–8.9971 (7.3353)	365	–6.4362 (7.9135)

BGMS, blood glucose monitoring system; SD, standard deviation; YSI, YSI glucose analyzer.

Figure 1.

Radar Plots of BGMS results compared with YSI reference results for (A) CN, (B) ACAN, (C) FSL, (D) OTU2, (E) OTVP, and (F) TT.

For ACAN, the mean difference was 2.4972 mg/dL and the mean relative difference was 0.5105%. The corresponding Radar Plot for ACAN (Figure 1B) showed that the data points clustered near the center of the plot, and 99.6% of results were within the 15-15 Zone.

Results for FSL showed that the mean difference was –2.2572 mg/dL and the mean relative difference was –11.3312%. As illustrated in the Radar Plot for the FSL (Figure 1C), the majority of data points were located below the horizontal line bisecting the plot, which was consistent with the corresponding negative mean difference and mean relative difference values. In addition, the data points were relatively dispersed and generally further away from the center of the plot than those in some of the other Radar Plots. Only 84.0% of FSL results were within the 15-15 Zone.

The mean difference and mean relative difference results for OTU2 were –9.5694 mg/dL and –6.4371%, respectively, and the corresponding Radar Plot is shown in Figure 1D. The majority of data points on the OTU2 Radar Plot were located below the horizontal line bisecting the plot, which was consistent with the corresponding negative mean difference and mean relative distance values, and 91.6% of data points were within the 15-15 Zone.

For OTVP, the mean difference was 3.8828 mg/dL and the mean relative difference was –1.7167%. The corresponding Radar Plot showed that the data points clustered around the center of the plot, and 99.8% of results were within the 15-15 Zone (Figure 1E).

Finally, the mean difference and mean relative difference for TT were –8.9971 mg/dL and –6.4362%, respectively. The Radar Plot for TT showed that the majority of data points were located below the horizontal line bisecting the plot, and these data points were relatively dispersed (Figure 1F). A total of 83.6% of TT results were within the 15-15 Zone.

Discussion

Several methods are available to provide visual representation of analytical performance data for BGMSs, such as regression plots, modified Bland-Altman plots, and error grids. While ISO 15197:2013 accuracy criteria are currently the standard for the acceptable accuracy of an individual BGMS, other types of analyses are more useful for comparing the accuracy of multiple BGMSs. MAD or MARD values may be used to compare the accuracy of multiple BGMSs in a single study, ⁸ with lower MAD and MARD values indicating less deviation from the reference value and thus, better accuracy. ¹⁰ However, MARD alone is not an adequate statistic to evaluate meter system quality. ⁹ Since no method to date provides the optimal visual representation of analytical performance data for BGMSs, a new way of illustrating these data was developed to better visualize analytical performance and the potential differences among BGMSs.

A Radar Plot provides a different method for capturing a distinct, single visualization of BGMS analytical performance, representing both accuracy (in terms of average error and based on ISO 15197:2013 accuracy criteria) and precision. Visualizing the data in this way simplifies the presentation by including multiple measures of analytical performance in a single graphic, which also facilitates comparison of several meters from a single study. Furthermore, Radar Plots convey this information in a recognizable way due to their resemblance to a target; intuitively, points closer to the center of the Radar Plot indicate that the BGMS and laboratory values are more similar. A high level of accuracy (based on ISO 15197:2013 accuracy criteria) is indicated by a large proportion of data points in the 15-15 Zone, and a high level of precision is indicated by data points clustered along a circle, such as a zone line. The first manuscript to include Radar Plots was an article that reviewed the various tools that are available for presenting meter system performance data. ¹¹ Since then, others have considered additional representations, ¹² and each method has its strengths and limitations.

Conclusions

Radar Plots provide a useful method for visually comparing the analytical performance of multiple BGMSs, as they allow for the inclusion of a variety of information, including accuracy (in terms of average error and ISO 15197:2013 accuracy criteria) and precision in a single, user-friendly visual representation. The tight clustering of data points in the center of the CN Radar Plot is a valuable visual indicator that illustrates the analytical performance of the CN BGMS and is especially demonstrative here to visually elucidate the analytical performance of the CN BGMS compared with 5 other BGMSs.