Impact of Blood Glucose Monitoring System Accuracy on Clinical Decision Making for Diabetes Management

Abstract

Background:

The accuracy of blood glucose monitoring systems (BGMS) is crucial for the safe and effective management of diabetes mellitus. Despite standardization of accuracy assessment procedures and requirements, various studies have shown that the accuracy of BGMS on the market can vary considerably. This article therefore provides health care professionals and users with an intuitive illustration of the impact of BGMS accuracy on clinical decision making.

Material and Methods:

Several hypothetical patient scenarios based on blood glucose (BG) levels in the low, normal, and high BG range are devised. Using data from a recent BGMS accuracy study, a method for calculating the expected range of BG readings from four examined BGMS at the selected BG levels is introduced. Based on these ranges, it is illustrated how clinical decisions and subsequent outcomes of the hypothetical patients are affected by the expected inaccuracies of the BGMS.

Results:

The range of expected BGMS readings for the same true BG level can vary considerably between different BGMS. The discussion of hypothetical patient scenarios revealed that the use of some BGMS could be associated with an increased risk of adverse events such as failure to detect hypoglycemia, driving with an unsafe BG level, delay of treatment intervention in diabetes during pregnancy, or the failure to prevent diabetic ketoacidosis.

Conclusions:

This article can support both health care professionals and patients to understand the impact of BGMS accuracy in a relatable, clinical context. Furthermore, it is suggested that current accuracy requirements might be insufficient for the prevention of adverse clinical outcomes in certain circumstances.

Introduction

Continuing advancements in diabetes technology, especially in the field of glucose monitoring, have enriched the tools to support clinical decision making for optimizing glycemic control of diabetes patients. Self-monitoring of blood glucose (BG) is an essential element of diabetes therapy, particularly for insulin-treated patients. The accuracy of blood glucose monitoring systems (BGMS) is therefore crucial for the safe and effective implementation of those therapies as BG measurement results are used for clinical decision making as well as monitoring glycemic control. This pertains to people being treated with and without insulin-based glucose lowering regimens.^1,2 Additionally, BGMS are used to calibrate some continuous glucose monitoring (CGM) systems, making BGMS accuracy a pivotal factor in CGM-based therapy approaches, including automated insulin delivery systems. ³ Inaccurate BGMS readings can lead to incorrect therapy decisions such as errors in insulin dosing or a failure to detect hypoglycemia, which can adversely affect clinical outcomes. In particular, simulation studies^4,5 and an observational study ⁶ have shown that inaccurate BG results have been associated with elevated HbA1c and an increased rate of hypoglycemic events. Aside from that, BGMS accuracy is pivotal to assure the safety of subjects and validity of glycemic outcomes in clinical trials aimed to assess various diabetes drugs and devices. ⁷

Accuracy requirements for BGMS are standardized through the ISO 15197:2013 ⁸ (ISO), which demands that at least 95% of BGMS readings fall within ±15 mg/dL of reference method results <100 mg/dL, or ±15% of reference method results ≥100 mg/dL. Furthermore, 99% of BGMS readings have to fall within clinically acceptable zones A and B of the consensus error grid. The Food and Drug Administration (FDA) self-monitoring BG test systems for over-the-counter use guideline from 2020 (FDA) requires 95% and 99% of BGMS readings to fall within ±15% and ±20% of reference method results, respectively. ⁹

Despite these standards for market authorization of BGMS, numerous post-market surveillance studies have found repeatedly that some BGMS on the market were unlikely to fulfill FDA/ISO requirements.^10-12 Even if the abovementioned requirements are met, it has been suggested that the ISO-allowed error margin of ±15 mg/dL in the hypoglycemic range may be too large, leading to a different interpretation of BG results in this critically important BG range. ¹³ Here, it is noteworthy that a recent trial comparing the safety and effectiveness of two insulin regimens had to be reorganized due to the suspicion that an FDA-cleared BGMS, that had been used in the study, was inaccurate and unsafe, especially in the hypoglycemic range.^14,15

Given the importance of BGMS accuracy and apparent issues with the current performance requirements, this article aims to examine the impact of various BGMS readings and their associated uncertainty in a clinical context. For that, a methodology to calculate a glucose expectation range (GER) is introduced. The GER provides a range in which the results of a specific BGMS at a given true BG level are expected to fall and is calculated from data collected from an ISO system accuracy study. Using data from such a study, ¹² the GER will be used to predict the expected readings of various marketed BGMS at different BG levels crucial for clinical decision making, such as carbohydrate intake or insulin dosing. These BG levels were based on critical values or turning points in diabetes management, either for treatment initiation or everyday critical decision making. Based on that, we devised several hypothetical patient scenarios that illustrate how clinical decisions and associated outcomes can change depending on the displayed BGMS result in comparison to the true value. This discussion is intended to empower health care professionals (HCP) and patients with helpful tools to understand the importance of BGMS accuracy as well as the differences that can occur in BG readings of various currently approved and used devices on the market.

Methods

Data Description

The data used in this article were taken from a recent surveillance study, ¹² where a single test strip lot of 18 current BGMS marketed in Europe was examined according to the ISO 15197:2013 standard. For that, 200 pairs of reference and BGMS measurements were generated from at least 100 different subjects. Depending on the reference measurement method specified by the BGMS manufacturer, either a glucose oxidase-based or a hexokinase-based reference measurement method was used. The distribution of reference results across the measurement range is predetermined by the ISO standard.

To illustrate the differences in expected BG readings between devices, four BGMS for which the examined test strip lot met ISO requirements were selected. These BGMS are anonymized and entitled BGMS A to D.

Calculation of Glucose Expectation Ranges

The GER indicates the range of expected BGMS results for a given true BG level. One possibility to calculate the GER is based on regression analysis, where data from the entire measurement range is used to estimate the parameters of a linear model with proportional errors. ¹⁶ While this is a straightforward method to calculate prediction intervals for expected readings at any given reference value, it is based on the assumption that the chosen model to describe the relationship between reference and BGMS measurements holds over the entire measurement range, which might not be the case for all BGMS. This could lead to biased predictions in regions of the BGMS measurement range where the linear model does not adequately describe the data.

To minimize the need for assumptions about the underlying relationship between reference and BGMS measurements, the statistical concept of distribution-free prediction intervals is used. These prediction intervals estimate a conservative interval in which a single future observation will fall with a certain level of confidence. ¹⁷ Applying this concept to the calculation of a GER, the following procedure is proposed (exemplified in Figure 1). First, a set S consisting of at least n data points (pairs of reference measurement and associated relative difference between reference and BGMS measurement) within a symmetric region around the chosen true glucose level is selected. Subsequently, the extreme values of the relative differences contained in this set are identified and used to define the endpoints of a new interval. To make the procedure robust against the presence of extreme outliers, any values in the set S falling below or above the threshold of Q1/Q3 ± 3*IQR (Q1/Q3: lower and upper quartiles, IQR: interquartile range) ¹⁸ are excluded before extreme value determination. The endpoints are then applied to the chosen true glucose level to obtain the GER. In addition, the median of set S is used to determine the median of the GER. The confidence level p of a single future measurement lying within the GER is given as p = (n − 1)/(n + 1). ¹⁷

Figure 1.

Exemplified calculation of the glucose expectation range of BGMS B for a true blood glucose level of 141 mg/dL. The red dots indicate data points in set S. The dashed horizontal line in the enlarged plot indicates the symmetric search region around the true blood glucose level. The vertical line indicates the range between the extreme values of the relative deviations in set S. BGMS, blood glucose monitoring system.

To keep the values in set S as compact around the chosen true glucose level as possible and at the same time obtain a large p, a value of n = 19 is chosen, giving confidence level for the GER of at least 90%.

Selection of Hypothetical Patient Scenarios

To illustrate the influence of BGMS readings on clinical decision making as well as the difference between BGMS A-D, five hypothetical patient scenarios, each based on a true BG level, are considered. For each true BG level and all four BGMS, a GER is calculated. Subsequently, the influence of a BG reading on clinical decision making at the expectation limit in comparison to the true value is discussed.

In scenario 1, a true BG level of 61 mg/dL has been selected. This level was selected as it is the midpoint between the upper limits of level 1 and level 2 hypoglycemia of 69 and 53 mg/dL, respectively. ¹⁹ Any BGMS should therefore be able to reliably detect the hypoglycemia, that is produce a reading below 70 mg/dL, but also be able to distinguish between levels 1 and 2, that is produce a reading of 54 mg/dL or higher.

In scenario 2, the BG threshold of 90 mg/dL or less, at which the authorities of the United Kingdom and Australia recommend people with diabetes not to drive, is considered. This is known as the rule “above five (90 mg/dL) to drive” ²⁰ and is also the intervention threshold for commercial pilots with diabetes. ²¹ Applying the ISO error margin of −15 mg/dL to this BG threshold of 90 mg/dL, a true BG level of 75 mg/dL is considered. By using a value below the critical value, it can be assessed whether BGMS can reliably produce BG readings below 90 mg/dL.

In scenario 3, a true BG value of 141 mg/dL is selected to analyze the case of a hypothetical patient with diabetes during pregancy. This BG level was calculated based on the recommended upper limit of the postprandial (2 hours after the meal) BG target of 120 mg/dL for pregnant women with diabetes ²² and the ISO-allowed deviation of 15% for BG levels above 100 mg/dL (141 mg/dL - 15% ≙ 120 mg/dL). In this case it would be imperative that any BGMS can reliably produce a BG reading above 120 mg/dL to inform the patient and HCP that an intervention to reduce postprandial glycemia should be considered.

For scenario 4, two situations with a true fasting BG level of 251 mg/dL are considered. In scenario 4A, a patient with type 1 diabetes using an insulin pump has true BG level of 251 mg/dL, which is the threshold for diabetic ketoacidosis, ²³ and administers an appropriate correction bolus. However, due to an undetected impairment of the infusion system, resulting in insulin insufficiency, after 2 hours the true BG level remains high at 251 mg/dL. Two consecutive BG readings above 250 mg/dL are considered to be a critical point for ketone testing due to increased risk for diabetic ketoacidosis. ²⁴ In this particular case, the BGMS should be able to provide BG readings at a similar level to prompt the patient to check for signs of ketoacidosis and a possible fault in the infusion device.

For scenario 4B, the same hypothetical patient with type 1 diabetes using functioning insulin pump experiences a true fasting BG level of 251 mg/dL. This BG level requires a correction bolus which is dependent on the measured BG level and insulin sensitivity factor. Therefore, the BGMS used by the patient should be able to produce a BG reading close to the true value in order to minimize over- or underdosing of insulin which may lead to hypoglycemia or hyperglycemia, respectively.

Results

Glucose Expectation Ranges

The GER of BGMS A-D in scenarios 1 to 4 are depicted in Figure 2. No outliers had to be removed and each sample size (n) was 20, ensuring a confidence level greater than 90%. The background of Figure 2 is color-coded to indicate the risk of adverse events related to BGMS errors. Based on previous studies^5,25 and ISO acceptance criteria, the risk zones are defined as follows: no or low risk of adverse events resulting from BGMS inaccuracies are assigned to deviations from the true value below 10 and between 10 and 15 mg/dL/% (green and light green), respectively; moderate and high risk is assigned to deviations between 15 and 20 and above 20 mg/dL/% (amber and red), respectively.

Figure 2.

Glucose expectation ranges (GER) for true glucose levels in scenarios 1-4. The GER give the interval in which a BG reading is expected to fall with a confidence level of at least 90%. The colored background indicates no, low, moderate, and high risk of adverse events resulting from BGMS readings. BGMS, blood glucose monitoring systems.

Hypothetical Patient Scenarios

For scenario 1 with a true BG level of 61 mg/dL, the BGMS display very different GER ranging overall from 50 to 76 mg/dL. In particular, BGMS C could fail to detect the presence of hypoglycemia, as it cannot reliably produce BG readings below 70 mg/dL. This could alter the clinical decisions taken by the patient or their caregiver, such as the delay or lack of carbohydrate intake and the failure to reduce the dose of short-acting insulin, when applicable. In contrast to that, BGMS B displays a lower GER limit of 50 mg/dL, which would be considered level 2 hypoglycemia. This could prompt the intake of excessive amounts of carbohydrates or even the use of glucagon, especially if the patient has a history of hypoglycemia unawareness, leading to subsequent hyperglycemia. Overall, all BGMS except A have GER extending either above 70 mg/dL or below 54 mg/dL, meaning that level 1 hypoglycemia could not be detected reliably.

In scenario 2, the BG level threshold for driving and flying of 90 mg/dL is considered by examining a true BG level of 75 mg/dL. Here, it can be observed that the upper limits of all GER are below 90 mg/dL, indicating that all BGMS would prompt the patient to intervene before or during driving or flying. However, it should be noted that the upper GER of BGMS C is at 89 mg/dL, meaning that even a small increase in the true BG level incurs an increased risk of giving a false reading above 90 mg/dL.

Scenario 3 deals with a hypothetical patient with diabetes during pregnancy and a true 2-hour postprandial BG level of 141 mg/dL. As this level is clearly above the upper target level of 120 mg/dL, appropriate treatment intervention to lower postprandial glucose should be considered. However, the GER of BGMS B has a lower limit of 121 mg/dL, meaning that there is an increased risk for giving the patient a false sense of security and delaying treatment intervention.

Scenario 4A considers a patient with a true BG level of 251 mg/dL developing ketoacidosis due to insulin infusion impairment. If that patient is using BGMS D, the initial measurement result could be close to the true value, but a 2-hour follow-up BG reading after a correction bolus, where the true value remained at 251 mg/dL, could be as low as 215 mg/dL, suggesting to the patient that the hyperglycemia is resolving and the insulin infusion is viable. This could delay the necessary interventions such as ketone measurement and a change of infusion set, with possible adverse events such as severe ketoacidosis.

For scenario 4B, it is assumed that the patient with true BG level of 251 mg/dL is using BGMS C. Assuming a BGMS reading of 291 mg/dL at the upper limit of the respective GER, an overcorrection of 40 mg/dL could occur. Considering a target of 100 mg/dL, this would result in hypoglycemia.

A summary of clinical outcomes of all discussed scenarios, as well as the 15 and 10 mg/dL/% agreement rates, is provided in Table 1.

Table 1.

Summary of Potential Clinical Outcomes in the Hypothetical Patient Scenarios.

Scenario	BGMS
	A	B	C	D
1: 61 mg/dL	No clinical impact	Hyperglycemia due to excessive food intake	Failure to detect hypoglycemia	No clinical impact
2: 75 mg/dL	No clinical impact	No clinical impact	Driving/flying below 90 mg/dL	No clinical impact
3: 141 mg/dL	No clinical impact	Delay of therapy intervention	No clinical impact	No clinical impact
4A: 251 mg/dL	No clinical impact	No clinical impact	No clinical impact	Failure to prevent ketoacidosis
4B: 251 mg/dL	No clinical impact	No clinical impact	Hypoglycemia due to insulin overcorrection	No clinical impact
15 mg/dL/% agreement rate	100.0%	100.0%	98.0%	99.5%
10 mg/dL/% agreement rate	99.5%	92.0%	86.0%	91.5%

The colors green, amber, and red indicate low, moderate, and high risk of the specified event. The last rows provide the percentage of BGMS readings that fall within ±15/10 mg/dL of reference values <100 mg/dL, and ±15/10% of reference values ≥100 mg/dL.

Abbreviation: BGMS, blood glucose monitoring system.

Discussion

The first part of this article described a novel method for calculating a distribution-free, data-driven prediction interval for expected glucose measurements from a given BG level. Compared to a model-based regression approach using data from the entire measurement range, the GER method is based on fewer assumptions and uses only data in a fairly narrow BG range around the level of interest. This means that the GER can only be calculated for BG levels with a sufficient number of neighboring reference BG measurements, therefore limiting the choice of BG levels. However, if the reference measurements follow the distribution mandated by ISO, we believe that any BG level in the range between 50 and 300 mg/dL should be eligible. As FDA guidelines have less stringent requirements for the reference value distribution but demand more data points, the same range of BG levels should be eligible for GER calculation in studies conducted according to FDA guidelines. An additional limitation is that the method for GER calculation has only been applied to datasets with 200 data points, therefore restricting the density of data points at the BG levels of interest. Validation of the method on larger datasets, for example, by pooling data from different test strip lots or using data from FDA compliant studies with 350 data points, will be considered in the future.

In general, it is assumed that the reference measurements reflect the true glucose level despite also being affected by measurement uncertainty. This means that the reference measurement uncertainty is attributed to the BGMS. However, it should be mentioned that the BG measurements in the surveillance study were performed by trained personnel and not by the subjects themselves. This reduces the influence of handling errors and following measurement inaccuracies which could occur when patients are measuring their BG.

The second part of the article used this methodology to calculate the GER at four chosen true BG levels for four current BGMS on the market fulfilling ISO criteria (Figure 2). Based on the resulting GER, five hypothetical patient scenarios were discussed. Using the limits of the GER, therefore representing worst-case assumptions, it was illustrated how different BGMS readings can influence clinical decision making and increase the likelihood of adverse events (Table 1). Especially scenario 1, discussing the detection of hypoglycemia with a true value of 61 mg/dL, revealed that a reliable distinction between no, level 1, and level 2 hypoglycemia is not always possible. Hypoglycemia is a major factor that limits the control of glucose in diabetes, making its reliable detection crucial for successful therapy. Furthermore, the occurrence of hypoglycemia could be considered as an endpoint in studies examining the safety and efficacy of insulin therapy. Using BGMS C in such a clinical study could thus distort the outcomes considerably by underestimating the occurrence of hypoglycemia.

However, it was also demonstrated that a device with high accuracy such as BGMS A is not associated with an increased risk of adverse outcomes. Another observation that can be made from the GER is that the accuracy of any BGMS can be different depending on whether the true BG level is in low, normal, or high range. BGMS B, for example, shows a negative bias for all chosen BG levels except 251 mg/dL.

Apart from the chosen patient scenarios, it is increasingly common to use a BGMS for the calibration of CGM devices. This was examined extensively in a previous simulation study. ³ The simulation study found that BGMS accuracy, in particular a systematic bias, can have a significant effect on clinical outcomes, additionally highlighting the importance of BGMS accuracy.

The agreement rates at the bottom of Table 1 suggest that the limits of ±15 mg/dL/%, used by ISO to define the threshold of acceptance (95%), may not necessarily prevent the occurrence of adverse events as all examined BGMS have similar agreement rates close to 100%. Only when the more stringent ±10 mg/dL/% limit is considered, significant differences between BGMS A and the other devices, found in the hypothetical scenarios, are revealed. When selecting a BGMS, either for a clinical study or for a specific patient through the HCP, it should therefore be ensured that the system has sufficient accuracy in the relevant glucose ranges. Only relying on ISO criteria compliance may lead to an inappropriate choice of BGMS and could be associated with an increased risk of adverse events. Patient groups that may benefit from more accurate BGMS include for example: patients with an increased risk of hypoglycemia, patients with reduced or absent hypoglycemia awareness, pregnant women with diabetes, commercial pilots or drivers with diabetes, and patients using CGM systems, where BGMS use is recommended for various reasons.

Conclusions

This article has demonstrated the importance of BGMS accuracy as well as the differences that can occur within current, market-approved BGMS. By using hypothetical patient scenarios instead of employing complex statistical analysis and graphical representations, we hope to support both HCP and patients to understand the impact of BGMS accuracy in a relatable, clinical context. Our results also suggest that the current ISO acceptance criteria may be insufficient for the prevention of adverse clinical outcomes in all diabetes patients. It might therefore be prudent to apply stricter accuracy requirements when the BGMS are used by certain patient groups.