Sage Journals: Discover world-class research

Abstract

Continuous glucose monitoring and flash glucose monitoring technologies measure glucose in the interstitial fluid and are increasingly used in diabetes care. Their accuracy, key to effective glycaemic management, is usually measured using the mean absolute relative difference of the interstitial fluid sensor compared to reference blood glucose readings. However, mean absolute relative difference is not standardised and has limitations. This review aims to provide a consensus opinion on assessing accuracy of interstitial fluid glucose sensing technologies. Mean absolute relative difference is influenced by glucose distribution and rate of change; hence, we express caution on the reliability of comparing mean absolute relative difference data from different study systems and conditions. We also review the pitfalls associated with mean absolute relative difference at different glucose levels and explore additional ways of assessing accuracy of interstitial fluid devices. Importantly, much data indicate that current practice of assessing accuracy of different systems based on individualised mean absolute relative difference results has limitations, which have potential clinical implications. Healthcare professionals must understand the factors that influence mean absolute relative difference as a metric for accuracy and look at additional assessments, such as consensus error grid analysis, when evaluating continuous glucose monitoring and flash glucose monitoring systems in diabetes care. This in turn will ensure that management decisions based on interstitial fluid sensor data are both effective and safe.

Keywords

Continuous glucose monitoring flash glucose monitoring mean average relative difference interstitial fluid glucose hypoglycaemia glycaemic variability

Introduction

Continuous glucose monitoring (CGM) and flash glucose monitoring are increasingly used in the management of patients with diabetes, particularly individuals receiving insulin therapy.¹ These systems measure glucose in the subcutaneous interstitial fluid (ISF), rather than in capillary blood as measured by traditional self-monitoring of blood glucose (SMBG) using finger stick meters.

Confidence in newer CGM systems and in flash glucose monitoring is gradually increasing, both with patients and healthcare professionals.^2,3 Clear advantages of CGM, and more recently flash glucose monitoring, are patient convenience and the comprehensive glycaemic data provided. For those on intensive insulin therapy, frequent glucose monitoring is a prerequisite for tight glycaemic control. Unfortunately, for many people with diabetes, poor engagement with SMBG can be a barrier to optimal glucose control.⁴

Encouragingly, clinical accuracy of ISF glucose monitoring systems has improved to the point that regulatory authorities are now approving systems for insulin dosing decisions, without the need for users to perform an adjunct SMBG test, except in defined situations (Table 1). Currently, the Abbott FreeStyle Libre system and the Dexcom G5 system are both approved in Europe (CE marked) and in the United States for non-adjunctive use.⁵ A key difference between the two systems is the need to calibrate Dexcom G5 twice daily, whereas Abbott FreeStyle Libre is factory calibrated. Infrequent or incorrect calibration by patients can potentially reduce the accuracy of Dexcom G5, an issue that does not affect FreeStyle Libre.

Table 1.

FreeStyle Libre and Dexcom G5 devices are approved for non-adjunctive insulin dosing with the following caveats.

The FreeStyle Libre system is approved for dosing insulin except when^a
- glucose reading is ⩽3.9 mmol/L
- glucose is changing rapidly
- symptoms do not match the readings on the system

In these situations, the user should test their glucose using a capillary sample (finger prick test)

Dexcom G5 data can be used to make treatment decisions without confirmation by blood glucose meter as long as certain criteria are met, including^b
- Dexcom G5 must be calibrated according to the manufacturer’s instructions (see below). This includes appropriate SMBG technique
- User has not taken acetaminophen (paracetamol)
- Dexcom G5 receiver (or Dexcom G5 Mobile App) must display both a glucose value AND a trend arrow
- User is not ‘stacking’ insulin doses (taking correction insulin doses too close together)
- User is not having symptoms that do not match the Dexcom G5 data

BG: blood glucose; SMBG: self-monitoring of blood glucose; MARD: mean absolute relative difference.

Manufacturer’s guidance, https://freestyleserver.com/Payloads/IFU/freestyle_libre/2017_mar/ART28697-501_rev-A_WEB.pdf (accessed January 2018).

Manufacturer’s guidance, www.dexcom.com/fingersticks, accessed October 2017.

Approval for non-adjunctive use reflects the findings of three key recent clinical studies which have assessed flash glucose monitoring and CGM devices for use independently of SMBG.

The Dexcom G4 Platinum CGM was assessed in the REPLACE-BG study.⁶ A total of 226 subjects with type 1 diabetes were randomised to CGM only (n = 149) or CGM plus SMBG (n = 77) for insulin dosing. At 26 weeks, individuals using CGM alone for insulin dosing had spent a mean of 63% ± 13% of time in the target glucose range [70–180 mg/dL (3.9–10.0 mmol/L)], and those using CGM plus SMBG had spent 65% ± 11% of time in the target range. In both arms, the time in target range was the same as at baseline. Metrics of glucose control, such as haemoglobin A_1c (HbA_1c), time in hypoglycaemia and glycaemic variability also showed little change from baseline to 26 weeks in either CGM alone or CGM plus SMBG groups. Thus, REPLACE-BG concluded that the Dexcom CGM system was safe for non-adjunctive use.

There are two main randomised studies to date investigating the effects of FreeStyle Libre on glycaemic markers. The first study is IMPACT, which was a 6-month trial in participants with well-controlled type 1 diabetes, randomised to SMBG alone or to Abbott FreeStyle Libre.⁷ In this study, adjunctive SMBG testing in the Libre group was reduced to a mean of 0.5 tests/day, while subjects scanned their Libre device an average of 15.1 times/day. Patients in the control group continued to perform SMBG more than 5 times/day. Time in hypoglycaemia (⩽3.9 mmol/L) was reduced in the Libre arm from 3.38 to 2.03 h/day (38% reduction compared with the control group; p < 0.0001). The second study is REPLACE, which randomised participants with type 2 diabetes on intensive insulin therapy to either FreeStyle Libre or SMBG alone for 6 months.⁸ In the Libre group, adjunctive SMBG was reduced to 0.3 tests/day. Overall, time in hypoglycaemia was reduced from 1.30 to 0.59 h/day (43% reduction compared with the control group; p = 0.0006). HbA_1c was similar in the two study groups but analysis of those younger than 65 years of age showed a significant improvement in HbA_1c compared with the SMBG group. Both studies support the safe use of the FreeStyle Libre system for non-adjunctive use. Moreover, both IMPACT and REPLACE showed significant improvement in quality of life measures and treatment satisfaction in the Libre arm compared with SMBG, indicating that the new sensor technology improves patient well-being in general.

It is worth highlighting that the above studies included patients at relatively low risk of severe hypoglycaemia. Nonetheless, there is growing evidence to support flash and CGM for non-adjunctive use in clinical practice.

As use of glucose sensing technologies increase, and their management extends beyond the expert clinical setting, it is important to maintain scrutiny of the accuracy of each system and understand how this accuracy is assessed. Unlike SMBG testing, there are currently no internationally agreed methods or standards for ISF glucose measurement. In this context, it is important for healthcare professionals to understand the benefits and limitations of assessing accuracy in ISF glucose monitoring because this has implications for the clinical decisions that are made using these technologies.

Lack of standardised assessment for ISF glucose sensors

In the European Union (EU), SMBG systems are assessed and compared by how well they meet the minimum standards for accuracy and reliability of glucose measurement as set out in ISO 15197:2013.⁹

The Clinical and Laboratory Standards Institute (CLSI) has published guidelines on Performance Metrics for Continuous Interstitial Glucose Monitoring (POCT05-A) that defines some aspects of CGM testing. These guidelines have pointed out the importance of assessing accuracy not only in steady states but also in two common scenarios: (1) during periods of rapid glucose change and (2) at different glucose concentrations, including extremes of glucose levels.¹⁰ However, there is no universally accepted protocol to compare performance among ISF glucose sensors without head-to-head trials based on simultaneous wear.

Accuracy, precision and concordance

CGM systems that sample ISF glucose need to be assessed for accuracy at the point of measurement and also accommodate the rate of change (RoC) of glucose. Currently, there is no accepted reference method that uses ISF glucose, mainly because it is not possible to get a large enough sample of ISF sufficient for in vitro analysis with a reference technology.

Therefore, the accuracy of ISF glucose readings is currently assessed by comparison with blood glucose readings taken at the same time. Any comparison between ISF glucose and blood glucose is primarily assessing the concordance of those readings, that is, how closely they match. This concordance between the two readings is therefore dependent both on the accuracy and the precision of the ISF device (CGM or flash glucose monitor) being tested and, importantly, the reference blood glucose device being used.

In this context, it must also be understood that blood and ISF are different physiological compartments that follow different dynamics.¹¹ Thus, the concordance between ISF glucose and blood glucose readings is also dependent on the physiological differences between the two compartments that are being sampled, including the lag time it takes the ISF to reflect blood glucose levels.

Defining accuracy: role of mean absolute relative difference

A number of metrics have been used to characterise accuracy in this context, and one, in particular, has emerged as a routine statement of sensor ‘accuracy’ – the mean absolute relative difference (MARD) of the ISF sensor readings when compared to a series of comparator blood glucose reference samples.

MARD is straightforward to calculate and is expressed as a single percentage number; therefore, it is an attractive measure of accuracy. In this context, a lower % MARD is seen as representing better sensor performance. An emerging view is that an arbitrary MARD of 10% represents the level of accuracy required for safe use of CGM readings to make insulin dosing decisions, without the need for an adjunct SMBG blood glucose reading.¹² Given that the MARD of well-established SMBG devices ranges between 4.4% and 13.4%,¹³ a cut-off of 10% does not sound unreasonable. However, it can be argued that this approach is too simplistic and requires closer scrutiny.

The MARD of 10% is based on flattening the curve in an in silico simulation of hypoglycaemia risk,¹² but this change in curve inflection may alternatively reflect limited additional benefit from MARD values lower than 10%. Furthermore, MARD is known to vary depending on a wide range of parameters, each of which can affect the ultimate MARD computation (detailed below). Also, the test device and study protocol variables are independent and will further compound each other. It should be noted that MARD varies during sensor life; for example, MARD is higher in the first day of sensor use^14,15 (believed to be due to the inflammatory reaction mediated by inserting the sensor subcutaneously¹⁶) and this inaccuracy may also be an issue towards the end of sensor life.¹⁷ Finally, the number of paired glucose data points will have a major influence on MARD outcome and this is yet to be standardised.

Taken together, accuracy and concordance of separate ISF glucose sensors can only be compared if study protocols and the inbuilt variation are identical. While the list of such studies is limited, these are on the increase and available head-to-head studies to date are shown in Table 2.

Table 2.

Head-to-head studies comparing CGM and flash glucose monitoring devices.

	Sensors in head-to-head study^a							Comments
	FreeStyle Navigator	FreeStyle Libre	MiniMed Enlite	DexCom Seven Plus	DexCom G4 Platinum	Dexcom G5	IWel
Luijf et al.¹⁸	14.5 (329)		18.9(312)		16.5(462)			n = 20; % MARD versus capillary blood (number of paired measurements); assessed during daily life
Freckmann et al.¹⁹	12.1(3021)		16.2(3304)	16.3(3761)				n = 12; % MARD versus capillary blood (number of paired measurements); assessed at clinical centre
Matuleviciene et al.²⁰			17.8(1012)		13.9(1012)			n = 38; % MARD versus capillary blood (number of paired measurements); assessed during daily life
Damiano et al.²¹	12.3(4645)		17.9(4521)		10.8(4634)			n = 24; % MARD versus venous blood (number of paired measurements); assessed at clinical centre
Kropff et al.²²			19.9(839)		12.2(987)			n = 24; % MARD versus capillary blood (number of paired measurements); assessed during daily life
Bonora et al.²³		16.6(10,020)			19.0(10,020)			n = 8; % MARD versus capillary blood (number of paired measurements); assessed during daily life
Aberer et al.²⁴		13.2(462)	21.4(502)		16.8(540)			n = 12; % MARD versus venous blood (number of paired measurements); assessed during daily life
Kamecke et al.²⁵		11.5(4217)				9.5(4463)		n = 20; % MAD/MARD versus capillary blood (number of paired measurements); assessed at clinical centre
Zeng et al.²⁶			15.9(241)				11.1(310)	n = 24; % MARD versus capillary blood (number of paired measurements); assessed at clinical centre

MARD: mean absolute relative difference; MAD: mean absolute difference; CGM: continuous glucose monitoring.

Numbers in grid = overall % MARD in head-to-head study (number of paired measurements), except Kamecke et al.²⁵ which reports combined MAD/MARD. All data gathered within manufacturers specified sensor lifespan. All studies are in subjects with type 1 diabetes, n = number of participants in each study.

It should be noted that some studies report the mean absolute difference (MAD) rather than MARD. The former gives an indication of the tendency of a glucose sensor to read high or low compared with a reference, whereas the latter is the relative deviation of a sensor from a reference. MAD is more commonly used to assess accuracy at low glucose levels.

Number of paired measurements

A key measure to inform accuracy of a system is paired glucose measurement between the test system and blood taken from subjects at the same time. This gives confidence that the two systems are providing readings that do not differ significantly from one another. However, MARD only provides a reliable value when the number of data points is sufficiently large. As with all averaging systems, the more data points for comparison, the more confidence can be placed in the computed MARD.

As an example, Figure 1 shows that the degree of uncertainty for a hypothetical MARD becomes tighter as the number of reference measurements increases. However, the larger the number of reference samples, the more burden is placed on patients and study personnel. To generate large sets of paired glucose data for a sensor over a 7- or 14-day wear life is therefore not straightforward.

Figure 1.

The impact of the number of paired points on the uncertainty of MARD: upper and lower bounds of the confidence interval with probability γ = 0.95. The constant line represents the value to which it would converge.

When considering the accuracy of CGM or flash glucose monitoring systems, the size of the data set that underpins the MARD calculation should be taken into account. To date, there are no clear guidelines as to the number of paired samples required to have confidence in the accuracy of a particular sensor.

Accuracy of the reference system

MARD is influenced by the method selected for comparing glucose data. When considering a MARD value, it is important to consider the blood glucose reference system used in the accuracy study.

All glucose reference methods have a measurement error of their own that must be taken into account when calculating the MARD. A common laboratory reference system is the Yellow Springs Instrument (YSI) glucose analyser, which provides accurate measurement of reference blood glucose samples and helps minimise MARD, because of its inherent low error.

SMBG meters make it possible to collect a larger number of paired readings based on capillary blood glucose, but SMBG meters also have a lower accuracy than a laboratory reference system. In addition, their accuracy varies widely between different manufacturers²⁷ and thus potentially modulate the computed MARD. However, SMBG is a ‘real-world’ comparison, used by most people with diabetes, and is the glucose measurement system that CGM would ideally replace, provided SMBG systems used have an appropriate quality control programme to ensure ongoing accuracy.

Accuracy at high RoC and extremes of glucose levels

It is well known that the rate of glucose changes in a study subject will affect MARD calculations.²⁸ CGM accuracy as defined by MARD is most reliable and lowest when glucose readings are stable. As the rate of glucose change increases, so does the computed MARD.

Figure 2 illustrates this for two specimen systems. As the rate of glucose falling or rising increases, so does the computed MARD. This is an important consideration when using CGM systems, as their accuracy varies as the RoC increases. When enrolling patients into studies who have high glucose variability, MARD will increase, and vice versa.

Figure 2.

MARD as defined by rate of change category (adapted from Pleus et al.²⁸). At low rate of change, the accuracy of system A and system B does not look that different. However, with increasing rate of change, the superior accuracy of system B over system A becomes evident.

An important consideration is the accuracy of ISF glucose readings at low glucose concentrations. Estimates of MARD are known to be subject to larger errors as glucose falls towards the hypoglycaemic range.²⁹ For example, in the study by Aberer et al.,²⁴ overall MARD of FreeStyle Libre was 13.2% with relatively little change in the hypoglycaemic (<3.9 mmol/L) and hyperglycaemic (>10 mmol/L) range (MARD of 14.6% and 10.1%, respectively). In contrast, Dexcom G4 Platinum had an MARD of 16.8%, with a larger difference comparing hypoglycaemic and hyperglycaemic range (MARD of 23.8% and 11.6%, respectively).²⁴ Another head-to-head study found even larger differences comparing MARD in hypoglycaemic and hyperglycaemic range (21.2% and 11.6% for Dexcom G4 Platinum and 36.5% and 18% for Enlite, respectively).²²

A low ISF glucose reading below 3.9 mmol/L can prompt corrective actions that may be unnecessary if actual blood glucose, as measured by SMBG, is significantly higher. For instance, a user may develop hypoglycaemia and take corrective action. Due to the time lag between blood glucose and ISF glucose, if the user continues to rely only on ISF glucose readings, there may be a lag in the rise of ISF over blood glucose, resulting in further and unnecessary treatment of hypoglycaemia. Similarly, experienced users may become less concerned with ISF low glucose readings than they would be with SMBG readings and take no immediate action. Each of these scenarios potentially creates unwanted risks.

However, the advantage of ISF glucose sensor devices is that they typically provide trend arrow support alongside the current glucose reading, indicating the direction and RoC, to assist decision-making. In the IMPACT and REPLACE studies using the FreeStyle Libre system in people with type 1 or type 2 diabetes on insulin,^7,8 ISF glucose sensor data helped users significantly reduce the amount of time that their glucose fell below 3.9 mmol/L, compared to subjects using SMBG. This reduction in hypoglycaemia was achieved without impacting HbA_1c or daily insulin doses.

Together, these outcomes suggest that any concerns about concordance of ISF sensor readings at low glucose levels are offset by the utility of having both a glucose reading and a trend arrow displayed on the reader, to support patient self-management and avoidance of hypoglycaemia.

Expert opinion is clear that definitions of hypoglycaemia should not differ, depending on the glucose sensing technology. However, changes in the accuracy of ISF sensors at low glucose must be clearly disclosed for each device, as part of overall reporting bias. Main variables that affect MARD levels are detailed in Table 3.

Table 3.

Variables that contribute to differences in concordance/MARD.

1. Test device–dependent variablesDevice performance• Intrinsic accuracy, that is, how close the test results are to a reference or standard• Intrinsic precision, that is, how close the test results are to each otherCalibration of device being tested (non-factory–calibrated devices only)• Discipline of calibration• Timing of calibration• Reference for calibration• Drift between moment of calibration and timing of comparison readings

2. Study protocol–dependent variablesDesignDistribution of glucose concentrations studied• Distribution of rate of glucose change• Reference BG method, for example, SMBG or laboratory instrumentStatistics and analysis• Number of paired measurements• Mean versus median• MAD versus MARD

BG: blood glucose; SMBG: self-monitoring of blood glucose; MARD: mean absolute relative difference; MAD: mean absolute difference.

Additional measures of accuracy and precision: Clarke and consensus error grids

Analytical performance of ISF sensor devices is divided into two important areas: accuracy and precision. Accuracy refers to how close the test results are to a reference or standard, and precision refers to the consistency of the system, that is, how close the test results are to each other. These concepts are illustrated in Figure 3.

Figure 3.

Precision and accuracy – what is the difference?

MARD is a reported metric for average accuracy but does not reflect any non-linear performance over the complete glucose range, nor does it provide any information about the precision of the system on which it reports.

To help better assessment of ISF sensing technologies, the Clarke error grid was devloped to compare readings of a particular system with reference samples (Figure 4(a)).³⁰ Readings in zone A represent high accuracy and those in zone B are acceptable accuracy, with the rest of zones showing reduced and clinically questionable accuracy. The Clarke error grid was further refined, and the more streamlined consensus error grid (CEG) was developed that can be generated in combination with MARD for additional accuracy reporting.³¹ The CEG compares glucose readings of the test device to reference blood glucose readings and plots them on a grid, which is divided into zones A–E. This aids in visualising accuracy as well as clinical impact of any errors of the system. Readings that fall into zones A and B are accepted for making clinical decisions.

Figure 4.

Error grid analysis. (a) Clarke error grid analysis. Zone A: clinically accurate values within 20% of the reference sample; zone B: values outside 20% of the reference sample but would not lead to inappropriate treatment; zone C: values that would lead to overcorrection of glucose levels; zone D: dangerous failure to detect and treat high or low glucose; zone E: values that could lead to treatment contradictory to that needed. Adapted from Clarke.³⁰ (b) Consensus error grid analysis, comparing the FreeStyle Libre sensor readings with capillary blood glucose reference values collected using the FreeStyle Precision BG meter built into the FreeStyle Libre reader.³²

In the example shown in Figure 4(b), 86.7% of the results are in the clinically accurate zone A of the CEG, and 99.7% of sensor results were in the clinically acceptable zones A and B of the CEG when compared to sensor capillary blood glucose result.³² The overall MARD in this system was 11.4% for sensor results when compared to capillary blood glucose reference samples.

The importance of looking both at MARD and CEG analysis is illustrated in Figure 5. In these comparisons of 2000 simulated paired measurements between test and reference samples, the MARD of 8.0% in Simulation 1 is achieved with a lower % of readings in zones A and B, compared to Simulation 2 with an MARD of 12.0% and 100.0% of readings in the clinically acceptable zones.

Figure 5.

Comparisons of simulated test and reference glucose samples. The MARD and CEG plots of 2000 paired readings can be modelled to illustrate that different methods of analysis may generate different assessments of ‘accuracy’.

Thus, using both these means of quantifying accuracy improves confidence in the efficacy and safety of ISF glucose sensing systems for making treatment decisions.

Areas of caution: when is adjunct blood glucose testing needed?

CGM and flash glucose monitoring systems are now being approved for insulin dosing based on a current glucose reading and RoC arrows, without the need for adjunct SMBG confirmation (see Table 1 for exceptions). This is testimony to the improved accuracy of ISF glucose sensing technologies.

In general, these systems bring an acknowledged benefit for users. They provide quick and discrete feedback on the current level and the direction of travel for glucose, and reduce the need for the inconvenient, and frequently described as anti-social, finger prick testing. This allows users to more confidently live their daily lives and make more informed decisions about diet, exercise, work and study, as well as the timing and dose of insulin injections.

However, in addition to the manufacturers’ own guidance (Table 1) expert opinion recognises that prudent clinical use of CGM and flash glucose monitoring systems should still involve use of SMBG capillary blood glucose testing as summarised in Table 4.

Table 4.

The continued need for adjunct blood glucose testing.

• During the first 24 h following sensor application when differences between blood glucose and ISF glucose are reportedly higher. This is hypothesised to be due to temporary local trauma at the site of application that affects ISF glucose concentration. Application of a new sensor 24 h before the old sensor ‘runs out’ represents a potential solution to this issue.• When a sensor glucose reading and trend arrow indicates a possible hypoglycaemic episode or when symptoms suggest a hypoglycaemic episode but the reader does not.• Driving: To comply with both EU and UK legislation, the UK Driving and Vehicle Licencing Authority (DVLA) does not consider ISF glucose readings to be sufficient on their own and drivers must also monitor their blood glucose levels using a traditional blood glucose test. Naturally, this may change once there is more confidence in the accuracy of modern CGM and flash monitoring systems.• Device-dependent interferences: When taking medications that have reported interference with ISF glucose values, for example, acetaminophen (paracetamol) or vitamin C (although this only applies to some ISF sensors).

ISF: interstitial fluid; CGM: continuous glucose monitoring; EU: European Union; UK: United Kingdom.

Conclusion and future directions

As real-time glucose monitoring systems become widespread in diabetes care, there is a tendency to focus on MARD as a defining metric for accuracy of a particular system. However, the consensus among a panel of diabetes experts is that, while MARD is useful, we should understand its drawbacks and relying solely on this metric for accuracy is not enough and more robust criteria are required.

The MARD of a system will vary depending on a wide range of parameters, each of which can affect the ultimate computation. Key factors include overall study design, choice of the reference system and number of paired readings that are analysed. It should be remembered that MARD is a metric for the concordance of glucose values from two different physiological compartments, measured with different systems, that is, ISF glucose and blood glucose. Furthermore, there is a dependence on whether the latter is capillary blood, venous blood (or venous plasma), and also on the reference method, which could vary from a handheld patient meter to a laboratory analyser.

The performance (‘accuracy’) of a CGM or flash glucose monitoring device is just one contributor to the concordance. When comparing devices, the only way to minimise or eliminate factors that contribute to non-concordance of each system is to conduct a head-to-head comparison when different ISF devices are worn simultaneously by the same subject and an appropriate, and identical, reference method is used.

In the absence of agreed standardised study protocols, comparison of MARD data obtained from various devices under different study conditions should be avoided, as such direct comparisons may lead to misleading conclusions. The main consensus points on accuracy assessment of CGM and flash glucose monitoring systems can be found in Table 5.

Table 5.

Consensus on assessments of accuracy of CGM and flash glucose monitoring systems.

• MARD is a useful measurement of accuracy of CGM systems. However, the lack of universally accepted protocols for assessing MARD means that it cannot be used in isolation as a statement of comparative accuracy in glucose measurement.• There is no nominal MARD threshold that represents the accuracy level required for safe use of CGM readings to make insulin dosing decisions.• The objective accuracy of any CGM or flash glucose monitoring system is dependent on the specific protocol used to test this accuracy. There is a need to develop standardised accuracy reporting protocols.• The accuracy of ISF sensors at low glucose should be clearly disclosed for each device.• With all ISF glucose sensing systems, readings in the low-glucose zone should be confirmed with an SMBG test, to provide confidence in what action to take, if any.• Comparisons between different ISF glucose sensing technologies require testing in a head-to-head study while worn simultaneously on the same subject and using the same reference method.• In the absence of agreed standardised study protocols, reliable comparison of MARD data obtained on different devices under different study conditions should be avoided, as such direct comparisons may lead to misleading conclusions.

BG: blood glucose; SMBG: self-monitoring of blood glucose; MARD: mean absolute relative difference; MAD: mean absolute difference.

When assessing the accuracy of any glucose sensing technology, MARD needs to be combined with additional objective measurements. An established tool in this regard is CEG analysis. This reflects not only the mean accuracy but also the utility of the system for making clinical decisions.

Footnotes

Declaration of conflicting interests

RAA declares: Institutional Research Grants from; Abbott Diabetes Care, Bayer, Eli Lilly, NovoNordisk, Roche, Takeda. Honoraria/education support and consultancy fees from; Abbott Diabetes Care, AstraZeneca, Bayer, Boehringer Ingelheim, Bristol-Myers Squibb, Eli Lilly, Glaxo SmithKline, Merch Sharp & Dohme, NovoNordisk, Takeda. MHC declares honoraria for advisory work for Abbott Diabetes Care. PJ declares personal fees from Abbott Diabetes Care outside the submitted work. LL declares speaker honoraria from Minimed Medtronic, Animas, Roche, Sanofi, Insulet and Novo Nordisk, advisory panel activities for: Abbott Diabetes Care, Roche, Sanofi, Minimed Medtronic, Animas and Novo Nordisk, grants to attend educational meetings from Sanofi, Novo Nordisk and Takeda. GR declares personal fees from Abbott Diabetes Care outside the submitted work. EGW declares speaker honoraria from Abbott Diabetes Care, Diasend, Dexcom, Eli Lilly, Minimed Medtronic, Novo Nordisk, Sanofi Aventis and has served on advisory panels for Abbott Diabetes Care, Eli Lilly, Sanofi Aventis, grants to attend educational meetings from Boehringer Ingelheim, Diasend, Novo Nordisk, Roche and Sanofi Aventis.

Funding

The author(s) received no financial support for the research, authorship and/or publication of this article.

References

Ajjan

RA.

How can we realize the clinical benefits of continuous glucose monitoring?

Diabetes Technol Ther 2017; 19: S27–S36.

Ólafsdóttir

Attvall

Sandgren

et al . A clinical trial of the accuracy and treatment experience of the flash glucose monitor FreeStyle Libre in adults with type 1 diabetes. Diabetes Technol Ther 2017; 19: 164–172.

Polonsky

Hessler

Ruedy

et al . The impact of continuous glucose monitoring on markers of quality of life in adults with type 1 diabetes: further findings from the DIAMOND randomized clinical trial. Diabetes Care 2017; 40: 736–741.

Ward

Stetson

Mokshagundam

SP.

Patient perspectives on self-monitoring of blood glucose: perceived recommendations, behaviors and barriers in a clinic sample of adults with type 2 diabetes. J Diabetes Metab Disord 2015; 14: 43–49.

FDA Advisory Panel votes to recommend non-adjunctive use of Dexcom G5 mobile CGM. Diabetes Technol Ther 2016; 18: 512–516.

Aleppo

Ruedy

Riddlesworth

et al . REPLACE-BG: a randomized trial comparing continuous glucose monitoring with and without routine blood glucose monitoring in adults with well-controlled type 1 diabetes. Diabetes Care 2017; 40: 538–545.

Bolinder

Antuna

Geelhoed-Duijvestijn

et al . Novel glucose-sensing technology and hypoglycaemia in type 1 diabetes: a multicentre, non-masked, randomised controlled trial. Lancet 2016; 388: 2254–2263.

Haak

Hanaire

Ajjan

et al . Use of flash glucose-sensing technology for 12 months as a replacement for blood glucose monitoring in insulin-treated type 2 diabetes. Diabetes Ther 2017; 8: 55–73.

ISO 15197:2013. In vitro diagnostic test systems – requirements for blood-glucose monitoring systems for self-testing in managing diabetes mellitus, 2013.

10.

Clinical and Laboratory Standards Institute (CLSI). CLSI performance metrics for continuous interstitial glucose monitoring; approved guideline. CLSI document POCT05-A. Wayne, PA: CLSI, 2008.

11.

Cengiz

Tamborlane

WV.

A tale of two compartments: interstitial versus blood glucose monitoring. Diabetes Technol Ther 2009; 11: S11–S16.

12.

Kovatchev

Patek

Ortiz

et al . Assessing sensor accuracy for non-adjunct use of continuous glucose monitoring. Diabetes Technol Ther 2015; 17: 177–186.

13.

Freckmann

Pleus

Link

et al . Accuracy evaluation of four blood glucose monitoring systems in unaltered blood samples in the low glycemic range and blood samples in the concentration range defined by ISO15197. Diab Technol Ther 2015; 17: 625–634.

14.

Laffel

Improved accuracy of continuous glucose monitoring systems in pediatric patients with diabetes mellitus: results from two studies. Diabetes Technol Ther 2016; 18: S2–S23.

15.

Christiansen

Garg

Brazg

et al . Accuracy of a fourth-generation subcutaneous continuous glucose sensor. Diab Technol Ther 2017; 19: 446–456.

16.

McGarraugh

Brazg

Weinstein

FreeStyle navigator continuous glucose monitoring system with TRUstart algorithm, a 1-hour warm-up time. J Diabetes Sci Technol 2011; 5: 99–106.

17.

Klueh

Kaur

Qiao

et al . Critical role of tissue mast cells in controlling long-term glucose sensor function in vivo. Biomaterials 2010; 31: 4540–4551.

18.

Luijf

Mader

Doll

et al . Accuracy and reliability of continuous glucose monitoring systems: a head-to-head comparison. Diabetes Technol Ther 2013; 15: 721–726.

19.

Freckmann

Pleus

Link

et al . Performance evaluation of three continuous glucose monitoring systems: comparison of six sensors per subject in parallel. J Diabetes Sci Technol 2013; 7: 842–853.

20.

Matuleviciene

Joseph

Andelin

et al . A clinical trial of the accuracy and treatment experience of the Dexcom G4 sensor (Dexcom G4 system) and Enlite sensor (guardian REAL-time system) tested simultaneously in ambulatory patients with type 1 diabetes. Diabetes Technol Ther 2014; 16: 759–767.

21.

Damiano

McKeon

El-Khatib

et al . A comparative effectiveness analysis of three continuous glucose monitors: the Navigator, G4 Platinum, and Enlite. J Diabetes Sci Technol 2014; 8: 699–708.

22.

Kropff

Bruttomesso

Doll

et al . Accuracy of two continuous glucose monitoring systems: a head-to-head comparison under clinical research centre and daily life conditions. Diabetes Obes Metab 2015; 17: 343–349.

23.

Bonora

Maran

Ciciliot

et al . Head-to-head comparison between flash and continuous glucose monitoring systems in outpatients with type 1 diabetes. J Endocrinol Invest 2016; 39: 1391–1399.

24.

Aberer

Hajnsek

Rumpler

et al . Standardised evaluation of three continuous glucose monitoring systems mimicking real-life conditions. Diabetes Obesity Metab 2017; 19: 1051–1055.

25.

Kamecke

Link

Pleus

et al . Performance of two tissue glucose monitoring systems intended for nonadjunctive use. In: American Diabetes Association 77th scientific sessions, San Diego, CA, 9–13 June 2017, poster no. 917.

26.

Zeng

Pagan

et al . Clinical comparison of iWel and Medtronic continuous glucose monitoring (CGM) devices. In: American Diabetes Association 77th scientific sessions, San Diego, CA, 9–13 June 2017, poster no. 121-LB.

27.

Freckmann

Schmid

Baumstark

et al . System accuracy evaluation of 43 blood glucose monitoring systems for self-monitoring of blood glucose according to DIN EN ISO 15197. J Diabetes Sci Technol 2012; 6: 1060–1075.

28.

Pleus

Schoemaker

Morgenstern

et al . Rate-of-change dependence of the performance of two CGM systems during induced glucose swings. J Diabetes Sci Technol 2015; 9: 801–807.

29.

Rodbard

Characterizing accuracy and precision of glucose sensors and meters. J Diabetes Sci Technol 2014; 8: 980–985.

30.

Clarke

WL.

The original Clarke error grid analysis (EGA). Diabetes Technol Ther 2005; 7: 776–779.

31.

Pfützner

Klonoff

Pardo

et al . Technical aspects of the Parkes error grid. J Diabetes Sci Technol 2013; 7: 1275–1281.

32.

Bailey

Bode

Christiansen

et al . The performance and usability of a factory-calibrated flash glucose monitoring system. Diabetes Technol Ther 2015; 17: 1–8.

33.

Reiterer

Polterauer

Schoemaker

et al . Significance and reliability of MARD for the accuracy of CGM systems. J Diabetes Sci Technol 2017; 11: 59–67.

Accuracy of flash glucose monitoring and continuous glucose monitoring technologies: Implications for clinical practice

Abstract

Keywords

Introduction

Lack of standardised assessment for ISF glucose sensors

Accuracy, precision and concordance

Defining accuracy: role of mean absolute relative difference

Number of paired measurements

Accuracy of the reference system

Accuracy at high RoC and extremes of glucose levels

Additional measures of accuracy and precision: Clarke and consensus error grids

Areas of caution: when is adjunct blood glucose testing needed?

Conclusion and future directions

Footnotes

Declaration of conflicting interests

Funding

References