Comparative Accuracy Analysis of a Real-time and an Intermittent-Scanning Continuous Glucose Monitoring System

Abstract

Background:

Currently, two different types of continuous glucose monitoring (CGM) systems are available: real time (rt) CGM systems that continuously provide glucose values and intermittent-scanning (is) CGM systems. This study compared accuracy of an rtCGM and an isCGM system when worn in parallel.

Methods:

Dexcom G5 Mobile (DG5) and FreeStyle Libre (FL) were worn in parallel by 27 subjects for 14 days including two clinic sessions with induced glucose excursions. The percentage of CGM values within ±20% or ±20 mg/dL of the laboratory comparison method results (YSI 2300 STAT Plus, YSI Inc., Yellow Springs, OH, United States; glucose oxidase based) or blood glucose meter values and mean absolute relative difference (MARD) were calculated. Consensus error grid and continuous glucose error grid analyses were performed to assess clinical accuracy.

Results:

Both systems displayed clinically accurate readings. Compared to laboratory comparison method results during clinic sessions, DG5 had 91.5% of values within ±20%/20 mg/dL and a MARD of 9.5%; FL had 82.5% of scanned values within ±20%/20 mg/dL and an MARD of 13.6%. Both systems showed a lower level of performance during the home phase and when using the blood glucose meter as reference.

Conclusion:

The two systems tested in this study represent two different principles of CGM. DG5 generally provided higher accordance with laboratory comparison method results than FL.

Keywords

accuracy continuous glucose monitoring mean absolute relative difference performance sensor

Introduction

In modern diabetes therapy, systems for continuous glucose monitoring (CGM) are becoming more and more common. In contrast to traditional self-monitoring of blood glucose (SMBG) that is performed several times a day, CGM systems can provide a complete picture of glycemic excursions during day and night, including meal excursions and effects of physical activity. Patients may utilize information about glycemic excursions of the past hours and prospective glucose trends for insulin dosing and, if possible with the used system, patients are alerted of low or high glucose levels. Recently, the Food and Drug Administration (FDA) approved the first CGM system for nonadjunctive use.¹ The nonadjunctive indication allows the user to use CGM as a replacement for SMBG measurements and thus serve as basis for all therapeutic decisions under certain conditions. This indication demands a high degree of accuracy and reliability of the displayed glucose values to enable an adequate glycemic control and not to compromise patient safety.² Currently, two manufacturers offer CGM systems that are intended for nonadjunctive use.

In general, two different types of CGM systems are available: real time (rt) CGM systems that continuously provide glucose values and intermittent-scanning (is) CGM systems that only display a glucose value when the sensor is scanned with the respective reading device. Additionally, the currently available isCGM systems are factory calibrated and thus do not have to be calibrated periodically by the user with SMBG measurements.

In this study, two commercially available CGM systems indicated for nonadjunctive use, one rtCGM system and one is CGM system, were tested in parallel for 14 days to evaluate and compare their measurement accuracy in a combined setting of in-clinic and real-life conditions.

Methods

This prospective, nonrandomized, single-arm clinical trial was performed following the Declaration of Helsinki, Good Clinical Practice (GCP), and local laws and regulations at the Institut für Diabetes-Technologie, Forschungs- und Entwicklungsgesellschaft mbH an der Universität Ulm, Ulm, Germany, between January and March 2017. The responsible independent ethics committee approved the study protocol; the trial was exempted from regulatory approval by the competent authority. All subjects gave written informed consent before any study procedures.

Study Population

Subjects aged 18 to 65 with diabetes mellitus and intensified insulin therapy using multiple daily injections (MDI) or continuous subcutaneous insulin infusion (CSII) were eligible for the study. The main exclusion criteria were pregnancy, known severe allergies to adhesives, presence of several cardiovascular risk factors or diseases that pose a risk for the subjects when participating in the study procedures.

Investigational Devices

The rtCGM system used was the Dexcom G5 Mobile (DG5; Dexcom Inc., San Diego, CA, United States). The DG5 can be worn for up to seven days and requires SMBG calibration every 12 hours. The isCGM system used was the FreeStyle Libre (FL; Abbott Diabetes Care, Alameda, CA, United States) system. The FL can be worn for up to 14 days and does not require user-entered SMBG calibrations.

To obtain data for 14 days from both systems, DG5 sensors and transmitters were replaced after seven days. Therefore, each subject wore two DG5 sensors and one FL sensor during the study period. As the CGM systems specified interferences with acetaminophen, subjects were advised not to take any medication containing this substance during the study.

Comparison Measurements

Comparison measurements during clinic session days were performed with a glucose oxidase (GOD)-based laboratory glucose analyzer (YSI; YSI 2300 STAT PLUS Glucose Analyser, YSI Incorporate, Yellow Springs, OH, United States) that claims conformity to the traceability requirements of ISO 17511.³ As a substitute for capillary samples, venous samples were gained with the heated-hand method, leading to hyperemia in the hand and arm, resulting in an “arterialization” of venous blood.^4,5 Therefore, a heating pad (~40°C) was applied to one hand and forearm of each subjects before and during venous sampling.

The blood glucose monitoring system (BGMS) Contour Next USB (Ascensia Diabetes Care GmbH, Basel, Switzerland) was used to calibrate the DG5 system and for additional capillary comparison measurements during home and clinic session days. Control solution measurements were performed for every test strip vial.

Study Design

The experiment phase lasted 14 days during which the subjects visited the study site several times. Each subject visited the study site for sensor insertion (day 1), replacement of the DG5 sensor (day 7), and for sensor removal (day 14). On two occasions, subjects visited the study site for clinic session days that were randomly distributed among the subjects. Clinic session days were scheduled for day 1, 2, 4, 7, 10, or 14. On the other study days, subjects were at home following their usual daily practices. All treatment decisions during the study were based on SMBG values and not on CGM values.

On the first study day, participants received the DG5 and FL systems and were trained on how to use the devices. All study participants applied the sensors themselves; the FL sensor on the upper arm and the DG5 sensor on the abdomen. DG5 sensors were calibrated two hours following sensor insertion and then every 12 hours. FreeStyle Libre sensors did not require calibration.

On clinic session days, rapid glucose changes with values in the hyper- and hypoglycemic range were induced. If a subject’s initial SMBG was below 140 mg/dL, low glucose values were induced first through insulin administration, and afterward high glucose values were induced by intake of a breakfast meal with high glycemic index and a delay in insulin bolusing. If the initial SMBG was 141 mg/dL or higher, high glucose levels were induced first followed by the induction of low glucose levels. Heating pads were applied to one arm prior to the inductions and after about one hour “arterializing” the venous blood, venous sampling was performed every 15 minutes for approximately eight hours for measurements with the GOD method. Capillary SMBG measurements were performed at least every 30 minutes on the other hand. During the clinic session days, CGM devices were blinded via the software or by covering the display.

During home phases, participants performed SMBG at least seven times per day (after getting up, before each major meal, one to two hours postprandial and before going to sleep). Additional SMBG measurements were performed for suspected hypoglycemia, 15 minutes after hypoglycemia treatment and in the case of low or high glucose alert (only provided by DG5). After each SMBG measurement, the FL sensor was scanned with the reader to obtain a glucose value (FL(s)).

After the 14-day wearing period, CGM systems were removed, data were downloaded, and all sensor sites were examined. Adverse events (AE) and device deficiencies were documented and analyzed.

Data Analysis

All data analyses were performed by Dexcom, Inc., the sponsor of the study. For both CGM systems, performance was evaluated in terms of point accuracy and rate accuracy based on paired CGM-GOD comparison method data points. For DG5, a paired data point was defined as a reference glucose value matched with the first CGM value within the following five minutes, or, if not available, the closest CGM value within 2.5 minutes prior to blood sampling. For FL, comparison method values were matched with the corresponding scan (FL(s)) or continuous value (FL(c)) which are both provided upon data download.

Point accuracy was calculated using the paired data points and was calculated as the percentage of CGM values within ±20% of the reference glucose value when the reference glucose value >100 mg/dL or within ±20 mg/dL when the reference glucose value ≤100 mg/dL. In addition, the mean absolute relative difference (MARD) and the mean relative difference (MRD) were calculated and Consensus Error Grid analyses were performed.

Rate accuracy was determined by means of a continuous glucose error grid analysis (CG-EGA).⁶

Furthermore, point accuracy compared to SMBG was evaluated by using paired data points of CGM values and SMBG values excluding calibration values. Continuous glucose monitoring-self-monitoring of blood glucose paired data points were matched as described above for GOD-based values.

For FL, all analyses were performed using both the stored glucose values following data download (FL(c)) and the glucose values displayed at the time of scanning of the sensor (FL(s)).

Technical and safety outcomes were based on data of the complete study. Device deficiencies and premature sensor removal were analyzed as technical outcome. Safety outcomes were AE and AE with possible, probable, or definitive causal relationship to an investigational device.

Results

Participants

Of 28 screened subjects, 27 (14 female, 13 male) were enrolled in the study. All subjects had type 1 diabetes and were either on CSII (63%) or on MDI (37%). The average age was 49.5 ± 12.6 years (mean ± standard deviation (SD)) and the mean diabetes duration was of 24.2 ± 13.2 years. The HbA1c level was 7.2% ± 0.8% (55.2 ± 8.7 mmol/mol). Most subjects (67%) did not have prior experience with rtCGM. Due to illness or device failures, four subjects could not complete the study as intended.

Performance

Point accuracy evaluation during clinic session days showed 91.5% of DG5 values and 82.5% of FL(s) values within ±20%/20 mg/dL of corresponding GOD-based values (see Table 1; Figures 1 and 2) with an MARD of 9.5% for DG5 and 13.6% for FL(s) (Table 1). Both systems had a negative bias compared to the GOD-based laboratory method, as shown by MRD (Table 1). There were no significant differences between the whole FL data (FL(c)) and scanned values only (FL(s)).

Table 1.

Point Accuracy During Clinic Session Days for Dexcom G5 and Scanned and Continuous FreeStyle Libre Results, Compared to “Arterialized” Venous Glucose Oxidase based Comparison Method Results.

	DG5	FL(s)	FL(c)
N	1524	973	1366
Within ±20%/20 mg/dL	91.5%	82.5%	81.8%
MARD	9.5%	13.6%	13.8%
MRD	–2.3%	–4.3%	–4.1%

Abbreviations: DG5, Dexcom G5 Mobile; FL(c); continuous FreeStyle Libre; FL(s), scanned FreeStyle Libre; MARD, mean absolute relative difference; MRD, mean relative difference.

Figure 1.

Differences between continuous glucose monitoring (top: Dexcom G5 Mobile, bottom: scanned FreeStyle Libre) and glucose oxidase based comparison method values. Dotted lines indicate the ±20 mg/dL/20% range.

Figure 2.

Distribution of values within ±20 mg/dL/20% of glucose oxidase based comparison method values (left: Dexcom G5 Mobile, right: scanned FreeStyle Libre) between individual subjects (n = 27).

Regarding different glucose ranges, both systems showed their highest accuracy at glucose levels between 301 and 350 mg/dL and the lowest accuracy at glucose levels of 40 to 60 mg/dL (Table 2). The largest differences between the two systems were observed in the low glucose range.

Table 2.

Point Accuracy During Clinic Session Days for Different Glucose Ranges for Dexcom G5 Mobile and Scanned and Continuous FreeStyle Libre Results, Compared to “Arterialized” Venous Glucose Oxidase based Comparison Method Results.

Glucose range (mg/dL)	DG5			FL(s)			FL(c)
Glucose range (mg/dL)	N	Within ±20%/20 mg/dL	MARD	N	Within ±20%/20 mg/dL	MARD	N	Within ±20%/20 mg/dL	MARD
40-60	111	85.6%	15.7%	165	73.9%	20.3%	200	68.5%	22.4%
61-80	176	92.0%	10.8%	123	87.8%	13.2%	149	86.6%	13.9%
81-180	598	91.1%	9.3%	292	77.1%	14.1%	447	77.6%	14.1%
181-300	420	90.2%	9.3%	214	83.2%	12.3%	342	84.8%	12.0%
301-350	151	98.7%	6.3%	100	98.0%	7.0%	124	96.0%	7.5%
351-400	68	95.6%	7.1%	63	93.7%	9.2%	84	94.0%	9.1%
401-500	–	–	–	16	81.3%	11.3%	20	80.0%	12.4%

Abbreviations: DG5, Dexcom G5; FL(c); continuous FreeStyle Libre; FL(s), scanned FreeStyle Libre; MARD, mean absolute relative difference.

The systems had the highest MARD on study day 1 (DG5 14.4%) or day 2 (FL 19.4%) which improved during the sensor life time. After seven days, DG5 and FL reached a comparable MARD of ~9% (Table 3).

Table 3.

Point Accuracy During Individual Clinic Session Days for Dexcom G5 Mobile and Scanned and Continuous FreeStyle Libre Results, Compared to “Arterialized” Venous Glucose Oxidase Based Comparison Method Results. Each Subject Attended Two of the Six Clinic Sessions Listed Below.

Study day	DG5			FL(s)			FL(c)
Study day	N	Within ±20%/20 mg/dL	MARD	N	Within ±20%/20 mg/dL	MARD	N	Within ±20%/20 mg/dL	MARD
1	212	75.0%	14.4%	145	71.0%	16.8%	209	63.2%	19.6%
2	214	96.3%	8.1%	137	69.3%	19.4%	193	72.5%	17.6%
4	239	92.9%	9.3%	145	77.9%	16.1%	219	77.2%	16.2%
7	239	92.1%	9.4%	161	93.8%	9.1%	229	96.9%	9.2%
10	407	96.1%	8.2%	252	86.9%	11.6%	341	85.0%	12.0%
14	213	92.5%	8.9%	133	91.7%	10.3%	175	93.7%	9.4%

Abbreviations: DG5, Dexcom G5; FL(c); continuous FreeStyle Libre; FL(s), scanned FreeStyle Libre; MARD, mean absolute relative difference.

When using SMBG as comparison method (Table 4), both systems had an approximately 1% higher MARD and a lower percentage of values within ±20%/20 mg/dL compared to the laboratory method.

Table 4.

Point Accuracy During Home and Clinic Session Days for Dexcom G5 Mobile and Scanned and Continuous FreeStyle Libre Results, Compared to Capillary Self-monitoring of Blood Glucose Values.

	DG5		FL(s)		FL(c)
	Home	Clinic	Home	Clinic	Home	Clinic
N	3211	1128	2750	1073	3081	1176
Within ±20%/20 mg/dL	85.8%	88.9%	78.2%	80.5%	79.0%	79.8%
MARD	11.4%	10.6%	15.2%	14.8%	14.9%	14.9%

Abbreviations: DG5, Dexcom G5; FL(c); continuous FreeStyle Libre; FL(s), scanned FreeStyle Libre; MARD, mean absolute relative difference.

Comparing the home phases to the clinic sessions, both systems showed a better performance during clinic sessions (88.9% vs 85.8% within ±20%/20 mg/dL for DG5 and 80.5% vs 78.2% for FL(s), Table 4).

Rate accuracy in terms of CG-EGA analysis is shown in Table 5. Both systems showed clinically accurate readings in all glucose ranges. However, in the hypoglycemic range, a substantial number of readings (3.9% for DG5 and 4.9% for FL(s)) were classified as erroneous.

Table 5.

Summarized Results of Continuous Glucose Error Grid Analysis for Dexcom G5 Mobile and Scanned FreeStyle Libre Results.

	DG5			FL(s)
	Hypoglycemia (BG ≤ 70 mg/dL)	Euglycemia(70 > BG ≤ 180 mg/dL)	Hyperglycemia(BG > 180 mg/dL)	Hypoglycemia (BG ≤ 70 mg/dL)	Euglycemia(70 > BG ≤ 180 mg/dL)	Hyperglycemia(BG > 180 mg/dL)
Accurate readings	95.6%	98.8%	98.5%	94.5%	96.9%	96.9%
Benign readings	0.6%	1.0%	0.6%	0.5%	2.8%	0.8%
Erroneousreadings	3.9%	0.1%	0.9%	4.9%	0.3%	2.3%

Abbreviations: BG, blood glucose; DG5, Dexcom G5 Mobile; FL(s), scanned FreeStyle Libre.

Consensus Error Grid analysis showed 91.7% of DG5 and 83.4% of FL(s) within zone A; 8.3% of G5 and 16.3% of FL(s) values were within zone B.

Technical and Safety Outcomes

In this study, 56 DG5 sensors, 56 transmitters, 30 receivers, 29 FL sensors, and 28 readers were used. Two DG5 sensors and two FL sensors were replaced within 12 hours of initial insertion as per protocol replacement guidelines.

In total, seven AE occurred in three subjects during the study. Of these, four were skin reactions related to the study devices (three DG5 and one FL). None of the AE was regarded as serious.

Discussion

This study aimed at directly comparing the performance of two CGM systems that represent the currently available CGM principles: steady data transmission and display including alerts in rtCGM and on-demand data display in isCGM. Both tested systems have a nonadjunctive indication; meaning that, apart from calibrating the DG5 twice per day, blood glucose monitoring may no longer be required. Accuracy and reliability of the values in conjunction with all additionally provided information is therefore more important than ever.

The two CGM systems DG5 and FL were compared in 27 subjects in this study. From a patient’s point of view, FL seems more attractive at first sight due to lower costs, longer wearing time, and factory calibration. However, the FL version available at the time of the study did not offer hypo- and hypoglycemia alerts, a valuable feature of rtCGM systems. Reported accuracy of FL was often lower than in different rtCGM systems.^7-12 Because measured accuracy is affected by the study setting,¹³ comparison of results from separate studies might distort the conclusions drawn. Therefore, in this study, both systems were tested in parallel in the same subjects to obtain comparable performance results.

Accuracy of CGM systems is more complex to assess than, eg, accuracy of a BGMS, because more than just a single value is provided. Accordingly, more than one parameter is required to characterize performance of a CGM system. In the absence of international regulations that demand specific accuracy requirements, different accuracy parameters that are common in the literature, but that are typically used for BGMS accuracy assessments, were applied in this study.

Point accuracy was notably higher in DG5 than in FL with 91.5% vs 82.5% of values within ±20%/20 mg/dL of the “arterialized” venous GOD-based comparison method results and an MARD of 9.5% vs 13.6%. This resulted from a lower bias and a higher precision. Mean absolute relative difference values at low glucose concentrations were high when compared with MARD values at higher glucose concentrations. This is expected, because low glucose concentrations in the denominator inflate MARD values. A limitation arising from evaluation procedures is the use of subsequent CGM values for matching with reference values as this might lessen the impact of the system-inherent time lag. Whereas DG5 showed a substantial improvement of accuracy already after the first day of use, MARD values in FL remained higher for several clinic session days. The study setting, however, limits comparability of individual days because each of the subjects only attended two of six possible clinic session days, and therefore, data for individual days comprise different subjects and sensors. Furthermore, other studies reported this obvious deviation for FL only on the first day.^7,9,14 However, it has to be noted that, while FL can be worn for 14 days, DG5 has to be replaced after seven days, and therefore, has an additional warm-up phase during 14 days of use. In this study, no relevant differences in MARD or bias between FL(s) and FL(c) were observed; however, in direct comparison of values at specific time points, there might be deviations.¹⁵

From a clinical perspective based on consensus error grids and CG-EGA, however, both systems were sufficiently accurate.

Both systems were found to exhibit better concurrence to “arterialized” venous samples as reference than to SMBG values, which emphasizes the impact of the study setting on the results. Accordingly, a similar study testing the same systems showed less pronounced differences between the two systems.⁷ Results from “arterialized” venous samples are more similar to capillary values and might therefore favor systems calibrated to capillary references. Depending on the specifics behind FL’s factory-calibration, it cannot be excluded that the choice of the comparison method might have impacted the differences observed between DG5 and FL. Generally, there should be consistent and mandatory guidelines on CGM accuracy testing with a focus on reference methods to enable unbiased testing and to allow comparison of results from different studies.

Conclusion

The two systems tested in this study represent two different embodiments of CGM, rtCGM and isCGM. Although both systems showed clinically acceptable accuracy, the DG5 generally provided closer agreement with the comparison methods than FL. Suitability of the systems will be determined by individual requirements and prioritization.

Footnotes

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: GF is general manager of the IDT (Institut für Diabetes-Technologie Forschungs- und Entwicklungsgesellschaft mbH an der Universität Ulm, Ulm, Germany), which carries out clinical studies on the evaluation of BG meters and medical devices for diabetes therapy on its own initiative and on behalf of various companies. GF/IDT have received speakers’ honoraria or consulting fees from Abbott, Ascensia, Bayer, Dexcom, LifeScan, Menarini Diagnostics, Metronom Health, Novo Nordisk, Roche, Sanofi, Sensile and Ypsomed. ML, UK, DW, SP, and CH are employees of IDT. AG is employee of Dexcom, Inc.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study and writing of the manuscript were funded by the study sponsor Dexcom Inc.

ORCID iDs

Delia Waldenmaier

Stefan Pleus

Guido Freckmann

References

FDA advisory panel votes to recommend non-adjunctive use of Dexcom G5 mobile CGM. Diabetes Technol Ther. 2016;18(8):512-516.

Shapiro

AR.

Nonadjunctive use of continuous glucose monitors for insulin dosing: is it safe?

J Diabetes Sci Technol. 2017;11(4):833-838.

International Organization for Standardization. In vitro diagnostic medical devices – Measurement of quantities in biological samples – Metrological traceability of values assigned to calibrators and control materials. EN ISO. 17511:2003. 2003.

Liu

Moberg

Kollind

Lins

Adamson

Macdonald

IA.

Arterial, arterialized venous, venous and capillary blood glucose measurements in normal man during hyperinsulinaemic euglycaemia and hypoglycaemia. Diabetologia. 1992;35(3):287-290.

Kropff

van Steen

deGraaff

Chan

van Amstel

RBE

DeVries

JH.

Venous, arterialized-venous, or capillary glucose reference measurements for the accuracy assessment of a continuous glucose monitoring system. Diabetes Technol Ther. 2017;19(11):609-617.

Kovatchev

Patek

Ortiz

Breton

MD.

Assessing sensor accuracy for non-adjunct use of continuous glucose monitoring. Diabetes Technol Ther. 2015;17(3):177-186.

Freckmann

Link

Pleus

Westhoff

Kamecke

Haug

Measurement performance of two continuous tissue glucose monitoring systems intended for replacement of blood glucose monitoring. Diabetes Technol Ther. 2018;20(8):541-549.

Fokkert

van Dijk

Edens

Abbes

de Jong

Slingerland

, et al. Performance of the FreeStyle Libre Flash glucose monitoring system in patients with type 1 and 2 diabetes mellitus. BMJ Open Diabetes Res Care. 2017;5(1):e000320.

Bailey

Bode

Christiansen

Klaff

Alva

The performance and usability of a factory-calibrated flash glucose monitoring system. Diabetes Technol Ther. 2015;17(11):787-794.

10.

Bailey

Chang

Christiansen

Clinical accuracy of a continuous glucose monitoring system with an advanced algorithm. J Diabetes Sci Technol. 2015;9(2):209-214.

11.

Faccioli

Del Favero

Visentin

, et al.; PedArPan Study Group. Accuracy of a CGM sensor in pediatric subjects with type 1 diabetes. Comparison of three insertion sites: arm, abdomen, and gluteus. J Diabetes Sci Technol. 2017;11(6):1147-1154.

12.

Laffel

Improved accuracy of continuous glucose monitoring systems in pediatric patients with diabetes mellitus: results from two studies. Diabetes Technol Ther. 2016;18(suppl 2):S223-S233.

13.

Kirchsteiger

Heinemann

Freckmann

, et al. Performance comparison of CGM systems: MARD values are not always a reliable indicator of CGM system accuracy. J Diabetes Sci Technol. 2015;9(5):1030-1040.

14.

Charleer

Mathieu

Nobels

Gillard

Accuracy and precision of flash glucose monitoring sensors inserted into the abdomen and upper thigh compared with the upper arm. Diabetes Obes Metab. 2018;20(6):1503-1507.

15.

Pleus

Kamecke

Link

Haug

Freckmann

Flash glucose monitoring: differences between intermittently scanned and continuously stored data. J Diabetes Sci Technol. 2018; 12(2):397-400.