Abstract
Background:
In this multicenter study, performance of a novel continuous glucose monitoring (CGM) system was evaluated.
Methods:
Adult participants with diabetes were included in the study. They each wore three sensors of the CGM system on the upper arms for up to 14 days. During four in-clinic visits, frequent comparison measurements with capillary blood glucose (BG) samples were performed. The primary endpoint was the 20/20 agreement rate (AR): the percentage of CGM readings within ±20 mg/dL (at BG values <100 mg/dL) or ±20% (at BG values ≥100 mg/dL) of the comparator. Further evaluations included mean absolute relative difference (MARD) and 20/20 AR in different BG ranges and across the wear time.
Results:
Data from 48 participants and 139 sensors were analyzed. During in-clinic sessions the 20/20 AR was 90.5% and the MARD was 9.2%. For BG ranges <70, 70-180, and >180 mg/dL, 20/20 AR was 94.3%, 89.0%, and 92.5%, respectively. At the beginning, middle, and end of sensor wear time, 20/20 AR was 92.8%, 91.5%, and 85.9%, respectively. The 14-day survival probability was 82.4%. Pain and bleeding after sensor insertion were within the expected range. Based on the study outcome, the use of the device is regarded as safe.
Conclusions:
The system showed a good performance compared to capillary BG measurements. This level of accuracy could be shown over the entire measurement range, especially in the low glycemic range, and the whole wear time of the sensors. The results of this study are supporting a non-adjunctive use of the device.
Introduction
During the last decade, the use of continuous glucose monitoring (CGM) systems has increased considerably due to advances in performance, evidence for clinical benefits, and subsequent integration into diabetes therapy guidelines.1-3 As a consequence, many innovative devices are being developed and new players are entering the market. Roche Diabetes Care (RDC) has a long history in glucose monitoring and has now developed a new CGM solution.
The CGM device is intended to be used by adults with diabetes. The sensor uses a glucose-specific enzyme in combination with an amperometric method to obtain a glucose-concentration-dependent signal. The system consists of the sensor, which comes preassembled in an applicator, and a smartphone application (Figure 1). The sensor can be worn on the back of the upper arm for up to 14 days. The system has a warm-up time of 1 hour and requires an initial calibration routine.
The Accu-Chek SmartGuide CGM system: applicator (left), sensor with adhesive (middle), and representative depiction of the app (right).
In this multicenter study, safety and performance of the CGM device were evaluated in adults with diabetes.
Methods
Study Design
This was an open-label, single-arm, prospective, non-randomized multicenter study to evaluate safety and performance of the Accu-Chek SmartGuide CGM device (RDC). The study was conducted at three study centers in Germany and Austria. The study was approved by the respective local ethics committees and responsible authorities and was registered at the International Standard Randomised Controlled Trial Number (ISRCTN) registry (ISRCTN89358739).
The primary endpoint of the study was the in-clinic agreement rate (AR) of CGM readings within ±20 mg/dL or ±20% of comparator blood glucose (BG) values (for BG values <100 or ≥100 mg/dL, respectively) obtained with the Accu-Chek Guide BG meter (RDC) from here on referred to as 20/20 AR. This BG meter was shown to be highly accurate with more than 99.7% of values within ±10 mg/dL or 10% of laboratory reference values when tested according to ISO 15197:2013. 4
The study was initiated and designed, and the data were processed by the manufacturer. The analysis of endpoints was performed by the manufacturer and the Institut für Diabetes-Technologie, Ulm, Germany.
Study Population
The study included adults with type 1 or type 2 diabetes diagnosed at least 12 months prior to the study, of whom at least 95% had to be on intensified insulin therapy. Main exclusion criteria were HbA1c >9.5% (80 mmol/mol), pregnancy or lactation period, serious acute or chronic concomitant diseases, or diabetes-related complications that might pose a risk to the participant, previous severe hypoglycemia or impaired hypoglycemia awareness, adhesive allergy or incompatibility, and intake of potentially interfering substances (hydroxyurea, levodopa, methyldopa, ascorbic acid, and acetylsalicylic acid).
Sample size calculation considering an expected 20/20 AR of ≥84%, a 90% power and a one-side confidence level of 0.025 yielded a required number of 14 517 measurement pairs. Anticipating 384 measurement pairs per participant and a drop-out rate of 20%, 48 participants were planned to be enrolled.
Study Procedures
All participants signed informed consent prior to any study procedures. The study was performed in a mixed in-clinic and home-use setting and was performed over 15 calendar days. After screening and enrollment, the participants were trained in sensor insertion and sensors were inserted at the study site on day 1. Participants were divided into two groups that had slightly differing visit schedules to cover a broader range of in-clinic days.
In this study, the app was used in a blinded mode and the sensors only stored raw signals. Three different sensor batches were tested.
Each participant received three sensors (from two different batches) which were inserted on the back of their upper arms following a predefined application scheme. One sensor was applied by the participants themselves and the other two by study personnel. After insertion, sensors were paired with the app and for the next 5 hours, capillary BG measurements were performed every 30 minutes. On day 1, sensors could be replaced in case of insertion failure or connection issues with the smartphone application. All comparator BG measurements were performed in duplicate and averaged.
On day 2, approximately 12 to 16 hours after sensor insertion, frequent comparator measurements were performed at the study site every 15 minutes over a period of 8 hours. On three additional days (day 5, 9, and 13 for group A; day 6, 10, and 14 for group B), participants visited the study sites for frequent comparator sampling and manipulation of BG levels with the intention to obtain BG levels <70 and ~300 mg/dL. Depending on the BG value (below or above 150 mg/dL) and trend in the morning, the investigators decided for each participant whether low or high BG values were targeted. Low BG levels were induced by the administration of insulin targeting 70 mg/dL based on the participants’ individual insulin sensitivity factors. High BG levels were induced by eating a meal with a predefined amount of carbohydrates and a delay of insulin administration. Meals were chosen from a selection of foods containing between 10 and 25 g carbohydrates per portion, depending on the starting BG value. Countermeasures (administration of carbohydrates or meal insulin, respectively) were taken when it was foreseeable that participants’ BG levels would be in the targeted range for ~1 hour. Comparator samples were taken every 15 minutes for a period of approximately 8 hours. If BG values fell below 70 mg/dL or rose above 250 mg/dL, the frequency of BG measurement was increased to every 10 minutes and participants could be kept in that range for up to 1 hour before countermeasures were initiated. All interventions could be modified at the discretion of the investigators.
On the other days (home-use setting), participants continued their usual daily life and were asked to perform at least eight duplicate BG measurements per day (after waking up, before and ~2 hours after every meal, before administration of insulin and before bedtime). Participants returned to the study sites on day 15 to remove all sensors after the runtime of 14 days. At each visit, CGM and BG data were downloaded, insertion sites were examined, and adverse events as well as device deficiencies were captured.
Data Analysis
The conversion of raw sensor signals into glucose concentrations was performed retrospectively. The algorithm included initial calibrations on day 1 (12 and 14 hours after insertion) and failsafe measures. If such a measure detected any sensor failure, no more values were generated. Calibration values were not used for performance analysis. The CGM values outside the measurement range of 40 to 400 mg/dL were excluded. Data from all sensors and batches were pooled.
The CGM and BG data points were paired by linear interpolation of CGM readings from 5-minute intervals if no matching CGM reading was available at the time of the BG measurement. For the primary performance analysis, only data from frequent sampling during in-clinic visits were used. The 20/20 AR was calculated by determining the number of CGM readings within ±20 mg/dL (for BG values <100 mg/dL) or ±20% (for BG values ≥100 mg/dL) of paired BG values. The threshold of 100 mg/dL was specified in agreement with recommendations in the POCT05 guideline 5 and to have a continuous transition between absolute and relative (percentage) differences.
Secondary endpoints included AR with other limits and for different BG ranges, mean absolute relative difference (MARD), or mean absolute difference (MAD), relative bias, accuracy at different CGM rates of change (RoC) and over sensor wear time, sensor-specific MARD, continuous glucose deviation interval and variability analysis (CG-DIVA), 6 consensus error grid analysis, 7 sensor survival, 8 and sensor data availability. Pain during sensor insertion was rated on a numeric scale (0: no pain to 10: maximal pain), and bleeding was classified as either no, mild, moderate, or severe bleeding.
Results
Study Population
After screening of 61 potential participants, 51 were deemed eligible. Of those, three did not have a sensor applied (one withdrew consent and two were recruited as potential replacement participants). In total, 166 CGM sensors (including replacements) were applied to 48 participants. One participant lost all three sensors during the warm-up phase and one participant was unable to complete the study due to an adverse event unrelated to the device; therefore, 46 participants completed the study.
Included participants were 44.4 ± 15.3 years of age, 42% were female, 58% were male, and 83% had type 1 diabetes. All participants were on insulin therapy, and some used additional anti-diabetic medication (Table 1).
Baseline Participant Characteristics.
AID: automated insulin delivery, CSII: continuous subcutaneous insulin infusion, MDI: multiple daily injections, SD: standard deviation.
Accuracy Evaluation
For the analysis, 139 sensors that were successfully started and delivered at least one valid value were included. After application of the algorithm and resultant data exclusions based on failsafe measures, 15 993 paired BG-CGM values (from 132 sensors) from in-clinic sessions and 11 187 values from the home-use setting were available for the performance evaluation.
Accuracy during in-clinic sessions as well as in-clinic sessions and home use combined are shown in Table 2. The primary endpoint, the 20/20 AR during in-clinic sessions, was 90.5% and the overall MARD (in-clinic) was 9.2%. During the whole study, the 20/20 AR was 89.8% and the overall MARD (in-clinic and home use) was 9.6%. The highest accuracy, measured by the 20/20 AR, was observed in the glycemic range below 70 mg/dL. The results of the CG-DIVA showing the bias and precision in different glucose ranges, as well as the sensor-to-sensor variability from in-clinic session data are shown in Figure 2.
Accuracy Results for In-Clinic Sessions (132 Sensors) as well as for In-Clinic and Home Use Combined (139 Sensors).
For BG values <100 mg/dL, absolute differences between BG and CGM values were calculated; for BG values ≥100 mg/dL, relative differences were calculated.
BG: blood glucose; AR: agreement rate; MARD: mean absolute relative difference; MAD: mean absolute difference.
Results of the continuous glucose deviation interval and variability analysis (CG-DIVA) of in-clinic session data. (a) Deviation intervals in different glucose ranges covering the Food and Drug Administration iCGM criteria: the dark gray boxes contain 85%, 70%, 80%, and 87% of expected deviations, respectively, and the light gray boxes contain 98%, 99%, and 99% of expected deviations. Black dashes show median deviations. (b) Sensor-to-sensor variability over the whole glucose range. Each sensor with at least three values is represented by median and 90% range. Sensors were ordered by median.
No relevant differences in these accuracy measures between the three sensor batches were observed (data on file). The accuracy results in different glucose level RoC as determined by the CGM system are shown in Table 3. Accuracy was the highest, when BG values changed no faster than −1 to +1 mg/dL/min. The mean absolute BG RoC during in-clinic sessions was 0.86 mg/dL/min.
Accuracy at Different Rates of Change Determined by the CGM System During In-Clinic Sessions.
For BG values <100 mg/dL, absolute differences between BG and CGM values were calculated; for BG values ≥100 mg/dL, relative differences were calculated.
BG: blood glucose; CGM: continuous glucose monitoring; RoC: rate of change; AR: agreement rate; MARD: mean absolute relative difference.
Over the course of the sensor wear time, the accuracy remained relatively stable with a 20/20 AR of more than 85% and MARDs between 8.3% and 10.8% during each in-clinic session (Table 4).
Accuracy at Different Sensor Wear Periods During In-Clinic Sessions.
For BG values <100 mg/dL, absolute differences between BG and CGM values were used; for BG values ≥100 mg/dL, relative differences were used.
In-clinic session without glucose manipulation.
AR: agreement rate; MARD: mean absolute relative difference.
The distribution of sensor-specific MARD is shown in Figure 3. The mean sensor-specific MARD was 9.8%.
Distribution of sensor-specific mean absolute relative differences (MARD) across sensors (n = 132) during in-clinic sessions.
The Consensus Error Grid Analysis showed 99.7% (in-clinic) and 99.8% (in-clinic and home use) of values within zones A and B and no values in zones D and E (Figure 4).
Consensus Error Grid during in-clinic sessions (a) and in-clinic sessions and home use combined (b).
Sensor Survival
Of 166 inserted CGM sensors, 26 (16%) were not started and were removed before the end of the warm-up time. Reasons for premature removal included obvious or suspected insertion failures, adhesive failures, or connection failures. One sensor did not deliver any valid values after the warm-up time. Therefore, 139 (84%) sensors that delivered at least one valid value were included in the survival analysis (Figure 5). If sensor data recording stopped due to a technical malfunction, it was considered as a sensor failure. In contrast, sensors that ended prematurely due to misfortune (eg, pulling out the sensor while changing clothes) and sensors that were removed due to the end of the study procedures were regarded as censored in the context of the Kaplan-Meier survival analysis. In total, 109 sensors completed and delivered values for the whole 14-day wear period, and the estimated 14-day survival probability was 82.4%.
Kaplan-Meier curve including all sensors that delivered at least one valid value (n = 139). Censored sensors are displayed as vertical dashes.
Data recovery rate with regard to the actual run time (first recorded value until sensor removal or failure) of each sensor was 92.7 ± 3.7%.
Safety
During the study, 15 adverse events related to the CGM system were recorded. All these events were related to the insertion site, such as itching, erythema, inflammation, bruising, bleeding, or hematoma. Of those, ten were classified as mild, two as moderate, and three as severe. None of the events was classified as serious, and all events resolved without sequelae.
Pain upon sensor insertion was rated on a scale from 0 (no pain) to 10 (highest imaginable pain). The mean rating was 1.7 ± 1.6, whereas 88% of 166 insertions induced no to mild pain (rating of 0-3). Bleeding was documented for 28% of insertions, with 20% being mild, 7% moderate, and 1% severe (two of the three severe adverse events were cases of bleeding for more than 5 minutes).
Discussion
This pivotal study evaluated performance and safety of a CGM device in adults with diabetes for 14 days. The system showed a good performance with an in-clinic 20/20 AR of 90.5%, an overall MARD of 9.2%, and a low systematic bias compared with capillary BG comparison measurements. This level of accuracy could be shown over the entire measurement range, especially in the low glycemic range, and the whole wear time of the sensors. The expected 20/20 AR of at least 84% used in the sample size calculation was exceeded by far. Individual sensor MARDs, however, showed that there were six markedly less accurate sensors with an MARD >20%. Five of these sensors had a short survival time, which speaks for an effective failure detection and subsequent shutdown of faulty sensors by the system. The CG-DIVA graphically displays the low bias and acceptable precision of CGM readings within different glucose ranges and highlights the low sensor-to-sensor variability. The accuracy during in-clinic sessions and during the home-use setting was very similar demonstrating the robustness of the system.
Successfully inserted sensors showed a survival rate of more than 80% in this study and a data recovery rate of more than 90%. There were no safety concerns regarding the use of the CGM system, and pain and bleeding upon insertion were within expectations. Since an in silico study concluded that it is feasible to use a CGM system with an MARD <10% for therapeutic decisions (including insulin dosing), 9 the tested CGM system can be regarded as safe for non-adjunctive use based on this study data.
There are many current performance studies evaluating other CGM systems. However, due to different study procedures, comparability of results between the studies is limited. In particular, variation is seen in regard to glucose and RoC distribution, comparator methods, and calculation of endpoints (eg, without a cut-off between absolute and relative differences in ARs). As described in a very recent publication, harmonized study procedures for CGM performance evaluations are desirable. 10
Alva et al 11 reported a similar 20/20 AR (but with a cut-off of 80 mg/dL between absolute and relative values) of 93.1% and an MARD of 9.2% for Abbott’s FreeStyle Libre 2 System compared with venous laboratory measurements in adults. From approval documents of this system, the mean absolute rate of change can be estimated as similar to the system evaluated in this study.12,13 Similar results were also described for Dexcom G6 with a 20/20 AR (cut-off of 80 mg/dL) of 92.3% and an MARD of 10.0% reported; estimated mean absolute RoC was slightly higher than in this study.14,15 Approval studies for the current generation systems of these two manufacturers apparently incorporated study procedures leading to lower RoCs, but demonstrated higher accuracy.16,17
Some limitations should be considered when interpreting the results of this study. First, the mean absolute comparator RoC was 0.86 mg/dL/min, which is similar to pivotal studies with other CGM systems as discussed above but in the middle compared with other CGM performance studies from the last years. 13 In general, more stringent glucose manipulation procedures to induce dynamics covering a sufficient number of higher RoCs would be desirable in clinical studies to analyze CGM performance as a whole. Second, many sensors were discarded directly after insertion because of suspected insertion failures. For those sensors, no performance data are available and they could not be included in the survival analysis. This study included considerably fewer participants than other pivotal studies. However, this was compensated by using three sensors per participant. Therefore, statistical independence can only be assumed for sensor-specific factors like sensor noise or batch variability. Potential subject-specific influences cannot be fully ruled out.
In this study, capillary samples were measured with a BG meter to obtain comparator values. Although many pivotal studies incorporate venous sampling, 13 capillary comparator values were deemed appropriate in this study for several reasons. First, the CGM system is manually calibrated using capillary BG values, and thus, CGM values are compared with samples from the same origin as used for calibration. Second, readings of the CGM system are intended to be an estimation of capillary BG levels because users are familiar with capillary BG values and all therapy guidelines are based on these kinds of values. Furthermore, capillary blood sampling, which requires only small volumes, is less burdensome for study participants over longer time periods than placement of a venous catheter and high-frequency draws of venous blood.
In summary, this study’s results support the intended non-adjunctive use of the examined CGM system (after initial calibration routine) and demonstrated similar results as the factory-calibrated Dexcom G6 and FreeStyle Libre 2 CGM systems.
Beyond the presented characteristics and results, the CGM system should be considered as part of a comprehensive solution, in this case, combined with the recently introduced Accu-Chek SmartGuide Predict app and the cloud-based Accu-Chek Care platform. Since the app indicates the risk of hypoglycemia during the next 30 minutes and during nighttime, provides glucose predictions for the next 2 hours to people with diabetes, and provides insights to glucose patterns, the transfer of CGM data to the platform enables health care professionals to gain instant access to glucose data of people with diabetes in their care to support them in their diabetes management.
Conclusions
In this study, 48 participants wore three sensors each of the CGM system for 14 days. The accuracy of the CGM system was good and comparable to that of previous-generation CGM systems of established manufacturers. Overall, very promising 14-day performance results of the CGM system in people with diabetes were demonstrated.
Footnotes
Acknowledgements
The authors thank the study participants and the staff in the study centers. The authors also thank Dr Stefan Pleus (Institut für Diabetes-Technologie GmbH) for his valuable feedback in interpreting and discussing the results, Dr Hans-Martin Kloetzer (RDC) for his great support in study data management, Dr Christian Ringemann (RDC) for his expertise in algorithm development and discussing the results, and the technical study coordinator Andreas Seidel (RDC) for his great contribution in preparation and conduct of the study.
Abbreviations
AID, automated insulin delivery; AR, agreement rate; BG, blood glucose; CG-DIVA, continuous glucose deviation interval and variability analysis; CGM, continuous glucose monitoring; CSII, continuous subcutaneous insulin infusion; MAD, mean absolute difference; MARD, mean absolute relative difference; MDI, multiple daily injections; RDC, Roche Diabetes Care; RoC, rate of change.
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: JKM is a member in the advisory boards of Abbott Diabetes Care, Becton-Dickinson/Embecta, Biomea, Eli Lilly, Medtronic, Novo Nordisk, PharmaSens, Roche Diabetes Care, Sanofi, and Viatris; received speaker honoraria from Abbott Diabetes Care, A. Menarini Diagnostics, Becton-Dickinson/Embecta, Boehringer-Ingelheim, Eli Lilly, MedTrust, Novo Nordisk, Roche Diabetes Care, Sanofi, Servier, and Ypsomed; and is shareholder of decide Clinical Software GmbH and elyte Diagnostics GmbH. GF is general manager and medical director of the IfDT (Institut für Diabetes-Technologie Forschungs- und Entwicklungsgesellschaft mbH an der Universität Ulm, Ulm, Germany), which carries out clinical studies, eg, with medical devices for diabetes therapy on its own initiative and on behalf of various companies. GF/IfDT have received speakers’ honoraria or consulting fees in the last three years from Abbott, Berlin Chemie, Boydsense, Dexcom, Lilly Deutschland, Novo Nordisk, Perfood, PharmaSens, Roche, Sinocare, Terumo, and Ypsomed. ME and DW are employees of IfDT. WM-H, KM, MA, GV, and CCR are employees of Roche Diabetes Care GmbH, Mannheim, Germany. TF is Chief Medical Officer and Chief Operations Officer of Clinical Research Services, Mannheim, Germany. TF received speaker honoraria or consulting fees in the last three years from Amarin, Astra Zeneca, Bayer, Boehringer-Ingelheim, Cipla, Daiichi-Sankyo, derCampus, Diabetes Academy Bad Mergentheim, Eli Lilly, Fortbildungskolleg, MSD, Novo Nordisk, Roche Diagnostics, Sanofi, Santis, and Sciarc. TF serves as associate editor for Endocrinology, Diabetes & Metabolism.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study and scientific writing were funded by Roche Diabetes Care GmbH, Mannheim, Germany.
