Performance and System Validation of a New Cellular-Enabled Blood Glucose Monitoring System Using a New Standard Reference Measurement Procedure of Isotope Dilution UPLC-MRM Mass Spectrometry

Abstract

Background:

We evaluated the accuracy, precision, and linearity of the In Touch® blood glucose monitoring system (BGMS), a new color touch screen and cellular-enabled blood glucose meter, using a new rapid, highly precise and accurate ¹³C₆ isotope-dilution liquid chromatography-mass spectrometry method (IDLC-MS).

Methods:

Blood glucose measurements from the In Touch® BGMS were referenced to a validated UPLC-MRM standard reference measurement procedure previously shown to be highly accurate and precise. Readings from the In Touch® BGMS were taken over the blood glucose range of 24-640 mg/dL using 12 concentrations of blood glucose. Ten In Touch® BGMS and 3 lots of test strips were used with 10 replicates at each concentration. A lay user study was also performed to assess the ease of use.

Results:

At blood glucose concentrations <75 mg/dL 100% of the measurements are within ±8 mg/dL from the true reference standard; at blood glucose levels >75 mg/dL 100% of the measurements are within ±15% of the true reference standard. 100% of the results are within category A of the consensus grid. Within-run precision show CV < 3.72% between 24-50 mg/dL and CV<2.22% between 500 and 600 mg/dL. The results show that the In Touch® meter exceeds the minimum criteria of both the ISO 15197:2003 and ISO 15197:2013 standards. The results from a user panel show that 100% of the respondents reported that the color touch screen, with its graphic user interface (GUI), is well labeled and easy to navigate.

Conclusions:

To our knowledge this is the first touch screen glucose meter and the first study where accuracy of a new BGMS has been measured against a true primary reference standard, namely IDLC-MS.

Keywords

accuracy blood glucose In Touch® BGMS self-monitoring isotope dilution mass spectrometry standardization

In the past 20 years there has been an explosion in the commercialization and marketing of over-the-counter blood glucose meters to help in the management of diabetes, a growing epidemic. It is known that the use of these blood glucose meters has beneficial effects in improving outcomes and maintaining glycemic control. It is therefore critical that these meters are easy to use so as to reduce user error, that they are accurate and precise, and that the results can be readily shared with the user’s health care team to render treatment decisions. Limitations in any of these features can impair the user’s ability to optimize self-management of their diabetes. In particular, difficulty in use and/or measurement inaccuracy can substantially impact glycemic control and in some cases potentially lead to unnecessary hypo/hyperglycemic events that result in ER visits and hospitalization, all of which are traumatic and have long term consequences on the quality of life.^1-4 Developing a blood glucose meter that is easy to use and has a graphic user interface (GUI) that is interactive, with ergonomic and human factors engineered that limit the number of multifunction buttons that need to be pressed, is important so that the user’s sequence of operation to obtain an accurate blood glucose reading is easier and reduces the risk of user error. In addition, establishing the accuracy and the reproducibility of this accuracy of glucose readings across the range of meter use is extremely important under normal everyday use in the self-management of diabetes so that patterns can be recognized and self-management decisions can be made with confidence.

A recent change in reimbursement rates for blood glucose meters and test strips in the United States has resulted in a shift to the distribution of more inexpensive meters and test strips to people with diabetes. Although these meters meet the FDA standards for clearance, there have been several reported instances of their inaccuracy postmarketing that raise issues of the safety of their use.^5,6 Until recently there have been no postmarketing surveillance programs that monitor the accuracy and use of blood glucose meters that once sold are in the hands of users although some new programs have begun to highlight the issue.^6-8 The current standard for clearance is that 95% of the blood glucose results must be within ±20% of a reference standard at blood glucose values greater than 75 mg/dL and within ±15 mg/dL for those less than 75 mg/dL. This means that under current FDA standards, a reading of 50 mg/dL could actually be 65 mg/dL or 35 mg/dL, the latter of which could have dangerous consequences.

The FDA has clearly recognized that accuracy is an important issue and in the guidance document issued January 14, 2014, has proposed higher standards for clearance and postmarket surveillance stating⁹ “Blood glucose test results are used by people with diabetes to make critical decisions about their treatment; therefore, it is important that the results are accurate so that nutritional and drug dosing errors are better avoided. To demonstrate that your SMBG device is sufficiently accurate to be used safely by diabetic patients for this purpose, you should demonstrate that 95% of all SMBG results in this study are within +/- 15% of the reference measurement across the entire claimed measuring range of the device and that 99% of all SMBG results are within +/- 20% of the reference measurement across the entire claimed measuring range of the device.” The current ISO 15197: 2013 minimum system accuracy criteria call for “95% of the measured glucose values shall fall within either ±15 mg/dl of the average measured values of the reference measurement procedure at glucose concentrations <100 mg/dl or within ± 15% at glucose concentrations ≥100 mg/dl and 99% of individual glucose measured values shall fall within categories A and B of the consensus error grid (CEG) for type 1 diabetes.”¹⁰

To date the most widely used “standard reference measurement procedure” use enzyme-based instruments like the YSI Glucose STAT analyzer which use the very same enzyme reactions used in the glucose meters being tested. However, these “reference measurements” are strictly comparative methods or comparative measurements rather than a true orthogonal and primary reference method which should be used to determine accuracy. In fact it has been reported that the YSI method is ~8% negatively biased for plasma glucose compared to a clinical laboratory improvement amendment recognized method.¹¹ In part, some of the “inaccuracies” may be due to the “corrections” for hematocrit needed to convert blood glucose values to the plasma glucose values displayed by glucose meters. The National Institute of Standards and Technology (NIST) assay for glucose in human serum uses isotopic GC-mass spectrometry as a standard reference measurement procedure.¹² GC-mass spectrometry, however, is laborious, with limited throughput that cannot readily be used in routine blood or serum glucose determinations as a reference standard for the assessment of blood glucose meter performance by most BGMS manufacturers. In the previous article,¹³ we reported on the development of a highly accurate and high throughput isotope-dilution liquid chromatography-mass spectrometry (ID-LC/MS) assay. In this article we have used this primary reference method to establish the accuracy of a new cellular-enabled glucose meter from Livongo Health, the In Touch® blood glucose monitoring system (BGMS). We have also detailed other performance characteristics of this new BGMS, and show that it represents a new generation of accurate, precise, and user-friendly meters for use by people with diabetes in their daily self-management.

The Livongo In Touch® BGMS

The design of the In Touch® BGMS (Livongo Health, Mountain View, CA) (Figure 1) was guided by human factors and usability engineering principles, with the goal of making self-monitoring of blood glucose easier. The concept was to design away the multifunction buttons on currently marketed glucose meters, the small and limited resolution black and white LCD display, and to eliminate written logbook or the uploading of blood glucose results through a cable into a PC, all of which are cumbersome and can add to the frustrations of regular checking of blood glucose.

Figure 1.

The In Touch® BGMS by Livongo. The In Touch® BGMS includes Check, Trends, Messages, Team, and Activity all selected through the color touch screen.

Accordingly, the In Touch® BGMS has no buttons but rather a color touch screen with colored panels and icons that correspond to each of its unique functions which users can simply tap to access (Figure 1). These functions include (1) CHECK, which initiates the sequence of checking blood glucose and includes a postresult sequence that tag the blood glucose value with mealtimes, carbs, and/or insulin, and how the user “feels” so the blood glucose values and trends are placed in context; (2) TRENDS, which is an on-board color tabular display of daily (hourly), 7, 14, 30, and 60 days logbook values, before and after meals, patterns and stats, and snapshot summary of important results like average, standard deviation, percentage within goals, over 14, 30, or 60 days, all of which eliminate pressing buttons and scrolling through the results as on current meters; (3) TEAM, which allows users to share and route their data to their designated and selected care team, a coach, their physician, or family member either by email, fax, or secure text; (4) MESSAGES, which provides 2-way messaging between care team and the user at the time of checking with messages that provide periodic general educational information, reminders (eg, running out of strips), or motivational guidance; and (5) ACTIVITY, which uses the built in accelerometer of In Touch® to measure the user’s steps and activity. When the user is finished checking their blood glucose, the data complete with tags and any activity information is sent automatically and securely through a proprietary encrypted algorithm through the GSM cellular network into the user’s secure cloud for review by the user and the user’s health care team. In addition, date and time are automatically set to where the user is anywhere in the world via the cellular network so that the user’s values are always correctly time-stamped.

In this article we assess the accuracy of the In Touch® BGMS using a true reference standard, IDLC-MS, which we have shown in the previous article¹³ has excellent traceability, sensitivity, and accuracy. We also include an assessment of precision (both within-run and intermediate), and linearity, and of the human factors–user experience of the system. The In Touch® BGMS was cleared by the FDA in September 2014 (k133584).

Methods

Preparation of Samples for Testing

Readings from were taken over the blood glucose range of 24-620 mg/dL, using whole blood spiked with exogenous glucose to cover this range. Twelve concentrations of blood glucose were used, as were 2 control solutions, level 1 and level 2. To achieve blood glucose concentrations less than 40 mg/dL, the samples were allowed to incubate in a water bath at 37C° and checked periodically for basal/endogenous glucose concentration by IDLC-MS.

Ten In Touch® BGMS and 3 lots of test strips were used (lots 890014, 890016, 890019) with 10 replicates at each concentration. Readings were analyzed for accuracy using the IDLC-MS determined values as reference standard, and for precision (within and between run), and linearity. For intermediate precision across multiple days, blood glucose values were measured at each of the concentration intervals twice daily for 15 days, for a total of 120 measurements per glucose level each day. Prior to the start of each test set, meters were verified that they were operating properly using the appropriate method per the owner’s manual. Test conditions were at 24.5-26.5ºC, with a humidity of less than 85%. One of the meters displayed the temperature which was recorded at the beginning and end of each test. The temperature did not deviate more than 4°C during the whole course and day of testing.

Isotope-Dilution LC/MS Standard Reference Method

An isotope dilution UPLC-MRM method was used as the standard reference measurement procedure, against which meter accuracy was established. In brief, the instrument set-up consisted of a Waters tandem quadrupole (TQ) mass spectrometer directly coupled in-line to a Waters acquity ultra performance liquid chromatography (UPLC) system and optimized in the multiple reaction-monitoring (MRM) mode using negative ion electrospray to quantify 4 unique precursor/product molecular ion transitions from both endogenous blood glucose levels and a heavy-isotope spiked internal standard consisting of ¹³C₆-D-glucose. The details are as described in the previous article.¹³

Data Analysis

For precision, the mean, standard deviation at 95% CI, and coefficient of variation were calculated at each glucose concentration. ANOVA, with a post hoc t test, was performed to determine any variation between meters. For analysis of BGMS accuracy, data were analyzed according to ISO 15197:2003 and ISO 15197:2013 and defined by “the closeness of agreement between the measured quantity value and a true quantity value of the measurand”^14-16 by measuring the difference between each individual measure by In Touch® and the mean of the reference standard and analyzed for correlation by regression analysis using Bland–Altman method.¹⁷

The results are reported as bias against the IDLC-MS reference standard and displayed in a Consensus Error grid according to ISO 15197:2013. Results were also compared to those done side by side with the YSI Glucose 2300 STAT analyzer.

All data were included in the analysis. All other statistical treatments were carried out as described.^14,15,18

Human Factors Study

In Touch® BGMS kits were provided to lay users. They had the opportunity to unwrap the sealed box and read the “About In Touch®” guide that is included in the box to get started. They then checked their blood glucose, tagged the results, submitted the results to their cloud-based In Touch® web-based data management system (DMS), and then viewed their results in their secure personal account. Immediately following the blood glucose test and viewing of their results, users were asked to respond to a survey of 80 questions on the physical appearance and function, features, use of the system, and their experience with the use of the DMS. Fifty-nine individuals completed the survey. Participants were between the ages of 14 and 85 years old, with 58% women and 42% men.

Results and Discussion

Validation of the Reference Method

The IDLC-MS method was validated as described in the previous article.¹³ The IDLC-MS method is both accurate (within 1% uncertainty) against the NIST SRM 965b and 909c glucose in serum reference standards and precise (CV ≤ 1.6%) and meets the traceability criteria of a primary method of measurement and reference standard.¹⁶

In Touch® Performance: Accuracy

The accuracy of the In Touch® BGMS was examined for the 2 criteria established by ISO 15197:2013: analysis of the bias of the individual measured values against a true reference standard, measured in our case by the IDLC-MS procedure (Figure 2), and through the consensus error grid⁹ of ISO 15197:2013 (Figure 3). Figure 2 shows that 100% of the values fall within ±15 mg/dL and exceeds the minimum 95% criteria of ISO 15197:2013.

Figure 2.

Bias of the measured values on In Touch® BGMS to the IDLC-MS reference standard. (A) Bland–Altman difference plot, in mg/dL to evaluate agreement and accuracy between In Touch® and the IDLC-MS reference standard. The horizontal dashed lines represent the ISO 15197: 2013 standards with 95% of the values falling within ±15 mg/dL throughout the range. (B) Bland–Altman difference plot,% bias. The dashed horizontal lines at 20% represent the current FDA standards at blood glucose >75 mg/dL for meter accuracy. (A) and (B) show that at blood glucose concentrations <75 mg/dL 100% of the blood glucose values measured by the In Touch® meter are within ±8 mg/dL from the true reference standard; at blood glucose levels >75 mg/dL 100% of the In Touch® measurements are within ±15% of the true reference standard with most data within ±10%, meeting and exceeding the minimum standard criteria of both the ISO 15197:2003 and 15197:2013 standards. All data were included in the analysis, and no points were excluded.

Figure 3.

Bench and clinical accuracy of the In Touch® BGMS. The measured values were placed in a consensus error grid with the IDLC-MS reference glucose values. The consensus grid divides the results into 5 risk categories. Each risk category represents the significance of the error in a glucose reading as it relates to making a clinical decision based on the glucose reading. Of the results, 100% are within category A of the consensus grid, indicating 100% of the glucose readings obtained in the study and for all of the In Touch® BGMS are within the clinically accurate category A. Panel A presents results from bench experiments, and panel B presents lay user studies that show that 100% of time first-time lay users are able to obtain results within the clinically accurate category A the first time.

Figure 3 shows the accuracy of the In Touch® BGMS in a consensus grid measured against the IDLC-MS reference standard at selected concentrations of blood glucose in side by side bench testing (panel A) and by first-time lay users (panel B) using finger sticks; 100% of the results are within category A of the consensus grid. The results show that accuracy is well within the In Touch(R) ISO 15197:2003 and ISO 15197:2013 minimum standard criteria (100% of the values within category A and category B) using the stringent IDLC-MS reference standard.

When the bench and lay accuracy results are compared at blood glucose levels ≤75 mg/dL and blood glucose levels ≥75 mg/dL, the data show that there is excellent correlation between the bench and lay user accuracy, with 100% of the lay values falling within ±15 mg/dL across all ranges and not less than 83% within ±5 mg/dL at values ≤75 mg/dL and greater than 83% at values ≥75 mg/dL (Figure 4). The data are also represented in the form of both ISO 15197: 2003 and ISO 15197: 2013 (Table 1).

Figure 4.

Bench and lay use of the In Touch® BGMS. Panel A shows the agreement between the bench and lay user accuracy at ≤75 mg/dL and panel B at ≥75 mg/dL.

Table 1.

Accuracy of the In Touch® BGMS According to ISO 15197:2003 and 15197:2013, All Lots, All Meters Combined.

Table 1: ISO 15197:2003
At <75 mg/dL			At >75 mg/dL
±15 mg/dL	±10 mg/dL	±5 mg/dL	±20%	±15%	±10%	±5%
100%	100%	98%	100%	100%	98.5%	87%
(600/600)	(600/600)	(588/600)	(2905/2905)	(2905/2905)	(2861/2905)	(2530/2905)
Table 1: ISO 15197: 2013
At <100 mg/dL			At >100 mg/dL
±15 mg/dL	±10 mg/dL	±5 mg/dL	±15%	±10%	±5%
100%	100%	93.3%	100%	99.2%	87.7%
(900/900)	(900/900)	(840/900)	(2605/2605)	(2583/2605)	(2285/2605)

Repeatability and Intermediate Precision

When the repeatability and reproducibility were examined (with accuracy in measured readings confirmed against the IDLC-MS reference standard), the data show excellent repeatability and reproducibility (Table 2). In repeatability studies, the CV for blood glucose values ≥100 mg/dL range from 1.97% to 3.32%; the CV for blood glucose levels <100 mg/dL range from 3.41% to 3.71%, with a CV of 3.71% at 24.9 mg/dL (Table 2). For intermediate precision, the CV for all values is less than 5%. ANOVA analysis of the imprecision between meters and between groups shows that there was no statistically significant difference in imprecision between In Touch® meters and between groups of BGMS with P > .05 at 95% confidence interval and range of 0.06 to 0.83.

Table 2.

Precision, Within-Run, and Intermediate Results for the In Touch® BGMS: (A) Within-Run Precision as Measured at Blood Glucose Concentrations 25-620 mg/dL; (B) Intermediate Precision, 15-Day Interval.

A
Interval	Standard, mg/dL mean glucose MS	In Touch®
Interval	Standard, mg/dL mean glucose MS	Mean	SD	CV (%)
Interval 1	24.91	24.75	0.92	3.71
Interval 2	52.59	52.92	1.80	3.41
Interval 3	78.85	78.95	2.69	3.41
Interval 4	113.69	114.39	3.79	3.32
Interval 5	181.61	180.70	4.21	2.33
Interval 6	230.31	228.84	5.85	2.56
Interval 7	339.04	334.83	8.71	2.60
Interval 8	366.12	364.83	9.85	2.37
Interval 9	437.61	437.03	11.40	2.61
Interval 10	480.05	483.29	11.35	2.35
Interval 11	561.28	557.85	11.99	2.15
Interval 12	620.34	628.33	12.44	1.97
Control 1	124.50	126.75	4.03	3.17
Control 2	384.55	384.64	10.77	2.83
B
Interval	Standard, mg/dL mean (glucose) std	In Touch®
Interval	Standard, mg/dL mean (glucose) std	Mean	SD	CV (%)
Interval 1	26.20	27.87	0.81	2.91
Interval 2	55.10	55.73	1.87	3.36
Interval 3	76.20	78.83	2.99	3.79
Interval 4	117.30	119.97	3.90	3.25
Interval 5	177.30	176.90	6.48	3.66
Interval 6	225.40	234.40	8.57	3.66
Interval 7	315.30	323.07	11.70	3.62
Interval 8	361.20	367.67	11.90	3.24
Interval 9	437.20	432.39	13.97	3.23
Interval 10	471.50	474.67	16.50	3.48
Interval 11	562.30	556.00	17.23	3.10
Interval 12	625.30	631.34	18.10	2.87
Control 1	127.00	127.30	3.02	2.37
Control 2	379.00	381.00	10.13	2.66

Linearity

Evaluation of the linearity¹⁸ of the In Touch® BGMS relative to the IDLC-MS standard shows that the measured values from the BGMS are highly linear over the range of 10-672 mg/dL, with an average correlation coefficient of r² = .9992 over the 3 lots of strips and 10 meters tested (Table 3).

Table 3.

Linearity Over the Blood Glucose Range 10-672 mg/dL for the In Touch® BGMS.

	r ²	Res SD	Slope	SD slope	Intercept	SD intercept	Significance slope
Strip lot 1	.9995	4.322	1.004	0.026	–1.057	16.42	P < .0001
Strip lot 2	.9990	7.35	0.995	0.045	–2.311	10.68	P < .0001
Strip lot 3	.9991	5.65	1.004	0.049	–1.05	12.4	P < .0001

Comparison Between YSI Measured Values and MS Reference Standard

When we compare the plasma glucose values measured by the YSI STAT analyzer with the plasma glucose values obtained by the IDLC-MS reference method, we find that while the YSI values are repeatable and reproducible with CV between 0.59% to 3.52% over the blood glucose range of 23 mg/dL to 561 mg/dL (compared to IDLC-MS determined CV values of 0.09% to 0.82%), there is significant difference in the YSI measured accuracy vs the IDLC-MS reference standard across the range (Figure 5).

Figure 5.

Bias of the YSI method vs the IDLC-MS reference standard method. This figure shows the bias in mg/dL throughout the range 8-500 mg/dL. In these experiments the known amount of ¹³C₆-glucose was added to the plasma samples and measured side by side by the YSI Stat Analzyer and by IDLC-MS. It is clear from the data that despite good precision of the YSI method, relative to the IDLC-MS reference standard the accuracy of the YSI method shows considerable bias, with measurements up to ~+10% bias relative to the true standard. This bias is most notable at the low plasma glucose concentrations.

The results show that at plasma glucose concentrations <75 mg/dL the YSI Stat method has a mean bias of +4.87% and SD (95% CI) of 3.29% from the IDLC-MS reference standard and at plasma glucose concentrations >75 mg/dL the YSI Stat method has a mean bias of +4.05 and SD (95% CI) of 2.39%.

Ergonomics and Human Factors

Because ergonomic features have been shown to play an important part in proper use of some medical devices,^19-21 we also surveyed first-time lay users for the ease of use of the In Touch® BGMS including handling, size, buttons, display readout size, and the sequence of operations required to perform a correct reading.

The results from a user panel show that 100% of the respondents reported that the size, color screen with its GUI are well-labeled and easy to use. Of the respondents, 100% preferred the use of the touch screen to the multifunction buttons on currently marketed blood glucose meters. In addition recall of values and viewing of trends and patterns was easy for 100% of the respondents as was submitting values. Of the respondents, 100% indicated that the “About In Touch®” quick start guide was sufficient to take blood glucose measurements and to understand the functions of the In Touch® meter. Of the respondents, 75% found the web-based DMS intuitive and easy to use without the need for instruction (Table 4).

Table 4.

Human Factors Survey

Physical Appearance and Function	100% Yes
Is the On/Off button easy to find
Is the On/Off button a good size
Relative to the size of the In Touch® meter is the display a good size?
Is the slot to insert the blood glucose testing strip visible and easy to find
Is the size of the In Touch® easy to carry and nold
Is it easy to find the charging connector and to charge In Touch®
In Touch® Features
Is the iconpanel “Check” your blood glucose easy to find and use
Is the “Trends” icoin and pane easy to understand and use
Is the “Team” icon and panel easy to understand and use
Is the “Message” icon and panel easy to understand and use
Is the “Settings” icon easy to understand and use?
Is the “Activity” icon and panel easy to find, understand and use
Is the color touch screen display easy to view and to use
Use of In Touch®
Is the In Touch® easy to turn on and off
Was the In Touch® blood glucose meter easy to set up
Did you find the automatic setting of correct time and date easy
Compared to other meters you have used did you find that the In Touch® was easier
When you checked your blood glucose, were the Touch® screen prompts easy
Was your blood glucose value easy to read on the display
Was sending your results to your personal health account easy to do
If your results were not sent, were you adequately informed
For those results that were not transmitted, were thety available in Trends
Were retrieving and reading messages from your account easy to read
When you tested with control solution, was it easy and clear
Did In Touch® guide you through with easy prompts to test with control
Were error messages clearly displayed on In Touch®
Were error messages easy to understand in simple languagge when displayed
In Settings was it easy and clear how to set reminders
In Settings was it easy and clear hbow to set your medications
In Setttings in the “About” panel was it clear to find contact and device information
In Trends could you view and access your blood glucose values easily
In Trends were the blood glucose results from 7,14,30, and 90 days available
Were your results in the Tables and graphs in the pattern and stats easy to read
Compared to other glucose meters was viewing of your blood glucose results easier
In Trends was the summary page easily found as well as a clear summary of your results
In Messages were your messages easily retrieved from your health account
In Messages were the messages easy to read and understand
In Team icon/panel was it clear and easy to send your results to your coach
In Team was it easy and clear how to send your results to your MD
Data Management System
Was the In Touch®, www.inTouch®.eoshealth.com web site easily accessible
Were you able to login to your personal accounts with a security password
Was the In Touch® web data management system clear and easy to view
Was the In Touch® web site and data management system easy to navigate
Did the In Touch® web data management system accurately match the blood glucose
Were you able to easily view your Trends and results on the In Touch® data system

Comparison to Other Blood Glucose Meters

Table 5 evaluates the performance of the In Touch® BGMS relative to other blood glucose meters available on the market today. The data for the other blood glucose meters were obtained from Freckmann et al²² and from the American Association of Diabetes Educators.²³

Table 5.

Comparison of Selected Commercially Available BGMS (Values in Parentheses Are Those Values According to ISO 15197: 2003)

	BG concentration <100 mg/dL (≤75 mg/dL)			BG concentration >100 mg/dL (≥75 mg/dL)
	± 15 mg/dL	±10 mg/dL	±5 mg/dL	±15%	±10%	±5%
Blood glucose system	%	%	%	%	%	%
Accuchek Nano-Roche^a	100 (100)	96 (100)	80 (84)	99 (100)	95 (94)	65 (65)
iBG Star^b	96 (100)	90 (99)	52 (57)	90 (90)	72 (74)	35 (36)
Contour USB-Bayer^a	90 (100)	69 (84)	45 (61)	91 (88)	68 (63)	34 (31)
Fora Bluetooth^b	83	41	14	65	40	14
Freestyle Lite—Abbott^a	100 (100)	100 (100)	93 (95)	100 (100)	100 (98)	86 (91)
Glucochek^a	97 (100)	84 (93)	55 (60)	55 (60)	66 (66)	38 (38)
Glucotel^a	89 (87)	70 (71)	33 (37)	92 (97)	72 (70)	41 (40)
In Touch®-Livongo	100 (100)	100 (100)	96 (98)	100 (100)	99 (99)	87 (87)
OneTouch® Verio Pro^a	88 (93)	53 (63)	21 (20)	93 (90)	75 (70)	40 (38)
Senior Line^a	15 (10)	0 (0)	0 (0)	79 (72)	55 (48)	29 (26)
Telcare BGM^c	(100)	(96)	(68)	(100)	(98)	(65)
iHealth Bluetooth BG^d	(100)	(95)	(68)	(95)	(74)	(53)
Wavesense Jazz-Agamatrix^a	100 (100)	75 (75)	38 (39)	95 (94)	80 (78)	47 (45)

From Freckmann et al.²²

From American Association of Diabetes Educators.²³

From FDA Decision Summary k110571.

From FDA Decision Summary k110017.

The results show that in spite of the more stringent IDLC-MS reference standard used to assess accuracy of the In Touch® BGMS, the meter performs very well compared to the others and is comparable to the Abbott Freestyle Lite, which uses an improved FAD-GDH enzyme system.

Conclusions

These are the first studies of a new BGMS where system accuracy has been measured against a truly orthogonal, highly accurate and precise reference measurement procedure, namely an IDLC-MS assay for glucose. In this study, we also utilized a well-documented calibration hierarchy that conforms to the metrological traceability requirements of ISO 17511,¹⁶ and is a true primary reference standard for the measurement of accuracy as demonstrated in the previous article.¹³ Given the current calls for more accurate reference standards as well as the need for postclearance surveillance,^6-8 we show that the In Touch® BGMS, as measured by using this IDLC-MS standard, meets and exceeds the minimum standard criteria of both the ISO 15197:2003 and 15197:2013 (yet to be adopted by the FDA) for BGMS accuracy. Given the critical nature of measuring accurate blood glucose values, particularly at low levels, this is the time for a gold standard.^24,25 We hope that other meters and systems on which people with diabetes rely will begin to adopt this IDLC-MS procedure as a standard reference system to ensure the determination of accuracy of BGMS used by people with diabetes.

In addition, the results show that the In Touch® BGMS is precise, highly linear in its reported values over its entire measurement range, and easy and straightforward to use with nominal set up steps required, unlike many of the current blood glucose meters on the market today. The color touch screen, easy to visualize icons, elimination setting the time and date manually, and the range of features including easy share with the user’s team are designed to help those with diabetes gain self-confidence and support in self-management of their diabetes. The results show that in addition to its ergonomic design and ease of use, the In Touch® BGMS performs at the highest standard measurable and establishes the In Touch® BGMS as a highly accurate and useful system.

Footnotes

Abbreviations

BGM, blood glucose monitor; BGMS, blood glucose monitoring system; CV, coefficient of variation; DMS, data management system; FAD-GDH, flavin adenine dinucleotide-glucose dehydrogenase; FDA, Food and Drug Administration; GC/MS, gas chromatography/mass spectrometry; GUI, graphic user interface; IDLC-MS, isotope dilution liquid chromatography–mass spectrometry; ISO, International Organization for Standardization; LC-MS, liquid chromatography/mass spectrometry; SD, standard deviation; SMBG, self-monitoring of blood glucose; UPLC, ultra performance liquid chromatography; YSI, Yellow Springs Instruments.

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: KA is a full-time employee of Livongo Health, Inc. and is the inventor on U.S. Patents 8,066,640, 8,568,309, 8,812,244, D726,205-210, D726,751-757, D727, 941-2, and D728,661, covering design and operation of the In Touch® BGMS.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was funded by Livongo Health, Inc and the Methodist Hospital Research Institute.

References

Breton

Kovatchev

. Impact of blood glucose self-monitoring errors on glucose variability, risk for hypoglycemia, and average glucose control in type 1 diabetes: an in silico study. J Diabetes Sci Technol. 2010;4(3):562-570.

Budiman

Samant

Resch

. Clinical implications and economic impact of accuracy differences among commercially available blood glucose monitoring systems. J Diabetes Sci Technol. 2013;7(4):365-380.

Geller

Shehab

Lovegrove

. National estimates of insulin-related hypoglycemia and errors leading to emergency department visits and hospitalizations. JAMA Intern Med. 2014;174(5):678-686.

Curkendall

Zhang

Williams

Pollack

Graham

. Incidence and cost of hypoglycemia among patients with Type 2 diabetes in the United States: analysis of a health insurance database. J Clin Outcomes Manage. 2011;18(10):455-462.

Klonoff

Reyes

. Do currently available blood glucose monitors meet regulatory standards? 1-day public meeting in Arlington, Virginia. J Diabetes Sci Technol. 2013;7:1071-1083.

Klonoff

. DTS cleared blood glucose monitor surveillance program: payer’s meeting. One day public meeting on performance, clinical outcomes from poorly performing blood glucose monitors, and need for surveillance for cleared blood glucose monitors. November 5, 2014.

Klonoff

Lias

Vigersky

. Error grid panel. J Diabetes Sci Technol. 2014;8(4):658-672.

AACE/ACE consensus conference on glucose monitoring. Washington, DC; September 28-29, 2014.

FDA draft guidance document: self-monitoring blood glucose test systems for over-the-counter use, draft guidance for industry and Food and Drug Administration staff; January 7, 2014.

10.

ISO 15197, 2nd ed., 2013-05-15. In vitro diagnostic test systems—requirements for blood glucose monitoring systems for self-testing in managing diabetes mellitus.

11.

Twomey

. Plasma glucose measurements with the Yellow Springs Glucose 2300 STAT and the Olympus AU640. J Clin Pathol. 2004;57:752-754.

12.

White

Welch

Sun

. The accurate determination of serum glucose by isotope dilution mass spectrometry—two methods. Biomed Mass Spectrom. 1982;9:395-405.

13.

Matsunami

Angelides

Engler

. Development and validation of a rapid 13C6-glucose isotope dilution UPLC-MRM mass spectrometry method for use in determining system accuracy and performance of blood glucose monitoring devices. J Diabetes Sci Technol. 2015:9:1051-1060.

14.

ISO 15725-1. Accuracy (trueness and precision) of measurement methods and results—part 1: Introduction and basic principles.

15.

Evaluation of precision performance of quantitative measurement methods; approved guideline-second edition; CLSI document EP5-A2. Wayne, PA: Clinical and Laboratory Standards Institute; 2004.

16.

ISO 17511:2003. Metrological traceability of values assigned to calibrators and control materials.

17.

Bland

Altman

. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986476:307-310.

18.

CLSI document EP6-A. Evaluation of the linearity of quantitative measurement procedures: a statistical approach v23, no 16. 2003.

19.

Liberman

Buckingham

Phillip

. Diabetes technology and the human factor. Int J Clin Pract Suppl. 2011;(170):83-90.

20.

NSI/AAMI HE48: human factors engineering guidelines and preferred practices for the design of medical devices. 1993.

21.

Story

. 2012. FDA perspective in human factor in device development. RAPS Webinar. June 7, 2012.

22.

Freckmann

Schmid

Baumstark

Pleus

Link

Haug

. System accuracy evaluation of 43 blood glucose monitoring systems for self-monitoring of blood glucose according to DIN EN ISO 15197. J Diabetes Sci Technol. 2012;6:1060-1075.

23.

American Association of Diabetes Educators. Practice advisory. Blood glucose meter accuracy. September 16, 2013.

24.

Hagvik

. Glucose measurement: time for a gold standard for glucose measurements. J Diabetes Sci Technol. 2007;2:169-172.

25.

Tonyushkina

Nichols

. Glucose meters: a review of technical challenges to obtaining accurate results. J Diabetes Sci Technol. 2009;3:971-980.