Abstract
Objective:
The objective was to simulate and compare clinical and economic outcomes of self-monitoring of blood glucose (SMBG) devices along error ranges and strip price.
Methods:
We programmed a type 1 diabetes natural history and treatment cost-effectiveness model. In phase 1, using past evidence from in silico modeling validated by the Food and Drug Administration, we associated changes in SMBG error to changes in hemoglobin A1c (HbA1c) and separately, changes in severe hypoglycemia requiring an inpatient stay. In phase 2, using Markov cohort simulation modeling, we estimated clinical and economic outcomes from the Canadian payer perspective. The primary comparison was a SMBG device with strip price $0.73 Canadian dollars (CAD) and 10% error (exceeding accuracy requirements by International Organization for Standardization (ISO) 15197:2013) versus a SMBG device with strip price $0.60 CAD and 15% error (accuracy meeting ISO 15197:2013). Outcomes for the average patient, were quality-adjusted life years (QALYs), incremental cost-effectiveness ratios (ICERs), and budget impact.
Results:
Assuming benefits translate into HbA1c improvements only, the ICER with 10% error versus 15% was $11 500 CAD per QALY. Assuming the benefits translate into reduced severe hypoglycemia requiring an inpatient stay only, an SMBG device with 10% error dominated (ie, less costly, more effective) an SMBG device with 15% error. The 3-year budget impact findings ranged from $0.004 CAD per member per month for HbA1c improvements to cost-savings for severe hypoglycemia reductions.
Conclusions:
From efficiency (cost-effectiveness) and affordability (budget impact) payer perspectives, investing in devices with improved accuracy (less error) appears to be an efficient and affordable strategy.
Glucose monitoring, frequent hemoglobin A1c testing (HbA1c), and intensive insulin treatment are of critical importance for managing type 1 diabetes. 1 Patient self-monitoring of blood glucose (SMBG) requires daily tracking of glucose concentrations, and should be carried out multiple times per day for patients using intensive insulin therapy. Frequent SMBG improves glycemic control and assists in avoiding acute complications such as hypoglycemia.1,2
However, not all SMBG devices have the same accuracy which can lead to inaccurate insulin dosing. Given that clinical trials assessing the relationship between SMBG accuracy and glycemic control are difficult to design and conduct, Breton and Kovatchev employed a realistic computer simulation to provide evidence on the relationship between SMBG accuracy and glycemic control. 3 Specifically, the authors found that as SMBG errors increase, HbA1c increases or separately, the risk of hypoglycemic episodes increase. Moreover, past evidence has suggested an association between improved glycemic control and cost savings as well as improved quality of life.4,5
Health Canada issued a notice on September 26, 2014 that it now recognizes International Organization for Standardization (ISO) 15197:2013 and has granted manufacturers a one-year transition period (to September 2015) to conform to the 2013 standards for new devices. ISO 15197:2013 differs from the previous 2003 version on the following points: increased accuracy for glucose meter systems, in particular for glucose values greater than 75 mg/dl (4.2 mmol/l); manufacturers of glucose meter systems must ensure their technology enables accuracy to improve from ±20% to ±15%; the new version accounts for 99% of results, as opposed to 95% for the previous 2003 version. Previously licensed devices are not subject to the new requirement.
There is a lack of evidence addressing the impact of SMBG device errors on long-term clinical and economic outcomes. We found only 1 previous analysis associating changes in SMBG error to economic outcomes applied to the German setting. 6 Using a 2-phased approach (Figure 1), the objective of this study was to estimate the cost-effectiveness of blood glucose monitoring devices with varying SMBG error (ie, SMBG devices meeting [15% error] and exceeding [10% error] standards allowed by ISO 15197:2013) and strip price from the Canadian payer perspective.

Evidence flow chart. Differences in the accuracy of self-monitoring blood glucose devices (SMBG) are associated with changes in HbA1c and hypoglycemia. 3 The analysis links these changes in hemoglobin A1c (HbA1c) and hypoglycemia to long-run diabetes disease progression. Disease progression in turn impacts costs, clinical event rates, and quality-adjusted life years (QALYs). Cost-effectiveness, budget impact, and cost-utility analyses are performed to investigate the long-run impact of accuracy differences in blood glucose monitoring devices.
Methods
Changes in SMBG Accuracy Associated With Short-Term Changes in HBA1c and Hypoglycemia
Breton and Kovatchev investigated the impact of errors from SMBG devices on multiple measures of glycemic control using in silico (computer) modeling validated by the Food and Drug Administration. 3 These errors were linked with changes in HbA1c or separately with changes in hypoglycemia, for example, a 5% error resulted in a 0.01% absolute increase in HbA1c, 10% error resulted in a 0.12% absolute increase in HbA1c, and 15% error resulted in a 0.26% absolute increase in HbA1c.
Two sources are used to generate rates of severe hypoglycemia requiring an inpatient stay per patient per year. First, Breton and Kovatchev associated errors from SMBG devices with changes in hypoglycemic events defined between 50 mg/dl and 70 mg/dl. 3 We used the information from Breton and Kovatchev to generate a relative risk between devices with different SMBG errors and applied those relative risks to a baseline number of events per year for each device. Second, we used past modeling evidence that links HbA1c to the number of baseline severe events (event per patient per year = exp(0.928 + [HbA1c*-0.113]), 7 and applied the relative risks to the baseline number of events to estimate per patient per year hypoglycemic events. With a baseline HbA1c of 7.6%, 8 the relative risk for an SMBG device with 10% versus 15% error was 0.84 applied to a baseline number of events of 1.024 per patient year (ie, 0.86 events per patient year associated with 10% error and 1.024 per patient year associated with 15% error).
Markov Model Structure
The Markov cohort simulation model estimates long-run clinical and economic outcomes for the average adult type 1 diabetes patient at an age of 50 years old with a mean duration of diabetes at 15 years and baseline HbA1c of 7.6%.9,10 The model reflects the biological process of type 1 diabetes and is applicable to a wide range of treatment settings. The model structure includes the major diabetes complication states categorized by the American Diabetes Association (ADA): coronary heart disease (CHD), nephropathy, neuropathy, retinopathy, and end-stage renal disease. A similar structure of disease progression has been used in many other validated diabetes modeling studies.7,10-12 The model structure uses a cycle length of 1 year with varying time horizons based on each scenario. Lifetime costs and outcomes are compared across SMBG error levels focusing on a SMBG device with 10% error as compared to 15% error and changes in HBA1c (scenario 1). A 3-year time horizon compares the difference in costs and outcomes for a SMBG device with 10% error as compared to 15% error and changes in severe hypoglycemia (scenario 2). For scenario 2, we used a shorter time horizon of 3 years because the model assumes severe hypoglycemia does not impact microvascular and macrovascular complications and their associated costs. The Markov model is the engine used to generate cumulative incidence estimates, quality-adjusted life years (QALYs), costs, and incremental cost-effectiveness ratios (ICERs) for the purposes of estimating the efficiency of SMBG devices with better accuracy (ie, less error) but higher strip prices.
Clinical Inputs
The model approach links HbA1c to the risk of long-term diabetes complications. Specifically, annual transition probabilities were derived from original Diabetes Control and Complications Trial (DCCT) prediction models, and cumulative probability estimates from the Wisconsin Epidemiologic Study of Diabetes Retinopathy.13-17 For microvascular intermediate disease states and end-stage complications, we used hazard rates from the DCCT calculated by Eastman et al.18,19 Due to a lack of data in type 1 diabetes, for CHD, we used hazard rates derived from the UKPDS population of type 2 diabetes patients. 20
Transition probabilities for different SMBG error rates are based on the changes in HbA1c from Breton and Kovatchev. 3 These transitions represent the movement into intermediate and end-stage diabetes-related complications. As the SMBG errors increase, HbA1c increases from baseline due to the errors, and thus increases the likelihood of entering intermediate complication states (see the appendix for transition probabilities).
Utility Inputs
Health state utilities were derived from a catalogue of EQ-5D preference scores by Sullivan et al, a report from the National Institute of Clinical Excellence (NICE), and from a recent article by McQueen et al.5,21,22 The relationship between HbA1c and health utility is characterized by McQueen et al as a linear relationship. For example, a 1% point increase in HbA1c was associated with a disutility of −0.03. This is incorporated in the model by adjusting the starting utility values at different HbA1c levels. As SMBG error increases, the link to increased HbA1c results in lower utility values. Disutilities by hypoglycemic event and age are not altered by changes in HbA1c (please see the appendix for utility values).
Health State Cost Inputs
Health state cost inputs from the Canadian perspective were derived from a report by the Canadian Agency for Drugs and Technologies in Health (CADTH), a research article by O’Brien et al, and the Ontario Case Costing Initiative (Table 1).23,24 Each health state cost includes a first year event cost depending on the complication. Subsequent costs are included for management of these complications. Health state costs were inflated at a rate of 1.48% using the average percentage changes in health and personal care spending of the consumer price index in Canada. 25 Costs and outcomes were discounted at 5% per annum.
Cost Inputs in 2014 Canadian Dollars.a
Uncertainty incorporated into cost inputs based on the 2.5 and 97.5 percentiles of gamma distributions.
Intervention Cost Inputs
Test strip prices were taken directly from the Ontario Drug Benefit Formulary as of June, 2014. Strip price for a device with 10% error was assumed to be at the maximum allowable per strip, at a price of $0.73 Canadian dollars (CAD) by the Ontario Drug Benefit Formulary/Comparative Drug Index. We found devices with more error (ie, 15-20%) with accompanying test strips with prices ranging from $0.40 to $0.73 CAD. We assumed a conservative price of $0.60 CAD. Utilization of strips per day in Canada was estimated by Roper Global Diabetes. 26 Based on 3.4 strips used per day, total costs for a device with 10% error was $907 CAD per year. Based on the same utilization, total costs for a device with 15% error was $745 CAD per year. SMBG meter costs were assumed to be the same across compared scenarios.
Budget Impact Inputs
A budget impact tool was built to address the expected changes in expenditures from a market mix of blood glucose monitoring devices in Canada.27,28 Some aspects of budget impact modeling include the perspective (Canada payer), size, and characteristics of the eligible population (type 1 diabetes), the use and cost of current and new interventions over time (mix of SMBG devices), impact on other costs (diabetes-related costs), and the ranges of inputs to conduct uncertainty and scenario analyses. We used prevalence and incidence statistics on diabetes from the Public Health Agency of Canada (Table 2). Approximately 0.86% of the Canadian population use SMBG devices to manage type 1 diabetes. The primary output of the budget impact analysis is cost per member per month (PMPM) in CAD over a 3-year time horizon without discounting, as recommended by budget impact guidelines in Canada. 28 If the incremental cost findings from the Markov model are positive, then the PMPM will be positive suggesting higher overall costs to the Canadian payer.
Budget Impact Inputs.
Markov Model Scenario and Sensitivity Analyses
One-way sensitivity analyses vary 1 input, while holding all other inputs constant, to assess the impact on the overall results. For example, for scenarios 1 and 2 we ran the following one-way sensitivity analyses: assumed the same rate of hypoglycemic events as competitor, assumed the same HbA1c as competitor, assumed same strip prices as competitor, and ran the “break-even” point for strip prices to have zero lifetime incremental costs. We also performed multivariate probabilistic sensitivity analyses (PSAs), varying all input parameters simultaneously over their possible ranges and plotting simulation values on a cost-effectiveness plane after assigning evidence-based probability distributions. 29
Results
Clinical Results
Increases in cumulative incidence of complications were observed when SMBG error increased (Table 3). For example, an SMBG error of 10% is associated with a risk of 17.88% of developing diabetic retinopathy over a lifetime horizon; an SMBG error of 15% is associated with a risk of 20.86% for a relative percentage difference of 14% in favor of an SMBG device with 10% error. A significant reduction in risk of 3-year hypoglycemic events requiring an inpatient visit (–16%) was also observed when comparing an SMBG device of 10% error versus a device with 15% error (scenario 2).
Estimates of Clinical Diabetes-Related Complications.
Mean start age 50, baseline HbA1c = 7.6%; 8 all changes in errors move up from 7.6%.
Assuming SMBG errors are associated only with changes in HbA1c.
Assuming SMBG errors are associated only with severe hypoglycemic events requiring an inpatient stay.
Cost-Utility Results
With SMBG errors associated with changes in HbA1c but NOT directly associated with severe hypoglycemic events, the ICER was $11 500 CAD per QALY (Table 4). In other words, it would cost $11 500 for an additional year of perfect health if SMBG errors were only associated with HbA1c changes. With SMBG errors associated with severe hypoglycemic events requiring an inpatient stay but NOT directly associated with changes in HbA1c, an SMBG device with 10% error and strip price of $0.73 is less costly (difference in CAD = −$1393) and more effective (fewer hypoglycemic events per patient = −0.5; more QALYs = 0.003) than an SMBG device with 15% error and strip price of $0.60.
Cost-Utility Results.
HbA1c was increased from a baseline of 7.6% (assuming no error). 8
Changes in HbA1c or separately hypoglycemic events derived from in silico modeling results from Breton and Kovatchev. 3
Dominant SMBG device with 10% error is less costly and more effective than SMBG device with 15% error.
Budget Impact Results
Assuming a market share of 16.3% for devices with 10% error, 30 we estimated the budget impact of scenarios 1 and 2 when increasing the market share to 20% (Table 5). When SMBG error was associated with HbA1c but not directly associated with severe hypoglycemic episodes requiring an inpatient stay, there was a PMPM impact of $0.004 CAD. Given the current Canadian population, type 1 diabetes prevalence, and utilization parameters, the $0.004 CAD PMPM finding translates into $1 600 000 CAD per year when increasing the devices with 10% error market share from 16.3% to 20%. The PMPM impact was cost saving when including the severe hypoglycemic events requiring an inpatient stay and excluding the impact of SMBG errors on changes in HbA1c.
Three-Year Budget Impact Results.
Market share assumed 16.3%. 30
Sensitivity Analyses
Univariate Sensitivity Analyses
For results specific to SMBG errors associated with changes in HbA1c (Table 6): assuming the same strip price between cohorts resulted in a cost-saving scenario (10% SMBG error less costly, more effective than 15% SMBG error). Incremental costs = 0 at a 10% error device with strip price of $0.65. For results specific to SMBG errors associated with changes in hypoglycemia: assuming the cost of a minor hypoglycemic event as estimated by O’Brien et al ($138 vs model assumption of $3728), the ICER is approximately $157 800 per QALY when comparing an SMBG device with 10% error versus 15%. Incremental costs = 0 when reducing the 15% SMBG error device strip price to $0.22.
One-Way Sensitivity Analyses (10% vs 15% Assuming SMBG Error Rates Associated With Changes in HbA1c, and Separately, Severe Hypoglycemic Events).
Cost of hypoglycemic episode based on weighted average of utilization including outpatient, inpatient, and emergency department visits from O’Brien et al. 24
Since the results were cost-saving for scenario 1 (assuming error associated with hypoglycemia only) and strip prices above $0.73 are not currently reimbursed in the Canadian market, we altered the strip price for device with 15% error downward until incremental costs = 0 (break-even point).
Scenario 1: Probabilistic Sensitivity Analyses
Figure 2 displays results from the PSA for scenario 1 assuming SMBG errors are associated with changes in HbA1c only. After varying all inputs simultaneously, the majority of simulations were in the southeast quadrant (67%). In other words, in the majority of simulations an SMBG device with 10% error and strip price of $0.73 is less costly and more effective versus an SMBG device with 15% error and strip price of $0.60. Of simulations, 22% were in the northeast quadrant where an SMBG device with 10% error and strip price of $0.73 is more effective and more costly than an SMBG device with 15% error and strip price of $0.60. Overall, 91% of simulations were cost-effective at a willingness to pay of $100 000 CAD per QALY for an SMBG device with 10% error and strip price of $0.73 versus an SMBG device with 15% error and strip price of $0.60.

Incremental cost-effectiveness scatterplot for scenario 1: SMBG error of 10% with strip price at $0.73 versus SMBG error of 15% with strip price at $0.60 assuming % error associated with HbA1c but NOT directly associated with severe hypoglycemic events requiring inpatient stay. Each point represents 1 Monte Carlo simulation.
Discussion
We estimated scenarios related to changes in blood glucose control (as measured by HbA1c or separately by hypoglycemic events) resulting from differences in SMBG device error. Results for scenario 1 and scenario 2 differed when key drivers of the model were varied, including changes in HbA1c or separately severe hypoglycemic events requiring an inpatient stay. Both scenarios resulted in higher average effectiveness as well as cost-saving scenarios (device with 10% error dominated device with 15% error) when SMBG device error was associated with only 3-year severe hypoglycemic events requiring an inpatient stay. In scenario 1 that included the impact on HbA1c, while holding severe hypoglycemia unchanged, the mean findings were cost-effective with 91% likelihood at a willingness to pay of $100 000 CAD per QALY. The budget impact findings were consistent with the cost findings from the cost-effectiveness analyses of scenarios 1 and 2.
Our results are similar to an analysis by Schnell and colleagues who analyzed potential cost savings from tighter accuracy standards for SMBG devices in Germany. 6 The authors used Breton and Kovatchev’s in silico modeling results to estimate cost savings and clinical effects in expectation of tighter accuracy standards in 2013. The results demonstrated better accuracy (ie, less error) led to reductions in diabetes-related complications and a significant potential for cost-savings for the German health care system. Our research differs in 2 critical points: (1) first we used experiment 4 of Miller et al, 2 which focuses on the glycemic impact over several weeks (Schnell et al focused on experiment 1 which discussed hypoglycemia detection rate); and (2) we focus on devices that meet or exceed the updated ISO 15197:2013 from a Canadian perspective. That is, devices exceeding ISO 15197:2013 have the potential for additional cost-savings over devices that simply meet the accuracy requirements of ISO 15197:2013.
Previous research has also linked better glycemic control to significant cost-savings.4,5,31,32 Using a retrospective analysis, Menzin et al found better glycemic control was associated with reduced admission rates and costs for short-term diabetes-related complications. 32 A more recent retrospective analysis by Aagren and Luo found that a 1% point increase in HbA1c will on average increase diabetes-related medical costs by 6% for individuals with type 1 diabetes. 4 Our analysis differs by the use of simulation modeling methods instead of data from individual patients. However, our study adds to the body of literature suggesting better glycemic control is associated with reduced medical costs.
The results should be interpreted with caution as the Markov model outputs depend on assumptions including the relationship between error in SMBG and both short-term and long-term complications. There is uncertainty in how: (1) error in SMBG impacts HbA1c or separately impacts hypoglycemic events and (2) HbA1c or hypoglycemic events impact on long-term costs and outcomes. The uncertainty in inputs in both phases 1 and 2 are propagated through the computer simulations to yield uncertainty in long-term costs and outcomes. Uncertainty in the modeling structure for phase 2 (HbA1c or hypoglycemic events association with long-term costs and outcomes) is a future line of research. We structured the Markov model (phase 2) to be consistent with prior cohort models evaluating the value of technologies within type 1 diabetes.7,10-12 However, alternative modeling structures could be considered including patient-level simulations where patient-level data are available from in silico modeling outputs.
While both phases relied on computer simulation modeling instead of actual patient data, testing differences in SMBG accuracy in clinical studies falls into the domain of prospective comparative effectiveness research. Given the multiple stakeholders involved in such a prospective study, society represented by government payers is the most likely stakeholder to fund such a study. Moreover, given the high cost to conduct a prospective comparative study and the possible ethical issues related to randomizing participants to less accurate SMBG devices, simulation modeling represents one of the present and likely future best available evidence to address comparative research questions on SMBG accuracy. Alternative clinical study designs, such as retrospective cohort studies, can also be used to investigate the accuracy of SMBG associated with changes in clinical outcomes. For example, Boettcher et al conducted a large multicenter retrospective study in a group of children and adolescents with type 1 diabetes using a mix of different SMBG devices. 33 Similar to the simulation modeling results generated in phase 1 by Breton and Kovatchev, the authors found real-world clinical evidence that SMBG device errors can lead to increases in hypoglycemic events and have a negative impact on HbA1c.
The relationship between HbA1c and complications may not capture all of the benefit that an SMBG device can provide to patients. In this case, the results would be considered conservative estimates of the value of reducing error in SMBG devices. Moreover, epidemiologic data from the DCCT 34 may not capture the current natural history of type 1 diabetes disease progression. In addition, the relationship between SMBG error and severe hypoglycemic episodes are currently based on severe inpatient stays only. Hypoglycemic events may be managed using a combination of multiple health care resources including outpatient visits, inpatient visits, emergency department visits, and even help from a family member. To address this limitation we used a weighted average calculation estimated from O’Brien et al ($138 vs the $3728 defined in the base-case analysis) that combines multiple health care utilization encounters including inpatient stays, outpatient visits, and emergency department visits. 24 The results of this sensitivity analysis suggest the value of investing in SMBG devices with less error is greatest for type 1 diabetes patients at high risk for severe hypoglycemic events.
Improved accuracy of SMBG, even at higher strip prices, ranges from cost saving to good value for money. Holding strip prices at current levels for a device with 10% error from the Canadian perspective is warranted as most analyses were cost-saving or good value for money. Under situations where a 10% error device increased costs (scenario 1), the budget impact analyses suggested the increase in cost per member to be minimal for a typical health insurance plan. From the efficiency (cost-effectiveness) and affordability (budget impact) payer perspectives, reducing the error in SMBG devices appears to be an efficient and affordable strategy.
Footnotes
Appendix
The objective of this study was to estimate the cost-effectiveness of blood glucose monitoring devices with varying blood glucose monitoring accuracy and strip price from the Canadian payer perspective. Specifically, we estimated scenarios related to changes in blood glucose control (as measured by HbA1c or separately, hypoglycemia) resulting from differences in SMBG accuracy. This technical appendix describes the decision-analytic methods, and inputs not described in the article, used to forecast clinical and economic outcomes.
Acknowledgements
We would like to thank Reema Malhorta for her help in collecting Canadian-based inputs for the model analyses.
Abbreviations
CAD, Canadian dollars; CADTH, Canadian Agency for Drugs and Technologies in Health; CHD, coronary heart disease; HbA1c, hemoglobin A1c; ICER, incremental cost-effectiveness ratio; ISO, International Organization for Standardization; QALY, quality-adjusted life year; PMPM, per member per month; PSA, probabilistic sensitivity analysis; SMBG, self-monitoring of blood glucose.
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: (1) Bayer HealthCare; (2) Bayer HealthCare manufactures self-monitoring of blood glucose devices; (3) unrestricted financial research support. RBM has received research support from Bayer HealthCare and served as an advisory board participant and consultant to Bayer HealthCare. MDB has served as an advisory board participant for Bayer HealthCare and Merck; consultant for Roche Diagnostics, The Epsilon Group, BD; honorarium from Roche and Sanofi; research support from Tandem Diabetes, Dexcom, Insulet, Animas, Inspark LLC, Medtronic, BD, Roche, Bayer HealthCare, Sanofi, and Senseonic; stocks with Inspark LLC, TypeZero Technologies LLC. JDC has received research support from Bayer HealthCare. MO, HK, and BB are employees of Bayer HealthCare.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Bayer HealthCare provided funding for this study.
