Comparative Simulation Study of Glucose Control Methods Designed for Use in the Intensive Care Unit Setting via a Novel Controller Scoring Metric

Abstract

Background:

Effective glucose control in the intensive care unit (ICU) setting has the potential to decrease morbidity and mortality rates and thereby decrease health care expenditures. To evaluate what constitutes effective glucose control, typically several metrics are reported, including time in range, time in mild and severe hypoglycemia, coefficient of variation, and others. To date, there is no one metric that combines all of these individual metrics to give a number indicative of overall performance. We proposed a composite metric that combines 5 commonly reported metrics, and we used this composite metric to compare 6 glucose controllers.

Methods:

We evaluated the following controllers: Ideal Medical Technologies (IMT) artificial-intelligence-based controller, Yale protocol, Glucommander, Wintergerst et al PID controller, GRIP, and NICE-SUGAR. We evaluated each controller across 80 simulated patients, 4 clinically relevant exogenous dextrose infusions, and one nonclinical infusion as a test of the controller’s ability to handle difficult situations. This gave a total of 2400 5-day simulations, and 585 604 individual glucose values for analysis. We used a random walk sensor error model that gave a 10% MARD. For each controller, we calculated severe hypoglycemia (<40 mg/dL), mild hypoglycemia (40-69 mg/dL), normoglycemia (70-140 mg/dL), hyperglycemia (>140 mg/dL), and coefficient of variation (CV), as well as our novel controller metric.

Results:

For the controllers tested, we achieved the following median values for our novel controller scoring metric: IMT: 88.1, YALE: 46.7, GLUC: 47.2, PID: 50, GRIP: 48.2, NICE: 46.4.

Conclusion:

The novel scoring metric employed in this study shows promise as a means for evaluating new and existing ICU-based glucose controllers, and it could be used in the future to compare results of glucose control studies in critical care. The IMT AI-based glucose controller demonstrated the most consistent performance results based on this new metric.

Keywords

artificial intelligence glucometrics closed loop control glucose intensive care unit knowledge-based system

Furnary and coauthors first introduced the importance of glucose control in the intensive care unit (ICU) setting in 1999,¹ which van den Berghe et al confirmed in 2001 in the first large prospective randomized trial of tight glucose control in an ICU setting.² Since that time, multiple tight glucose control studies have been reported;^3-6 however, the studies report different glucose metrics, making it difficult to compare study results. A recent invitation-only meeting developed a consensus statement on reporting of individual ICU glucose control metrics.⁷ However, these recommendations have not yet been uniformly adopted. To compare the overall results of the various glucose control methods used in different studies, a comprehensive glucose score measurement is needed to incorporate the individual glucose metrics commonly reported. This comprehensive glucose score will allow for a direct comparison of the global glucose control achieved by different glucose control methods. In addition, a global score will be important for future studies that attempt to utilize machine learning as a means to enhance the effectiveness of glucose controllers, as many machine learning techniques require an objective function to maximize.⁸ This study is an attempt to develop such a global glucose score, while at the same time testing the overall effectiveness of Ideal Medical Technologies novel AI-based glucose controller (IMT),⁹ the Yale protocol (YALE),¹⁰ Glucommander (GLUC),¹¹ Wintergerst et al PID controller (PID),¹² GRIP,¹³ and the NICE-SUGAR (NICE) study protocol¹⁴ through simulation testing.

Methods

We propose a new metric for scoring controller performance called the Glucose Scoring Metric, abbreviated GSM. The GSM offers a flexible framework for combining multiple individual glucose scoring metrics into one readable number by taking each individual metric $m$ from a set of metrics $M$ , with result $p_{m}$ , to which a scoring function $f_{m} (p_{m})$ is applied. Finally, each metric is given a weight $w_{m}$ , and the GSM is computed using a linear combination of the individual scores, which is then scaled to a more readable number:

G S M \equiv 100 \sum_{m \in M} w_{m} f_{m} (p_{m})

To be a valid scoring metric, the following restrictions are applied:

\sum_{m \in M} w_{m} = 1, w_{m} > 0

This ensures that the total weight of all metrics represents 100% of the available score. The second one concerns the scoring functions:

\forall m \in M, p_{m} \in R : 0 \leq f_{m} (p_{m}) \leq 1

Restricting scoring functions to give a result between 0 and 1 ensures that the formerly mentioned linear combination always gives a result between 0 and 100.

In our study, we used the metrics, weights, and scoring functions noted in Table 1.

Table 1.

Glucose Scoring Metric Component Definitions.

Metric $p_{m}$	Weight $w_{m}$	Scoring function $f_{m} (p)$
$p_{s e v} =$ % of values <40 mg/dL	$\frac{2}{19}$	$f_{s e v} (p) = {\begin{matrix} 1 : p_{s e v} = 0 \\ 0 : p_{s e v} \neq 0 \end{matrix}$
$p_{m i l d} =$ % of values 40-69 mg/dL	$\frac{5}{19}$	$f_{m i l d} (p) = \max (h (p), 0)$ $h (p) = 1 + \frac{p}{100} (- 41 + 23 p - 5 p^{2})$
$p_{n o r m} =$ % of values 70-140 mg/dL	$\frac{5}{19}$	$f_{n o r m} (p) = \frac{1}{30} \max (0, p - 70)$
$p_{h y p} =$ % of values >140 mg/dL	$\frac{2}{19}$	$f_{h y p} (p) = \frac{1}{10} \max (0, \min (10, 10 - p))$
$p_{c v} =$ Coefficient of variation (% CV)	$\frac{5}{19}$	$f_{c v} (p) = \frac{1}{30} \min (30, \max (0, 34 - p))$

See the implementation notes in the online supplement for relevant information about reproducing these scoring functions.

We derived each scoring function and weight from previous literature on the subject alongside clinical insights from one of the authors (LD). It is increasingly rare that glucose controllers will allow a severe hypoglycemic event, so if one is observed, the controller is awarded no score for this component of the total score. As this is such a rare occurrence (<3% of simulations experienced a severe hypoglycemic event), the weight for this category is low.

In the case of mild hypoglycemia, the scoring system mentioned in our previous simulation study¹⁵ bins all hypoglycemic (eg, <70 mg/dL) values into their integer floor, and it compares the percentage of hypoglycemic values in each bin to the percentage in each bin of the cumulative distribution given by the ideal distribution of hypoglycemic values. If the percentage observed is greater than the so called “allowed percentage,” the score is reduced by a certain amount, up to and including 0. Otherwise, it is increased by a certain amount, up to and including 0. This method is sound in principle, but it is flawed in 2 subtle ways. The first is that the score will be higher than expected if all of the hypoglycemic values are clustered around a few distinct values; in this case, the observed percentage for all values exterior to these clusters is 0, so all points are awarded in these categories, and points are only lost in categories where the hypoglycemic values are clustered. Second, the method cannot be easily calculated retrospectively, as it requires all individual glucose values and not just the overall incidence of hypoglycemia.

To rectify this issue, we propose that a polynomial be fit to the desired score for overall mild hypoglycemia, heavily rewarding mild hypoglycemia incidence of $< 1 %$ , giving even score to hypoglycemia between $1 %$ and $2 %$ , and heavily penalizing mild hypoglycemia $> 2 %$ . The resulting curve is reduced to integer coefficients for ease of implementation. The scoring function is then scaled and shifted to fit the restrictions on scoring functions. The polynomial we use in this study is seen in Table 1 as $h (p)$ . A plot of this function is shown in Figure 1. A cubic function is chosen so as to achieve the shape which steeply rewards high performance, and even more steeply punishes poor performance, while maintaining ease of implementation. Mild hypoglycemia is given equal weight in the GSM, based on prior clinical studies of glucose metrics that affect mortality rates.^16-18

Figure 1.

The point at (0, 1) is a fixed point in the fit we used. The rational coefficient cubic is obtained by taking the coefficients of the best fit and rounding them to the nearest hundredth, for ease of implementation. The rational coefficient cubic is the one we chose, seen as $h (p)$ in Table 1.

Normoglycemia, here defined as the percentage of values falling between 70 to 140 mg/dL, is scored in a linear fashion based on the integer floor of the reported percentage. Maximum score is awarded for 100% time in normoglycemia, with 0 score being awarded for percentages <70%. Normoglycemia is awarded a larger weight than severe hypoglycemia, given the rarity of severe hypoglycemia with current controllers and that a high time in the 70-140 mg/dL range has been shown to be correlated with decreasing mortality rates.¹⁹

Hyperglycemia, defined as the percentage of values above 140 mg/dL, is given the same lower weight as severe hypoglycemia, as we believe that as overall tight glucose control improves through use of newer control methodologies, mild hypoglycemia, time in range (70-140 mg/dL), and a measurement of dispersion such as coefficient of variation (CV) will be the most important measurements of overall glucose control.¹⁸ The score we award is linear in the integer floor of this percentage, being maximal for no observed hyperglycemia, with no score awarded for hyperglycemia of larger than 10%.

The CV is used as a measure of dispersion. It is given the same weight as mild hypoglycemia and time in normoglycemia, as variability is a key contributor to patient outcomes.^18,20 Given that reducing glucose variability is important to improving outcomes, and because reducing variance to $0 %$ is impossible in a critical care situation, a maximum score is awarded for a CV of $< 5 %$ , with no score awarded for CV $\geq 34 %$ .

Under this implementation of the GSM, we define the “minimum viable controller” as the controller which achieves the best metrics that still results in a GSM score of 0. For our GSM, the minimum viable controller attains the results shown in Table 2.

Table 2.

Minimum Viable Controller Results.

Metric	% < 40 mg/dL	% 40-69 mg/dL	% 70-140 mg/dL	% > 140 mg/dL	% CV
Result (%)	>0	3.83	70	10	34

We introduce this term as a way to contextualize what our scoring system expects from clinical controllers at the bare minimum. Poor performance in the values expressed here has been linked to increased mortality rates.^17,19,21

The most important feature of the GSM is that it is modular. The current metrics, scoring functions, and weights are initial suggestions for this comprehensive metric, and are meant to be modified according to which metrics are most important for patient outcomes, a topic that is still under debate in the critical care community.¹⁸ Alternative metrics could also be included as components in the GSM, such as the letter grading given by Chassin et al.²² Perhaps the most difficult part of adding a new metric $m$ to the set of included metrics $M$ is the design of the scoring function $f_{m} (p_{m})$ . In general, the function must map the entire real line $(R)$ to the unit interval $[0, 1]$ , $f : R \to [0, 1]$ , and in most cases must be monotonic, which means that it is either always increasing or always decreasing.

We employed a method to identify a critical strip $[a, b]$ inside which the score changes, and outside which the score is either 0 or 1. Inside this critical strip, if the desire is to have the function be piecewise constant, or have a “staircase” shape, having a linear transformation of the floor function $⎣ p ⎦,$ ceiling function $⎡ p ⎤$ , or rounding function $[p]$ can be used to “step” when transitioning between integers, or to step at the midpoints between integers. This design can be seen in 4 of the 5 metrics we employed, with the direction of the staircase reflecting whether or not a high value for that metric was more desirable than a low value. If the desire is to have a more complex shape, there are many ways to fit a function to a desired set of points as demonstrated with our mild hypoglycemia scoring function. Generally speaking, cubic functions offer an excellent balance between fit and ease of implementation. Once the critical strip function $c (p)$ is chosen, the full scoring function can be realized in a piecewise fashion. For a metric where smaller outcomes are desired, such as CV, such a function comes in the form:

f_{m} (p) = \max (0, \min (1, c (p))) = {\begin{matrix} 1, p < a \\ c (p), a \leq p \leq b \\ 0, p > b \end{matrix}

With a metric where larger outcomes are desirable, like time in range 70-140 mg/dL, the values outside the critical strip are swapped, but the end result is the same:

f_{m} (p) = \max (0, \min (1, c (p))) = {\begin{matrix} 0, p < a \\ c (p), a \leq p \leq b \\ 1, p > b \end{matrix}

To compare our new scoring method to more established scoring indices, we also calculated the glycemic lability index,²³ glycemic penalty index,²⁴ hyperglycemic index,²⁵ J index,²⁶ and MAGE score.²⁷ To accommodate for variation in sampling frequencies, we normalized the glycemic lability index to represent a uniform 1-hour sampling interval for all 120 hours of simulation:

G L I {(m m o l / l)}^{2} \cdot {(5 d a y s)}^{- 1} = \frac{120}{N} \sum_{n = 1}^{N} \frac{{(G l u c_{n} - G l u c_{n + 1})}^{2}}{(h_{n + 1} - h_{n})}

As the original article allowed for a minimum interval of every 1 hour, this normalization brings our score in line with the original method.²³ As our control range was 100-140 mg/dL, we adjusted the hyperglycemic index to score for area under the curve for values >140 mg/dL.

Simulator

We designed and implemented a new simulator to accommodate the modularity required in implementing additional glucose controllers. We adopted an object-oriented framework, with several abstract classes. The Controller class implements methods by which a given glucose controller can read glucose values from the Sensor class as often as desired, and also make control recommendations via the Prescriptions class. We designed the Prescriptions class such that controllers can make an arbitrary number of fixed control recommendations in between glucose samples, including recommendations to stop all insulin or dextrose, which is sometimes needed by controllers like the Yale protocol. The prescriptions and the cycle length, or time elapsed until the next glucose sample, are then passed to the Model class, which forward-propagates the chosen glucose insulin model²⁸ to the desired time in the future. The resultant glucose value is then passed through a time-variant sensor error model before being passed into the glucose controller. The general layout is shown in Figure 2.

Figure 2.

Class interfaces.

We tasked each controller with controlling 80 different simulated patients across 5 exogenous dextrose infusions for 5 in-silico days to a range of 100-140 mg/dL, except for the NICE-SUGAR controller, which was designed to control to a range of 81-108 mg/dL. We presented details of the patients and infusions used in a previous publication.¹⁵ We employed a modified version of the Van Herpe ICU minimal model, which is described in more detail in our prior publication.¹⁵ The modified Van Herpe model is noted in equations 4-7.

\frac{d G (t)}{d t} = (P_{1} - i_{s} (t) X (t)) G (t) - P_{1} G_{b} + \frac{F_{C G} + F_{E G}}{V (t) V_{G}},

\frac{d X (t)}{d t} = P_{2} X (t) + P_{3} (I_{1} (t) - I_{b})

\frac{d I_{1} (t)}{d t} = α \max (0, I_{2}) - n i_{h} (t) (I_{1} (t) - I_{b}) + \frac{F_{C I}}{V (t) V_{I}}

\frac{d I_{2} (t)}{d t} = β γ (G (t) - h) - n Δ i_{h} (t) I_{2} (t)

In our previous simulation study,¹⁵ we employed a sensor error model that involved sampling from a $U n i [- a, a]$ probability distribution, where $a$ is the maximum allowed relative difference for that simulation. We use this value, denoted $r$ , as a relative difference to perturb the glucose signal as follows: $X = (1 + r) G$ , where $X$ is the sensor value read by the controller, and $G$ is the glucose value derived by the model. For any distribution of $r$ , the mean absolute relative difference (MARD) can be calculated directly with the distribution of $r$ :

\begin{array}{l} M A R D = E [\frac{| X - G |}{G}] = E [\frac{| (1 + r) G - G |}{G}] \\ = E [\frac{| r G |}{G}] = E [| r |] \end{array}

For the memoryless uniform random variable employed previously,¹⁵ it can be seen that

M A R D_{m e m o r y l e s s} = E [| U n i [- a, a] |] = \frac{| a |}{2}

While this approach does give a well perturbed signal, there is no memory inherent to the process, so it is possible to jump from $- a$ to $a$ between consecutive samples, regardless of the temporal distance between those samples. To ground this sensor error model in reality, we propose the use of a random process as a means of generating the relative difference $r$ via a random walk with reflection.

By using a random process, the relative difference has memory, and can thus be used to simulate the drift of sensor error with time. We denote the relative difference at time step $n$ by $r_{n}$ . Between samples, we allow the relative difference to drift by up to ${\frac{d r}{d t}}_{m}$ percentage per minute, which generates a cone of uncertainty from which the next relative difference is uniformly chosen. With a time between samples of ∆t minutes, let $R = {\frac{d r}{d t}}_{m} Δ t$ denote the half width of our cone. The process can then be defined as follows:

y_{n} = r_{n - 1} + U n i [- R, R]

r_{n} = {\begin{matrix} y_{n}, - a \leq y_{n} \leq a \\ 2 a - y_{n}, y_{n} > a \\ - 2 a - y_{n}, y_{n} < - a \end{matrix}

If the resultant relative difference after sampling ( $y_{n})$ is > $a$ , or < $- a$ , the reflection across the nearest barrier is taken, giving $r_{n}$ . The benefit is 2-fold: first, all relative differences are guaranteed to be between $- a$ and $a$ , and second, the random walk taken with reflection at these 2 barriers has an identical MARD to the memoryless uniform. The proof is outside the scope of this article.

M A R D_{c o n e} = \frac{| a |}{2}

The relative difference profiles that results from this process therefore does not compromise the MARD of the sensor being modelled, but produces a profile that wanders in time. This means that consecutive samples can expect to see similar relative differences, but over time, the relative difference of the sensor drifts. An extreme example of the reflection process is seen in Figure 3.

Figure 3.

Relative difference profile of a sensor with a large drift shows frequent barrier reflections (depicted in red), with all relative differences remaining inside the expected range.

When comparing Figures 4 and 5, the benefits of the random walk method over the memoryless uniform method become apparent, as the random walk method more closely resembles the curves taken from actual continuous glucose monitors.²⁹

Figure 4.

Over time, the random walk method allows the relative difference to wander within the allowed range.

Figure 5.

The memoryless uniform exhibits a much noisier relative difference profile.

While the random walk method of sensor error modelling is not a replacement for the autoregressive sensor error modelling seen in other publications,^29,30 it does offer a lower computational overhead, a more simplistic implementation, and an MARD that is mathematically guaranteed. For the purposes of this study, we used the values in Table 3 in the sensor error model.

Table 3.

Random Walk Method Study Parameters.

$a$	${\frac{d r}{d t}}_{m}$	∆t	$r_{0}$
.2	.004	Controller-dependent	$0$

The merits and limitations of this sensor error model will have to be studied in further detail to determine applicability to clinical scenarios.

Statistical Methods

For each of the 2400 unique simulations, we calculated the mean glucose, coefficient of variation of glucose, percentage of glucose values <40 mg/dL, percentage of glucose values between 40 to 69 mg/dL, percentage of values in controller-defined range (100-140 mg/dL, with the exception of NICE which uses 81-108 mg/dL), percentage of glucose values in the normoglycemic range of 70-140 mg/dL, and percentage of glucose values >140 mg/dL. With the exception of incidence of severe hypoglycemia, all values are reported as the median and interquartile range. Severe hypoglycemia is reported as the percentage of simulations that experienced one or more severe hypoglycemic events (<40 mg/dL).

Time-Variant Model Parameters

In addition to controller flows, we perturbed our model with a time-variant insulin sensitivity, time-variant insulin half-life, time-variant insulin/glucose volume of distribution, and unannounced exogenous dextrose infusions as described in the supplement of our previous paper.¹⁵ Time-variant model parameters were important for transforming the 2-compartment model used in the original ICU minimal model²⁸ into a model that features the complexity of clinical situations. As an example, consider the glucose trace achieved in Figure 6 when an exogenous dextrose infusion of 5 mg/kg/minute is applied.

Figure 6.

Native response with time-variant sensitivity (TVS) and without (CS). CS, constant sensitivity. Time 0 glucose = 200 mg/dL. Continuous dextrose infusion of 5 mg/kg/min.

As can be seen from Figure 6, the time-variant insulin sensitivity has significant effects on the native response to the exogenous dextrose infusion. Because the insulin sensitivity curve used was <100%, the native system was much less capable of responding to the incoming dextrose infusion. Figures 7 and 8 demonstrate the effects of time-variant insulin half-life and volume of distribution, with the latter having the least effect of the time-variant parameters.

Figure 7.

Native response with time-variant half-life (TVHL) and without (CHL). CHL, constant half-life. Time 0 glucose = 200 mg/dL. Continuous dextrose infusion of 5 mg/kg/min.

Figure 8.

Native response with time-variant volume of distribution (TVVD) and without (CVD). CVD, constant volume of distribution. Time 0 glucose = 200 mg/dL. Continuous dextrose infusion of 5 mg/kg/min.

The insulin sensitivity underwent between day changes of 0-67%, except for 2 outliers of 150 and 600%. These insulin sensitivity changes were within the range reported by Pretty et al.³¹ The insulin half-life and volume of distribution underwent between day changes in the range of 0-33%, except for one simulated trauma patient whose volume of distribution increased by 62% on day 1.

A total of 5 exogenous dextrose infusions were used; 4 were clinically relevant and 1 was a “stress test” used to evaluate the controller’s ability to deal with frequent, unannounced, random changes in the exogenous infusion. In addition, we employed 4 time-variant sensitivity curves, 3 time-variant volume of distribution curves, and 3 time-variant insulin half-life curves. We also included the unmodified original parameters, giving a total of 80 unique simulated patients to control. The details of these simulated patients can be found in the supplementary material of our previous simulation study.¹⁵ This resulted in 400 total simulations per controller. The comparator controllers were set to cycle according to their design documents.

Results

The time in range results are presented in Table 4, with mean glucose, coefficient of variation, average infusion rates and MARD results presented in Table 5.

Table 4.

Comparative Simulation Study Results.

	Severe hypoglycemia incidence (%)	Mild hypoglycemia (40-69 mg/dL)	Control range (100-140 mg/dL)*	Normoglycemia (70-140 mg/dL)	Hyperglycemia (>140 mg/dL)
IMT	0.00	0.00 (0.00-0.00)	92.6 (89.8-96.2)	96.7 (94.6-97.9)	3.3 (2.1-5.4)
Yale	0.25	0.81 (0.00-1.61)	64.1 (54.3-72.7)	71.5 (63.8-77.7)	27.8 (22.1-34.4)
Glucommander	4.50	1.63 (0.82-3.23)	67.6 (57.6-76.4)	78.0 (70.7-84.1)	20.2 (14.8-26.4)
PID	4.50	0.80 (0.00-2.43)	63.8 (52.5-71.2)	75.7 (67.8-81.4)	22.6 (17.0-30.6)
GRIP	6.25	1.66 (0.00-3.01)	65.1 (51.3-75.9)	76.1 (65.6-83.8)	20.7 (15.3-30.7)
NICE	2.50	3.97 (0.00-6.40)	42.8* (33.6-52.6)	89.0 (80.9-95.1)	6.7 (3.3-12.8)

Values are median (25-75) percentage of all glucose values in given range, except for severe hypoglycemia, which is percentage of simulations.

NICE results are for time in range 81-108 mg/dL.

Table 5.

Central Tendency, Dispersion, Infusion Rate, and MARD Results of Comparative Simulation Study.

	Mean (mg/dL)	CV (%)	Dextrose rate (mg/kg/min)	Insulin rate (U/kg/hr)	MARD (%)
IMT	119 (118-120)	9.9 (8.4-11.5)	0.28 (0.16-0.49)	0.074 (0.055-0.101)	9.74 (8.96-10.60)
Yale	127 (125-130)	18.3 (15.1-21.2)	0.35 (0.00-0.52)	0.056 (0.035-0.081)	10.01 (9.62-10.45)
Glucommander	123 (121-126)	18.2 (14.7-21.5)	0.08 (0.03-0.13)	0.060 (0.040-0.085)	9.99 (9.62-10.36)
PID	125 (121-129)	19.1 (15.9-22.3)	0.19 (0.12-0.29)	0.057 (0.039-0.075)	9.99 (9.48-10.55)
GRIP	123 (122-128)	19.0 (14.7-22.0)	0.00* (0.00-0.00)	0.057 (0.040-0.076)	9.98 (9.57-10.45)
NICE	108 (105-112)	20.6 (17.0-24.2)	0.02 (0.00-0.05)	0.079 (0.058-0.105)	10.06 (9.71-10.39)

GRIP does not have an automated dextrose component in its specification. Results are median (25-75).

In addition, we computed the GSM defined in Table 1 for every controller. We present the overall score and the individual metric scores as the median and interquartile range, with the exception of the severe hypoglycemia metric, for which every controller received the same median and interquartile score of 1 (1-1). Overall score is given as a raw value, and individual metric scores are converted to a percentage of total available score for readability. The overall GSM score is noted in Figure 9, mild hypoglycemia in Figure 10, normoglycemia in Figure 11, hyperglycemia in Figure 12, and coefficient of variation in Figure 13.

Figure 9.

Overall GSM scores by controller. Results are median (25-75).

Figure 10.

Mild hypoglycemia (40-69 mg/dL) scores by controller. Results are median (25-75).

Figure 11.

Normoglycemia (70-140 mg/dL) scores by controller. Results are median (25-75).

Figure 12.

Hyperglycemia (>140 mg/dL) scores by controller. Results are median (25-75).

Figure 13.

Coefficient of variation scores by controller. Results are median (25-75).

Table 6 shows the results of the overall GSM score in addition to the glycemic penalty index (GPI), glycemic lability index (GLI), hyperglycemic index (HI), J index, and mean amplitude of glycemic excursion (MAGE) score.

Table 6.

All Controllers Controlled to a Range of 100-140 mg/dL, Except NICE, Which Controlled to a Range of 81-108 mg/dL.

	GSM (unitless, range 0-100)	GPI* [C_Hypo (%) – C_Hyper (%)]	GLI** (mmol/L)² × (5 days)^-1	HI*** (mg/dL)	J index (mg/dL)²	MAGE (mg/dL)
IMT	88 (82-92)	19 (18-20)	32 (26-39)	0.5 (0.3-0.9)	17 (16.7-17.7)	27 (23-31)
Yale	47 (40-58)	30 (27-32)	414 (97-701)	3.3 (2.1-5.3)	22.8 (21.1-24.7)	53 (42-67)
GLUC	47 (35-59)	27 (24-29)	171 (122-232)	2.1 (1-3.4)	21.4 (20.1-22.9)	45 (36-54)
PID	50 (38-58)	27 (24-30)	158 (124-196)	2.3 (1.4-4.4)	21.8 (20.7-23.6)	49 (41-58)
GRIP	48 (32-62)	27 (24-31)	97 (71-130)	3.2 (1.7-5.9)	21.8 (20-24.2)	49 (40-64)
NICE	46 (29-77)	15 (10-20)	107 (75-143)	0.8 (0.4-1.9)	17 (15.5-18.9)	44 (36-54)

Penalizes for time outside of range 80-110 mg/dL. **Normalized to hourly glucose samples. ***Used area under the curve for data >140 mg/dL. Results are median (25-75).

Figure 14 shows the glucose versus time curves for all 6 controllers for a single simulation scenario. The sensor error model was turned off for this scenario to allow for smoother curves and improved legibility.

Figure 14.

Individual glucose traces for one test scenario with starting glucose 200 mg/dL. All controllers controlled to a range of 100-140 mg/dL, except NICE, which controlled to a range of 81-108 mg/dL. CRMin, control range minimum = 100 mg/dL; CRMax, control range maximum = 140 mg/dL.

Discussion

Effective glucose control in the ICU setting has the potential to save lives³ and to decrease the economic burden of health care.^32-36 However, it is difficult to compare the numerous publications on this subject as they frequently report different results. While there is some consensus on how to report individual glucose metrics,⁷ having a universal glucose control metric will be instrumental in future development of closed loop glucose controllers. While there has been some early work done on developing a universal glucose control metric for type 1 diabetes,²² the only global index attempted for ICU patients was based on a normoglycemia range of 80-110 mg/dL,²⁴ which is no longer the recommended control range.³⁷ As glucose control in the ICU setting continues to evolve toward closed loop control, a global score of overall glucose control will allow researchers to compare different glucose control methods as they attempt to find the most efficacious control strategy. To be useful, such a score should utilize the most commonly reported glucose metrics that have also been shown to be predictive of patient outcomes.¹⁸

In creating the GSM, we chose the commonly reported individual glucose metrics of severe hypoglycemia (<40 mg/dL), mild hypoglycemia (40-69 mg/dL), time in normoglycemic range (70-140 mg/dL), time in hyperglycemic range (>140 mg/dL), and percentage coefficient of variation. We weighted the GSM more heavily toward mild hypoglycemia, time in normoglycemic range, and coefficient of variation as previous publications have indicated these are the most important in determining mortality rates,^18,19 and we believe that as future glucose control methodologies become more effective, severe hypoglycemia and severe hyperglycemia will mostly be eliminated. Our global GSM score represents an attempt to create a universal score that can be utilized by different authors to report on the overall effectiveness of their glucose control. We expect that the weighting of this score will change once it is validated against clinical results such as mortality rates. The modular nature of our scoring system will allow for easy substitution or addition of different individual glucose metrics, assuming such modifications have been proven through robust clinical studies to be superior to those we have proposed. The GSM also has the potential to have differing realizations according to the group being evaluated, including an individual GSM for individuals with type 1 or type 2 diabetes, general hospital admissions, ICU care, and so on.

We performed a simulation study to compare 5 currently utilized ICU glucose controllers to our novel AI-based glucose controller using the GSM. For each controller we measured the 5 individual glucose metrics that are part of the GSM, and calculated the overall GSM. As can be seen from Tables 4 and 5, our AI-based glucose controller outperformed all 5 of the comparator controllers across the majority of the individual metrics. The AI-based controller achieved a high time in range and low CV while at the same time avoiding any hypoglycemic events when the control range was set to 100-140 mg/dL. Our overall GSM score was 76% better than the next best controller: the Wintergerst PID controller. As noted in Table 6, our AI-based controller on the whole outperformed the other controllers using currently accepted and validated scoring metrics. Although the results in Table 6 do not serve to validate the GSM scoring metric, they do show that it is at least consistent with the other indices in assessing overall controller performance. It should be noted that the NICE-SUGAR controller was the least effective controller tested based on the GSM score, which may potentially explain the lack of positive findings in the NICE-SUGAR study.³⁸

The AI controller achieved its enhanced control through improved dosing methods of insulin and/or dextrose, as it did not have the maximal infusion rates for either insulin or dextrose as noted in Table 5. One of the reasons the AI-based glucose controller outperforms the other controllers is its short cycle interval of every 5-10 minutes. The other controllers cycle themselves only once every 20-120 minutes, thus they are incapable of maintaining control when there are rapid changes in the glucose versus time curve, such as during rapid changes in exogenous dextrose infusion rates or in insulin sensitivity. As the AI controller was patterned after the native system, which releases variable quantities of insulin every 5 minutes and can quickly change hepatic glucose output in an effort to maintain euglycemia,³⁹ from its inception it was designed with a rapid cycle interval in mind. In internal testing the AI controller remains significantly more effective than Glucommander and PID, even when these controllers are set to cycle every 10 minutes. The latter results are reflective of the AI controllers control to range strategy, and use of current glucose value in relation to the desired control range, glucose rate of change, and weight-based insulin/dextrose doses when making all control decisions.

Effective glucose control in the ICU setting that achieves a high time in range while at the same time avoiding any hypoglycemia and minimizing variability has the potential to significantly decrease morbidity and mortality rates, while at the same time lowering overall length of stay and thus health care costs. To date, only one study has been performed that has achieved these goals.⁴⁰ In this Japanese study on hepatobiliary-pancreatic surgery patients, the tight glucose control group had a 58% reduction of their surgical site infection rate, and a 21% reduction of their length of stay. As our AI-based controller achieves results in simulation testing that are at least as good as those achieved in the Japanese study, we are hopeful that it will hold up to the challenging environment of the ICU setting and be available as the control arm of an artificial pancreas system.

Conclusion

The GSM proposed in this study successfully allows for direct comparison of different glucose control methodologies. The AI-based glucose controller that we tested outperformed all of the currently in-use ICU-based glucose controllers to which we compared it in this simulation study.

Footnotes

Abbreviations

AI, artificial intelligence; CHL, constant half-life; CS, constant sensitivity; CV, coefficient of variation; CVD, constant volume of distribution; CRMax, control range maximum; CRMin, control range minimum; GLI, glycemic lability index; GLUC, Glucommander; GPI, glycemic penalty index; GSM, Glucose Scoring Metric; HI, hyperglycemic index; ICU, intensive care unit; IMT, Ideal Medical Technologies; MAGE, mean amplitude of glycemic excursion; MARD, mean absolute relative difference; NICE, NICE-SUGAR; PID, Wintergerst et al PID controller; TVHL, time-variant half-life; TVS, time-variant sensitivity; TVVD, time-variant volume of distribution; YALE, Yale protocol.

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: LD and JD are stock holders in Ideal Medical Technologies Inc.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Supplementary Material

Supplementary material is available for this article online.

References

Furnary

Zerr

Grunkemeier

Starr

Continuous intravenous insulin infusion reduces the incidence of deep sternal wound infection in diabetic patients after cardiac surgical procedures. Ann Thorac Surg. 1999;67:352-360; discussion 60-62.

van den Berghe

Wouters

Weekers

et al . Intensive insulin therapy in critically ill patients. N Engl J Med. 2001;345:1359-1367.

Krinsley

Glucose control reduces ICU stay and mortality. Perform Improv Advis. 2005;9:4-6, 1.

Aldawood

Tamim

Alsultan

Rishu

Arabi

YM.

Intensive insulin therapy versus conventional insulin therapy for critically ill trauma patients admitted to ICU. Middle East J Anaesthesiol. 2010;20:659-666.

Lazar

Chipkin

Fitzgerald

Bao

Cabral

Apstein

CS.

Tight glycemic control in diabetic coronary artery bypass graft patients improves perioperative outcomes and decreases recurrent ischemic events. Circulation. 2004;109:1497-1502.

Macrae

Grieve

Allen

et al . A clinical and economic evaluation of Control of Hyperglycaemia in Paediatric intensive care (CHiP): a randomised controlled trial. Health Technol Assess. 2014;18:1-210.

Finfer

Wernerman

Preiser

et al . Clinical review: consensus recommendations on measurement of blood glucose and reporting glycemic control in critically ill adults. Crit Care. 2013;17:229.

Bothe

Dickens

Reichel

et al . The use of reinforcement learning algorithms to meet the challenges of an artificial pancreas. Expert Rev Med Devices. 2013;10:661-673.

Computerized system for blood chemistry monitoring. USPTO. 2015. Available at: http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PALL&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.htm&r=1&f=G&l=50&s1=8956321.PN.&OS=PN/8956321&RS=PN/8956321. Accessed December 6, 2016.

10.

Yale-New Haven Hospital. ICU Insulin Infusion Protocol (IIP) for Adults. 2011. Available at: http://inpatient.aace.com/sites/all/files/Yale_IIP_MICU120-160_2011.pdf. Accessed December 28, 2016.

11.

Davidson

Steed

Bode

BW.

Glucommander: a computer-directed intravenous insulin system shown to be safe, simple, and effective in 120,618 h of operation. Diabetes Care. 2005;28:2418-2423.

12.

Wintergerst

Deiss

Buckingham

et al . Glucose control in pediatric intensive care unit patients using an insulin-glucose algorithm. Diabetes Technol Ther. 2007;9:211-222.

13.

Vogelzang

Zijlstra

Nijsten

MW.

Design and implementation of GRIP: a computerized glucose control system at a surgical intensive care unit. BMC Med Inform Decis Mak. 2005;5:38.

14.

NICE SUGAR study treatment algorithm. 2009. Available at: https://studies.thegeorgeinstitute.org/nice/docs/ALGORITHM.pdf. Accessed October 4, 2016.

15.

DeJournett

In silico testing of an artificial-intelligence-based artificial pancreas designed for use in the intensive care unit setting. J Diabetes Sci Technol. 2016;10:1360-1371.

16.

Hulkower

Pollack

Zonszein

Understanding hypoglycemia in hospitalized patients. Diabetes Manag (Lond). 2014;4:165-176.

17.

Egi

Bellomo

Stachowski

et al . Hypoglycemia and outcome in critically ill patients. Mayo Clin Proc. 2010;85:217-224.

18.

Mackenzie

Whitehouse

Nightingale

PG.

The metrics of glycaemic control in critical care. Intensive Care Med. 2011;37:435-443.

19.

Krinsley

Preiser

JC.

Time in blood glucose range 70 to 140 mg/dl >80% is strongly associated with increased survival in non-diabetic critically ill adults. Crit Care. 2015;19:179.

20.

Krinsley

JS.

Glycemic variability and mortality in critically ill patients: the impact of diabetes. J Diabetes Sci Technol. 2009;3:1292-1301.

21.

Falciglia

Freyberg

Almenoff

D’Alessio

Render

ML.

Hyperglycemia-related mortality in critically ill patients varies with admission diagnosis. Crit Care Med. 2009;37:3001-3009.

22.

Chassin

Wilinska

Hovorka

Grading system to assess clinical performance of closed-loop glucose control. Diabetes Technol Ther. 2005;7:72-82.

23.

Ryan

Shandro

Green

et al . Assessment of the severity of hypoglycemia and glycemic lability in type 1 diabetic subjects undergoing islet transplantation. Diabetes. 2004;53:955-962.

24.

Van Herpe

De Brabanter

Beullens

De Moor

Van den Berghe

. Glycemic penalty index for adequately assessing and comparing different blood glucose control algorithms. Crit Care. 2008;12:R24.

25.

Vogelzang

van der Horst

Nijsten

MW.

Hyperglycaemic index as a tool to assess glucose control: a retrospective study. Crit Care 2004;8:R122-R127.

26.

Service

FJ.

Glucose variability. Diabetes. 2013;62:1398-1404.

27.

Baghurst

PA.

Calculating the mean amplitude of glycemic excursion from continuous glucose monitoring data: an automated algorithm. Diabetes Technol Ther. 2011;13:296-302.

28.

Herpe

Espinoza

Haverbeke

Moor

den Berghe

GV.

Glycemia prediction in critically ill patients using an adaptive modeling approach. J Diabetes Sci Technol. 2007;1:348-356.

29.

Facchinetti

Del Favero

Sparacino

Castle

Ward

Cobelli

Modeling the glucose sensor error. IEEE Trans Biomed Eng. 2014;61:620-629.

30.

Facchinetti

Sparacino

Cobelli

Modeling the error of continuous glucose monitoring sensor data: critical aspects discussed through simulation studies. J Diabetes Sci Technol. 2010;4:4-14.

31.

Pretty

Le Compte

Chase

et al . Variability of insulin sensitivity during the first 4 days of critical illness: implications for tight glycemic control. Ann Intensive Care. 2012;2:17.

32.

Scurlock

Raikhelkar

Mechanick

JI.

The economics of glycemic control in the ICU in the United States. Curr Opin Clin Nutr Metab Care. 2011;14:209-212.

33.

Krinsley

JS.

Is glycemic control of the critically ill cost-effective?

Hosp Pract (1995). 2014;42:53-58.

34.

Van den Berghe

Wouters

Kesteloot

Hilleman

DE.

Analysis of healthcare resource utilization with intensive insulin therapy in critically ill patients. Crit Care Med. 2006;34:612-616.

35.

Sadhu

Ang

Ingram-Drake

Martinez

Hsueh

Ettner

SL.

Economic benefits of intensive insulin therapy in critically Ill patients: the targeted insulin therapy to improve hospital outcomes (TRIUMPH) project. Diabetes Care. 2008;31:1556-1561.

36.

Cardona

Pasquel

Fayfman

et al . Hospitalization costs and clinical outcomes in CABG patients treated with intensive insulin therapy. J Diabetes Complications. 2017;31(4):742-747.

37.

American Diabetes Association. 13. Diabetes Care in the Hospital. Diabetes Care. 2016;39(suppl 1):S99-S104.

38.

NICE-SUGAR Study Investigators, Finfer

Chittock

et al . Intensive versus conventional glucose control in critically ill patients. N Engl J Med. 2009;360:1283-1297.

39.

DeJournett

Essential elements of the native glucoregulatory system, which, if appreciated, may help improve the function of glucose controllers in the intensive care unit setting. J Diabetes Sci Technol. 2010;4:190-198.

40.

Okabayashi

Shima

Sumiyoshi

et al . Intensive versus intermediate glucose control in surgical intensive care unit patients. Diabetes Care. 2014;37:1516-1524.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.14 MB